.. _local_data_skels_tut:

Skeletons
---------

``navis`` lets you import neurons from a variety of local and remote sources. In this tutorial you will learn how to load skeletons from local data and - failing that - construct them from scratch. For loading remote data (e.g. the MICrONS, neuromorpho, Virtual Fly Brain or Janelia hemibrain datasets), ``navis`` has dedicated interfaces. See the :ref:`example gallery <example_gallery>` for tutorials on these.


Skeletons in ``navis`` are represented by the :class:`navis.TreeNeuron` class. You can either construct these manually (see bottom of this page) or use one of the built-in functions to them from one of the various file formats:

SWC files
=========

SWC is a common format for storing neuron skeletons. Thus ``navis`` provides functions to both read and write SWC files. To demo these, we will be using supplemental data from Bates, Schlegel et al. (Current Biology, 2020). If you want to follow along, please download Supplemental Data S1 (`link <https://doi.org/10.1016/j.cub.2020.06.042>`_). If you do, make sure to adjust the filepath accordingly.

.. code:: ipython3

    import navis

I extracted the archive with the supplemental data inside my downloads folder. It contains a bunch of CSV files with meta data but the important file for us is the ``skeletons_swc.zip``. Now you could extract that zip archive too but ``navis`` can actually read directly from (and write to) zip files!

.. code:: ipython3

    skeletons = navis.read_swc('~/Downloads/mmc2/skeletons_swc.zip', include_subdirs=True)
    skeletons








.. raw:: html

    &lt;class 'navis.core.neuronlist.NeuronList'&gt; containing 480 neurons (101.1MiB)<div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>type</th>
          <th>name</th>
          <th>n_nodes</th>
          <th>n_connectors</th>
          <th>n_branches</th>
          <th>n_leafs</th>
          <th>cable_length</th>
          <th>soma</th>
          <th>units</th>
          <th>created_at</th>
          <th>origin</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>navis.TreeNeuron</td>
          <td>1313071</td>
          <td>11634</td>
          <td>None</td>
          <td>600</td>
          <td>611</td>
          <td>2899338.75</td>
          <td>None</td>
          <td>1 dimensionless</td>
          <td>2022-05-09 08:44:26.013419</td>
          <td>/Users/philipps/Downloads/mmc2/skeletons_swc.zip</td>
        </tr>
        <tr>
          <th>1</th>
          <td>navis.TreeNeuron</td>
          <td>5002989</td>
          <td>1670</td>
          <td>None</td>
          <td>24</td>
          <td>25</td>
          <td>628291.75</td>
          <td>None</td>
          <td>1 dimensionless</td>
          <td>2022-05-09 08:44:26.009716</td>
          <td>/Users/philipps/Downloads/mmc2/skeletons_swc.zip</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>478</th>
          <td>navis.TreeNeuron</td>
          <td>11519759</td>
          <td>14782</td>
          <td>None</td>
          <td>663</td>
          <td>673</td>
          <td>2714858.25</td>
          <td>None</td>
          <td>1 dimensionless</td>
          <td>2022-05-09 08:44:29.017471</td>
          <td>/Users/philipps/Downloads/mmc2/skeletons_swc.zip</td>
        </tr>
        <tr>
          <th>479</th>
          <td>navis.TreeNeuron</td>
          <td>11543484</td>
          <td>6256</td>
          <td>None</td>
          <td>367</td>
          <td>371</td>
          <td>1229948.00</td>
          <td>None</td>
          <td>1 dimensionless</td>
          <td>2022-05-09 08:44:29.010490</td>
          <td>/Users/philipps/Downloads/mmc2/skeletons_swc.zip</td>
        </tr>
      </tbody>
    </table>
    </div>



Let's say you are looking at a huge collection of SWC files and you only want to sample a few of them:

.. code:: ipython3

    # Load only the first 10 skeletons
    sample = navis.read_swc('~/Downloads/mmc2/skeletons_swc.zip', include_subdirs=True, limit=10)
    sample








.. raw:: html

    &lt;class 'navis.core.neuronlist.NeuronList'&gt; containing 10 neurons (1.1MiB)<div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>type</th>
          <th>name</th>
          <th>n_nodes</th>
          <th>n_connectors</th>
          <th>n_branches</th>
          <th>n_leafs</th>
          <th>cable_length</th>
          <th>soma</th>
          <th>units</th>
          <th>created_at</th>
          <th>origin</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>navis.TreeNeuron</td>
          <td>1313071</td>
          <td>11634</td>
          <td>None</td>
          <td>600</td>
          <td>611</td>
          <td>2.899339e+06</td>
          <td>None</td>
          <td>1 dimensionless</td>
          <td>2022-05-09 08:44:44.072213</td>
          <td>/Users/philipps/Downloads/mmc2/skeletons_swc.zip</td>
        </tr>
        <tr>
          <th>1</th>
          <td>navis.TreeNeuron</td>
          <td>5002989</td>
          <td>1670</td>
          <td>None</td>
          <td>24</td>
          <td>25</td>
          <td>6.282918e+05</td>
          <td>None</td>
          <td>1 dimensionless</td>
          <td>2022-05-09 08:44:44.099230</td>
          <td>/Users/philipps/Downloads/mmc2/skeletons_swc.zip</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>8</th>
          <td>navis.TreeNeuron</td>
          <td>6082926</td>
          <td>3112</td>
          <td>None</td>
          <td>164</td>
          <td>171</td>
          <td>8.933154e+05</td>
          <td>None</td>
          <td>1 dimensionless</td>
          <td>2022-05-09 08:44:44.289857</td>
          <td>/Users/philipps/Downloads/mmc2/skeletons_swc.zip</td>
        </tr>
        <tr>
          <th>9</th>
          <td>navis.TreeNeuron</td>
          <td>2122447</td>
          <td>2694</td>
          <td>None</td>
          <td>112</td>
          <td>115</td>
          <td>8.098385e+05</td>
          <td>None</td>
          <td>1 dimensionless</td>
          <td>2022-05-09 08:44:44.312675</td>
          <td>/Users/philipps/Downloads/mmc2/skeletons_swc.zip</td>
        </tr>
      </tbody>
    </table>
    </div>



We can also point :func:`navis.read_swc` at single files instead of folders or zip archives:

.. code:: ipython3

    # For this I extraced the skeletons_swc.zip archive
    s = navis.read_swc('~/Downloads/mmc2/swc/CENT/11519759.swc')
    s




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>type</th>
          <td>navis.TreeNeuron</td>
        </tr>
        <tr>
          <th>name</th>
          <td>11519759</td>
        </tr>
        <tr>
          <th>n_nodes</th>
          <td>14782</td>
        </tr>
        <tr>
          <th>n_connectors</th>
          <td>None</td>
        </tr>
        <tr>
          <th>n_branches</th>
          <td>663</td>
        </tr>
        <tr>
          <th>n_leafs</th>
          <td>673</td>
        </tr>
        <tr>
          <th>cable_length</th>
          <td>2714858.25</td>
        </tr>
        <tr>
          <th>soma</th>
          <td>None</td>
        </tr>
        <tr>
          <th>units</th>
          <td>1 dimensionless</td>
        </tr>
        <tr>
          <th>created_at</th>
          <td>2022-04-14 22:01:20.455672</td>
        </tr>
        <tr>
          <th>origin</th>
          <td>/Users/philipps/Downloads/mmc2/swc/CENT/115197...</td>
        </tr>
      </tbody>
    </table>
    </div>



You can even use URLs directly:

.. code:: ipython3

    s = navis.read_swc('http://neuromorpho.org/dableFiles/jefferis/CNG%20version/AV4c1_140213c1.CNG.swc')

Now let's say you have skeletons and you want to save them to disk. Easy!

.. code:: ipython3

    # Write a single neuron 
    navis.write_swc(s, '~/Downloads/mmc2/my_neuron.swc')

.. code:: ipython3

    # Write a whole list of skeletons to a folder and use the neurons' `name` property as filename
    navis.write_swc(sample, '~/Downloads/mmc2/{neuron.name}.swc')






.. code:: ipython3

    # Write directly to a zip file 
    navis.write_swc(sample, '~/Downloads/mmc2/skeletons.zip')






.. code:: ipython3

    # Write directly to a zip file and use the neuron name as filename
    navis.write_swc(sample, '~/Downloads/mmc2/{neuron.name}.swc@skeletons.zip')






See :func:`navis.write_swc` for further details! 

NMX files
=========

NMX is a xml-based format used e.g. by pyKNOSSOS to store skeletons (+ meta data). ``navis`` supports reading (but not writing) this format. If you want to follow along download `this dataset <https://doi.org/10.5281/zenodo.58985>`_ by Wanner et al. (2016). Same procedure as with the SWCs: I extracted the archive to my downloads folder. 

.. code:: ipython3

    # Read a single file
    s = navis.read_nmx('~/Downloads/WannerAA201605_SkeletonsGlomeruli/Neuron_id0001.nmx')
    s




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>type</th>
          <td>navis.TreeNeuron</td>
        </tr>
        <tr>
          <th>name</th>
          <td>NML</td>
        </tr>
        <tr>
          <th>id</th>
          <td>Neuron_id0001</td>
        </tr>
        <tr>
          <th>n_nodes</th>
          <td>3369</td>
        </tr>
        <tr>
          <th>n_connectors</th>
          <td>None</td>
        </tr>
        <tr>
          <th>n_branches</th>
          <td>64</td>
        </tr>
        <tr>
          <th>n_leafs</th>
          <td>69</td>
        </tr>
        <tr>
          <th>cable_length</th>
          <td>151639.171875</td>
        </tr>
        <tr>
          <th>soma</th>
          <td>None</td>
        </tr>
        <tr>
          <th>units</th>
          <td>1 dimensionless</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    # Read all files in folder
    nl = navis.read_nmx('~/Downloads/WannerAA201605_SkeletonsGlomeruli/')
    nl






.. parsed-literal::

    WARNING : Skipped "Glomeruli.nmx": failed to import skeleton. (navis)




.. raw:: html

    &lt;class 'navis.core.neuronlist.NeuronList'&gt; containing 1022 neurons (142.1MiB)<div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>type</th>
          <th>name</th>
          <th>id</th>
          <th>n_nodes</th>
          <th>n_connectors</th>
          <th>n_branches</th>
          <th>n_leafs</th>
          <th>cable_length</th>
          <th>soma</th>
          <th>units</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>navis.TreeNeuron</td>
          <td>NML</td>
          <td>Neuron_id0742</td>
          <td>4351</td>
          <td>None</td>
          <td>91</td>
          <td>96</td>
          <td>200111.703125</td>
          <td>None</td>
          <td>1 dimensionless</td>
        </tr>
        <tr>
          <th>1</th>
          <td>navis.TreeNeuron</td>
          <td>NML</td>
          <td>Neuron_id0756</td>
          <td>4779</td>
          <td>None</td>
          <td>0</td>
          <td>3</td>
          <td>9099.673828</td>
          <td>None</td>
          <td>1 dimensionless</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>1020</th>
          <td>navis.TreeNeuron</td>
          <td>NML</td>
          <td>Neuron_id0997</td>
          <td>2395</td>
          <td>None</td>
          <td>61</td>
          <td>71</td>
          <td>136052.546875</td>
          <td>None</td>
          <td>1 dimensionless</td>
        </tr>
        <tr>
          <th>1021</th>
          <td>navis.TreeNeuron</td>
          <td>NML</td>
          <td>Neuron_id0983</td>
          <td>3332</td>
          <td>None</td>
          <td>115</td>
          <td>123</td>
          <td>223207.265625</td>
          <td>None</td>
          <td>1 dimensionless</td>
        </tr>
      </tbody>
    </table>
    </div>



Note the error message? NMX files don't always contain skeletons. If ``navis`` comes across one that can't be turned into a :class:`~navis.TreeNeuron`, it will skip the file and produce a warning.


Neuroglancer precomputed format
===============================

Among other formats, neuroglancer supports a "precomputed" format for skeletons (see specs `here <https://github.com/google/neuroglancer/blob/master/src/neuroglancer/datasource/precomputed/skeletons.md>`_). This binary format is more compact than uncompressed SWC files but probably is not used outside of neuroglancer afaik. That said: ``navis`` lets you read and write skeletons from/to precomputed format using :func:`navis.read_precomputed` and :func:`navis.write_precomputed`. Note that these functions work on both precomputed skeletons and meshes.

Manually constructing skeletons
===============================

What if you have some obscure data format for which ``navis`` does not have a read function? The data underlying a :class:`~navis.TreeNeuron` is a simple SWC table - so as long as you can produce that from your data, you can create your own skeletons. Here's a quick & dirty example:

.. code:: ipython3

    import pandas as pd 
    
    # Create a mock SWC table for a 2-node skeleton
    swc = pd.DataFrame()
    swc['node_id'] = [0, 1]
    swc['parent_id'] = [-1, 0]   # negative indices indicate roots 
    swc['x'] = [0, 1]
    swc['y'] = [0, 1]
    swc['z'] = [0, 1]
    swc['radius'] = 0
    
    swc




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>node_id</th>
          <th>parent_id</th>
          <th>x</th>
          <th>y</th>
          <th>z</th>
          <th>radius</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>-1</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
          <td>0</td>
        </tr>
        <tr>
          <th>1</th>
          <td>1</td>
          <td>0</td>
          <td>1</td>
          <td>1</td>
          <td>1</td>
          <td>0</td>
        </tr>
      </tbody>
    </table>
    </div>



This SWC can now be used to construct a :class:`~navis.TreeNeuron`:

.. code:: ipython3

    s = navis.TreeNeuron(swc, name='my_neuron', units='microns')
    s




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>type</th>
          <td>navis.TreeNeuron</td>
        </tr>
        <tr>
          <th>name</th>
          <td>my_neuron</td>
        </tr>
        <tr>
          <th>n_nodes</th>
          <td>2</td>
        </tr>
        <tr>
          <th>n_connectors</th>
          <td>None</td>
        </tr>
        <tr>
          <th>n_branches</th>
          <td>0</td>
        </tr>
        <tr>
          <th>n_leafs</th>
          <td>1</td>
        </tr>
        <tr>
          <th>cable_length</th>
          <td>1.732051</td>
        </tr>
        <tr>
          <th>soma</th>
          <td>None</td>
        </tr>
        <tr>
          <th>units</th>
          <td>1 micrometer</td>
        </tr>
      </tbody>
    </table>
    </div>



There are a few other ways to construct a :class:`navis.TreeNeuron` (e.g. using a graph) - see the docstring for details.

Also note that all navis neurons can be stored to disk using ``pickle`` - see the :ref:`pickling tutorial <local_data_pickle_tut>`. 

Hopefully the above has given you some entry points on how to load your data. See also the :ref:`I/O API reference api_io>`. Please keep in mind that you can also convert one neuron type into another - for example by skeletonizing ``MeshNeurons`` (see :ref:`neuron_conversion`).