.. _cloudvolume_tut:

Neuroglancer & CloudVolume
**************************

`Neuroglancer <https://github.com/google/neuroglancer>`_ is a WebGL-based viewer for volumetric data. You may have used it to browse some of the recent large EM datasets. If you want to programmatically access/download these data, you want `CloudVolume <https://github.com/seung-lab/cloud-volume>`_. `CloudVolume` is an excellent Python library developed by William Silversmith (Seung lab, Princeton) and others. While `CloudVolume` is not directly related to `Neuroglancer`, it shares much of its functionality. As a rule of thumb: if you can view a dataset in `Neuroglancer`, you can download that data using `CloudVolume`. Some examples:

1. `FlyWire <https://flywire.ai/>`_ is a segmentation of an entire *Drosophila* brain. This dataset is very much work in progress and you will to register and apply for access. Check out `FAFBseg <https://fafbseg-py.readthedocs.io>`_ for a fairly mature interface built using navis. 
2. `Google's flood-filling segmentation <http://fafb-ffn1.storage.googleapis.com/landing.html>`_ of an entire *Drosophila* brain.
3. The Allen Institute's `MICrONs datasets <https://www.microns-explorer.org/>`_. 
4. The Janelia `hemibrain connectome <https://neuprint.janelia.org>`_. 

`CloudVolume` supports the backends/data formats of these and many up-and-coming datasets. You can use it to query the segmentation directly, and to fetch meshes and skeletons (if available). ``navis`` & friends provide simple interfaces for some of the datasets (see e.g. the neuPrint and the MICrONs tutorials) but there is also some lower-level option to pull neurons into ``navis`` via `CloudVolume`. 

First of all, you will want to make sure to ``cloudvolume`` is installed and up-to-date:

.. code-block:: bash

   pip3 install cloud-volume -U
   
Once that's done we can start pulling data using ``cloudvolume``. In this example here, we will use the Google segmentation of the FAFB dataset:

.. code:: ipython3

    import navis
    import cloudvolume as cv

*Before* we connect to the database we have to "monkey patch" `cloudvolume` such that it returns ``navis`` neurons:

.. code:: ipython3

    # This needs to be run only once at the beginning of each session
    navis.patch_cloudvolume()


.. parsed-literal::

    INFO  : cloud-volume successfully patched! (navis)


Now we can connect to our data source:

.. code:: ipython3

    # Don't forget to set `use_https=True` to avoid having to setup Google credentials
    vol = cv.CloudVolume('precomputed://gs://fafb-ffn1-20200412/segmentation', use_https=True, progress=False)

Fetch a some (mesh) neurons:

.. code:: ipython3

    # If we set `as_navis=True` we will get MeshNeurons 
    m = vol.mesh.get([4335355146, 2913913713, 2137190164, 2268989790], as_navis=True, lod=3)
    m




.. raw:: html

    &lt;class 'navis.core.neuronlist.NeuronList'&gt; containing 4 neurons (3.4MiB)<div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>type</th>
          <th>name</th>
          <th>id</th>
          <th>units</th>
          <th>n_vertices</th>
          <th>n_faces</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>navis.MeshNeuron</td>
          <td>None</td>
          <td>4335355146</td>
          <td>1 nanometer</td>
          <td>14784</td>
          <td>30420</td>
        </tr>
        <tr>
          <th>1</th>
          <td>navis.MeshNeuron</td>
          <td>None</td>
          <td>2913913713</td>
          <td>1 nanometer</td>
          <td>15388</td>
          <td>31213</td>
        </tr>
        <tr>
          <th>2</th>
          <td>navis.MeshNeuron</td>
          <td>None</td>
          <td>2137190164</td>
          <td>1 nanometer</td>
          <td>8631</td>
          <td>17612</td>
        </tr>
        <tr>
          <th>3</th>
          <td>navis.MeshNeuron</td>
          <td>None</td>
          <td>2268989790</td>
          <td>1 nanometer</td>
          <td>9450</td>
          <td>19189</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    fig = m.plot3d()

.. raw:: html
   :file: figures/google_meshes.html
   
|
|
|
|   

This also works for skeletons. Note though that not all datasets contain precomputed skeletons. For those cases you might want to check out :func:`navis.skeletonize`.

.. code:: ipython3

    sk = vol.skeleton.get([4335355146, 2913913713, 2137190164, 2268989790], as_navis=True)
    sk




.. raw:: html

    &lt;class 'navis.core.neuronlist.NeuronList'&gt; containing 4 neurons (2.1MiB)<div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>type</th>
          <th>name</th>
          <th>id</th>
          <th>n_nodes</th>
          <th>n_connectors</th>
          <th>n_branches</th>
          <th>n_leafs</th>
          <th>cable_length</th>
          <th>soma</th>
          <th>units</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>navis.TreeNeuron</td>
          <td>SWC</td>
          <td>4335355146</td>
          <td>27460</td>
          <td>None</td>
          <td>2184</td>
          <td>2192</td>
          <td>8219293.0</td>
          <td>None</td>
          <td>1 nanometer</td>
        </tr>
        <tr>
          <th>1</th>
          <td>navis.TreeNeuron</td>
          <td>SWC</td>
          <td>2913913713</td>
          <td>28640</td>
          <td>None</td>
          <td>2369</td>
          <td>2381</td>
          <td>8557471.0</td>
          <td>None</td>
          <td>1 nanometer</td>
        </tr>
        <tr>
          <th>2</th>
          <td>navis.TreeNeuron</td>
          <td>SWC</td>
          <td>2137190164</td>
          <td>15404</td>
          <td>None</td>
          <td>867</td>
          <td>870</td>
          <td>4712362.0</td>
          <td>None</td>
          <td>1 nanometer</td>
        </tr>
        <tr>
          <th>3</th>
          <td>navis.TreeNeuron</td>
          <td>SWC</td>
          <td>2268989790</td>
          <td>18105</td>
          <td>None</td>
          <td>945</td>
          <td>946</td>
          <td>5545407.0</td>
          <td>None</td>
          <td>1 nanometer</td>
        </tr>
      </tbody>
    </table>
    </div>