.. _trns:


Transformations
***************

Introduction
------------
As of version 0.5.0, ``navis`` includes functions that let you transform and mirror spatial data (e.g. neurons). This new functionality broadly splits into high and low level functions. In this tutorial, we will start by exploring the higher level functions that most users will use and then take a sneak peak at the low level functions.

At the moment, navis supports the following transform types:
 - `CMTK <https://www.nitrc.org/projects/cmtk/>`_ warp transforms 
 - `Hdf5 <https://github.com/saalfeldlab/template-building/wiki/Hdf5-Deformation-fields>`_ deformation fields 
 - `Elastix <https://elastix.lumc.nl/>`_ transforms
 - landmark-based thin spline plate transforms 
 - affine transforms 


flybrains
--------- 
Since ``navis`` brings the utility but does not ship with any transforms, we have to either generate those ourselves or get them elsewhere. Here, we will showcase the `flybrains <https://github.com/navis-org/navis-flybrains>`_ library that provides a number of different transforms directly to ``navis``. Setting up and registering your own custom transforms will be discussed further down.

First, you need to get `flybrains <https://github.com/navis-org/navis-flybrains>`_. Please follow the instructions to install and download the bridging registrations before you continue.

.. code:: ipython3

    import flybrains



.. raw:: html

    <script type="text/javascript">
    window.PlotlyConfig = {MathJaxConfig: 'local'};
    if (window.MathJax) {MathJax.Hub.Config({SVG: {font: "STIX-Web"}});}
    if (typeof require !== 'undefined') {
    require.undef("plotly");
    requirejs.config({
        paths: {
            'plotly': ['https://cdn.plot.ly/plotly-latest.min']
        }
    });
    require(['plotly'], function(Plotly) {
        window._Plotly = Plotly;
    });
    }
    </script>



Importing ``flybrains`` automatically registers the transforms with ``navis``. That allows ``navis`` to plot a sequence of bridging transformations to map between these template spaces.


.. image:: https://github.com/navis-org/navis-flybrains/blob/main/_static/bridging_graph.png?raw=true
   :width: 1000
   :alt: Flybrain Bridging Graph
   :align: center
   
In addition to those bridging transforms, ``flybrains`` also contains mirror registrations (we will cover those later), meta data and meshes for the template brains:

.. code:: ipython3

    # This is the Janelia "hemibrain" template brain
    flybrains.JRCFIB2018F




.. parsed-literal::

    Template brain
    --------------
    Name: JRCFIB2018F
    Short Name: JRCFIB2018F
    Type: None
    Sex:  None
    Dimensions: NA
    Voxel size:
      x = 0.008 m
      y = 0.008 i
      z = 0.008 c
    Bounding box (m):
      NA
    Description: Calibrated version of Janelia FIB hemibrain dataset
    DOI: https://doi.org/10.1101/2020.01.21.911859



.. code:: ipython3

    import navis
    
    # This is the hemibrain neuropil surface mesh
    fig, ax = navis.plot2d(flybrains.JRCFIB2018F)
    ax.azim, ax.elev = -90, 180



.. image:: transforming_files/transforming_4_0.png


You can check the registered transforms like so:

.. code:: ipython3

    navis.transforms.registry.summary()




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>source</th>
          <th>target</th>
          <th>transform</th>
          <th>type</th>
          <th>invertible</th>
          <th>weight</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>IBNWB</td>
          <td>JFRC2</td>
          <td>CMTKtransform with 1 transform(s)</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>1</th>
          <td>JFRC2013</td>
          <td>JFRC2013DS</td>
          <td>CMTKtransform with 1 transform(s)</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>2</th>
          <td>JFRC2013DS</td>
          <td>JFRC2013</td>
          <td>CMTKtransform with 1 transform(s)</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>3</th>
          <td>JFRC2014</td>
          <td>JFRC2013</td>
          <td>CMTKtransform with 1 transform(s)</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>4</th>
          <td>JFRC2</td>
          <td>T1</td>
          <td>CMTKtransform with 1 transform(s)</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>97</th>
          <td>JRCFIB2018Fraw</td>
          <td>JRCFIB2018F</td>
          <td>&lt;navis.transforms.affine.AffineTransform objec...</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>98</th>
          <td>JRCFIB2018F</td>
          <td>JRCFIB2018Fum</td>
          <td>&lt;navis.transforms.affine.AffineTransform objec...</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>99</th>
          <td>FAFB14um</td>
          <td>FAFB14</td>
          <td>&lt;navis.transforms.affine.AffineTransform objec...</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>100</th>
          <td>FAFB</td>
          <td>FAFB14</td>
          <td>&lt;navis.transforms.base.AliasTransform object a...</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
        <tr>
          <th>101</th>
          <td>JFRC2</td>
          <td>JFRC2010</td>
          <td>&lt;navis.transforms.affine.AffineTransform objec...</td>
          <td>bridging</td>
          <td>True</td>
          <td>1</td>
        </tr>
      </tbody>
    </table>
    <p>102 rows × 6 columns</p>
    </div>



Using ``xform_brain``
---------------------
For high level transforming, you will want to use :func:`navis.xform_brain`. This function takes a ``source`` and ``target`` argument tries to find a bridging sequence that gets you to where you want. Let's try it out:

Incidentally, the example neurons that ``navis`` ships with are from the Janelia hemibrain project and are therefore in `JRCFIB2018raw` (raw = 8nm voxel coordinates). We will be using those but there is nothing stopping you from using ``navis'`` interface with neuPrint (:ref:`tutorial <neuprint_intro>`) to fetch your favourite hemibrain neurons and transform those.

.. code:: ipython3

    # Load the example hemibrain neurons (JRCFIB2018raw space)
    nl = navis.example_neurons()
    nl




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>type</th>
          <th>name</th>
          <th>id</th>
          <th>n_nodes</th>
          <th>n_connectors</th>
          <th>n_branches</th>
          <th>n_leafs</th>
          <th>cable_length</th>
          <th>soma</th>
          <th>units</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>navis.TreeNeuron</td>
          <td>1734350788</td>
          <td>1734350788</td>
          <td>4465</td>
          <td>None</td>
          <td>603</td>
          <td>None</td>
          <td>266458.0000</td>
          <td>[4176]</td>
          <td>8 nanometer</td>
        </tr>
        <tr>
          <th>1</th>
          <td>navis.TreeNeuron</td>
          <td>1734350908</td>
          <td>1734350908</td>
          <td>4845</td>
          <td>None</td>
          <td>733</td>
          <td>None</td>
          <td>304277.0000</td>
          <td>[6]</td>
          <td>8 nanometer</td>
        </tr>
        <tr>
          <th>2</th>
          <td>navis.TreeNeuron</td>
          <td>722817260</td>
          <td>722817260</td>
          <td>4336</td>
          <td>None</td>
          <td>635</td>
          <td>None</td>
          <td>274910.5625</td>
          <td>None</td>
          <td>8 nanometer</td>
        </tr>
        <tr>
          <th>3</th>
          <td>navis.TreeNeuron</td>
          <td>754534424</td>
          <td>754534424</td>
          <td>4702</td>
          <td>None</td>
          <td>697</td>
          <td>None</td>
          <td>286743.0000</td>
          <td>[4]</td>
          <td>8 nanometer</td>
        </tr>
        <tr>
          <th>4</th>
          <td>navis.TreeNeuron</td>
          <td>754538881</td>
          <td>754538881</td>
          <td>4890</td>
          <td>None</td>
          <td>626</td>
          <td>None</td>
          <td>291435.0000</td>
          <td>[703]</td>
          <td>8 nanometer</td>
        </tr>
      </tbody>
    </table>
    </div>



.. code:: ipython3

    fig, ax = navis.plot2d([nl, flybrains.JRCFIB2018Fraw], linewidth=2, figsize=(8, 8))
    ax.azim, ax.elev = -90, 180


.. parsed-literal::

                                                               


.. image:: transforming_files/transforming_9_1.png


Let's say we want these neurons in JRC2018F template space. Before we do the actual transform it's useful to quickly check above bridging graph to see what we expect to happen:

First, we know that we are starting in `JRCFIB2018Fraw` space. From there, there is are two simple affine transforms to go from voxels to nanometers and from nanometers to micrometers. Once we are in micrometers, we can use a Hdf5 transform generated by the Saalfeld lab to map to ``JRC2018F`` (`Bogovic et al., 2019 <https://www.biorxiv.org/content/10.1101/376384v2>`_). Note that the arrows in the bridging graph indicate the transforms' forward directions but they can all be inversed to traverse the graph.

OK, with that out of the way:

.. code:: ipython3

    xf = navis.xform_brain(nl, source='JRCFIB2018Fraw', target='JRC2018F')


.. parsed-literal::

    Transform path: JRCFIB2018Fraw->JRCFIB2018F->JRCFIB2018Fum->JRC2018F


.. parsed-literal::

                                                           

Painless, wasn't it? Let's see if it worked:

.. code:: ipython3

    # Plot the transformed neurons and the JRC2018F template brain
    fig, ax = navis.plot2d([xf, flybrains.JRC2018F], linewidth=2, color='r', figsize=(8, 8))
    ax.azim, ax.elev = -90, -90
    ax.dist = 5


.. parsed-literal::

                                                               


.. image:: transforming_files/transforming_13_1.png


Worked like a charm! I highly recommend you read through the documentation for :func:`navis.xform_brain` and check out the parameters you can use to fine-tune it. 


Using ``mirror_brain``
----------------------
Another useful type of transform is mirroring using :func:`navis.mirror_brain`. The way this works is this:

1. Reflect coordinates about the midpoint of the mirror axis (affine transformation)
2. Apply a warping transformation to compensate for e.g. left/right asymmetries

For the first step, we need to know the length of the mirror axis. This is why - similar to having registered transforms - we need to have meta data about the template space (i.e. the bounding box) available to ``navis``.

The second step is optional. For example, `JRC2018F` and `JRC2018U` are templates generate from averaging multiple fly brains and are therefore already mirror symmetrical. ``flybrains`` does include some mirror transforms though: e.g. for `FCWB`, `VNCIS1` or `JFRC2`. 

Since our neurons are already in `JRC2018F` space, let's try mirroring them:

.. code:: ipython3

    mirrored = navis.mirror_brain(xf, template='JRC2018F')


.. parsed-literal::

                                                            

.. code:: ipython3

    fig, ax = navis.plot2d([xf, mirrored, flybrains.JRC2018F], linewidth=2, color='r', figsize=(8, 8))
    ax.azim, ax.elev = -90, -90
    ax.dist = 5


.. parsed-literal::

                                                                 


.. image:: transforming_files/transforming_16_1.png


Perfect! As noted above, this only works if the ``template`` is registered with ``navis`` and if it contains info about its bounding box. If you only have the bounding box at hand but no template brain, check out the lower level function :func:`navis.transforms.mirror`.


Low-level functions
-------------------
Let's assume you want to add your own transforms. There are four different transform types:

.. autosummary::
    :toctree: generated/

    navis.transforms.affine.AffineTransform
    navis.transforms.cmtk.CMTKtransform
    navis.transforms.h5reg.H5transform
    navis.transforms.thinplate.TPStransform

To show you how to use them, we will create a new thin plate spline transform using :class:`~navis.transforms.thinplate.TPStransform`. If you look at the bridging graph again, you might note the `"FAFB14"` template brain: it stands for `"Full Adult Fly Brain"` and this is is the 14th registration of the data. We will use landmarks to generate a mapping between this 14th and the previous 13th iteration.   

First grab the landmarks from the Saalfeld's lab `elm <https://github.com/saalfeldlab/elm>`_ repository:

.. code:: ipython3

    import pandas as pd
    
    # These landmarks map betweet FAFB (v14 and v13) and a light level template
    # We will use only the v13 and v14 landmarks
    landmarks_v14 = pd.read_csv('https://github.com/saalfeldlab/elm/raw/master/lm-em-landmarks_v14.csv', header=None)
    landmarks_v13 = pd.read_csv('https://github.com/saalfeldlab/elm/raw/master/lm-em-landmarks_v13.csv', header=None)
    
    # Name the columns
    landmarks_v14.columns = landmarks_v13.columns = ["label", "use", "lm_x","lm_y","lm_z", "fafb_x","fafb_y","fafb_z"]
    
    landmarks_v13.head()




.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }
    
        .dataframe tbody tr th {
            vertical-align: top;
        }
    
        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>label</th>
          <th>use</th>
          <th>lm_x</th>
          <th>lm_y</th>
          <th>lm_z</th>
          <th>fafb_x</th>
          <th>fafb_y</th>
          <th>fafb_z</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>Pt-1</td>
          <td>True</td>
          <td>571.400083</td>
          <td>38.859963</td>
          <td>287.059544</td>
          <td>525666.465856</td>
          <td>172470.413167</td>
          <td>80994.733289</td>
        </tr>
        <tr>
          <th>1</th>
          <td>Pt-2</td>
          <td>True</td>
          <td>715.811344</td>
          <td>213.299356</td>
          <td>217.393493</td>
          <td>595391.597008</td>
          <td>263523.121958</td>
          <td>84156.773677</td>
        </tr>
        <tr>
          <th>2</th>
          <td>Pt-3</td>
          <td>True</td>
          <td>513.002196</td>
          <td>198.001970</td>
          <td>217.794090</td>
          <td>501716.347872</td>
          <td>253223.667163</td>
          <td>98413.701578</td>
        </tr>
        <tr>
          <th>3</th>
          <td>Pt-6</td>
          <td>True</td>
          <td>867.012542</td>
          <td>31.919253</td>
          <td>276.223437</td>
          <td>670999.903156</td>
          <td>179097.916778</td>
          <td>67561.691416</td>
        </tr>
        <tr>
          <th>4</th>
          <td>Pt-7</td>
          <td>True</td>
          <td>935.210895</td>
          <td>234.229522</td>
          <td>351.518068</td>
          <td>702703.909963</td>
          <td>251846.384054</td>
          <td>127865.886146</td>
        </tr>
      </tbody>
    </table>
    </div>



Now we can use those landmarks to generate a thin plate spine transform:

.. code:: ipython3

    from navis.transforms.thinplate import TPStransform
    
    tr = TPStransform(landmarks_source=landmarks_v14[["fafb_x","fafb_y","fafb_z"]].values,
                      landmarks_target=landmarks_v13[["fafb_x","fafb_y","fafb_z"]].values)
    
    # navis.transforms.MovingLeastSquaresTransform has similar properties

The transform has a method that we can use to transform points but first we need some data in FAFB 14 space:

.. code:: ipython3

    # Transform our neurons into FAFB 14 space
    xf_fafb14 = navis.xform_brain(nl, source='JRCFIB2018Fraw', target='FAFB14')


.. parsed-literal::

    Transform path: JRCFIB2018Fraw->JRCFIB2018F->JRCFIB2018Fum->JRC2018F->FAFB14um->FAFB14


.. parsed-literal::

                                                           

Now let's see if we can use the v14 -> v13 transform:

.. code:: ipython3

    # Transform the nodes of the first two neurons 
    pts_v14 = xf_fafb14[:2].nodes[['x', 'y', 'z']].values
    
    # The actual transform
    pts_v13 = tr.xform(pts_v14)

Quick check how the v14 and v13 coordinates compare

.. code:: ipython3

    fig, ax = navis.plot2d(pts_v14, scatter_kws=dict(c='k'))
    _ = navis.plot2d(pts_v13, scatter_kws=dict(c='r'), ax=ax)
    ax.azim = ax.elev = -90



.. image:: transforming_files/transforming_26_0.png


So that did something... to be honest, I'm not sure what to expect for the FAFB 14->13 transform but let's assume this is correct and move on.

Next, we will register this new transform with ``navis`` so that we can use it with higher level functions:

.. code:: ipython3

    # Register the transform
    navis.transforms.registry.register_transform(tr, source='FAFB14', target='FAFB13', transform_type='bridging')

Now that's done we can use `FAFB13` with :func:`navis.xform_brain`:

.. code:: ipython3

    # Transform our neurons into FAFB 14 space
    xf_fafb13 = navis.xform_brain(xf_fafb14, source='FAFB14', target='FAFB13')


.. parsed-literal::

    Transform path: FAFB14->FAFB13


.. parsed-literal::

                                                           

.. code:: ipython3

    fig, ax = navis.plot2d(xf_fafb14, c='k')
    _ = navis.plot2d(xf_fafb13, c='r', ax=ax, lw=1.5)
    ax.azim = ax.elev = -90


.. parsed-literal::

                                                               


.. image:: transforming_files/transforming_31_1.png


For completeness, lets also have a quick look at registering additional template brains.

Template brains are represented in navis as :class:`navis.transforms.templates.TemplateBrain` and there is currently no canonical way of constructing them: you can associate as much or as little data with them as you like. However, for them to be useful they should have a ``name``, a ``label`` and a ``boundingbox`` property. 

Minimally, you could do something like this:

.. code:: ipython3

    # Construct template brain from base class
    my_brain = navis.transforms.templates.TemplateBrain(name='My template brain',
                                                        label='my_brain',
                                                        boundingbox=[[0, 100], [0, 100], [0, 100]])
    
    # Register with navis
    navis.transforms.registry.register_templatebrain(my_brain)
    
    # Now you can use it with mirror_brain:
    import numpy as np
    pts = np.array([[10, 10, 10]])
    pts_mirrored = navis.mirror_brain(pts, template='my_brain')

.. code:: ipython3

    # Plot the points
    fig, ax = navis.plot2d(pts, scatter_kws=dict(c='k', alpha=1, s=50))
    fig, ax = navis.plot2d(pts_mirrored, scatter_kws=dict(c='r', alpha=1, s=50), ax=ax)



.. image:: transforming_files/transforming_34_0.png


While this is a working solution, it's not very pretty: for example, ``my_brain`` does have the default docstring and no fancy string representation (e.g. for ``print(my_brain)``). I highly recommend you take a look at how `flybrains <https://github.com/navis-org/navis-flybrains>`_ constructs the templates.

**Acknowledgments**

Much of the transform module is modelled after functions written by Greg Jefferis for the `natverse <http://natverse.org>`_. Likewise, `flybrains <https://github.com/navis-org/navis-flybrains>`_ is a port of data collected by Greg Jefferis for ``nat.flybrains`` and ``nat.jrcbrains``.