By default, most
navis functions use only a single core (although some third-party functions used under the hood might). Distributing expensive computations across multiple cores can speed things up considerable.
As of version
navis functions natively support parallel processing. This notebook will illustrate various ways to use parallelism. Importantly,
pathos for multiprocessing:
$ pip install pathos -U
Parallelizing generic functions¶
For non-navis function you can use NeuronList.apply to parallelize them.
First, let’s write a mock function that simply waits one second and then returns the number of nodes:
def my_func(x): import time time.sleep(1) return x.n_nodes
%time n_nodes = nl.apply(my_func)
CPU times: user 46.3 ms, sys: 6.43 ms, total: 52.8 ms Wall time: 5.05 s
%time n_nodes = nl.apply(my_func, parallel=True)
CPU times: user 94.7 ms, sys: 5.18 ms, total: 99.9 ms Wall time: 1.1 s