Add scripts and files to process Vi's screen

[jni]

Uses dask-jobqueue for the main workflow.

[jni]

@GenevieveBuckley please annotate the bits of code that need clarification and I'll add comments, as I think these scripts should be used as templates to generalise to new screens, so they should be well documented.

[GenevieveBuckley]

Same again, scripts in shared spaces would be better.

[jni]

I forgot to include the output, which made me very happy (after seeing many outputs with full tracebacks and half-finished progress bars =P):

In [1]: %run -i vi-start-to-finish.py
illum: 100%|████████████████████████████████████████████████████████████████████████████████| 284/284 [42:31<00:00,  8.99s/it]
illumination estimated in 42.0m31.0s
illum-corr: 100%|████████████████████████████████████████████████████████████████████████| 284/284 [2:51:22<00:00, 172.00s/it]
illumination corrected in 2.0h51.0m22.0s
montage: 100%|██████████████████████████████████████████████████████████████████████████| 54528/54528 [44:46<00:00,  7.50it/s]
montaged in 46.0m8.0s
features:  50%|██████████████████████████████████▍                                  | 27183/54528 [1:02:34<1:02:27,  7.30it/s]distributed.scheduler - ERROR - Workers don't have promised key: ['tcp://172.16.193.188:41686'], features-55f478c8aca9dcbd97f98cdeb1d49b36
NoneType: None
distributed.client - WARNING - Couldn't gather 1 keys, rescheduling {'features-55f478c8aca9dcbd97f98cdeb1d49b36': ('tcp://172.16.193.188:41686',)}
features:  99%|██████████████████████████████████████████████████████████████████████▎| 54046/54528 [2:04:04<01:36,  4.99it/s]distributed.scheduler - ERROR - Workers don't have promised key: ['tcp://172.16.193.190:39647'], features-15f2d90c870cd1ef0dc775bbf90374d1
NoneType: None
distributed.client - WARNING - Couldn't gather 1 keys, rescheduling {'features-15f2d90c870cd1ef0dc775bbf90374d1': ('tcp://172.16.193.190:39647',)}
features: 100%|███████████████████████████████████████████████████████████████████████| 54528/54528 [2:05:46<00:00,  7.23it/s]
features computed in 2.0h6.0m22.0s

Needless to say I don't know yet what those "Workers don't have promised key", but it doesn't seem to have affected any feature computation.

[GenevieveBuckley]

I think now that UMAP is being included more routinely in these scripts (eg: c236e06) we should update requirements.txt and environment.yml in the microscopium repo. Thoughts?

[jni]

Hmm, I thought I'd responded to this, sorry, my bad! Thoughts: when we pull these things into microscopium proper, we can update the requirements. As we've discussed recently, one thing I want to do pretty short term is to go from images + info-csv + config file to the full feature computation and dimensionality reduction. That PR is the place to update the requirements.

[jni]

btw assuming you have no further questions I think this is ready to merge.