Installing bips into python environment

[zhenghanQ]

I am having trouble installing bips into my local python environment:

This is what I have done:
conda create -n zqi2 python=2.7 anaconda
pip install -t /mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages nibabel
pip install -t /mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages git+https://github.com/akeshavan/BrainImagingPipelines.git

However, I saw the following:

Downloading/unpacking git+https://github.com/akeshavan/BrainImagingPipelines.git
Cloning https://github.com/akeshavan/BrainImagingPipelines.git to /tmp/pip-WIovG3-build
Running setup.py (path:/tmp/pip-WIovG3-build/setup.py) egg_info for package from git+https://github.com/akeshavan/BrainImagingPipelines.git
fatal: No names found, cannot describe anything.
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip-WIovG3-build/setup.py", line 45, in
from setup_egg import extra_setuptools_args
ImportError: No module named setup_egg
Complete output from command python setup.py egg_info:
fatal: No names found, cannot describe anything.

Traceback (most recent call last):

File "", line 17, in

File "/tmp/pip-WIovG3-build/setup.py", line 45, in

from setup_egg import extra_setuptools_args

ImportError: No module named setup_egg

---

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-WIovG3-build
Storing debug log for failure in /mindhive/gablab/u/zqi/.pip/pip.log

Any ideas? Thanks!!

[zhenghanQ]

Sorry, git has a special way of formatting my message... Hopefully it looks less striking in your email..

[satra]

@zhenghanQ - i believe the setup file needs to be fixed. but for now you can do this:

conda create -n test python pip
source activate test
conda install numpy scipy networkx nose traits traitsui sphinx
pip install https://github.com/nipy/nipype/archive/master.zip
wget https://github.com/INCF/BrainImagingPipelines/archive/master.zip
unzip master.zip
cd master
python setup.py install

[zhenghanQ]

Thanks, Satra!
I followed your instruction. The master.zip extracted to a folder called BrainImagingPipelines-master folder. I unzipped the folder and installed bips without problem.

I realized bips also requires another package: reportlab. After I added that into my environment, bips worked fine! :) Yay! Problem solved!!

[zhenghanQ]

Hey, Satra,
I noticed there seems to be a bug in the workflow df49 (dicom conversion) in the bips version here https://github.com/INCF/BrainImagingPipelines/archive/master.zip. I was only able to get dicom_info but not the other nifti files. However, if I use the version of bips installed in /software/python/EPD/7.3/bin, the workflow works fine.
Zhenghan

[satra]

did you install pydicom and dcmstack into your new environment?

[zhenghanQ]

ah, no... are they also python packages that I can install by just using pip?

[satra]

yup

pip install pydicom dcmstack

[zhenghanQ]

Great! Thank you, Satra! I just added these two packages and now the dicom conversion workflow is working! 👍

[zhenghanQ]

Here is one more problem related with bips and python environment.

bips workflow 3a2e (normalize functional image) does not run. It looks like WarpTimeSeriesImageMultiTransform was not able to run. Am I missing another python package? Or is this something inside of the version of bips in the master archive?

I tried running this workflow using bips in /software/python/EPD/7.3/bin. The workflow works without any problem.

Sorry that I kept asking questions on this issue. I can later write a short document with step to step instruction for people who would like to use bips in their own python environment. Thanks!!

Here is the crash report:
File: /gablab/p/BILD/tmp/crash-20140503-163026-zqi-_warp_images1.npz
Node: _warp_images1
Working directory: /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1

Node inputs:

args =
dimension = 3
environ = {}
ignore_exception = False
input_image = /gablab/p/BILD/data_kids/syntax/MRIdata/subjects/SLI3081/fixedfx/fwhm_6.0/cope_GE_C.nii.gz
invert_affine =
num_threads = -1
out_postfix = _wtsimt
reference_image = /gablab/p/BILD/data_kids/MRI/ants/ANTS_pediatric_mask.nii
reslice_by_header =
terminal_output = stream
tightest_box =
transformation_series = ['/gablab/p/BILD/data_kids/MRI/ants/SLI3081/smri/warped_field/ants_Warp.nii.gz', '/gablab/p/BILD/data_kids/MRI/ants/SLI3081/smri/affine_transformation/ants_Affine.txt', '/mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/fsl_reg_2_itk/fsl2antsAffine.txt']
use_bspline =
use_nearest = False

Traceback:
Traceback (most recent call last):
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 18, in run_node
result['result'] = node.run(updatehash=updatehash)
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/pipeline/engine.py", line 1386, in run
self._run_interface()
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/pipeline/engine.py", line 1496, in _run_interface
self._result = self._run_command(execute)
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/pipeline/engine.py", line 1622, in _run_command
result = self._interface.run()
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 946, in run
runtime = self._run_interface(runtime)
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/interfaces/ants/resampling.py", line 106, in _run_interface
self.raise_exception(runtime)
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1313, in raise_exception
raise RuntimeError(message)
RuntimeError: Command:
WarpTimeSeriesImageMultiTransform 3 /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/cope_GE_C.nii.gz cope_GE_C_wtsimt.nii.gz -R /gablab/p/BILD/data_kids/MRI/ants/ANTS_pediatric_mask.nii /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/ants_Warp.nii.gz /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/ants_Affine.txt /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/fsl2antsAffine.txt
Standard output:
FIELD: /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/ants_Warp.nii.gz
AFFINE: /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/ants_Affine.txt
AFFINE: /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/fsl2antsAffine.txt
moving_image_filename: /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/cope_GE_C.nii.gz
output_image_filename: cope_GE_C_wtsimt.nii.gz
reference_image_filename: /gablab/p/BILD/data_kids/MRI/ants/ANTS_pediatric_mask.nii
[0/3]: FIELD: /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/ants_Warp.nii.gz
[1/3]: AFFINE: /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/ants_Affine.txt
[2/3]: AFFINE: /mindhive/scratch/Sun/zqi/BILD/syntax/normalizefunc/normalize_post_struct/_imgtype_cope/_subject_id_SLI3081/warp_images/mapflow/_warp_images1/fsl2antsAffine.txt
read veclength as:: 1

Standard error:

Return code: 0
Interface WarpTimeSeriesImageMultiTransform failed to run.

[zhenghanQ]

This might be obvious to people who use python frequently. I just noticed another package is necessary for bips: pygraphviz.

However I have a hard time installing this package into my own python environment...
There are lots of suggestions online (e.g. http://stackoverflow.com/questions/15661384/python-does-not-see-pygraphviz), including manually installing graphviz and graphviz-dev. I got the former but cannot locate the latter. Any suggestions will be appreciated. :)

[satra]

the ants error might have to do with which version of ants you are using.

for pygraphviz, graphviz should already be on the system.

[zhenghanQ]

Thanks, Satra! The ants version I am using is /software/ANTS/versions/120222/bin/. Is this out of dated for the version of bips on the master repository?

I am currently trying to build a python environment on my own local account (as you suggested). Therefore, I need to manually install a number of dependencies for bips. pygraphviz doesn't seem to come with python 2.7 from anaconda.

[zhenghanQ]

One of the two earlier problems has been solved!

I copied pygraphviz folder from /software/python/anachona/.../pygraphviz to my own python environment folder and it works!

But ANTS still seems to have an compatibility issue with my local bips, when running normalization workflows. The weirdest thing is that the central python environment doesn't seem to have the same issue (/software/python/EPD/virtualenvs/7.3/nipype0.7/bin/python).

With Carlo's help, we have done the following and still haven't figured out what was the problem:

1. compared the normalization workflows in bips at the central python environment vs. my python environment
2. upgraded ANTS from 120222 to 140107
3. compared the python scripts generated by bips run in the central python env vs. my local python env
4. Ran WarpTimeSeriesImageMultiTransform in the command line and it also failed.

Here is the error message:
File "/mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/batch/pyscript_20140617_174907__warp_images6.py", line 26, in
result = info['node'].run(updatehash=info['updatehash'])
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/pipeline/engine.py", line 1386, in run
self._run_interface()
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/pipeline/engine.py", line 1496, in _run_interface
self._result = self._run_command(execute)
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/pipeline/engine.py", line 1622, in _run_command
result = self._interface.run()
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 946, in run
runtime = self._run_interface(runtime)
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/interfaces/ants/resampling.py", line 104, in _run_interface
runtime = super(WarpTimeSeriesImageMultiTransform, self)._run_interface(runtime, correct_return_codes = [0,1])
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1383, in _run_interface
self.raise_exception(runtime)
File "/mindhive/gablab/u/zqi/envs/zqi2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1313, in raise_exception
raise RuntimeError(message)
RuntimeError: Command:
WarpTimeSeriesImageMultiTransform 3 /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/zstat_Task_Rest.nii.gz zstat_Task_Rest_wtsimt.nii.gz -R /gablab/p/BILD/data_kids/MRI/ants/ANTS_pediatric_mask.nii /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/ants_Warp.nii.gz /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/ants_Affine.txt /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/fsl2antsAffine.txt
Standard output:
FIELD: /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/ants_Warp.nii.gz
AFFINE: /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/ants_Affine.txt
AFFINE: /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/fsl2antsAffine.txt
moving_image_filename: /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/zstat_Task_Rest.nii.gz
output_image_filename: zstat_Task_Rest_wtsimt.nii.gz
reference_image_filename: /gablab/p/BILD/data_kids/MRI/ants/ANTS_pediatric_mask.nii
[0/3]: FIELD: /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/ants_Warp.nii.gz
[1/3]: AFFINE: /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/ants_Affine.txt
[2/3]: AFFINE: /mindhive/scratch/Tue/zqi/BILD/phono/normalizefunc/normalize_post_struct/_imgtype_zstat/_subject_id_BILDC3223/warp_images/mapflow/_warp_images6/fsl2antsAffine.txt
read veclength as:: 1
Standard error:
terminate called after throwing an instance of 'itk::ExceptionObject'
what(): /software/src/build/ANTS/ITKv4-install/include/ITK-4.6/itkVectorImage.hxx:54:
itk::ERROR: VectorImage(0x18ad3d0): Cannot allocate VectorImage with VectorLength = 0
Aborted (core dumped)
Return code: 134
Interface WarpTimeSeriesImageMultiTransform failed to run.

Zhenghan