DIR-lab 4DCT dataset

[markemus]

I'm trying to load the DIR-lab 4DCT dataset, but I've run into some trouble.

I downloaded the dataset from dir-lab.com, and applied the following pre-processing steps:

-downloaded all 10 data sets.
-renamed *-ssm.img to *_s.img (to silence errors from dirlab.py).
-ran the dirlab.py module on the img files.
-copied the resulting mha directory to /srv/mymachine/hsokooti/Data/DIR-Lab/4DCT/.

However, running RegNet3D.py throws the following error:

[MainThread  ] ---------------------------------Fri Nov  9 11:21:44 2018--------------------------------
[MainThread  ] ----------------------------------start experiment------------------------------
[MainThread  ] number of images in the last chunk=12
[MainThread  ] SyntheticDeformation[generation]: start DIR-Lab_4D/3D_max7_D9//T40/case2/Dsmooth0/DNext3/nextIm.mha
[MainThread  ] Process Process-1:
[MainThread  ] Traceback (most recent call last):
[MainThread  ]   File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
[MainThread  ]     self.run()
[MainThread  ]   File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
[MainThread  ]     self._target(*self._args, **self._kwargs)
[MainThread  ]   File "/home/markemus/dev/RegNet/functions/reading/direct_1st_epoch.py", line 49, in run
[MainThread  ]     self.fill()
[MainThread  ]   File "/home/markemus/dev/RegNet/functions/reading/direct_1st_epoch.py", line 85, in fill
[MainThread  ]     mode_synthetic_dvf='generation'
[MainThread  ]   File "/home/markemus/dev/RegNet/functions/synthetic_deformation.py", line 63, in get_dvf_and_deformed_images
[MainThread  ]     generate_next_im(setting, im_info=im_info)
[MainThread  ]   File "/home/markemus/dev/RegNet/functions/synthetic_deformation.py", line 495, in generate_next_im
[MainThread  ]     original_im_sitk = sitk.ReadImage(su.address_generator(setting, 'originalIm', **im_info_su))
[MainThread  ]   File "/home/markemus/.local/lib/python3.5/site-packages/SimpleITK/SimpleITK.py", line 8614, in ReadImage
[MainThread  ]     return _SimpleITK.ReadImage(*args)
[MainThread  ] RuntimeError: Exception thrown in SimpleITK ReadImage: /tmp/SimpleITK/Code/IO/src/sitkImageReaderBase.cxx:89:
[MainThread  ] sitk::ERROR: The file "/srv/markemus-VirtualBox/hsokooti/Data/DIR-Lab/4DCT/mha/case2/case2_T40_RS1.mha" does not exist.
[MainThread  ] total number of variables 1116164
[Thread-4    ] SyntheticDeformation[reading]: waiting 5s for DIR-Lab_4D/3D_max7_D9//T40/case2/Dsmooth0/DNext3/nextIm.mha

Is there an additional/different preprocessing step I need to do? Renaming */case2_T40.mha to */case2_T40_RS1.mha seems to fix this error and creates a nextIm.mha file, but there are more errors after that.

[hsokooti]

The script functions.preprocessing.dirlab.py is just a conversion from binary files to mha extension. In order to make the voxel spacing isotropic, you can run the script functions.preprocessing.resampling_isotropic.py. The image case2_T40.mha has the voxel spacing of [1.16, 1.16, 2.5] mm, but after resampling the voxel size of case2_T40_RS1 is [1, 1, 1] mm. RS1 stands for resampling to 1 mm.

[markemus]

Thank you, much appreciated.

New error:

Traceback (most recent call last):
  File "/home/markemus/dev/RegNet/functions/preprocessing/resampling_isotropic.py", line 34, in <module>
    resampling(data='DIR-Lab_4D', spacing=[1, 1, 1], requested_im_list=['Im', 'Mask', 'Torso'])
  File "/home/markemus/dev/RegNet/functions/preprocessing/resampling_isotropic.py", line 25, in resampling
    im_raw_sitk = sitk.ReadImage(su.address_generator(setting, 'original'+requested_im+'Raw', **im_info_su))
  File "/home/markemus/.local/lib/python3.5/site-packages/SimpleITK/SimpleITK.py", line 8614, in ReadImage
    return _SimpleITK.ReadImage(*args)
RuntimeError: Exception thrown in SimpleITK ReadImage: /tmp/SimpleITK/Code/IO/src/sitkImageReaderBase.cxx:89:
sitk::ERROR: The file "/srv/markemus-VirtualBox/hsokooti/Data/DIR-Lab/4DCT/mha/case1/Lung_Filled/case1_T00_Lung_Filled.mha" does not exist.

I figured chest_segmentation.py was my next step, so I ran that and got:

/usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
  return f(*args, **kwds)
/usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
  return f(*args, **kwds)
sh: 1: E:/PHD/Software/Works_Other/LungSegmentation/ptpulmo.exe: not found

Aside from the path, it looks like there's a missing dependency ptpulmo.exe. I couldn't find it online, is that proprietary?

[hsokooti]

Thank you for pointing out to this issue. The package ptpulmo is a commercial software. However, several free packages for lung segmentation are available. For instance, lung-segmentation-3d. You can also take a look at the LOLA11 challenge.

The accuracy of torso or lung segmentation is not that important for the RegNet.

[markemus]

Our data is formatted very differently so we're going to rebuild the graph on our own in the end, but thank you for all your help.