PARIETAL: Yet another deeP leARnIng brain ExTrAtion tooL 👯👯
During the last years, we have been using available state-of-the-art skull-stripping tools happily. However, in deep learning pipelines, most of the internal processes are computed very fast thanks to the use of GPUs (lesion segmentation, tissue segmentation, etc..), so brain extraction tends to be orders of magnitude slower than the rest of the GPU based pipeline processes. The main motivation behind PARIETAL is to have a fast and robust skull-stripping method that could be incorporated in our deep learning pipelines.
Fortunately, various brain MRI datasets have been released, such as the Calgary-Campinas-359 dataset, permitting researchers to train deep learning models that will hopefully improve both performance and processing time. Although different deep learning methods have been proposed already for accurate brain extraction. PARIETAL is yet another one, yielding fast and accurate outputs, regardless of the imaging brain extraction protocol. In order to validate our proposed method, we have carried different experiments using the trained model on the Campinas dataset, analyzing the capability of the learned architecture on unseen data and varied image acquisition protocols.
PARIETAL is a patch-based residual network 3D-UNET with ~10M parameters (see Figure). We have trained the model using the silver-masks provided by the Calgary-Campinas-359 dataset. This dataset consists of 359 images of healthy adults (29-80 years) acquired on Siemens, Philips and General Electric scanners at both 1.5T and 3T (see Souza et al. 2017 for more information about the dataset).
- Input modalities:
T1-w - Training patch size:
32x32x32 - Training sampling: balanced training, same number of brain and skull samples (non-brain) after sampling at
16x16x16 - Optimizer:
Adadelta - Training batch size:
32 - Training epochs:
200 - Train loss:
cross entropy - Early stopping:
50 epochs (based on validation DSC) - Inference patch size:
32x32x32 - Inference sampling:
16x16x16
We implemented PARIETAL in Python using the Pytorch deep learning toolkit. All necessary packages can be installed from pip as follows:
pip install -r requirementsTo use PARIETAL as an standalone script, just run ./parietal --help to see all the available options:
/path/to/parietal --help- input_scan (
--input_image): T1-w nifti image to process - output_scan (
--out_name): Output name for the skull-stripped image
- binary threshold (
--threshold): output threshold used to binarize the skull-stripped image (default=0.5) - gpu use (
--gpu): use GPU for faster inference (default=No) - gpu number (
--gpu_number): which GPU number to use (default=0) - verbose (
--verbose): show useful information
To use PARIETAL as a Python library, just import the BrainExtraction class from your script.
from brain_extraction import BrainExtraction
b = BrainExtraction()
input_scan = 'tests/example/T1.nii.gz'
output_scan = 'tests/example/parietal_brainmask.nii.gz'
# The result is stored both in disk at output_scan path and returned
# as np.array
brainmask = b.run(input_scan, ouput_scan)In order to facilitate its use in larger experiments, the method’s options can be set by default from a configuration file stored at config/config.cfg. Class declaration arguments overwrite the default configuration:
[data]
normalize = True
out_threshold = 0.5
workers = 10
[model]
model_name = campinas_baseline_s2_multires
sampling_step = 16
patch_shape = 32
use_gpu = True
gpu_number = 0The model can run with both GPU or a decent CPU. In most of our experiments, PARIETAL can extract the brain out of T1-w image in less than 10 seconds when using GPU and about 2 minutes when running on the CPU (see performance experiments for a more complete analysis).
In order to reduce the hassle to install all the dependencies in your local machine, we also provide a Docker version. Please follow the guide to install Docker for your operating system. If you are on Linux and you want to use the GPU capabilities of your local machine, please be sure that you install the nvidia-docker (version 2.0) packages.
Once Docker is available in your system, install the minimum Python dependencies as:
pip install pyfiglet dockerThen, running PARIETAL is as easy as an standalone script: (note: the first time you run the script, this may take some time to run because it will download the Docker image locally in your system).
/path/to/parietal_docker --help- input_scan (
--input_image): T1-w nifti image to process - output_scan (
--out_name): Output name for the skull-stripped image
- binary threshold (
--threshold): output threshold used to binarize the skull-stripped image (default=0.5) - gpu use (
--gpu): use GPU for faster inference (default=No) - gpu number (
--gpu_number): which GPU number to use (default=0) - verbose (
--verbose): show useful information
We have compared the performance of PARIETAL with several publicly available state-of-the-art tools and also against some other deep learning methods. To do so, we have run PARIETAL on different public available datasets such as OASIS, LPBA40 and the Campinas dataset.
Performance evaluation against the 12 manual masks from the Campinas dasaset. We extract values for other methods from the Lucena et al. 2019 paper:
| method | Dice | Sensitivity | Specificity |
|---|---|---|---|
| ANTs | 95.93 | 94.51 | 99.70 |
| BEAST | 95.77 | 93.84 | 99.76 |
| BET | 95.22 | 98.26 | 99.13 |
| BSE | 90.48 | 91.44 | 98.64 |
| HWA | 91.66 | 99.93 | 97.83 |
| MBWSS | 95.57 | 92.78 | 99.48 |
| OPTIBET | 95.43 | 96.13 | 99.37 |
| ROBEX | 95.61 | 98.42 | 99.13 |
| STAPLE (previous) | 96.80 | 98.98 | 99.38 |
| Silver-masks | 97.13 | 96.82 | 99.70 |
| CONSNet | 97.18 | 98.91 | 99.46 |
| PARIETAL | 97.23 | 96.73 | 97.75 |
Performance evaluation against the 40 manual masks from the LPBA40 dasaset. Values for the rest of the methods are extracted from the Lucena et al. 2019 paper:
| method | Dice | Sensitivity | Specificity |
|---|---|---|---|
| ANTs | 97.25 | 98.98 | 99.17 |
| BEAST | 96.30 | 94.06 | 99.76 |
| BET | 96.62 | 97.23 | 99.27 |
| HWA | 92.51 | 99.89 | 97.02 |
| MBWSS | 96.24 | 94.40 | 99.68 |
| OPTIBET | 95.87 | 93.35 | 99.74 |
| ROBEX | 96.77 | 96.50 | 99.50 |
| STAPLE (previous) | 97.59 | 98.14 | 99.46 |
| CONSNet (Campinas model) | 97.35 | 98.14 | 99.45 |
| CONSNet (trained on LPBA40) | 98.47 | 98.55 | 99.75 |
| auto UNET Salehi (trained on LPBA40) | 97.73 | 98.31 | 99.48 |
| Unet Salehi (trained on LPBA40) | 96.79 | 97.22 | 99.34 |
| 3DCNN Kleesiek (trained on LPBA40) | 96.96 | 97.46 | 99.41 |
| PARIETAL (Campinas model) | 97.25 | 96.10 | 98.40 |
Similar to the previous datasets, we also show the performance of PARIETAL against the 77 brainmasks of the OASIS dataset. Values for the rest of the methods are extracted from the Lucena et al. 2019 paper:
| method | Dice | Sensitivity | Specificity |
|---|---|---|---|
| ANTs | 95.30 | 94.39 | 98.73 |
| BEAST | 92.46 | 86.76 | 99.70 |
| BET | 93.50 | 92.63 | 98.10 |
| HWA | 93.95 | 98.36 | 96.12 |
| MBWSS | 90.24 | 84.09 | 99.35 |
| OPTIBET | 94.45 | 91.51 | 9.22 |
| ROBEX | 95.55 | 93.95 | 99.06 |
| STAPLE (previous) | 96.09 | 95.18 | 98.98 |
| CONSNet (Campinas model) | 95.54 | 93.98 | 99.05 |
| CONSNet (trained on OASIS) | 97.14 | 97.45 | 98.88 |
| auto UNET Salehi (trained on OASIS) | 97.62 | 98.66 | 98.77 |
| Unet Salehi (trained on OASIS) | 96.22 | 97.29 | 98.27 |
| 3DCNN Kleesiek (trained on OASIS) | 95.02 | 92.40 | 99.28 |
| PARIETAL (Campinas model) | 92.55 | 87.40 | 98.51 |
In contrast to the previous datasets, OASIS masks were not manually annotated, so the results of PARIETAL using the Campinas trained model were limited, mostly due to inconsistencies between labelling protocols 🤷♂️ (see Figure):
To further illustrate such an issue, we retrained the model using the 77 brain masks of the OASIS dataset using a two-fold cross-validation methodology. We followed the same approach done in Kleesiek et al. 2016, Salehi et al. 2017 and Lucena et al. 2019, i.e. a two-fold cross-validation strategy for assessing our model. After retraining, the performance of PARIETAL was similar or better than other deep learning methods:
| method | Dice | Sensitivity | Specificity |
|---|---|---|---|
| CONSNet (Campinas model) | 95.54 | 93.98 | 99.05 |
| CONSNet (trained on OASIS) | 97.14 | 97.45 | 98.88 |
| auto UNET Salehi (trained on OASIS) | 97.62 | 98.66 | 98.77 |
| Unet Salehi (trained on OASIS) | 96.22 | 97.29 | 98.27 |
| 3DCNN Kleesiek (trained on OASIS) | 95.02 | 92.40 | 99.28 |
| PARIETAL (Campinas model) | 92.55 | 87.40 | 98.51 |
| PARIETAL (trained on OASIS) | 97.99 | 97.84 | 98.14 |
Finally, we analyze the processing time (in seconds) of the proposed architecture against other methods in the field. For the PARIETAL method, we show the processing times with/without loading the model in the GPU for each new sample. This is the case when the model is not used in batch mode (to implement).
Processing times from all methods, but PARIETAL, have been extracted from Lucena et al. 2019 paper, where the authors report the use of a workstation equipped with a Xeon E3-1220 v3, 4x3.10Ghz, Intel). GPU resources are identical for all the deep learning methods (NVIDIA TITAN-X GPU, 12GB).
| method | Campinas | OASIS | LPBA40 |
|---|---|---|---|
| ANTs | 1378 | 1025 | 1135 |
| BEAST | 1128 | 944 | 905 |
| BET | 9 | 5 | 7 |
| BSE | 2 | 1 | 1 |
| HWA | 846 | 248 | 281 |
| MBWSS | 135 | 66 | 79 |
| OPTIBET | 773 | 579 | 679 |
| ROBEX | 60 | 53 | 57 |
| CONSNet (GPU) | 25 | 18 | 36 |
| CONSNet (CPU) | 516 | 214 | 301 |
| PARIETAL (GPU) | 12 | 7 | 9 |
| PARIETAL (GPU + model load) | 17 | 12 | 14 |
| PARIETAL (CPU) | 129 | 122 | 141 |
- Souza, R., Lucena, O., Garrafa, J., Gobbi, D., Saluzzi, M., Appenzeller, S., … Lotufo, R. (2017). An open, multi-vendor, multi-field-strength brain MR dataset and analysis of publicly available skull stripping methods agreement. NeuroImage, 170, 482–494. (link)
- Lucena, O., Souza, R., Rittner, L., Frayne, R., & Lotufo, R. (2019). Convolutional neural networks for skull-stripping in brain MR imaging using silver standard masks. Artificial Intelligence in Medicine, 98(August 2018), 48–58. (link)
- Sadegh, S., Salehi, M., Member, S., Erdogmus, D., Member, S., Gholipour, A., & Member, S. (2017). Auto-context Convolutional Neural Network (Auto-Net) for Brain Extraction in Magnetic Resonance Imaging, 0062(c), 1–12. (link)
- Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-Hein, K., Bendszus, M., & Biller, A. (2016). Deep MRI brain extraction: A 3D convolutional neural network for skull stripping. NeuroImage, 129, 460–469. (link)
- v0.1: first usable version
- v0.2: multi-resolution training
- V0.3: docker capabilities and paper cleanup