Skip to content

GENFI to BIDS converter #865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
Feb 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
c2103a1
First version of the genfi to bids converter
MatthieuJoulot Jan 11, 2023
2ad7720
Add weight to T2
MatthieuJoulot Jan 11, 2023
6816b45
Add naming correction for fieldmaps
MatthieuJoulot Jan 16, 2023
dc3bf18
Add clinical data
MatthieuJoulot Jan 17, 2023
16bded2
Add gif volumes and change column selection function
MatthieuJoulot Jan 17, 2023
c5b66a8
Add description + comment asl related stuff
MatthieuJoulot Jan 18, 2023
4d55573
Add Type hints and docstrings
MatthieuJoulot Jan 18, 2023
6607336
Add documentation for genfi-to-bids
MatthieuJoulot Jan 18, 2023
2ebf961
Non-regression test + corrections
MatthieuJoulot Jan 18, 2023
12b8eba
Add a unit test
MatthieuJoulot Jan 18, 2023
e73c2a5
Make the clinical data optional
MatthieuJoulot Jan 18, 2023
6b0f012
Apply suggestion from review (part 1)
MatthieuJoulot Jan 19, 2023
7d9e673
Factorize find_clinical_data (part 2)
MatthieuJoulot Jan 19, 2023
0ea27c7
Suggestion from review (part3)
MatthieuJoulot Jan 19, 2023
970fb64
Add gif argument
MatthieuJoulot Jan 19, 2023
c463913
Factorize correct_fieldmaps_name following suggestion from review
MatthieuJoulot Jan 20, 2023
4b8914e
Change variable name in identify modality for clarity
MatthieuJoulot Jan 20, 2023
a13d860
Suggestion from review (part 4)
MatthieuJoulot Jan 20, 2023
4afdfde
Fetch readme data from Genfitobids.md instead of hardcoding it
MatthieuJoulot Jan 20, 2023
9290a83
Apply suggestions from code review
MatthieuJoulot Jan 20, 2023
91ad74f
Apply suggestions from code review par
MatthieuJoulot Jan 20, 2023
115b109
reduce read_imaging_data
MatthieuJoulot Jan 20, 2023
3d1d29b
Suggestion of the review
MatthieuJoulot Jan 23, 2023
ad756cc
Add Doctstrings and type hints to functions from review
MatthieuJoulot Jan 23, 2023
3878853
Apply suggestions from code review
MatthieuJoulot Jan 24, 2023
7c58f9f
Reverse change from review
MatthieuJoulot Jan 24, 2023
e1411c8
Apply suggestion from review
MatthieuJoulot Jan 24, 2023
47ce632
Apply suggestions from code review
MatthieuJoulot Jan 24, 2023
eabc029
Select columns using gif
MatthieuJoulot Jan 24, 2023
cf08ef0
Factorize bids_path and bids_name generation following suggestion
MatthieuJoulot Jan 24, 2023
5ec3f95
Apply suggestions from code review
MatthieuJoulot Jan 25, 2023
8a40630
Apply suggestions
MatthieuJoulot Jan 25, 2023
60391ca
Refactor session_id computation following review
MatthieuJoulot Jan 25, 2023
84fcfa5
Apply suggestions from code review
MatthieuJoulot Jan 25, 2023
7162d75
Remove useless code
MatthieuJoulot Jan 25, 2023
8394825
Apply suggestions from code review
MatthieuJoulot Jan 26, 2023
72d4c72
Merge branch 'dev' into genfi-to-bids
MatthieuJoulot Jan 26, 2023
879f0bf
make format
MatthieuJoulot Jan 26, 2023
a337588
Change test to reflect changes in command line
MatthieuJoulot Jan 27, 2023
355d891
Update test/nonregression/iotools/test_run_converters.py
NicolasGensollen Feb 3, 2023
342e2f3
Update test/unittests/iotools/converters/genfi_to_bids/test_genfi_to_…
NicolasGensollen Feb 3, 2023
d63491f
Update test/unittests/iotools/converters/genfi_to_bids/test_genfi_to_…
NicolasGensollen Feb 3, 2023
ff1a8ea
Update test/nonregression/iotools/test_run_converters.py
NicolasGensollen Feb 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions clinica/iotools/bids_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -906,6 +906,80 @@ def run_dcm2niix(
)


def identify_modality(filename: str) -> Optional[str]:
"""Identifies the modality of a file given its name.

Parameters
----------
filename: str
Input filename

Returns
-------
Optional[str]:
Modality or None if parsing uns
"""
filename = filename.lower()
if "dwi" in filename:
return "dwi"
if "t1" in filename:
return "T1"
if "t2" in filename:
return "T2w"
if "fieldmap" in filename:
return "fieldmap"
if "fmri" in filename:
return "rsfmri"
else:
return None


def parse_description(filepath: PathLike, start_line: int, end_line: int) -> str:
"""Parse the description of the dataset from the readme in the documentation.

Parameters
----------
filepath : PathLike
Path to the readme file from which to extract the description.

start_line : int
Line number where the description starts.

end_line : int
Line number where the description ends.

Returns
-------
str :
The description extracted from the readme file.
"""
with open(filepath, "r") as fp:
txt = fp.readlines()
return "\n".join(txt[start_line:end_line])


def parse_url(filepath: PathLike) -> List[str]:
"""Parse the URLs from the readme in the documentation.

Parameters
----------
filepath : PathLike
Path to the readme file from which to extract the URLs.

Returns
-------
List[str] :
The list of found URLs.
"""
import re

with open(filepath, "r") as fp:
txt = fp.readlines()
txt = "".join(txt)
url_extract_pattern = "https?:\\/\\/(?:www\\.)?[-a-zA-Z0-9@:%._\\+~#=]{1,256}\\.[a-zA-Z0-9()]{1,6}\\b(?:[-a-zA-Z0-9()@:%_\\+.~#?&\\/=]*)"
return re.findall(url_extract_pattern, txt)


def write_to_tsv(df: DataFrame, buffer: Union[PathLike, BinaryIO]) -> None:
"""Save dataframe as a BIDS-compliant TSV file.

Expand Down
2 changes: 2 additions & 0 deletions clinica/iotools/converters/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from .adni_to_bids import adni_to_bids_cli
from .aibl_to_bids import aibl_to_bids_cli
from .genfi_to_bids import genfi_to_bids_cli
from .habs_to_bids import habs_to_bids_cli
from .nifd_to_bids import nifd_to_bids_cli
from .oasis3_to_bids import oasis3_to_bids_cli
Expand All @@ -22,6 +23,7 @@ def cli() -> None:
cli.add_command(oasis_to_bids_cli.cli)
cli.add_command(oasis3_to_bids_cli.cli)
cli.add_command(ukb_to_bids_cli.cli)
cli.add_command(genfi_to_bids_cli.cli)

if __name__ == "__main__":
cli()
95 changes: 95 additions & 0 deletions clinica/iotools/converters/genfi_to_bids/genfi_to_bids.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
"""Convert the GENFI dataset into BIDS."""

from os import PathLike
from typing import Optional


def convert_images(
path_to_dataset: PathLike,
bids_dir: PathLike,
path_to_clinical: Optional[PathLike],
gif: bool,
) -> None:
"""Convert the entire dataset to BIDS.

Scans available files in the path_to_dataset,
identifies the patients that have images described by the JSON file,
converts the image with the highest quality for each category.

Parameters
----------
path_to_dataset: PathLike
Path to the raw images

bids_dir: PathLike
Path to directory where the bids will be written

path_to_clinical: PathLike, optional
Path to the clinical data associated with the dataset.
If None, the clinical data won't be converted.

gif: bool
If True, indicates the user wants to have the values of the gif parcellation
"""
import os

import clinica.iotools.bids_utils as bids

from .genfi_to_bids_utils import (
complete_clinical_data,
dataset_to_bids,
find_clinical_data,
intersect_data,
merge_imaging_data,
read_imaging_data,
write_bids,
)

# read the clinical data files
if path_to_clinical:
df_demographics, df_imaging, df_clinical = find_clinical_data(path_to_clinical)
# makes a df of the imaging data
imaging_data = read_imaging_data(path_to_dataset)

# complete the data extracted
imaging_data = merge_imaging_data(imaging_data)

# complete clinical data
if path_to_clinical:
df_clinical_complete = complete_clinical_data(
df_demographics, df_imaging, df_clinical
)

# intersect the data
if path_to_clinical:
df_complete = intersect_data(imaging_data, df_clinical_complete)
else:
df_complete = imaging_data
# build the tsv
results = dataset_to_bids(df_complete, gif)

write_bids(
to=bids_dir,
participants=results["participants"],
sessions=results["sessions"],
scans=results["scans"],
)
# fetch data for readme from markdown
readme_path = os.path.join(
os.path.dirname(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__))))
),
"docs",
"Converters",
"GENFItoBIDS.md",
)
try:
readme_data = {
"link": bids.parse_url(readme_path)[0],
"desc": bids.parse_description(readme_path, 4, 5),
}
except IndexError:
raise ValueError("Could not parse information for dataset.")
bids.write_modality_agnostic_files(
study_name="GENFI", readme_data=readme_data, bids_dir=bids_dir
)
49 changes: 49 additions & 0 deletions clinica/iotools/converters/genfi_to_bids/genfi_to_bids_cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
from os import PathLike
from typing import Optional

import click

from clinica.iotools.converters import cli_param

clinical_data_directory = click.option(
"-cdd",
"--clinical-data-dir",
"clinical_data_directory",
type=click.Path(exists=True, file_okay=False, resolve_path=True),
help="Path to the clinical data directory",
)

gif = click.option("-gif", is_flag=True, help="Add values from gif to session.tsv")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what gif is doing. It's not explained in the docs and the help string is a bit too shy of details.

I would also avoid short option flags only. Modern CLI design tends to favor long option flags by default, and long + short option flags for popular use cases. Since this option maps to a boolean variable, I'd expect something along the lines of --with-gif or --add-gif maybe?



@click.command(name="genfi-to-bids")
@cli_param.dataset_directory
@cli_param.bids_directory
@clinical_data_directory
@gif
def cli(
dataset_directory: PathLike,
bids_directory: PathLike,
clinical_data_directory: Optional[PathLike] = None,
gif: bool = False,
) -> None:
"""GENFI to BIDS converter.

Convert the imaging and clinical data of GENFI, located in DATASET_DIRECTORY and
CLINICAL_DATA_DIRECTORY respectively, to a BIDS dataset in the target BIDS_DIRECTORY.
"""
from clinica.iotools.bids_utils import _write_bidsignore
from clinica.iotools.converters.genfi_to_bids.genfi_to_bids import convert_images
from clinica.utils.check_dependency import check_dcm2niix
from clinica.utils.stream import cprint

check_dcm2niix()

convert_images(dataset_directory, bids_directory, clinical_data_directory, gif)
_write_bidsignore(str(bids_directory))

cprint("Conversion to BIDS succeeded.")


if __name__ == "__main__":
cli()
Loading