Overview

The analysis pipeline provided by these classes is designed to streamline the processing of synchrotron scattering data. The core functionality is divided among several key classes:

• Mask:
  Generates and converts detector mask files from Foxtrot (.txt) or Swing/HDF5 (.h5) input, preparing them for subsequent data analysis.

• SwingData:
  Processes individual HDF5 (or NXS) files by extracting experimental parameters, processing Eiger detector images, and performing both 1D and 2D azimuthal integration using pyFAI. It also supports saving integration results and generating summary plots.

• SwingBatch:
  Automates the batch processing of multiple HDF5 files, providing progress feedback, logging, and optional video creation from generated plots.

• TextFileProcessor:
  Loads and processes Foxtrot-format text files to extract q and i data, and supports data subtraction between files with interpolation when necessary.

Together, these classes enable efficient, automated conversion of raw synchrotron data into quantitative results and visualizations, ensuring robust handling of various file formats and experimental conditions.

Classes

---

1. Mask

Purpose:

The Mask class is design to generate mask files in a format compatible with the library. It can handle 2 type of files:

• *.txt files, corresponding to maks files generated by foxtrot. The method convertfoxtrotmask() can be used to convert the mask in the suitable 'edf' format
• *.h5 files corresponding to data file from Swing. The method drawmask() uses the scattering data as input of pyFAI-drawmask so that the user can generate its own mask.

An instance is created by typing:

mask=Mask(filepath)

Key Methods:

draw_mask()

This method opens pyFAI-drawmask GUI to create your own mask file, in a format compatible with the library. Do not forget to save the mask on your drive before closing pyFAI-drawmask.

Parameters:

path to nxs/h5 file used to create a mask file in the suitable format using pyFAI-drawmask

convertfoxtrotmask()

This method converts foxtrot masks (*.txt) files in the suitable *.edf format.

Parameters:

path to foxtrot mask

---

2. SwingData

Purpose:

The SwingData class is designed to manipulate individual HDF5 (or NXS) data files, which may contain multiple scattering frames. Its main functionalities include:

• Extracting experimental parameters, metadata, and images from the HDF5 file.
• Loading and processing scattering (Eiger) detector data, with options for frame averaging and anisotropic corrections.
• Performing 1D azimuthal integration (and 2D integration for anisotropic data) using the pyFAI library.
• Saving integration results (as text files) and generating summary plots that combine Basler images, Eiger images, and integration curves.

A SwingData instance can be initialized with the following parameters:

• filename: str
  Path to the data file (*.h5 or *.nxs).

• maskfile: str
  Path to the mask file (in *.edf format).

• basler_coords: tuple
  Pixel coordinates of the X-ray beam on the Basler image. (Provided by the beamline scientist.)

• basler_calibration: tuple
  Calibration factors (in µm/pixel) for the Basler image along X and Z. (Ask if you intend to perform XZ mapping.)

• anisotropic_data: bool
  Set to True to optimize integration of anisotropic data (note: this operation may slow down computation).
  (Default: False)

• sectors: list of tuple
  List of angular sectors defined as (angle, delta_angle) for azimuthal integration.
  (Default: None, which corresponds to isotropic averaging)

• average_frames: bool
  Tag to perform averaging of scattering frames contained in the HDF5 file.
  (Default: True)

• transmission_correction: bool
  Tag to perform intensity normalization and transmission corrections.
  (Default: True)

For anisotropic data, additional operations are applied:

• Caving: Detector frames are reconstructed using inversion symmetry to partially suppress beamstop effects, dead pixels, and detector gaps. A new mask file is generated and applied automatically.
• 2D Integration: Generates 2D maps (q, chi, and intensity) for further analysis of azimuthal profiles.

Key Methods:

extract_from_h5()

• Purpose:
  Extracts experimental parameters and metadata from the HDF5 file.
• Details:
  Retrieves sample name, Basler image, positions (start and end for X and Z), detector geometry parameters (distance, pixel size, beam center, etc.), transmission data, time stamps, and generates a reduced filename for output naming.

extract_scatteringdata()

• Purpose:
  Loads and processes the Eiger detector data from the HDF5 file.
• Details:
  • If frame averaging is enabled, it averages the scattering frames.
  • If anisotropic data is flagged (caving enabled), it applies a “caving” procedure: masked pixels are tagged with a large value and then replaced based on inversion symmetry.
  • Updates the mask file if caving is applied.

writeedf(name, data)

• Purpose:
  Writes a given data array to an EDF file.
• Details:
  Creates an EDF header from the extracted HDF5 parameters (distance, wavelength, detector geometry, etc.) and saves the file using the fabio library.

integrate()

• Purpose:
  Performs 1D azimuthal integration on the scattering data.
• Details:
  • Uses pyFAI’s AzimuthalIntegrator to integrate the Eiger images.
  • Supports integration over defined sectors.
  • Returns a dictionary with keys (angle, delta_angle) that contain:
    • "q": The scattering vector values.
    • "i_raw": Raw integrated intensity.
    • "i_normalize": Normalized intensity (after applying transmission corrections).
  • Computes a chi-map (azimuthal angle array) and stores it in self.chi_array.

integrate2D()

• Purpose:
  Performs 2D integration for anisotropic data.
• Details:
  • Computes 2D maps for q, chi, and intensity for each individual frame.
  • Stores the integrated maps in a dictionary (keys: 'qmap', 'chimap', 'imap').

save_integration()

• Purpose:
  Saves the integration results as text files.
• Details:
  • Creates a subdirectory called integration to store 1D integration results.
  • Filenames include the sample’s reduced filename, sector information, and frame number if applicable.
  • For anisotropic data, an additional directory is created for 2D maps (saving separate files for q, chi, and intensity).

plot_data()

• Purpose:
  Generates plots combining:
  • The Basler image (with a marker indicating the measurement point).
  • The Eiger mean image (with an overlay of the chi-map showing integration sectors).
  • A log–log plot of the integrated intensity versus q.
• Details:
  • For anisotropic data, a 2D integration map (q, chi, intensity) is also plotted.
  • Plots are saved in a subdirectory called plots.

plot_and_save()

• Purpose:
  A convenience method that calls both save_integration() and plot_data() to perform integration and plot generation in one command.

---

3. SwingBatch

Purpose:

The SwingBatch class processes a batch of HDF5 files from synchrotron experiments by leveraging the SwingData class. It is designed to:

• Process all HDF5 files in a specified directory.
• Provide progress feedback (supporting both terminal and Jupyter Notebook environments).
• Generate integration results and plots for each file.
• Write a log summarizing the processing outcomes (successful and failed files).
• Optionally create a video from the generated plot images.

A SwingBatch instance can be initialized with the following parameters:

• data_dir (str): Path to the directory containing the HDF5 files (*.h5 or *.nxs).
• mask_path (str): Path to the mask file (in *.edf format).
• anisotropic_data (bool): Set to True to enable data caving (reconstruction using inversion symmetry, which is time consuming). (Default: False)
• average_frames (bool): If True, integrate the mean image; if False, integrate each frame individually. (Default: True)
• transmission_correction (bool): Set to True to perform intensity normalization and transmission corrections. (Default: True)
• basler_coords (tuple): Pixel coordinates of the X-ray beam on the Basler image. (Default: [0, 0])
• basler_calibration (tuple): Calibration factors (in µm/pixels) for the Basler image along X and Z. (Default: [3.73, 3.67])
• log_file (str): Name of the log file to be created in the data directory. (Default: "processing_log.txt")
• sectors (list of tuple): List of angular sectors defined as (angle, delta_angle) for azimuthal integration. (Default: [(0, 180)] if not provided)

Key Methods:

get_h5_files()

• Purpose:
  Scans the specified data directory and returns a list of all HDF5 files (*.h5 and *.nxs).
• Usage:
  This method is used internally by process() to determine which files to process.

process()

• Purpose:
  Processes all HDF5 files in the data directory.
• Details:
  • Iterates through each file (as returned by get_h5_files()).
  • Calls _process_single_file() for extraction, integration, and plotting.
  • Displays progress updates (supports both terminal and Jupyter Notebook environments).
  • Logs processing results via _log_results().

_process_single_file(file_path)

• Purpose:
  Processes a single HDF5 file.
• Details:
  • Instantiates a SwingData object with the specified parameters.
  • Calls save_integration() and plot_data() on the SwingData instance to generate outputs.

_log_results()

• Purpose:
  Writes a log file summarizing the processing outcomes.
• Details:
  • Logs the number of successfully processed files.
  • Records any failures (with error details) into a log file stored in the data directory.

get_report()

• Purpose:
  Generates a textual report summarizing the processing results.
• Details:
  • Returns a string detailing the number of files processed and lists any files that failed processing along with their error messages.

create_video_from_images(output_filename="output_video.mp4", fps=3)

• Purpose:
  Creates an MP4 video by combining plot images generated during processing.
• Details:
  • Searches for PNG files in the "plots" subdirectory of the data directory.
  • Combines these images into a video using OpenCV.
  • The default frames per second is set to 3 (this can be adjusted via the fps parameter).

---

4. TextFileProcessor

Purpose:

• This class is designed to load, store, and process data from text files in the Foxtrot format.
• It supports loading multiple files with different skiprows values to correctly extract q and i data.
• It also enables subtracting data from two files and saving the result in a designated subfolder.

Key Methods:

__init__(self)

Initializes the TextFileProcessor object with an empty dictionary for storing file data.

load_txt(self, path)

Loads q and i data from a Foxtrot-format text file by attempting various skiprows values.

• Parameters:
  path – file path of the text file.
• Returns:
  A tuple (q, i) containing the extracted data, or (None, None) if unsuccessful.

get_data(self, path)

Retrieves the stored q and i data for a given file path.

• Parameters:
  path – file path of the text file.
• Returns:
  The stored (q, i) data if available; otherwise, (None, None).

subtract_files(self, file1, file2)

Subtracts the i data of the second file from the first (interpolating if necessary) and saves the result in a new text file within a sub folder.

• Parameters:
  file1 – the first file's path.
  file2 – the second file's path.
• Returns:
  None. The result is saved as a new text file named in the format file1_name-file2_name_subtracted.txt.

---