Our study aims to advance the application of dimensionality reduction, object detection, and segmentation in Hyperspectral Imaging (HSI), specifically for tissue ablation monitoring. We evaluate different modalities for ablation detection and segmentation in hyperspectral images, focusing on thermal effects induced by laser ablation treatment in vivo.
During our experimental study, we utilized a TIVITA hyperspectral camera with a spectral range of 500-995 nm to capture hypercubes of 640x480x100 voxels, encompassing 100 bands, alongside regular RGB images. The acquisition process, synchronized to mitigate breathing motion, involved placing polyurethane markers around the target area for spatial reference. To minimize extraneous light, the camera was positioned vertically at a 40 cm distance from the surgical field. A 20 W Halogen lamp was used as the light source. Two distinct imaging modes, reflectance-based and absorbance-based (Table 1 and Table 2), were employed to provide comprehensive insight into sample properties. A dataset comprising 233 hyperspectral cubes from 20 experiments, spanning pre-laparotomy, temperature escalation, and post-ablation phases, was collected, offering a robust foundation for analysis. Temperature thresholds recorded during the experiments delineate the thermal effects produced, as illustrated in accompanying figures.
Table 1. Example of hyperspectral images taken at different wavelengths.
| Absorbance | HSV | Reflectance |
|---|---|---|
abs.mp4 |
hsv.mp4 |
ref.mp4 |
Table 2. Ablated area at different temperatures.
| T | Absorbance | HSV | Reflectance |
|---|---|---|---|
| 35 |  |
 |
 |
| 60 |  |
 |
 |
| 110 |  |
 |
 |
The workflow proposed in this study, presented in Figure 1, consists of several key steps to extract valuable information from hyperspectral data in the laser-mediated thermal treatment scenario. First, dimensionality reduction is applied to reduce the high-dimensional data to a manageable size. Next, a supervised learning technique based on neural networks is used to detect the ablation area, i.e., the region of the image where tissue has been treated by laser irradiation. Finally, an unsupervised learning technique based on clustering is used to segment the spectral signature of the ablation area, allowing the identification of specific tissue types or classes of thermal damage. The proposed workflow provides a comprehensive approach to the analysis of hyperspectral data and has the potential to improve the accuracy and efficiency of diseased tissue analysis in the thermal treatment scenario.
Figure 1. Proposed workflow for hyperspectral image processing and analysis. The workflow consists of three main components: dimensionality reduction, ablation area detection using supervised learning, and spectral signature segmentation based on unsupervised learning.
The segmentation of the ablation area in hyperspectral images was meticulously examined through various clustering algorithms (Figure 2). While DBSCAN, OPTICS, and affinity propagation resulted in oversimplification, k-means, BIRCH, agglomerative clustering, spectral clustering, and GMM showcased superior performance, albeit requiring manual cluster input. Notably, Mean Shift emerged as a standout performer, offering high-quality segmentation without manual cluster definition, thanks to its adaptability, autonomous cluster center determination, and robustness to noise. Our analysis revealed significant variation in cluster numbers across reflectance and absorbance modalities, influenced by tissue-specific spectral characteristics and temperature-dependent variations, underscoring the necessity for adaptable segmentation approaches tailored to spectral complexities.
Figure 2. Comparison of ablation segmentation performed with different unsupervised algorithms. The top row represents the input data for clustering algorithms.
This study introduces a robust workflow for analyzing ablation detection and segmentation in hyperspectral images from laser treatment on in-vivo tissues. Leveraging PCA, t-SNE, and Faster R-CNN, we enhance hyperspectral data analysis, facilitating accurate ablation identification and localization. Mean Shift stands out for automated, high-quality segmentation. Our findings guide future research in refining techniques and extending applications to diverse medical scenarios, improving analysis and decision-making in laser cancer therapy and beyond.
- Operating System
- macOS
- Linux
- Windows (limited testing carried out)
- Python 3.8.x
- Required core packages: dev.txt
Step 1: Download and install Miniconda
wget https://repo.anaconda.com/miniconda/Miniconda3-py38_22.11.1-1-Linux-x86_64.sh
chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.shStep 2: Install FFmpeg and verify that the installation is correct
- Linux
sudo apt update
sudo apt upgrade
sudo apt install ffmpeg
ffmpeg -version- macOS
brew update
brew upgrade
brew install ffmpeg
ffmpegStep 3: Clone the repository, create a conda environment, and install the requirements for the repository
git clone https://github.com/ViacheslavDanilov/hsi_analysis.git
cd hsi_analysis
chmod +x create_env.sh
source create_env.shAll essential components of the study, including the curated source code, dataset, and trained models, are publicly available:
- Source code: https://github.com/ViacheslavDanilov/hsi_analysis
- Dataset: https://doi.org/10.5281/zenodo.10444212
- Models: https://doi.org/10.5281/zenodo.10444269
Please cite our paper if you found our data, methods, or results helpful for your research:
Danilov V.V., De Landro M., Felli E., Barberio M., Diana M., Saccomandi P. (2024). Advancing laser ablation assessment in hyperspectral imaging through machine learning. Computers in Biology and Medicine. DOI: https://doi.org/10.1016/j.compbiomed.2024.108849