Skip to content

3D LiDAR Semantic Segmentation with range images and Retentive Networks

Notifications You must be signed in to change notification settings

SiMoM0/RangeRet

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

64 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RangeRet

Master's Thesis about 3D LiDAR Semantic Segmentation using Range Images and Retentive Networks.

The purpose of the Thesis is the development of a lightweight model that is able to achieve real-time performance while requiring low amount of memory and an optimized training process.

Installation

Requirements

  • Pytorch
  • yaml
  • tqdm

Data Preparation

SemanticKITTI

Download the files from the SemanticKITTI website

./semanticKITTI/
├── 
├── ...
└── dataset/
    ├──sequences
        ├── 00/           
        │   ├── velodyne/	
        |   |	├── 000000.bin
        |   |	├── 000001.bin
        |   |	└── ...
        │   └── labels/ 
        |       ├── 000000.label
        |       ├── 000001.label
        |       └── ...
        ├── 08/ # for validation
        ├── 11/ # 11-21 for testing
        └── 21/
	    └── ...

Training

Run the following commands, specifying the SemanticKITTI dataset path

cd RangeRet
python3 main.py <config_path> <data_path>

Example: python3 main.py config/RangeRet-semantickitti.yaml /semanticKITTI/dataset/

Checkpoint will be saved in checkpoints/model-checkpoint.pt, the final model will be saved as rangeret-model.pt and training logs in log/ with the current date and time.

Inference

Run the following script, specifying the SemanticKITTI dataset path

cd RangeRet
python3 infer.py <config_path> <model_path> <data_path> <pred_path> <split>

where split can be train, valid or test.

Example of inference on SemanticKITTI validation set: python3 infer.py config/RangeRet-semantickitti.yaml rangeret-model.pt /semanticKITTI/dataset predictions/ valid

To correctly evaluate the results, please refer to the scripts available at semantic-kitti-api.

Results

Parameters Inference (ms) Memory (MB) Val mIoU (%) Test mIoU (%)
RangeRet 1.7M 49 2.0 46.9 45.2

Ablation Study

Models trained on a subest of SemanticKITTI with 5000 scans, equals to the 25% of the entire dataset, and evaluated on the complete validation set, sequence 08.

REM Architecture Decay Matrix Residual Connection Params mIoU (%)
Linear Transformers 1.49M 30.1
Conv Transformers 1.56M 37.3
Conv RetNet Standard 1.69M 37.8
Conv RetNet Our 1.69M 40.3
Conv RetNet Our 1.69M 42.3

Thesis

Master Thesis document is available here

Acknowledgements

Code is built on RangeNet++ and Torchscale.