Yijie Zheng, Jinxuan Yang, Yu Chen, Yaxuan Wang,
[News]: Our paper (arXiv) is accepted for oral presentation at IGARSS 2025! Welcome to meet us at the session (TH3.P9: Detection and Tracking of Marine Animals using Spaceborne Remote Sensing Data) in August 7
Our work detected beluga whales in the Arctic region from very high resolution satellite imagery with manually created point labels.
We created bounding box labels from point labels using SAM, significantly improved annotaion efficiency.
Based on the high-quality automated annotation process, the YOLOv8s model trained on SAM-annotated data (YOLO-SAM) produced bbounding boxes that fits the actual whales shape.
YOLO-SAM significantly surpassed the model trained on point labels (YOLO-Buffer) in terms of precision and
Our YOLO-SAM distinguished whales from seals in high accuracy.
Please install PyTorch first. torch<2.1.0 is recomended for compatibility with other dependencies.
This repo is developed using python 3.10 and torch 2.0.0+cu118
conda create -n belugaSeeker python=3.10
conda activate belugaSeeker
pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118
For training and inference, please install the mmyolo codebase.
pip install openmim
mim install "mmengine>=0.6.0" "mmcv>=2.0.0rc4,<2.1.0" "mmdet>=3.0.0rc6,<3.1.0" "mmyolo"
The following depencdencies are required for data preprocess and annotation.
pip install rasterio geopandas
pip install git+https://github.com/facebookresearch/segment-anything.git
The checkpoints for SAM can be downloaded via SAM-ViT-H.
Click the links below to download the checkpoints for beluga whale detector:
It's easy to get started with inference code. We provide inference_demo.ipynb to help you get started.
Please refer to crop.ipynb for cropping the images and create_whale_masks.ipynb for generating whale masks and bounding boxes.
The file structure follows the mmyolo convention. Use the command below to specify the GPU devices. Automatic mixed precision training is enabled to accelerate training while maintaining the performance.
CUDA_VISIBLE_DEVICES=0 python tools/train.py configs_beluga/yolov8_s_b24-100e.py --amp
Please refer to evaluate.ipynb for evaluating on all the metrics.
This work is supported by the 4th IEEE GRSS Student Challenge. The authors gratefully acknowledge Amou for the assistance with ArcGIS Pro, and Zori for dilivering the data.
This project uses the following open source libraries: