BACH: the Behavior Analysis maCHine

Description

BACH is a command line tool to monitor the behavior of ants in experimental nests.

Publications

• Alessio Sclocco, Shirlyn Jia Yun Ong, Sai Yan Pyay Aung, Serafino Teseo. Integrating real-time data analysis into automatic tracking of social insect behavior. bioRxiv 2020.11.03.366195; doi: https://doi.org/10.1101/2020.11.03.366195
• Sclocco, A., Teseo, S. Microbial associates and social behavior in ants. Artif Life Robotics 25, 552–560 (2020). https://doi.org/10.1007/s10015-020-00645-z

Compatibility and Dependencies

• Python >= 3.7
• numpy >= 1.19.2
• opencv_python >= 4.5.1.48
• opencv-contrib-python >= 4.5.1.48
• Darknet

Data

• YOLO/Darknet BACH model nest area
• YOLO/Darknet BACH model foraging area
• Example videos of ants in the experimental setup

Install and Update

To install BACH from the command line using pip, type the following command from BACH's main directory:

python -m pip install -e .

Video Recording

To record videos with BACH it is possible to use the bach.recorder module. This is how to use this functionality from the command line:

python -m bach.recorder --webcam <int> --output <string> --width <int> --height <int> --frames <int>

Where:

• --webcam <int> is the ID of the webcam
• --output <string> is the location and file name of the output video
• --width <int> is the width of the video
• --height <int> is the height of the video
• --frames <int> is the number of frames to record, in total

Example of execution:

python -m bach.recorder --webcam 0 --output /mnt/data/videos/recorded/today --width 4096 --height 2160 --frames 250

In this example BACH records a video from webcam 0 and stores it in the file today.mp4 in the directory /mnt/data/videos/recorded. The video will be 10 seconds long (because the default number of samples is 25 per second, and we are storing 250), and the resolution will be 4096x2160 (4K).

Frame Extraction

To extract frames with BACH it is possible to use the bach.frame_extractor module; it is possible to extract frames from a recorded video, or a live stream from a webcam. This is how to extract frames from a recorded video using the command line:

python -m bach.frame_extractor --file <string> --reduction <int> --frame_file <string>

Where:

• --file <string> is the recorded video
• --reduction <int> is the reduction factor, used to avoid saving all frames
• --frame_file <string> is the location where to save the frames, including the prefix for the file names

Example of execution:

python -m bach.frame_extractor --file /mnt/data/videos/recorded/today.mp4 --reduction 25 --frame_file /mnt/data/videos/frames/today

In this example the recorded video from which frames are extracted is today.mp4 in the directory /mnt/data/videos/recorded. Of the whole video, only 1 every 25 frames will be saved, that is one frame every second if the frame rate of the video is 25 fps. Each frame is saved as a file named today_.jpg in the directory /mnt/data/videos/frames, where is the id of the frame.

Tracking

It is possible to track a video using the bach.tracker module; tracking works on a recorded video, or a live stream from a webcam. This is how to perform tracking on a recorded video, while generating a real-time annotated video, using the command line:

python -m bach.tracker --file <string> -c <string> -m <string> -w <string> --show_video --video_width <int> --video_height <int> --threshold <float> --output_file <string>

Where:

• --file <string> is the recorded video
• -c <string> is the Darknet/Yolo configuration file
• -m <string> is the Darknet/Yolo metadata file
• -w <string> is the Darknet/Yolo trained model
• --show_video is a flag used to enable visualization
• --video_width <int> is the width of the annotated video
• --video_height <int> is the height of the annotated video
• --threshold <float> is the confidence threshold for a detection
• --output_file <string> is the location of the output file

Example of execution:

python -m bach.tracker --file /mnt/data/videos/recorded/today.mp4 -c /mnt/data/models/2020-15-01/yolo-ants.cfg -m /mnt/data/models/2020-15-01/yolo-ants.data -w /mnt/data/models/2020-15-01/yolo-ants.weights --show_video --video_width 1920 --video_height 1080 --threshold 0.25 --output_file /mnt/data/experiments/today.out

In this example the recorded video to track is today.mp4 in the directory /mnt/data/videos/recorded. The Yolo model is stored in the directory /mnt/data/models/2020-15-01 and is composed of three distinct files: 1) configuration file yolo-ants.cfg, 2) metadata file yolo-ants.data, and 3) weights file yolo-ants.weights. The flag --show_video is used to tell BACH to show a video of the tracking, in real-time, with a requested resolution of 1920x1080. The threshold set for the Yolo model is 0.25, i.e. the model will recognize anything that has a confidence score of 25% or more. Finally, a text file containing the position of each tracked ant will be saved in the file today.out in the directory /mnt/data/experiments.

Example of execution:

python -m bach.tracker --file /mnt/data/videos/recorded/today.mp4 -c /mnt/data/models/2020-15-01/yolo-ants.cfg -m /mnt/data/models/2020-15-01/yolo-ants.data -w /mnt/data/models/2020-15-01/yolo-ants.weights --threshold 0.25 --output_file /mnt/data/experiments/today.out --video_output /mnt/data/videos/tracking/today --width 4096 --height 2160 --fps 10

In this example the recorded video to track is today.mp4 in the directory /mnt/data/videos/recorded. The Yolo model is stored in the directory /mnt/data/models/2020-15-01 and is composed of three distinct files: 1) configuration file yolo-ants.cfg, 2) metadata file yolo-ants.data, and 3) weights file yolo-ants.weights. A video containing the tracking is saved in the file today.mp4 in the directory /mnt/data/videos/tracking, with a resolution of 4096x2160 and at 10 FPS. The threshold set for the Yolo model is 0.25, i.e. the model will recognize anything that has a confidence score of 25% or more. Finally, a text file containing the position of each tracked ant will be saved in the file today.out in the directory /mnt/data/experiments.

Debug

To enable debugging mode, simply add --debug as a command line option to bach.tracker.

Ghost Threshold

Entities are becoming ghosts when they are not seen for a certain number of frames. Ghosts are eliminated from the list of entities when they reach a certain threshold, expressed in frames, that is passed to the bach.tracker module via the --ghost_threshold command line option. The option takes an integer as parameter, and the integer is the number of frames necessary to become a ghost.