Exporting object tracking task annotation from multiple jobs is not merging overlapping tracks

[corentin87]

Actions before raising this issue

• I searched the existing issues and did not find anything similar.
• I read/searched the docs

Steps to Reproduce

1. Create a new task and load a cars video of 100 frames
2. Select Advanced Configuration and split the task into 3 jobs of 50 frames
3. Draw a track on a car in job#1 for frames [0, 40]
4. Draw a track on the same car in job#2 for frames [40, 90]
5. Export task annotations to MOT 1.1

Expected Behavior

When performing tracking annotations on a long video, the user wants to be able to split the task into multiple jobs. The jobs need to have some overlapping frames in oder to connect the tracks during the export.

If there is 10 frames overlapping between job#1 and job#2 (frames 40 to 49) as mentioned above, the user is expecting to have only one track with the same ID (here id=1) in the exported MOT 1.1 gt.txt file, such as:

40,1,366.45,162.22,46.96,42.81,1,1,1.0
41,1,365.18,160.14,46.19,42.19,1,1,1.0
42,1,363.9,158.07,45.44,41.54,1,1,1.0
43,1,362.62,155.99,44.68,40.91,1,1,1.0
44,1,361.93,154.57,43.65,40.03,1,1,1.0
45,1,361.24,153.15,42.62,39.15,1,1,1.0
46,1,360.55,151.72,41.61,38.28,1,1,1.0
47,1,359.85,150.3,40.58,37.39,1,1,1.00
48,1,359.16,148.88,39.56,36.52,1,1,1.0
49,1,358.47,147.46,38.53,35.63,1,1,1.0

First issue, this is not what is happening. The track id=1 from job#1 and job#2 is not being merged. Instead, we are ending up with two different tracks (id=1 and id=2), with really similar coordinates.

40,1,366.45,162.22,46.96,42.81,1,1,1.0

41,2,363.4,158.3,48.25,42.11,1,1,1.0
41,1,365.18,160.14,46.19,42.19,1,1,1.0

42,2,362.78,156.87,47.08,41.37,1,1,1.0
42,1,363.9,158.07,45.44,41.54,1,1,1.0

43,2,362.16,155.45,45.89,40.60,1,1,1.0
43,1,362.62,155.99,44.68,40.91,1,1,1.0

44,2,361.54,154.03,44.72,39.83,1,1,1.0
44,1,361.93,154.57,43.65,40.03,1,1,1.0

45,2,360.92,152.61,43.56,39.05,1,1,1.0
45,1,361.24,153.15,42.62,39.15,1,1,1.0

46,2,360.3,151.18,42.38,38.29,1,1,1.0
46,1,360.55,151.72,41.61,38.28,1,1,1.0

47,2,359.68,149.76,41.20,37.53,1,1,1.0
47,1,359.85,150.3,40.58,37.39,1,1,1.0

48,2,359.06,148.34,40.04,36.75,1,1,1.0
48,1,359.16,148.88,39.56,36.52,1,1,1.0

49,2,358.26,147.09,39.53,36.19,1,1,1.0
49,1,358.47,147.46,38.53,35.63,1,1,1.0

Second issue, track id=1 becomes a static ghost track in job#2 after the overlapping frames until the end of the video:

50,2,357.46,145.85,39.02,35.59,1,1,1.0
50,1,357.77,146.03,37.53,34.77,1,1,1.0
51,2,356.66,144.6,38.5,35.03,1,1,1.0
51,1,357.77,146.03,37.53,34.77,1,1,1.0
52,2,355.87,143.36,37.98,34.44,1,1,1.0
52,1,357.77,146.03,37.53,34.77,1,1,1.0
53,2,355.07,142.11,37.47,33.86,1,1,1.0
53,1,357.77,146.03,37.53,34.77,1,1,1.0
54,2,354.27,140.87,36.96,33.28,1,1,1.0
54,1,357.77,146.03,37.53,34.771,1,1.0
55,2,353.47,139.63,36.44,32.70,1,1,1.0
55,1,357.77,146.03,37.53,34.77,1,1,1.0
56,2,352.67,138.38,35.93,32.12,1,1,1.0

And you can't know because it's visually not visible.

I am assuming, an IOU is used between the 10 overlapping frames and if the average is satisfying then the track id is kept. But here the coordinates are almost the same, so either there is a bug or either iOU overlap ration is set to too high.

Possible Solution

Merging overlapping tracks IDs should use a lower IOU threshold.

Context

I discover the problem, while checking my MOT gt.txt exported file where I saw overlapping tracks and some static ghost tracks that lived for thousand of frames. To confirm it, I draw the exported tracks back on the video. I was lucky to find the issue but that affected some of my other work because I was trusting CVAT would merge overlapping track properly.

Environment

No response