Generator extremely slow

[Michelvl92]

Hi,

I have used your generator, for video clips of resolution: 640,480 (mp4), which are between 6sec - 5min long. When the generator is configured to output a batchsize of 2, and nb frames of 24. But to take the first batch of the generator, and can take up to 24sec. I don’t have a slow server. It’s a relatively new Intel Xeon with 12 cores, up to 4Ghz, 64GB of RAM and a 1080 gpu.

Training will take up to multiple days with such slow loading times, and this framework is not usable.

Would you have any idea how to speed this up? E.g. should I divide my videos to multiple smaller videos of e.g. 5 sec?

kind regards.

[metal3d]

There are severa possibilities that makes the video reading so slow. The first one is that your video meta values (inside header) doesn't give the right frame count, so the generator needs to read the entire video to count the frames. MP4 is sometimes a problem for me too.
I have not a lot of times for a few months to work on the project and to enhance it. I will try to find some times to make better benchmarks.