Real Time Video Processing

[imirzayev]

Hello,

Does it support real time video streaming or works only with pre-recorded videos?

[guohengkai]

Hi,
The current version doesn't support real time video streaming because of the sliding window implementation. I think it is possible to support streaming but we haven't test it yet.

[Yaoyaolingbro]

It is truly possible to realize the code. You can refer the vllm (simple idea, just need a queue to process while kv-cache would immensely increase the inference time.)

[nagadomi]

If a video stream contains many scene changes or camera switches, still no need to reset the model (input) state?

[guker]

Hi, The current version doesn't support real time video streaming because of the sliding window implementation. I think it is possible to support streaming but we haven't test it yet.

if i want to support real time video streaming, how i should modify the code? @guohengkai

[SiliChen321]

If a video stream contains many scene changes or camera switches, still no need to reset the model (input) state?

Our model can process videos with scene changes without requiring any modifications.

[nagadomi]

Thank you for answer to my quick question.