How to read/write SEI data of keyframes in FLV files using libmedia, similar to ffmpeg?

[Kuo-TingKai]

Hello,

I'm working on a project that involves handling FLV (Flash Video) files, and I need to read and write SEI (Supplemental Enhancement Information) data for keyframes. I'm currently using libmedia, but I'm not sure how to achieve this functionality.

I know that ffmpeg has methods to handle SEI data in video streams. I'm looking for a similar approach using libmedia. Specifically, I'd like to know:

1. How can I read SEI data from keyframes in an FLV file using libmedia?
2. What's the process for writing SEI data to keyframes in an FLV file with libmedia?
3. Are there any specific functions or classes in libmedia that are designed for handling SEI data?
4. If libmedia doesn't have direct support for this, are there any workarounds or alternative libraries you'd recommend?

Any code examples or detailed explanations would be greatly appreciated. Thank you in advance for your help!

[zhaohappy]

You can process avpacket data by yourself to handle SEI data. I had commit a code example, you can check sei for detail.

[ikigai-kevin-k]

Thanks for your informative example!🙏
Unfortunately I am a newbie for both typescript and ffmpeg😔.
Could you further explain how to integrate the readSEI function into /product/player.html.
What I want to implement is, when I load and play the video (.flv, .mp4, .mp3) then the console log will show the SEI data for each frame without interruption.
In my imagination, the implentation logic in product/player.html would be like:

   ...
   import { readSEI } from '../../examples/sei.js';
   ...
   function run() {
       player = new AVplayer{
       ...
           player.on('packet', async (packet, stream) => {
               try {
                   await readSEI(packet, stream);
               } catch (error) {
               console.error('Error reading SEI:', error);
               }
           });
       }
   }

[zhaohappy]

AVplayer has no packet event, but you can add logic in line, for example:

if (task.formatContext.streams[streamIndex].codecpar.codecType === AVMediaType.AVMEDIA_TYPE_AUDIO
   && task.cacheAVPackets.has(streamIndex)
) {
  ...
}
else if (task.formatContext.streams[streamIndex].codecpar.codecType === AVMediaType.AVMEDIA_TYPE_VIDEO
  && task.cacheAVPackets.has(streamIndex)
) {
   readSEI(avpacket, task.formatContext.streams[streamIndex])
}

Then compile your own AVplayer and use it.

[ikigai-kevin-k]

I just finally customized your example to fit my encountered scenario, with a lot of gratefulness!