Bad stream.frameRate

[telamon]

Loading a .webm file

const data = readFileSync('test.webm')

const io = new ffmpeg.IOContext(data)

const inFormat = new ffmpeg.InputFormatContext(io)

console.log('=== ffmpeg dump ===')
ffmpeg.log.level = ffmpeg.log.INFO
inFormat.dump()

console.log('\n\n=== log stream[0] ===')
console.log(format.streams[0])

Outputs:

=== ffmpeg dump ===
Input #0, matroska,webm, from '':
  Metadata:
    ENCODER         : Lavf60.16.100
  Duration: 00:00:15.01, start: -0.007000, bitrate: N/A
  Stream #0:0: Video: av1 (libdav1d) (Main), yuv420p(tv, progressive), 1280x720, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc60.31.102 libsvtav1
        DURATION        : 00:00:15.000000000
  Stream #0:1: Audio: opus, 48000 Hz, mono, fltp
      Metadata:
        ENCODER         : Lavc60.31.102 libopus
        DURATION        : 00:00:15.008000000
  - stream=0 timebase=(1 / 1000)
  - stream=1 timebase=(1 / 1000)

=== log stream[0] ===
FFmpegStream {
  _handle: ArrayBuffer { byteLength: 8 },
  _codecParameters: FFmpegCodecParameters {
    codecType: 0,
    codecId: 225,
    width: 1280,
    height: 720,
    bitRate: 0,
    bitsPerCodedSample: 0,
    bitsPerRawSample: 0,
    format: 0,
    sampleRate: 0,
    nbChannels: 0,
    channelLayout: FFmpegChannelLayout { nbChannels: 0 },
    frameRate: FFmpegRational { numerator: 0, denominator: 1 },
    extraData: <Buffer 81 05 0c 00 0a 0b 00 00 00 2d 4c ff b3 c0 2f 80 04>
  }
}

The stream.frameRate is uninitialized, and stream.timeBase uses the normalized webm-format timebase (1 / 1000).
But from the format.dump() call we can see that it prints 30fps for "Stream #0:0".

How do i get the fps?

[ryanramage]

@sethvincent an interesting ticket for you. Also check you keet dm for more info on this

[sethvincent]

After looking into this my impression is we should use stream.avgFramerate to get the frame rate because AVCodecParameters::framerate isn't always set for containers like webm: https://ffmpeg.org/doxygen/trunk/structAVCodecParameters.html#a6a9cc50670962595c117ceda4e7d0ee0

But stream.avgFramerate should provide what we need.

We could optionally expose something like an fps getter:

get fps() {
  return this.avgFramerate.toNumber()
}

That would be handy.

We might also want to expose AVStream's r_frame_rate property: https://ffmpeg.org/doxygen/4.3/structAVStream.html#ad63fb11cc1415e278e09ddc676e8a1ad

I'm not sure how the use cases differ between r_frame_rate and the average frame rate.

I noticed we didn't have stream.avgFramerate documented in the readme so added it and a couple other properties here: #100

[tony-go]

I'm not sure how the use cases differ between r_frame_rate and the average frame rate.

      * Real base framerate of the stream.
      * This is the lowest framerate with which all timestamps can be
      * represented accurately (it is the least common multiple of all
      * framerates in the stream). Note, this value is just a guess!
      * For example, if the time base is 1/90000 and all frames have either
      * approximately 3600 or 1800 timer ticks, then r_frame_rate will be 50/1.

Ref: https://ffmpeg.org/doxygen/4.3/avformat_8h_source.html#l00989

[tony-go]

Also I wonder if after a few frame read the stream could not be updated. 🤔