4k h265 to 720p h264

[levski]

i got go2rtc installed in a container with an nvidia gpu presented.
nvidia-smi is working
image is go2rtc:master-hardware

from the same stream, i want to forward the original for record and downscaled 720p version for detect.
record is working tested via go2rtc UI and ffmpeg from a remote machine
detect stream plays tested via UI, but still shows 4k and avframe (is avframe h264?)

streams:
  # Input Stream: Original H.265 stream from the camera
  3F_Hallway_record:
    - rtsp://user:[email protected]:554/h265Preview_01_main
      # Native H.265 recording, no transcoding
      # Hardware decoding and encoding (NVIDIA)
      # Using 'copy' for both video and audio streams
    - ffmpeg:rtsp://user:[email protected]:554/h265Preview_01_main#video=copy#audio=copy

  # Transcoded H.264 for detection
  3F_Hallway_detect:
    - ffmpeg:rtsp://user:[email protected]:554/h265Preview_01_main
      # Convert to H.264, scale to 720p, limit to 15 FPS
      # Hardware-accelerated decoding and encoding
    - ffmpeg:rtsp://user:[email protected]:554/h265Preview_01_main#hwaccel=cuda#video=h264_nvenc#width=1280#height=720#fps=15#media=video

    $ ffmpeg -i rtsp://user:[email protected]:8554/3F_Hallway_detect -f null -
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  WARNING: library configuration mismatch
  avcodec     configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libaribb24 --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc --enable-libsmbclient
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[rtsp @ 0x6330b8044680] method SETUP failed: 461 Unsupported transport
Input #0, rtsp, from 'rtsp://user:[email protected]:8554/3F_Hallway_detect':
  Metadata:
    title           : go2rtc/1.9.7
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Audio: aac (LC), 16000 Hz, mono, fltp
  Stream #0:1: Video: hevc (Main), yuv420p(tv), 3840x2160, 15 fps, 90k tbr, 90k tbn, 90k tbc
Stream mapping:
  Stream #0:1 -> #0:0 (hevc (native) -> wrapped_avframe (native))
  Stream #0:0 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[hevc @ 0x6330b8088dc0] Could not find ref with POC 12
Output #0, null, to 'pipe:':
  Metadata:
    title           : go2rtc/1.9.7
    encoder         : Lavf58.76.100
  Stream #0:0: Video: wrapped_avframe, yuv420p(tv, progressive), 3840x2160, q=2-31, 200 kb/s, 15 fps, 15 tbn
    Metadata:
      encoder         : Lavc58.134.100 wrapped_avframe
  Stream #0:1: Audio: pcm_s16le, 16000 Hz, mono, s16, 256 kb/s
    Metadata:
      encoder         : Lavc58.134.100 pcm_s16le
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 49 >= 49
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 65 >= 65
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 70 >= 70
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 94 >= 94
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 106 >= 106
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 109 >= 109
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 115 >= 115
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 118 >= 118
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 121 >= 121
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 125 >= 125
[null @ 0x6330b8077740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 130 >= 130
frame=  132 fps= 17 q=-0.0 Lsize=N/A time=00:00:09.00 bitrate=N/A speed=1.18x    
video:69kB audio:284kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Exiting normally, received signal 2.

some guidance is appreciated.

[AlexxIT]

You have many unsupported params for ffmpeg source. I don't know where you get them.
If you have changed default ffmpeg source params - it may broke hardware support.

[levski]

after more testing you right, i definitely mixed up lots of the stuff i googled and read from reddit, frigate, and github.
i got it cleaned up now, the record stream works but the detect just shows all green

rtsp:
  listen: ":8554"    # RTSP Server TCP port, default - 8554
  username: "user"  # Optional, default - disabled
  password: "password"   # Optional, default - disabled
  default_query: "video&audio"  # Optional, default codecs filters 

streams:
  # Input Stream: Original H.265 stream from the camera
  3F_Hallway_record:
    - rtsp://cam_user:[email protected]:554/h265Preview_01_main

  3F_Hallway_detect:
    - ffmpeg:rtsp://cam_user:[email protected]:554/h265Preview_01_main#video=h264#hardware#width=1280#height=720#fps=15

_record stream working as expected
_detect however is getting transcoded to h264 but has all green screen on the stream

error log:

h265Preview_01_main#video=h264#hardware#width=1280#height=720#fps=15 caller=github.com/AlexxIT/go2rtc/internal/streams/producer.go:170

ffmpeg from laptop to go2rtc
  Metadata:
    title           : go2rtc/1.9.7
  Duration: N/A, start: 43856.377767, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 24.58 fps, 24.58 tbr, 90k tbn, 49.17 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    title           : go2rtc/1.9.7
    encoder         : Lavf58.76.100
  Stream #0:0: Video: wrapped_avframe, yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 24.58 fps, 24.58 tbn
    Metadata:
      encoder         : Lavc58.134.100 wrapped_avframe

am close to the h264 720p 15 fps target. perhaps you can point me to what am doing wrong.

[levski]

made another tweak and the green screen cleared out and it's getting transcoded to 720p:

    - ffmpeg:rtsp://cam_user:[email protected]:554/h265Preview_01_main#video=h264#width=1280#height=720#fps=15

however i am uncertain if this is using the gpu to transcode.
nvidia-smi inside the go2rtc container doesn't show any ffmpeg process
nvidia-smi at the host level didn't show an another ffmpeg show up either.
what flag does one use for nvidia gpu transcode?

[AlexxIT]

You definitely have some kind of hardware transcoding problem. That's why it's turned off by default. It is unpredictable in different environments.

First try hardware transcoding without resizing the picture.

PS. fps parameter is not supported. I don't know where you got it from.

[levski]

hardware trancoding works with the cpu but can't seem to figure out the correct flags to get the gpu to do the work.
tried #video=264#hardware=cuda but that didn't do the trick as well so i googled and tested what i found that includes the fps lol

i found this from nvidia's docs hub:
ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -c:a copy -c:v h264_nvenc -preset p6 -tune ll -b:v 5M -bufsize 5M -maxrate 10M -qmin 0 -g 250 -bf 3 -b_ref_mode middle -temporal-aq 1 -rc-lookahead 20 -i_qfactor 0.75 -b_qfactor 1.1 output.mp4

from that i gathered these flags would make sense testing:
#hwaccel=cuda#hwaccel_output_format=cuda#c:a=copy#c:v=h264_nvenc#b:v=4M

enable hardware acceleration, use cuda, copy audio, transcode video to h264, bitrate of 4Mbit... do correct me if how i converted those flags with # is wrong. am clearly over my head. lol