WebRTC and two way audio

[jaccky]

Hi and thank you for your amazing work.
I'm in a scenario in wich I'm trying to configure a RPI3 with RPI camera module and raspiudio mic v3 as

• a smart video doorbell, the client has to be chrome on android smartphones
• at the same time the video feed will be used via rtsp by a frigate instance installed of ciurse on another dedicated machine
  Go2RTC is installed on the Raspberry PI 3

Everything works fine, but I can't understand if two way audio is possible or not.
In the Readme.md is written that two way audio is possible via browser/WebRTC and with exec source. But I don't understand and I've not been able to redirect the talkback channel or the {input} to ffmpeg to reproduce the audio on the raspberry audio card (alsa)

Do you think such a configuration would be possible ? If there is a "talkback" stream coming from the browser is it possible to redirect to the local audio card even using ffmpeg or exe ??

Many thanks,
Gianantonio

[AlexxIT]

Exec source support backchannel. You can check examples in the readme
https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-exec

[jaccky]

Hi,
sorry to bore you again, finally I came up with this configuration:

streams:
  cam:
    - exec:rpicam-vid --nopreview -t 0   -b 10000000 --width 1024 --height 768 --vflip  --inline -o -
  mic:
    exec:ffmpeg -hide_banner -f alsa -sample_fmt s16 -i plughw:1,0 -c:a aac -ar 16000 -ac 1 -b:a 64k  -f rtsp {output}
  dueway:
    - exec:ffmpeg -hide_banner -i rtsp://localhost:8554/cam -i rtsp://localhost:8554/mic -c:v copy -c:a copy -f rtsp {output}
    - exec:ffplay -fflags nobuffer -nodisp -f alaw -ar 8000 -i -#backchannel=1

api:
  listen: ":1984"    # default ":1984", HTTP API port ("" - disabled)
  origin: "*"        # default "", allow CORS requests (only * supported)
  tls_listen: ":8443" # default "", enable HTTPS server
  tls_cert: "/home/gng/ssl/videophone.crt"       # default "", PEM-encoded fullchain certificate for HTTPS
  tls_key: "/home/gng/ssl/videophone.key"        # default "", PEM-encoded private key for HTTPS

webrtc:
  allow_talk: true

when I open the webrtc link: https://XXXXXXX.XXXXXX.net:8443/webrtc.html?src=dueway&media=video+audio+microphone

I can see this info for the stream:

{
  "producers": [
    {
      "id": 170,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "127.0.0.1:34610 forwarded localhost:8554",
      "source": "exec:ffmpeg -hide_banner -i rtsp://localhost:8554/cam -i rtsp://localhost:8554/mic -c:v copy -c:a copy -f rtsp rtsp://127.0.0.1:8554/ed9ab454ae90b16924358fadb4d3a495",
      "url": "rtsp://localhost:8554/cam",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=go2rtc/1.9.7\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1; sprop-parameter-sets=J2QAKKwrQCADDQgAAAMACAAAAwHnJQABMSwAAvrze9wDxImo,KO4CXLA=; profile-level-id=640028\r\na=control:streamid=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/16000/1\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=140856E500\r\na=control:streamid=1\r\n",
      "user_agent": "Lavf59.27.100",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, MPEG4-GENERIC/16000"
      ],
      "receivers": [
        {
          "id": 171,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video",
            "level": 40,
            "profile": "High"
          },
          "childs": [
            173
          ],
          "bytes": 9521606,
          "packets": 6929
        },
        {
          "id": 172,
          "codec": {
            "codec_name": "aac",
            "codec_type": "audio",
            "sample_rate": 16000
          },
          "bytes": 27922,
          "packets": 28
        }
      ],
      "bytes_recv": 9633292
    },
    {
      "id": 2154619888,
      "format_name": "exec",
      "protocol": "pipe",
      "medias": [
        "audio, sendonly, PCMA/8000, L16"
      ],
      "senders": [
        {
          "id": 181,
          "codec": {
            "codec_name": "pcm_alaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "parent": 174,
          "bytes": 4960,
          "packets": 31
        }
      ],
      "bytes_send": 4960
    }
  ],
  "consumers": [
    {
      "id": 160,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "10.0.0.159:61552 host",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, VP8, VP9, H264, AV1",
        "audio, recvonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000",
        "audio, sendonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000, L16, PCML"
      ],
      "receivers": [
        {
          "id": 174,
          "codec": {
            "codec_name": "pcm_alaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "childs": [
            181
          ],
          "bytes": 187520,
          "packets": 1172
        }
      ],
      "senders": [
        {
          "id": 173,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 171,
          "bytes": 9521606,
          "packets": 6929
        }
      ],
      "bytes_recv": 211660,
      "bytes_send": 9624634
    }
  ]
}

But I still can't hear any audio from either the raspberry or the browser client....

Any suggestion on how to fix this ?

Thanks, Gianantonio.

[AlexxIT]

The settings seem to be fine. Data is being transmitted. Bytes values are increasing.
Probably some problem with alsa settings in the OS.

[chip131001]

@jaccky
Did you manage to solve the problem?