Audio mix source

[noho501]

Hi @pedroSG94 ,

Do you have any plans to improve the sound mixing quality when using both the microphone and in-app sound?
#1571

Currently, if both are used, the sound is not very clear. It seems like the microphone is picking up sound from both the speaker and microphone, making the live stream sound noisy and unpleasant.

If there is a way to mix the app sound and microphone audio more transparently, it would be great (I have already enabled echo cancellation and noise cancellation). I tried using the Larix app, and there is a noticeable difference in the sound quality.

Best regards,

[pedroSG94]

Hello,

What do you means with noisy and unpleasant?
Do you have audio pops? echo?
Can you reproduce it in all devices?

[noho501]

Yes, that's correct. I have tested both the RootEncoder and Larix apps, and below are the test links. The RootEncoder does not perform as well as Larix in terms of sound quality.

Larix: https://youtube.com/live/9tu5ujjUPxk
RootEncoder: https://youtube.com/live/ReBu0ijdaVc

[pedroSG94]

I see, I will check it.

For now, try add this line after create the stream class:

setTimestampMode(TimestampMode.BUFFER, TimestampMode.BUFFER)

This is only available in the master branch so you can use the app example with screen example and add that line here:
https://github.com/pedroSG94/RootEncoder/blob/master/app/src/main/java/com/pedro/streamer/screen/ScreenService.kt#L89

[pedroSG94]

I was doing tests using MixAudioSource and the audio sometimes produce noise but in general the quality is fine in my case. I tested using Pixel 6a and Xioami A3 (Android 14 and Android 10).

Also, after use TimestampMode.BUFFER, the quality is better and then I can't reproduce audio noise so this seem to solve the problem.

Let me know if you was able to fix the audio using that.

[noho501]

Hi,

We will run a test and provide you with the results. Thank you for your help!

[noho501]

Hi,

Thank you for your help. It seems to be working better than before.

[noho501]

Hi @pedroSG94 ,

We've encountered an issue with screen broadcasting that occurs after 20 minutes. At that point, an audio delay appears; for example, gunshots sound before they are visible in the video. You can see an example here: Delay.

Additionally, there's noticeable noise when the microphone is on, even though Noise Cancellation is enabled. Here’s an example: Noise.

Do you have any suggestions on how to address these issues?

[pedroSG94]

Hello,

Ok, I will check it.
Thank you for the report

[noho501]

Hi @pedroSG94 ,

How is it going?

[pedroSG94]

Hello,

Sorry for no report back. I was testing with 4 differents devices but I'm not able to reproduce it.
The sync and audio is the same that at the start doing tests of 40min+ (quality is good)

Tested devices: Pixel 7, Pixel 6a, Xiaomi A3, Samsung A50

I was using the app example by default and adding this line:

setTimestampMode(TimestampMode.BUFFER, TimestampMode.BUFFER)

For now, I'm testing in a local server

Did you tried with a local server instead YouTube to discard that the problem only happens in YouTube? (I will check this now too)
Can you tell me your video and audio configuration to do the test with the same values?
Did you modify the volume of the microphone or internal?
Did you tried remove TimestampMode.BUFFER to know if the delay problem is related with that? (You maybe will have pops on YouTube but you can discard the other problem)

[noho501]

Hi @pedroSG94

I just tested to the latest version and noticed an improvement in the delayed sound compared to the previous version. However, the sound still becomes distorted when the microphone is turned on.

You can listen to an example of the issue in this live stream: [YouTube Link]

I set this line on my code:
setTimestampMode(TimestampMode.BUFFER, TimestampMode.BUFFER)

Audio: 48Khz/128kps

Best regards,

[pedroSG94]

Hello,

Thank you for the info. I will check it using that audio config.
If you are using this to increase the volume of the microphone try to remove the line:

microphoneVolume = 2f

If the device audio is fine but only fail the microphone audio using MixAudioSource and it is working using MicrophoneSource, the problem could be related with the audio volumen filter or the audio mixer.

[pedroSG94]

Hello,

Finally, I was able to reproduce it. I'm working on it but it seems that the problem is related with the way audio is mixed

[pedroSG94]

Hello,

I was doing multiple tests but I'm no able to find the reason and fix it for now. I will continue with it because it is important but it could take time.

For now, I only dicovered that the bug can be reproduced after start and stop the stream multiple times. If you start stream the first time, all is working fine, at least in my case.
Also, if you reproduce this bug and change to MicrophoneSource, the audio is totally clear and return to MixAudioSource produce again the audio problem so I can say that the audio timestamp calculation should be fine and the problem should be in the mix algorithm of MixAudioSource or maybe the mix is too slow sometimes.

[noho501]

Hi,

Thank you. I will also review it on my end, and if I discover anything, I will be sure to inform you.

Best regards,