Support for HDR

[ThibaultBee]

Since 2.6.0, an experimental support of HDR is available for AVC, HEVC, VP9 and AV1.
Unfortunately, I don't have a phone with an HDR encoder + HDR camera and I could not properly test it.

How do I know my phone support HDR

In the demo-camera, you will see a profile that supports HDR: it will ends by 10 (such as Main10) or HDR or HDR10Plus

Here is the list of profile that supports HDR:

    private val avcProfilesMap = mapOf(
            AVCProfileHigh10 to hdr
        )

        private val hevcProfilesMap = mapOf(
            HEVCProfileMain10 to hdr,
            HEVCProfileMain10HDR10 to hdr10,
            HEVCProfileMain10HDR10Plus to hdr10Plus,
        )

        private val vp9ProfilesMap = mapOf(
            VP9Profile1 to hdr,
            VP9Profile2 to hdr,
            VP9Profile2HDR to hdr,
            VP9Profile2HDR10Plus to hdr10Plus,
            VP9Profile3 to hdr,
            VP9Profile3HDR to hdr10,
            VP9Profile3HDR10Plus to hdr10Plus,
        )

        private val av1ProfilesMap = mapOf(
            AV1ProfileMain10 to hdr,
            AV1ProfileMain10HDR10 to hdr10,
            AV1ProfileMain10HDR10Plus to hdr10Plus,
        )

Alternalively, you can call getSupportedHdrProfiles from the streamer helper

How can I help

I am looking for some testing.

• Could you tell me if the stream is okay?
• Brand/Model of devices for reference purpose
• Codec/Profile used
• Server

Note: Be sure that your server can properly handle HDR stream before commenting.

Thanks 🙏 🙏

[sharmashivanu91187]

I tested this on Samsung galaxy ultra 23, But HDR settings are not visible. In Google Pixel devices, HDR options are visible. In Samsung S23 ultra MediaCodecHelper.isFeatureSupported(mimeType, FEATURE_HdrEditing) returns false. Is this the reason for HDR options not visible.

[sharmashivanu91187]

I have tested on Pixel 7 and HDR stream is working in that(using SRT, HEVC encoder).

[sharmashivanu91187]

Also, I am getting following error when i stop stream:

main (blocked):tid=1 systid=3898 | waiting to lock <0x0c22df33> (java.lang.Object) held by thread 49
at io.github.thibaultbee.streampack.internal.encoders.MediaCodecEncoder.stopStream(MediaCodecEncoder.kt:222)
at io.github.thibaultbee.streampack.internal.encoders.VideoMediaCodecEncoder.stopStream(VideoMediaCodecEncoder.kt:120)
at io.github.thibaultbee.streampack.streamers.bases.BaseStreamer.stopStreamImpl(BaseStreamer.kt:332)
at io.github.thibaultbee.streampack.streamers.bases.BaseStreamer.stopStream$suspendImpl(BaseStreamer.kt:318)
at io.github.thibaultbee.streampack.streamers.bases.BaseStreamer.stopStream(unavailable)
at io.github.thibaultbee.streampack.ext.srt.streamers.CameraSrtLiveStreamer.stopStream(CameraSrtLiveStreamer.kt:249)
at com.latakoo.androidflight.capture.srt.utils.StreamerManager$stopStream$1.invokeSuspend(StreamerManager.kt:180)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:280)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
at kotlinx.coroutines.BuildersKt.runBlocking(unavailable:1)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
at kotlinx.coroutines.BuildersKt.runBlocking$default(unavailable:1)
at com.latakoo.androidflight.capture.srt.utils.StreamerManager.stopStream(StreamerManager.kt:179)
at com.latakoo.androidflight.capture.srt.ui.main.PreviewViewModel$stopStream$1.invokeSuspend(PreviewViewModel.kt:131)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.internal.DispatchedContinuationKt.resumeCancellableWith(DispatchedContinuation.kt:367)
at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:30)
at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable$default(Cancellable.kt:25)
at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:110)
at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:126)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch(Builders.common.kt:56)
at kotlinx.coroutines.BuildersKt.launch(unavailable:1)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.launch$default(Builders.common.kt:47)
at kotlinx.coroutines.BuildersKt.launch$default(unavailable:1)
at com.latakoo.androidflight.capture.srt.ui.main.PreviewViewModel.stopStream(PreviewViewModel.kt:129)
at com.latakoo.androidflight.capture.srt.ui.main.PreviewFragment.stopStream(PreviewFragment.kt:344)
at com.latakoo.androidflight.capture.srt.ui.main.PreviewFragment.access$stopStream(PreviewFragment.kt:63)
at com.latakoo.androidflight.capture.srt.ui.main.PreviewFragment$startStopLive$1.onFinish(PreviewFragment.kt:304)
at android.os.CountDownTimer$1.handleMessage(CountDownTimer.java:142)
at android.os.Handler.dispatchMessage(Handler.java:107)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.app.ActivityThread.main(ActivityThread.java:8501)
at java.lang.reflect.Method.invoke(Native method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:552)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:878)

   **Root blocking**
 c2.exynos.hevc.encoder.thread (native):tid=49 systid=4504 

pc 0x5a98c libc.so (syscall + 28) (BuildId: 33ad5959e2b38fc822cda3c642e16c94)
pc 0x5fcc0 libc.so (__futex_wait_ex + 144) (BuildId: 33ad5959e2b38fc822cda3c642e16c94)
pc 0xc9af8 libc.so (pthread_cond_wait + 72) (BuildId: 33ad5959e2b38fc822cda3c642e16c94)
pc 0xd5d04 base.apk (std::__ndk1::condition_variable::wait + 20) (BuildId: f259cea2e2f407141cffea3c63db584332ec18b9)
pc 0x9a990 base.apk (srt::CUDT::sendmsg2 + 1420) (BuildId: f259cea2e2f407141cffea3c63db584332ec18b9)
pc 0x73b40 base.apk (srt::CUDT::sendmsg2 + 84) (BuildId: f259cea2e2f407141cffea3c63db584332ec18b9)
pc 0xd0310 base.apk (srt_sendmsg2 + 64) (BuildId: f259cea2e2f407141cffea3c63db584332ec18b9)
pc 0x144e8 base.apk (nativeSendMsgCtrl2 + 288) (BuildId: 3f52e5fbacdd1de5ecef44ffde3a3659269b6cae)
pc 0x351e30 libart.so (art_quick_generic_jni_trampoline + 144) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x2012998 memfd:jit-cache (io.github.thibaultbee.srtdroid.models.Socket.send + 344)
pc 0x20109ac memfd:jit-cache (io.github.thibaultbee.streampack.ext.srt.internal.endpoints.SrtProducer.write + 1292)
pc 0x2011f2c memfd:jit-cache (io.github.thibaultbee.streampack.streamers.bases.BaseStreamer$muxListener$1.onOutputFrame + 236)
pc 0x2011c20 memfd:jit-cache (io.github.thibaultbee.streampack.internal.muxers.ts.utils.TSOutputCallback.writePacket + 320)
pc 0x2014c2c memfd:jit-cache (io.github.thibaultbee.streampack.internal.muxers.ts.packets.TS.write + 1772)
pc 0x2014470 memfd:jit-cache (io.github.thibaultbee.streampack.internal.muxers.ts.packets.Pes.write + 1328)
pc 0x33b3a4 libart.so (art_quick_invoke_stub + 612) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x511590 libart.so (bool art::interpreter::DoCall + 1976) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x490ffc libart.so (void art::interpreter::ExecuteSwitchImplCpp + 1012) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x3545d8 libart.so (ExecuteSwitchImplAsm + 8) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x4b26ec base.apk (io.github.thibaultbee.streampack.internal.muxers.ts.TSMuxer.generateStreams)
pc 0x36e6ec libart.so (art::interpreter::Execute + 232) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x36dfe4 libart.so (artQuickToInterpreterBridge + 964) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x351f68 libart.so (art_quick_to_interpreter_bridge + 88) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x2018200 memfd:jit-cache (io.github.thibaultbee.streampack.internal.muxers.ts.TSMuxer.encode + 4752)
pc 0x33b3a4 libart.so (art_quick_invoke_stub + 612) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x511590 libart.so (bool art::interpreter::DoCall + 1976) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x491eac libart.so (void art::interpreter::ExecuteSwitchImplCpp + 4772) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x3545d8 libart.so (ExecuteSwitchImplAsm + 8) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x4cb5c4 base.apk (io.github.thibaultbee.streampack.streamers.bases.BaseStreamer$videoEncoderListener$1.onOutputFrame)
pc 0x36e6ec libart.so (art::interpreter::Execute + 232) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x36dfe4 libart.so (artQuickToInterpreterBridge + 964) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x351f68 libart.so (art_quick_to_interpreter_bridge + 88) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x2015930 memfd:jit-cache (io.github.thibaultbee.streampack.internal.encoders.MediaCodecEncoder$encoderCallback$1.onOutputBufferAvailable + 880)
pc 0x5b98b0 libart.so (nterp_helper + 4016) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x3b7a90 framework.jar (android.media.MediaCodec$EventHandler.handleCallback + 352)
pc 0x5b9854 libart.so (nterp_helper + 3924) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x3b7d84 framework.jar (android.media.MediaCodec$EventHandler.handleMessage + 288)
pc 0x33b3a4 libart.so (art_quick_invoke_stub + 612) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x511590 libart.so (bool art::interpreter::DoCall + 1976) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x490ffc libart.so (void art::interpreter::ExecuteSwitchImplCpp + 1012) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x3545d8 libart.so (ExecuteSwitchImplAsm + 8) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x1b7f18 framework.jar (android.os.Handler.dispatchMessage)
pc 0x36e6ec libart.so (art::interpreter::Execute + 232) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x51225c libart.so (bool art::interpreter::DoCall + 5252) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x490ffc libart.so (void art::interpreter::ExecuteSwitchImplCpp + 1012) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x3545d8 libart.so (ExecuteSwitchImplAsm + 8) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x1dbe94 framework.jar (android.os.Looper.loopOnce)
pc 0x36e6ec libart.so (art::interpreter::Execute + 232) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x51225c libart.so (bool art::interpreter::DoCall + 5252) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x49136c libart.so (void art::interpreter::ExecuteSwitchImplCpp + 1892) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x3545d8 libart.so (ExecuteSwitchImplAsm + 8) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x1dc6d0 framework.jar (android.os.Looper.loop)
pc 0x36e6ec libart.so (art::interpreter::Execute + 232) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x51225c libart.so (bool art::interpreter::DoCall + 5252) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x49136c libart.so (void art::interpreter::ExecuteSwitchImplCpp + 1892) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x3545d8 libart.so (ExecuteSwitchImplAsm + 8) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x1b7558 framework.jar (android.os.HandlerThread.run)
pc 0x36e6ec libart.so (art::interpreter::Execute + 232) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x36dfe4 libart.so (artQuickToInterpreterBridge + 964) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x351f68 libart.so (art_quick_to_interpreter_bridge + 88) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x33b3a4 libart.so (art_quick_invoke_stub + 612) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x23a5d0 libart.so (art::ArtMethod::Invoke + 144) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x539a7c libart.so (art::Thread::CreateCallback + 1600) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0x53942c libart.so (art::Thread::CreateCallbackWithUffdGc + 8) (BuildId: 4801adadf1dae7e020ba05f3204efc9c)
pc 0xca7cc libc.so (__pthread_start + 204) (BuildId: 33ad5959e2b38fc822cda3c642e16c94)
pc 0x607b0 libc.so (__start_thread + 64) (BuildId: 33ad5959e2b38fc822cda3c642e16c94)
at io.github.thibaultbee.srtdroid.models.Socket.nativeSend(Native method)
at io.github.thibaultbee.srtdroid.models.Socket.send(Socket.kt:644)
at io.github.thibaultbee.streampack.ext.srt.internal.endpoints.SrtProducer.write(SrtProducer.kt:307)
at io.github.thibaultbee.streampack.streamers.bases.BaseStreamer$muxListener$1.onOutputFrame(BaseStreamer.kt:134)
at io.github.thibaultbee.streampack.internal.muxers.ts.utils.TSOutputCallback.writePacket(TSOutputCallback.kt:24)
at io.github.thibaultbee.streampack.internal.muxers.ts.packets.TS.write(TS.kt:130)
at io.github.thibaultbee.streampack.internal.muxers.ts.packets.Pes.write(Pes.kt:51)
at io.github.thibaultbee.streampack.internal.muxers.ts.TSMuxer.generateStreams(TSMuxer.kt:170)
at io.github.thibaultbee.streampack.internal.muxers.ts.TSMuxer.encode(TSMuxer.kt:159)
at io.github.thibaultbee.streampack.streamers.bases.BaseStreamer$videoEncoderListener$1.onOutputFrame(BaseStreamer.kt:122)
at io.github.thibaultbee.streampack.internal.encoders.MediaCodecEncoder$encoderCallback$1.onOutputBufferAvailable(MediaCodecEncoder.kt:79)
at android.media.MediaCodec$EventHandler.handleCallback(MediaCodec.java:1943)
at android.media.MediaCodec$EventHandler.handleMessage(MediaCodec.java:1841)
at android.os.Handler.dispatchMessage(Handler.java:107)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:68)

[sharmashivanu91187]

This Only happens in case of HDR streaming

[ThibaultBee]

The detection of hdr failed but I am pretty sure if you forced to use the main10 profile it will work.

[ThibaultBee]

About the HDR on the Samsung S23, the only reason it is not detected neither it is applied is because of FEATURE_HdrEditing.

According to the documentation of FEATURE_HdrEditing:

video encoder only: codec supports HDR editing.

HDR editing support means that the codec accepts 10-bit HDR input surface, and it is capable of generating any HDR metadata required from both YUV and RGB input when the metadata is not present. This feature is only meaningful when using an HDR capable profile (and 10-bit HDR input).

This feature implies that the codec is capable of encoding at least one HDR format, and that it supports RGBA_1010102 as well as P010, and optionally RGBA_FP16 input formats, and that the encoder can generate HDR metadata for all supported HDR input formats.

And from what I read on other dedicated articles: I expect FEATURE_HdrEditing to be true for c2.qti.hevc.encoder.
It is not the case. From my point of view, it is a Samsung or Qualcomm bug. The metadata of c2.qti.hevc.encoder have not been properly registered.

Now the question is: should I make an exception for c2.qti.hevc.encoder, delete calls to FEATURE_HdrEditing as a workaround>

[sharmashivanu91187]

Regarding Samsung S23 ultra, I am able to record HDR videos using this sample: https://github.com/android/camera-samples/tree/main/Camera2Video (Its Dynamic range profile is HLG10)

[ThibaultBee]

Could you get the name of the encoder? Be sure to NOT use the mediarecorder.
And log mEncoder.name

[sharmashivanu91187]

type=video/hevc
Avaliable Encoders:

1. c2.qti.hevc.encoder.hdr
2. c2.qti.hevc.encoder
3. OMX.qcom.video.encoder.hevc
4. c2.qti.hevc.encoder.cq
5. OMX.qcom.video.encoder.hevc.cq
6. c2.android.hevc.encoder

[sharmashivanu91187]

@ThibaultBee I got the reply from Samsung:

"Samsung has exposed HLG capture capability for 3rd party developers. You can use that for your application. However, HDR10+ is not supported for 3rd party applications at this time."

[ThibaultBee]

Ok, but why FEATURE_HdrEditing is set to false on encoders that support HDR?

[ThibaultBee]

Tested successfully on Pixel 8 with HEVC encoder on 3.0.0 RC1.