Getting access to a texture or media stream of the WebGL scene data?

[udbhav-s]

I've been trying to capture the user's AR session and made a small demo using the Raw Camera Access API and it succeeds in capturing the camera frame, but how should I access the framebuffer for the 3D scene (drawn by Three.js in my case)? Calling readPixels() on the rendering context still returns a black frame. I am not sure if the secondary-views feature mentioned here is meant for this purpose, but on my device the feature is not recognized even with WebXR Incubations enabled in the chrome flags (camera-access works fine). Is my best bet rerendering the scene twice in another canvas using the same pose information to be able to capture a stream?

[tangobravo]

I assume you want to capture the combined camera frame and scene content? I don't think that's particularly well supported with the current API - my suggestion in immersive-web/webxr-ar-module#78 would mean the site would do all the rendering into a standard canvas, which could then be captured just with captureStream().

Right now, a separate WebGL canvas for the capture might be the best way.

I wonder if it would work to set that separate canvas as a DOM Overlay... then you might not need to render it to the XRSession context at all, though the browser will likely still do lots of unnecessary compositing work I imagine.