Fix --video producing black frames without --viz kit (issue #5052)#5056
Fix --video producing black frames without --viz kit (issue #5052)#5056pascal-roth wants to merge 3 commits intodevelopfrom
Conversation
SimulationContext.render() was not calling omni.kit.app.get_app().update() when running with Isaac Sim (Kit) and no active visualizer pumped the Kit app loop. This meant replicator render products (used by gym.wrappers.RecordVideo via render_mode="rgb_array") were never refreshed, producing black video frames. The fix calls app.update() in render() when has_kit() is True and no visualizer with pumps_app_update() == True is active. KitVisualizer already calls app.update() in its own step(), so we skip the call to avoid double-rendering when --viz kit is used. Adds two regression tests to verify: 1. render() calls app.update() when no visualizer pumps the Kit loop 2. render() does not call app.update() when a visualizer already does Fixes #5052
|
Hi @pascal-roth . Thanks for this! I tested it out. It does fix the blank videos, but it seems the camera position is not updated properly according to the |
|
Thanks for testing and reporting this, @nblauch! The camera perspective mismatch you're seeing (headless Good news: that exact problem is already being fixed in PR #5011 (Add perspective video recording via Newton GL viewer for Kitless backends; Fix for Kitfull backends camera position). That PR explicitly reads To avoid duplicating the fix here and keeping both diffs focused, I'd suggest we track resolution in #5011. Once that lands, the Let me know if the scope there differs from what you experienced! |
Summary
SimulationContext.render()was not callingomni.kit.app.get_app().update()when no active visualizer pumped the Kit app loop. Replicator render products (used bygym.wrappers.RecordVideoviarender_mode="rgb_array") were never refreshed, producing black video frames.app.update()inrender()whenhas_kit()isTrueand no visualizer withpumps_app_update() == Trueis active.KitVisualizeralready callsapp.update()in its ownstep(), so we skip the call to avoid double-rendering when--viz kitis used.test_simulation_context.py— one verifyingapp.update()IS called without a visualizer, one verifying it is NOT called when a visualizer already pumps the Kit loop.Test plan
test_render_pumps_app_update_without_visualizer— fails without fix, passes with fixtest_render_skips_app_update_when_visualizer_pumps_it— passes with fix (no double-rendering)./isaaclab.sh -f)./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Velocity-Rough-Anymal-C-v0 --videoshould produce correct (non-black) videoCloses #5052
🤖 Generated with Claude Code