dataforge: robocap end-to-end slice (download/convert/register/view) - #112
Open
pablo-rerun wants to merge 3 commits into
Open
dataforge: robocap end-to-end slice (download/convert/register/view)#112pablo-rerun wants to merge 3 commits into
pablo-rerun wants to merge 3 commits into
Conversation
…gister / view RobocapDataset converts one (device, session, segment) into one exoego:v2 base-layer rrd: 6 fisheye62 cameras (Kalibr calibration reused from simplecv, video remuxed chunk-native via Mp4Reader and retimed onto the absolute comment-tag epoch) plus the dev0 IMU parsed raw from its SQLite db (basalt's scale constants and 14_902_432 ns camera->IMU offset; dev1/dev2 TODO). Timeline is video_time on the raw camera clock. Blueprint (3D rig | named camera panes | gyro+accel strip) is embedded at convert and registered as the catalog dataset default at register. IMU db failures are recoverable (session_1 ships a malformed dev2 db). Four tyro verbs under dataforge/apis with thin tools/apps shims and pixi tasks; exoego_schema.md §8 IMU section flips from reserved to emitted. Validated on real data: session_1 seg1 converted (6 cams, 336 frames), registered into 'robocap' on a local rerun server catalog, and pixel-verified in a headless viewer from the catalog URI — video decoding, frustums, IMU plots, dataforge:v1 properties, and the default blueprint (evidence in /tmp/rerun-viewer-validation/20260819-dataforge-robocap/).
Three independent reviews (API-minimalist, correctness, altitude) all upheld the chunk-retime design — the Mp4Reader docstring itself prescribes the downstream map, and no offset option or lens path exists. Fixes from the pass: preserve row/chunk ids in retime (no index=/entity_path= overrides), close the IMU sqlite connection (sqlite3's context manager only ends the transaction), reword the 'absolute epoch' docs to the boot-relative device clock and pin the duration-timeline choice, comment the send_chunks/sample_count eagerness coupling, and record start_time_ns as a capture property so the clock origin is queryable without scanning chunks.
…form, add cap mesh A static Transform3D on /world/rig_00 permanently shadows any temporal world_T_rig a derived pose layer stacks on the segment (verified against a live catalog: poses query back as identity, and static translation + temporal quaternion compose into silently wrong poses). Per exoego:v2 a world-anchored rig carries no transform, so the base now logs none, and also cedes the root ViewCoordinates to the pose layer (whose world is gravity-aligned Z-up; the base's rig frame is LFD, not the RDF previously declared). The textured cap scan (robocap-mesh/3DModel.glb, optional, skipped when unreadable) is logged as a static child at /world/rig_00/mesh with the hand-tuned scan-to-IMU-frame alignment carried over from robocap-slam, so the rig reads as a physical object and rides any pose layer for free.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #111 (contracts). The walking skeleton every dataset copies: robocap → one exoego:v2 base-layer rrd per
(device, session, segment).dataforge/datasets/robocap.py): verify-onlydownload(corpus lives on the NAS), sequence discovery (23 sessions / 88 segments,-olddirs excluded), andconvert:log_pinhole, video remuxed chunk-native (Mp4Reader→send_chunks, retimed onto the absolute epoch from the MP4commenttag)./world/rig_NN/imu_MM/{gyro,accel}: raw native-rate samples from the SQLite db (stdlibsqlite3), basalt's measured scale constants and 14 902 432 ns camera↔IMU offset;video_timeis the raw camera clock. dev1/dev2 + multi-IMU blueprint panes are TODO. Malformed dbs (session_1's dev2) degrade to a warning, never abort.dataforge/apis/) + thintools/apps/shims + pixi tasks:pixi run -e dataforge dataforge-{download,convert,register,view}.exoego_schema.md§8: IMU flips from reserved → emitted.Validated end-to-end on real data: session_1 seg1 converted in ~2 s (pure remux; 6 cams, 336 frames, 36 MiB), registered into a
robocapdataset on a localrerun servercatalog (port 51235), and pixel-verified headless from the catalog URI — video decode, frustums, IMU plots,dataforge:v1properties, and the default blueprint all render. Evidence:/tmp/rerun-viewer-validation/20260819-dataforge-robocap/.Gates:
pixi run -e dataforge-dev {tests,lint,typecheck,deadcode}all green (26 tests).Next: HOCap +
hf_fetch+ thegtlayer.