How to get the gripper's state

[Jay-Ye]

Thanks so much for this amazing work.
I'm wondering if it's possible to get the gripper's state information that is aligned with the dataset.
I checked env.agent.get_gripper_closedness(), but it seems not to be similar to the real gripper state recorded episode_steps[i]['observation']['state'][6]

[xuanlinli17]

In this case you need env.agent.robot.get_qpos() and retrieve the gripper qpos and remap to the dataset range. Gripper qpos ranges can be obtained from env.agent.robot.get_qlimits().

You can also visualize the robot urdfs in https://github.com/simpler-env/SimplerEnv/tree/main/tools/robot_object_visualization to understand the meaning of each joint's qpos.

[Jay-Ye]

Thank you so much for the clarification.
I have a side question regarding the camera poses.
To get good real-to-sim visual matching, we do need the camera extrinsic w.r.t the robot base to be consistent across real rollout and simulated environments. As far as I know, neither Bridge nor RT-1 data provided camera extrinsic.
I noticed says here, that you can manually tune the camera pose to make it better align with real demonstrations. Is this also how you get the camera extrinsic for the Bridge and RT-1 settings?

[xuanlinli17]

I use fSpy to get the camera extrinsics for Bridge. Google Robot's camera extrinsics are already known.

[Jay-Ye]

Thank you so much for patiently answering my questions.
Congratulations again on this amazing work!

[JaehongJaehongMin]

Hello, @Jay-Ye
Could you please explain how did you record the environment data like in rlds (tfds) fornat?
I was trying to use EnvLogger, but it didn't work that easy that I thought. :(

Thank you!