What is the difference between the diffusion_unet_hybrid_image_policy and diffusion_unet_image_policy ?

[KIM-HYEDO]

What is the difference between the diffusion_unet_hybrid_image_policy and diffusion_unet_image_policy ?

[YanJiaHuan]

diffusion_unet_image_policy use only image input while diffusion_unet_hybrid_image_policy uses both camera and the robot's low_dim data like joints or endpose.
By the way, they are both used in sim env, if one policy is ready to go in real world, they will add prefix 'real'

[Fchaubard]

Same kindof question. I understand the difference between the two but the "vision-based" model they have in the demo is NOT ONLY vision-based, its the hybrid model which uses vision + state (even though the notebook says vision-based.. its not.. its vision+state based)..

And so I am trying to see eval code + checkpoints for the vision only.

I see here the training code they used for the vision only training...
https://github.com/real-stanford/diffusion_policy/blob/main/diffusion_policy/policy/diffusion_unet_hybrid_image_policy.py#L284.

But seems like all of the image checkpoints have a hydra config which says hybrid!

See here:
https://diffusion-policy.cs.columbia.edu/data/experiments/image/pusht/diffusion_policy_cnn/train_2/checkpoints/

and the config here:

https://diffusion-policy.cs.columbia.edu/data/experiments/image/pusht/diffusion_policy_cnn/config.yaml that says:

target: diffusion_policy.workspace.train_diffusion_unet_hybrid_workspace.TrainDiffusionUnetHybridWorkspace
checkpoint:
save_last_ckpt: true
save_last_snapshot: false
topk:
format_str: epoch={epoch:04d}-test_mean_score={test_mean_score:.3f}.ckpt
k: 5
mode: max
monitor_key: test_mean_score
dataloader:
batch_size: 64
num_workers: 8
persistent_workers: false
pin_memory: true
shuffle: true
dataset_obs_steps: 2
ema:
target: diffusion_policy.model.diffusion.ema_model.EMAModel
inv_gamma: 1.0
max_value: 0.9999
min_value: 0.0
power: 0.75
update_after_step: 0
exp_name: default
horizon: 16
keypoint_visible_rate: 1.0
logging:
group: null
id: null
mode: online
name: 2023.01.16-20.20.06_train_diffusion_unet_hybrid_pusht_image
project: diffusion_policy_debug
resume: true
tags:

• train_diffusion_unet_hybrid
• pusht_image
• default
  multi_run:
  run_dir: data/outputs/2023.01.16/20.20.06_train_diffusion_unet_hybrid_pusht_image
  wandb_name_base: 2023.01.16-20.20.06_train_diffusion_unet_hybrid_pusht_image
  n_action_steps: 8
  n_latency_steps: 0
  n_obs_steps: 2
  name: train_diffusion_unet_hybrid

Where are the image only checkpoints? And how to eval it?