Is there examples about using multiple Inputs observations?

[akux2021]

Hi @Toni-SM ,

Are there examples of using multiple Inputs observations?

For example, one input is an image and another input is a vector.

Something similar in stablebaseline3 here https://stable-baselines3.readthedocs.io/en/master/guide/custom_policy.html#multiple-inputs-and-dictionary-observations

[Toni-SM]

Hi @akux2021

Yes, skrl allows using complex spaces like gym.spaces.Dict.

The observations are automatically converted to flatten tensors to store them in memory.
The flattened tensor is forward to the models .compute() method.
Models have a method to convert the flattened tensor to the original spaces: Model.tensor_to_space.

You can see the method documentation example code for more details.
Also, you can see its usage in the DeepMind's reach_site_vision example. Specifically, this feature is highlighted and commented in this example

[akux2021]

Thanks, it`s very helpful!

[elle-miller]

Hi @Toni-SM, I am having difficulty with using dictionary observation spaces. I am using SKRL with Isaac Lab environments.

Where/how are the dictionary observations flattened? I have looked through the code but can't find evidence of this. At the moment the observation is arriving as a dictionary to compute(), and not a flattened tensor.

This would be fine, except I am getting errors since the memory class expects flattened tensors for storing. Would it be possible for the memory class to store dictionaries?

Specifically, the error comes from add_samples() in Memory

# dimensions and shapes of the tensors (assume all tensors have the dimensions of the first tensor)
tmp = tensors.get("states", tensors[next(iter(tensors))])  # ask for states first
dim, shape = tmp.ndim, tmp.shape
...
AttributeError: 'dict' object has no attribute 'ndim'

[Toni-SM]

Hi @elle-miller

Due to Isaac Lab that only supports Box spaces at the moment, the automatic conversion from space to tensor is not implemented in the wrapper (in contrast to the gym/gymnasium wrapper). This is why the error is generated when trying to store the information in memory.

I am working (on the toni/space_tensor_conversion branch) on support it... If you can wait, I can have a working version by the end of next week 😅.

Btw, are you using the skrl integration in Isaac Lab (skrl_ppo_cfg.yaml) or the skrl's Python scripts examples?

[elle-miller]

@Toni-SM Thanks very much, that is great to hear! Looking forward to the new version 😄

I am using the skrl integration in Isaac Lab skrl_ppo_cfg.yaml

[Toni-SM]

Hi @elle-miller Support for different observation spaces should be work now with the toni/space_tensor_conversion branch

You can install and try it via

./isaaclab.sh -p -m pip install git+https://github.com/Toni-SM/skrl.git@toni/space_tensor_conversion