Alternative input representations to RGB

[willis-richard]

Is it possible to get an alternative input representation? For example, rather than an RBG input, to have one-hot matrices encoding features such as square contains:

• Nothing
• Apple
• Agent facing N
• Agent facing S
• etc

I am not familiar with lab2d so I wouldn't know how to do this.

[jzleibo]

You would have to add custom observations in the Lua for whatever information you are looking to pull out. There are lots of examples of how to add custom observations in the existing substrates. You can follow those examples. Take a look at Allelopathic Harvest, I believe that one has quite a few implemented. You'll probably find this tutorial helpful: https://github.com/google-deepmind/meltingpot/blob/main/docs/substrate_tutorial/index.md

Btw, this is a meltingpot question, not a lab2d question. Also: if you do look into lab2d (the underlying engine) you may come across the "layers" representation, which will sound like what you want. But for meltingpot it will give misleading and wrong answers. Don't use it. The right way to get new custom observations is to add them in the same way we add all the custom observations.

[duenez]

Closing. Feel free to re-open if you have more questions.