Episodes support

[andreagentili]

Is it possible to train this network with episodes? E.g. in Karpathy Waterworld, one can add a rule "the character dies if it eats a green pill". After death the "character" shall be revived, how do I tell the network to switch to a new episode? The same could be done for cars example, if a car "explodes" when it touches a wall.

Great lib anyway, and code is well written.

[janhuenermann]

Currently there is no support for episodes and I don't plan to add it. But I think, what you can maybe do, is try to just remove the character from the map and then put it in a kind of "death" state, which should lead to low reward. In this way, the character tries to avoid this state. I'll mark this issue as enhancement, so someone can implement this in the future. Maybe this helps.

[andreagentili]

Thanks for your reply!
But I did't exactly get what you mean :) What do you mean by a "death state"?
Maybe:

1. character executes agent's actions and hits a green ball
2. agent gets a bad reward (-1000) and learns
3. character and world is reset (e.g. position)
4. goto 1

Is this it?

[janhuenermann]

Yes about right ;)

Instead of step 2 you could also just remove the agent and give it like -10 reward for a few timesteps (1-2 seconds, I don't know just guessing). But really, I never tried this and it's just a suggestion.

[js08]

@andreagentili hey your link doesnt work https://github.com/janhuenermann/neurojs/issues/cs.stanford.edu/people/karpathy/reinforcejs/waterworld.html

[js08]

@janhuenermann can you tell the file name where we need to make the changes

[andreagentili]

Fixed the link, was just missing the starting http://.