Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to understand the following implementation of loss in theano? #88

Open
yxchng opened this issue Feb 26, 2017 · 4 comments
Open

How to understand the following implementation of loss in theano? #88

yxchng opened this issue Feb 26, 2017 · 4 comments

Comments

@yxchng
Copy link

yxchng commented Feb 26, 2017 •
edited
Loading

screen shot 2017-02-26 at 2 33 16 pm

I don't see how the stochasticity of the actions 1/T*... is implemented in the theano line above. Isn't log_ likelihood_sym only computing one distribution? and not T of them and taking the average.
@yxchng
Copy link
Author

yxchng commented Feb 26, 2017 •
edited
Loading

Also, I think the right expression should be this, meaning it should take the gradient of the loglikelihood and not average

screen shot 2017-02-26 at 2 40 12 pm

Isn't it?

@dementrock
Copy link
Member

I don't understand your question. Do you mean it should be 1/N * ... instead of 1/NT * ... ?

@yxchng
Copy link
Author

yxchng commented Feb 27, 2017

I think so. Not sure

@dementrock
Copy link
Member

Generally you want your loss / gradient updates to be scale invariant. Also it does not really matter if you use rmsprop / adam etc. since they automatically rescale your gradients.

jonashen pushed a commit to jonashen/rllab that referenced this issue May 29, 2018
@CatherineSue
Support for tensor logging to tensorboard (rll#88)
4d2417e 
Add a customized tensor scalar to tensorboard by using the
custom_scalar plugin in tensorboard. Each line in the scalar
corresponds to an element in the tensor.

Wrap the tensorboard logging module into a new class `Summary`
in file rllab/misc/tensor_summary.py. It supports both the
simple value and tensor logging. It also saves the
computation graph created by rllab.

To record the tensor into tensorboard, use the
`record_tensor` function in file rllab/misc/logger.py.

Refer to: rll#39, rll#38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dementrock @yxchng