PyTensor intro walkthrough #34
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
One thing that may be useful is to wrap up with an applied simple example. Logistic regression with a naive gradient descent? |
|
@ricardoV94 ping for review on your own PR |
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
fused log add is incorrect (as we'll eventually mention fusion optimizations one way or another). The owner is log, which takes as input the output of an add operation
Reply via ReviewNB
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
I wouldn't use the word side effects.Perhaps say it will change the meaning of the graph_
Reply via ReviewNB
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
loop lifting -> loop fusion
More interesting is cos writing directly on the squeezed output. In numpy this would be doing the optimiztaino np.cos(x, out=x) which quickly becomes unreadable. Also squeeze is a view of CGemv so no intermediate allocation there either, but we can omit that. If you want to show use print_memory_map in the new version that shows both the destroy and view maps
Reply via ReviewNB
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
The use of two functions inadvertently highlights one of the advantages of the pytensor graph-way of building computations. You don't need to think in advance how to define intermediate functions so that you can get intermediate outputs from specific inputs to JIT (as you would need to in Numba/JAX). You just grab the relevant variable and plug them as inputs/outputs. May be worth to mention.
Reply via ReviewNB
| @@ -0,0 +1,2873 @@ | |||
| { | |||
There was a problem hiding this comment.
There's no second step, we just pass exp(new_input) instead of old_input . I just called that log_x because once you exponentiate it you get x (old input)
Reply via ReviewNB
There was a problem hiding this comment.
I think it's too big brain, I simplified it.
|
Looks good @jessegrabowski , left some small notes |
|
Left one minor comment, looks great otherwise |
Closes #12
Closes #13