Dot products (Linear Algebra for Machine Learning)

[vasilygrz]

Hi Jon,

Thank you for creating these courses. I'm currently doing the first one, Linear Algebra for Machine Learning.

In 3.4 The Dot Product, you said the following:

1. (...) a lot of the PyTorch operations (...) don't work on integer-typed PyTorch tensors.

   I've found this open issue in PyTorch issue where they mention that torch.dot can be calculated with tensors of integer type on a CPU, but not on a GPU. The issue is that you can't use integers to calculate gradients, like you said.

   I'd like to propose adding a comment here to put this information into context:

   ML-foundations/notebooks/1-intro-to-linear-algebra.ipynb

   Line 3212 in f6e244f

   "torch.dot(torch.tensor([25, 2, 5.]), torch.tensor([0, 1, 2.]))"

2. (...) in TensorFlow, as far as am aware, there's a not a built-in dot product method.

   I've searched and found the tf.tensordot function. It can be calculated as:

   tf.tensordot(x_tf, y_tf, axes=1)
   

   Similarly, I'd like to propose adding the command below the existing one with a comment:

   ML-foundations/notebooks/1-intro-to-linear-algebra.ipynb

   Line 3291 in f6e244f

   "tf.reduce_sum(tf.multiply(x_tf, y_tf))"

[jonkrohn]

Sounds great, @vasilygrz! Some of these may be developments since I first created the content, but — if so — they are developments everyone should know about! Would you like to make a pull request?

[vasilygrz]

Thank you for accepting my ideas, @jonkrohn. Yes, I'd like to.

Questions:

• Should I only add a comment in the code cell or can I also create a text cell?
• After that, should I restart and run the whole notebook?

Comments:

Today I've discovered that the tensordot function also exists in NumPy, which allows you to choose the axis, and the einsum function, which provides you more flexibility with matrix operations.

Have you thought about adding these functions to the notebooks?

[jonkrohn]

Great!

I'd recommend adding the new functions as comments below the existing command, within the same code cell. I think that would minimize confusion for folks who are watching the video content.

Hmm, how about I restart and run the whole notebook after the pull request? Otherwise it'll be a mess to see what you changed (Jupyter will change tons of lines).

Feel free to add comments about the NumPy functions you found today too!