01-tensor-operations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Quick Blitz on PyTorch tensor operations\n",
    "\n",
    "PyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing.\n",
    "\n",
    "The torch package contains data structures for multi-dimensional tensors and mathematical operation over these are defined. Additionally, it provides many utilities for efficient serializing of Tensors and arbitrary types, and other useful utilities.\n",
    "\n",
    "In this notebook, I am going to enlighten following five fuctions with suitable examples:\n",
    "\n",
    "1. torch.arange()\n",
    "> initalizes a tensor with a range of value\n",
    "2. torch.complex()\n",
    "> constructs a complex tensor with real and imaginary part\n",
    "3. torch.squeeze()\n",
    "> returns a tensor with all the dimensions of `input` size 1 removed\n",
    "4. torch.as_strided()\n",
    "> creates a view of an existing *torch.Tensor* `input`\n",
    "5. torch.randn()\n",
    "> returns a tensor defined by the variable argument size, containing random numbers from standard normal distribution.\n",
    "\n",
    "Before we begin, let's install and import PyTorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------------------------------------------------------------------\n",
    "## Function 1 - torch.arange\n",
    "\n",
    ">torch.arange(start=0, end, step=1, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor\n",
    "\n",
    "This funtion returns a 1-D tensor of size `{(end-start)/step}` with values from the interval `[start, end)` taken with common difference `step` beginning from *start*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 1, 2, 3, 4, 5, 6, 7])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 1 - working\n",
    "\n",
    "torch.arange(8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example clearly shows, if only one parameter is passed through the function then that parameter will be considered as `end`. This implies `start` will be taken as 0 and `step` will be 1, by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([2.0000, 2.4500, 2.9000, 3.3500, 3.8000, 4.2500, 4.7000, 5.1500])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 2 - working\n",
    "\n",
    "torch.arange(2, 5.5, 0.45)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If dtype is not given, infer the data type from the other input arguments. If any of start, end, or stop are floating-point, the dtype is inferred to be the default dtype. Otherwise, the dtype is inferred to be torch.int64."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "upper bound and larger bound inconsistent with step sign",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-4-e7598494595c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Example 3 - breaking (to illustrate when it breaks)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m8\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2.5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m: upper bound and larger bound inconsistent with step sign"
     ]
    }
   ],
   "source": [
    "# Example 3 - breaking (to illustrate when it breaks)\n",
    "\n",
    "torch.arange(8, 2.5, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if the `start` is greater than the `end`, then `step` can't be positive.\n",
    "To fix this, you need to set the `step` as a negative number in order to traverse back."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`torch.arange()` can be used whenever there is a need to create a 1-D tensor with a specified range and step."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------------------------------------------------------------------\n",
    "## Function 2 - torch.complex\n",
    "\n",
    "> torch.complex(real, imag, *, out=None) → Tensor\n",
    "\n",
    "Constructs a complex tensor with its real part equal to `real` and its imaginary part equal to `imag`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1.+2.j, 1.+3.j])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 1 - working\n",
    "\n",
    "real1 = torch.tensor([1, 1], dtype = torch.float32)\n",
    "imag1 = torch.tensor([2, 3], dtype = torch.float32)\n",
    "\n",
    "z = torch.complex(real1, imag1)\n",
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The elements of `real1` tensor forms a complex tensor with the corresponding elements of `imag1` tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.+3.j, 2.+4.j],\n",
       "        [3.+5.j, 4.+6.j]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 2 - working\n",
    "\n",
    "real2 = torch.tensor([[1, 2],\n",
    "                      [3, 4]], dtype = torch.float32)\n",
    "imag2 = torch.tensor([[3, 4],\n",
    "                      [5, 6]], dtype = torch.float32)\n",
    "\n",
    "z2 = torch.complex(real2, imag2)\n",
    "z2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`torch.complex` also works for multi-dimensional tensors having same shape."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "Expected both inputs to be Float or Double tensors but got Long and Long",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-29a0af03c321>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      6\u001b[0m                     [8, 9]])\n\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 8\u001b[0;31m \u001b[0mz3\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcomplex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreal3\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mimag3\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      9\u001b[0m \u001b[0mz3\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: Expected both inputs to be Float or Double tensors but got Long and Long"
     ]
    }
   ],
   "source": [
    "# Example 3 - breaking (to illustrate when it breaks)\n",
    "\n",
    "real3 = torch.tensor([[1, 2],\n",
    "                     [3, 4]])\n",
    "imag3 = torch.tensor([[6, 7],\n",
    "                    [8, 9]])\n",
    "\n",
    "z3 = torch.complex(real3, imag3)\n",
    "z3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`torch.complex(real, imag)` only accepts float or double datatype values. If we do not specify the `dtype` while creating a tensor then it will accept `long` as the datatype. Hence, using `torch.complex` on that tensor will give the RunTime Error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function can be used when we need to create a complexed tensor. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------------------------------------------------------------------\n",
    "## Function 3 - torch.squeeze\n",
    "\n",
    ">torch.squeeze(input, dim=None, *, out=None) → Tensor\n",
    "\n",
    "Returns a tensor with all the dimensions of input of size 1 removed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1., 1., 2., 2., 2., 2., 3., 3., 3., 3.]])\n",
      "torch.Size([1, 12])\n"
     ]
    }
   ],
   "source": [
    "# Example 1 - working\n",
    "\n",
    "t = torch.tensor([[1, 1, 1, 1],\n",
    "                 [2, 2, 2, 2],\n",
    "                 [3, 3, 3, 3]], dtype = torch.float32)\n",
    "\n",
    "print(t.reshape(1, 12))\n",
    "print(t.reshape(1, 12).shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1., 1., 1., 1., 2., 2., 2., 2., 3., 3., 3., 3.])\n",
      "torch.Size([12])\n",
      "torch.Size([1, 12])\n"
     ]
    }
   ],
   "source": [
    "t2 = t.reshape(1, 12).squeeze()\n",
    "print(t2)\n",
    "print(t2.shape)\n",
    "print(t2.squeeze().unsqueeze(dim = 0).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Squeezing a tensor removes all of the axes that have a length of 1. Also unsqueezing a tensor adds a dimension with a length of 1. These functions allow us to expand or shrink the rank of our tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 1, 2, 1, 3])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 2 - working\n",
    "\n",
    "t3 = torch.zeros(2, 1, 2, 1, 3)\n",
    "t3.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 2, 3])\n",
      "torch.Size([2, 1, 2, 1, 3])\n",
      "torch.Size([2, 2, 1, 3])\n",
      "torch.Size([2, 1, 2, 1, 3])\n",
      "torch.Size([2, 1, 2, 3])\n",
      "torch.Size([2, 1, 2, 1, 3])\n"
     ]
    }
   ],
   "source": [
    "print(torch.squeeze(t3).size())\n",
    "print(torch.squeeze(t3, 0).size())\n",
    "print(torch.squeeze(t3, 1).size())\n",
    "print(torch.squeeze(t3, 2).size())\n",
    "print(torch.squeeze(t3, 3).size())\n",
    "print(torch.squeeze(t3, 4).size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example shows squeezing of the tensor with given dimension. It can be seen that squeezing only works for shrinking 1 dimension, it does not have any effect on others in the given tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3])\n"
     ]
    },
    {
     "ename": "TypeError",
     "evalue": "unsqueeze() missing 1 required positional arguments: \"dim\"",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-12-7660ae04fa35>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mt4\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfloat32\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt4\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt4\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munsqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: unsqueeze() missing 1 required positional arguments: \"dim\""
     ]
    }
   ],
   "source": [
    "# Example 3 - breaking (to illustrate when it breaks)\n",
    "\n",
    "t4 = torch.tensor([1, 5, 3], dtype = torch.float32)\n",
    "print(t4.shape)\n",
    "print(t4.squeeze().unsqueeze().shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unsqueezing always requires dimension as an argument `dim`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the tensor has a batch dimension of size 1, then `squeeze(input)` will also remove the batch dimension, which can lead to unexpected errors.\n",
    "\n",
    "This function can be used when there is a need to shrink or expand the rank of a tensor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------------------------------------------------------------------\n",
    "## Function 4 - torch.as_strided\n",
    "\n",
    ">torch.as_strided(input, size, stride, storage_offset=0) → Tensor\n",
    "\n",
    "Create a view of an existing *torch.Tensor* `input` with specified `size`, `stride` and `storage_offset`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[5., 3.],\n",
       "        [6., 4.]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 1 - working\n",
    "\n",
    "t1 = torch.tensor([[5, 3],\n",
    "                   [6, 4],\n",
    "                   [7, 8]], dtype = torch.float32)\n",
    "\n",
    "t2 = torch.as_strided(t1,(2,2),(2,1))\n",
    "t2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above example, we are creating a view of size (2,2) out of (3,2) tensor by using the stride value (2,1).\n",
    "For the first dimension, every 2nd value from the `start` will be picked and for the 2nd dimension, every 1st value from the already picked 1st dimension will be picked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.5939, -1.6919,  0.9064],\n",
      "        [-1.5749,  0.8807, -0.3661],\n",
      "        [ 1.4907,  0.9837,  1.0228]])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.5939, -1.5749],\n",
       "        [ 0.9064, -0.3661],\n",
       "        [ 0.8807,  0.9837]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 2 - working\n",
    "\n",
    "x = torch.randn(3, 3)\n",
    "print(x)\n",
    "\n",
    "y = torch.as_strided(x,(3,2),(2,3))\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we are creating a view of size (3,2) out of (3,3) tensor by using the stride value (2,3).\n",
    "For the first dimension, every 2nd value from the `start` will be picked and for the 2nd dimension, every 3rd value from the already picked 1st dimension will be picked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.7991, -1.3464, -0.0839],\n",
      "        [ 0.5485, -0.2519, -0.9006],\n",
      "        [-0.7210, -1.7063, -1.4901]])\n"
     ]
    },
    {
     "ename": "RuntimeError",
     "evalue": "setStorage: sizes [3, 2], strides [2, 6], storage offset 0, and itemsize 4 requiring a storage size of 44 are out of bounds for storage of size 36",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-15-4e82b14b585e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mas_strided\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m6\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: setStorage: sizes [3, 2], strides [2, 6], storage offset 0, and itemsize 4 requiring a storage size of 44 are out of bounds for storage of size 36"
     ]
    }
   ],
   "source": [
    "# Example 3 - breaking (to illustrate when it breaks)\n",
    "\n",
    "x = torch.randn(3, 3)\n",
    "print(x)\n",
    "\n",
    "y = torch.as_strided(x,(3,2),(2,6))\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we are striding through the 1st dimension `3` and the 2nd dimension `2` of the output matrix, their product `3x2=6`, cannot be less than the sum of stride indices `2+6=8`. This means we cannot make long skips such that we exceed the last element of the original tensor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function is useful in directly accessing a specific set of values in a tensor based on the storage memory format for direct elementary operations with more efficiency."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------------------------------------------------------------------\n",
    "## Function 5 - torch.randn()\n",
    "\n",
    "> torch.randn(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)\n",
    "\n",
    "PyTorch torch.randn() returns a tensor defined by the variable argument size (sequence of integers defining the shape of the output tensor), containing random numbers from standard normal distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[1., 2., 3.],\n",
       "         [5., 6., 7.],\n",
       "         [8., 6., 4.]]),\n",
       " tensor([[ 1.1783, -0.0732,  0.2186],\n",
       "         [ 0.0207,  0.0675,  0.4593],\n",
       "         [-0.5369, -0.4181, -1.4149]]))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 1 - working\n",
    "\n",
    "t1 = torch.tensor([[1, 2, 3],\n",
    "                 [5, 6, 7],\n",
    "                 [8, 6, 4]], dtype = torch.float32)\n",
    "t2 = torch.randn(t1.shape)\n",
    "t1, t2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This returns a tensor of size of the tensor x, filled with values from standard normal distribution, i.e mean is 0 and variance is 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 0.6480, -1.7293, -0.3584,  0.2468],\n",
       "         [ 0.0075, -0.7489, -0.7530, -0.4006],\n",
       "         [-0.4935,  0.1592, -0.1691,  1.8577]],\n",
       "\n",
       "        [[-2.7374,  0.0997,  0.0680, -0.1719],\n",
       "         [-0.0843,  0.2185,  0.3678, -1.6001],\n",
       "         [-0.7214,  1.1170, -0.3394, -0.8283]]], requires_grad=True)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Example 2 - working\n",
    "\n",
    "t2 = torch.randn(2, 3, 4, requires_grad = True)\n",
    "t2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This returns a tensor of size `2x3x4`, filled with random numbers, also recording the gradient values, when performed. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "randn() received an invalid combination of arguments - got (float, float, float, requires_grad=bool), but expected one of:\n * (tuple of ints size, *, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)\n * (tuple of ints size, *, torch.Generator generator, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)\n * (tuple of ints size, *, torch.Generator generator, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)\n * (tuple of ints size, *, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-18-878868f826cc>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Example 3 - breaking (to illustrate when it breaks)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mt3\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2.\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m3.\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4.\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrequires_grad\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0mt3\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: randn() received an invalid combination of arguments - got (float, float, float, requires_grad=bool), but expected one of:\n * (tuple of ints size, *, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)\n * (tuple of ints size, *, torch.Generator generator, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)\n * (tuple of ints size, *, torch.Generator generator, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)\n * (tuple of ints size, *, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)\n"
     ]
    }
   ],
   "source": [
    "# Example 3 - breaking (to illustrate when it breaks)\n",
    "\n",
    "t3 = torch.randn(2., 3., 4., requires_grad = True)\n",
    "t3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This error is generated because `size` parameter cannot accept floating point value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `randn()` function is useful for selecting weights as a starting point when beginning to train a predictive program."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------------------------------------------------------------------------------------------\n",
    "## Conclusion\n",
    "\n",
    "In this notebook, I tried to explain just 5 different PyTorch tensor operations. Each of these has unique paramaters and methods that allow users to play around with data and deriving insights.\n",
    "Having now completed this assignment, and spent significant time looking at the subtle differences in these functions it is my conclusion that Pytorch is a valuable tool for Deep Learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reference Links\n",
    "\n",
    "* Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install jovian --upgrade --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "if (window.IPython && IPython.notebook.kernel) IPython.notebook.kernel.execute('jovian.utils.jupyter.get_notebook_name_saved = lambda: \"' + IPython.notebook.notebook_name + '\"')"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import jovian"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "window.require && require([\"base/js/namespace\"],function(Jupyter){Jupyter.notebook.save_checkpoint()})"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[jovian] Attempting to save notebook..\u001b[0m\n",
      "[jovian] Updating notebook \"priyanshu-kr/01-tensor-operations\" on https://jovian.ai/\u001b[0m\n",
      "[jovian] Uploading notebook..\u001b[0m\n",
      "[jovian] Capturing environment..\u001b[0m\n",
      "[jovian] Committed successfully! https://jovian.ai/priyanshu-kr/01-tensor-operations\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'https://jovian.ai/priyanshu-kr/01-tensor-operations'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jovian.commit(project='01-tensor-operations')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}