adding tensorboard examples

extra/Getting_Started/tensorboard/tensorboard_with_pytorch.ipynb

@@ -0,0 +1,289 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "%matplotlib inline"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "# How to use TensorBoard with PyTorch\n",
+    "\n",
+    "Copyrights see https://pytorch.org/tutorials/recipes/recipes/tensorboard_with_pytorch.html\n",
+    "\n",
+    "TensorBoard is a visualization toolkit for machine learning experimentation. \n",
+    "TensorBoard allows tracking and visualizing metrics such as loss and accuracy, \n",
+    "visualizing the model graph, viewing histograms, displaying images and much more. \n",
+    "In this tutorial we are going to cover TensorBoard installation, \n",
+    "basic usage with PyTorch, and how to visualize data you logged in TensorBoard UI.\n",
+    "\n",
+    "## Installation\n",
+    "PyTorch should be installed to log models and metrics into TensorBoard log \n",
+    "directory. The following command will install PyTorch 1.4+ via \n",
+    "Anaconda (recommended):\n",
+    "\n",
+    "::\n",
+    "\n",
+    "   $ conda install pytorch torchvision -c pytorch \n",
+    "   \n",
+    "\n",
+    "or pip\n",
+    "\n",
+    "::\n",
+    "\n",
+    "   $ pip install torch torchvision\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Using TensorBoard in PyTorch\n",
+    "\n",
+    "Let’s now try using TensorBoard with PyTorch! Before logging anything, \n",
+    "we need to create a ``SummaryWriter`` instance.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    },
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2023-03-06 10:53:12.601699: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA\n",
+      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
+      "2023-03-06 10:53:12.711871: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
+      "2023-03-06 10:53:13.121665: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n",
+      "2023-03-06 10:53:13.121710: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n",
+      "2023-03-06 10:53:13.121717: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "from torch.utils.tensorboard import SummaryWriter\n",
+    "writer = SummaryWriter()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Writer will output to ``./runs/`` directory by default.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Log scalars\n",
+    "\n",
+    "In machine learning, it’s important to understand key metrics such as \n",
+    "loss and how they change during training. Scalar helps to save \n",
+    "the loss value of each training step, or the accuracy after each epoch. \n",
+    "\n",
+    "To log a scalar value, use \n",
+    "``add_scalar(tag, scalar_value, global_step=None, walltime=None)``. \n",
+    "For example, lets create a simple linear regression training, and \n",
+    "log loss value using ``add_scalar``\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "x = torch.arange(-5, 5, 0.1).view(-1, 1)\n",
+    "y = -5 * x + 0.1 * torch.randn(x.size())\n",
+    "\n",
+    "model = torch.nn.Linear(1, 1)\n",
+    "criterion = torch.nn.MSELoss()\n",
+    "optimizer = torch.optim.SGD(model.parameters(), lr = 0.1)\n",
+    "\n",
+    "def train_model(iter):\n",
+    "    for epoch in range(iter):\n",
+    "        y1 = model(x)\n",
+    "        loss = criterion(y1, y)\n",
+    "        writer.add_scalar(\"Loss/train\", loss, epoch)\n",
+    "        optimizer.zero_grad()\n",
+    "        loss.backward()\n",
+    "        optimizer.step()\n",
+    "        \n",
+    "train_model(10)\n",
+    "writer.flush()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Call ``flush()`` method to make sure that all pending events \n",
+    "have been written to disk.\n",
+    "\n",
+    "See [torch.utils.tensorboard tutorials](https://pytorch.org/docs/stable/tensorboard.html) \n",
+    "to find more TensorBoard visualization types you can log.\n",
+    "\n",
+    "If you do not need the summary writer anymore, call ``close()`` method.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false,
+    "jupyter": {
+     "outputs_hidden": false
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "writer.close()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Run TensorBoard\n",
+    "\n",
+    "Install TensorBoard through the command line to visualize data you logged\n",
+    "\n",
+    "::\n",
+    "\n",
+    "   $ pip install tensorboard\n",
+    "\n",
+    "\n",
+    "Now, start TensorBoard, specifying the root log directory you used above. \n",
+    "Argument ``logdir`` points to directory where TensorBoard will look to find \n",
+    "event files that it can display. TensorBoard will recursively walk \n",
+    "the directory structure rooted at logdir, looking for .*tfevents.* files.\n",
+    "\n",
+    "::\n",
+    "\n",
+    "   $ tensorboard --logdir=runs\n",
+    "\n",
+    "Go to the URL it provides OR to [http://localhost:6006/](http://localhost:6006/)\n",
+    "\n",
+    "<img src=\"file://../../_static/img/thumbnails/tensorboard_scalars.png\" scale=\"40 %\">\n",
+    "\n",
+    "This dashboard shows how the loss and accuracy change with every epoch. \n",
+    "You can use it to also track training speed, learning rate, and other \n",
+    "scalar values. It’s helpful to compare these metrics across different \n",
+    "training runs to improve your model.\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Share TensorBoard dashboards\n",
+    "\n",
+    "[TensorBoard.dev](https://tensorboard.dev/) lets you upload and share \n",
+    "your ML experiment results with anyone. Use TensorBoard.dev to host, \n",
+    "track, and share your TensorBoard dashboards.\n",
+    "\n",
+    "Install the latest version of TensorBoard to use the uploader.\n",
+    "\n",
+    ":: \n",
+    "\n",
+    "   $ pip install tensorboard --upgrade\n",
+    "\n",
+    "Use a simple command to upload and share your TensorBoard.\n",
+    "\n",
+    ":: \n",
+    "\n",
+    "  $ tensorboard dev upload --logdir runs \\\n",
+    "  --name \"My latest experiment\" \\ # optional\n",
+    "  --description \"Simple comparison of several hyperparameters\" # optional\n",
+    "\n",
+    "For help, run ``$ tensorboard dev --help``.\n",
+    "\n",
+    "**Note:** Uploaded TensorBoards are public and visible to everyone. \n",
+    "Do not upload sensitive data.\n",
+    "\n",
+    "View your TensorBoard live at URL provided in your terminal. \n",
+    "E.g. [https://tensorboard.dev/experiment/AdYd1TgeTlaLWXx6I8JUbA](https://tensorboard.dev/experiment/AdYd1TgeTlaLWXx6I8JUbA)\n",
+    "\n",
+    "\n",
+    "<img src=\"file://../../_static/img/thumbnails/tensorboard_dev.png\" scale=\"40 %\">\n",
+    "\n",
+    "\n",
+    "<div class=\"alert alert-info\"><h4>Note</h4><p>TensorBoard.dev currently supports scalars, graphs, histograms, distributions, hparams, and text dashboards.</p></div>\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Learn More\n",
+    "\n",
+    "-  [torch.utils.tensorboard](https://pytorch.org/docs/stable/tensorboard.html) docs\n",
+    "-  [Visualizing models, data, and training with TensorBoard](https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html) tutorial\n",
+    "\n",
+    "\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}