GridSearchCV(cv=2, estimator=KernelRidge(),\n",
- " param_grid=[{'alpha': array([0.0001, 0.001 , 0.01 ]),\n",
- " 'gamma': array([0.0001, 0.001 , 0.01 ]),\n",
- " 'kernel': ['laplacian']}],\n",
- " scoring='neg_mean_absolute_error', verbose=1000)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
GridSearchCV(cv=2, estimator=KernelRidge(),\n",
- " param_grid=[{'alpha': array([0.0001, 0.001 , 0.01 ]),\n",
- " 'gamma': array([0.0001, 0.001 , 0.01 ]),\n",
- " 'kernel': ['laplacian']}],\n",
- " scoring='neg_mean_absolute_error', verbose=1000)KernelRidge()
KernelRidge()
![](\"https://raw.githubusercontent.com/milicasan/TutorialImages/89238bfff57a38ec78ecbd08765b63bd15ff9e2c/AI4spec.png\")
\n",
+ "\n",
+ "In the exercise below, our AI objective is to train a model to predict a single electronic energy level, HOMO for example. This is a classic example of supervised machine learning regression, where the objective (label) is a single floating-point number. Many AI models can address this task, from decision trees, via kernel-based learning to neural networks. Here, we demonstrate the use of Kernel Ridge Regression, a relatively simple regression approach that has performed well on numerous prediction tasks in physics and chemistry **{Cite Mathias Rupp}**\n",
+ "\n",
+ "We use a large dataset of molecular structures and their computed HOMO energies to train the AI. Once the model is trained, it can be used as a tool: one can input a new molecule (previously unseen by the model) and instantly receive a value for its HOMO, within the expected error of the trained model.\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "XegnO6aTwrmX"
+ },
+ "source": [
+ "## 1. HOMO energy prediction with kernel ridge regression\n",
+ "\n",
+ "\n",
+ "Here we will machine-learn the relationship between molecular structure (represented by the Coulomb matrix CM) and their HOMO (Highest Occupied Molecular Orbital) energy using KRR.\n",
+ "\n",
+ "This tutorial shows step by step how to load the data, visualize them, select the hyperparameters, train the model and validate it. We use the QM7 dataset of 7000 small organic molecules. The HOMO energies of all molecules were pre-computed with first principles quantum mechanical methods (DFT) to obtain the target data that our model can be trained on. Detailed descriptions and results for a similar dataset (QM9) can be found in [A. Stuke, et al. \"Chemical diversity in molecular orbital energy predictions with kernel ridge regression.\" J. Chem. Phys. 150. 204121 (2019)](https://aip.scitation.org/doi/10.1063/1.5086105).\n",
+ "\n",
+ "\n",
+ "## Setup"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "qdphsPPRwwNF"
+ },
+ "outputs": [],
+ "source": [
+ "from sklearn.model_selection import GridSearchCV\n",
+ "from sklearn.model_selection import cross_validate\n",
+ "from sklearn.kernel_ridge import KernelRidge\n",
+ "from sklearn.metrics import r2_score\n",
+ "\n",
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "import pandas as pd\n",
+ "import seaborn as sns\n",
+ "import json\n",
+ "import math, random\n",
+ "from scipy.sparse import load_npz\n",
+ "from matplotlib.colors import LinearSegmentedColormap\n",
+ "import time"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "hi4ISia7wrmc"
+ },
+ "source": [
+ "### Load and visualize data\n",
+ "\n",
+ "At first, we load the data.\n",
+ "\n",
+ "The input data x is an array that contains all 7000 molecules of the QM7 dataset, represented by their Coulomb matrices, which were computed with the [Dscribe](https://www.sciencedirect.com/science/article/pii/S0010465519303042?via%3Dihub) package.\n",
+ "\n",
+ "The output data y is a list that contains the corresponding (pre-computed) HOMO energies."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
"colab": {
- "provenance": [],
- "collapsed_sections": [
- "0FxS6Kazwrmo"
- ],
- "toc_visible": true
- },
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.6.3"
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "jTZ_1hpqLBK5",
+ "outputId": "a15f3c92-4b6c-453a-e0d6-1a9e91e72112"
+ },
+ "outputs": [],
+ "source": [
+ "!wget https://github.com/fullmetalfelix/ML-CSC-tutorial/raw/master/data/qm7/cm.npz\n",
+ "!wget https://raw.githubusercontent.com/fullmetalfelix/ML-CSC-tutorial/master/data/qm7/HOMO.txt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "6tTcc8Mtwrmd",
+ "outputId": "baf6dc4d-80fb-4b36-f3ba-026677ad11ee"
+ },
+ "outputs": [],
+ "source": [
+ "x = load_npz(\"cm.npz\").toarray()\n",
+ "y = np.genfromtxt(\"HOMO.txt\")\n",
+ "\n",
+ "print(\"Number of molecules:\", len(y))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "yWa4pRHxwrmf"
+ },
+ "source": [
+ "Print the Coulomb matrix of a random molecule in the dataset."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "7CovKOElwrmg",
+ "outputId": "68c822c8-6e05-431d-8223-791fc3851287"
+ },
+ "outputs": [],
+ "source": [
+ "rand_mol = random.randint(0, len(y))\n",
+ "\n",
+ "print(x[rand_mol])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "BKo-SoN2wrmh"
+ },
+ "source": [
+ "Visualize the Coulomb matrix of the random molecule."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 502
+ },
+ "id": "GJu2RtUhwrmh",
+ "outputId": "1466b332-185d-4ad7-e2f6-c3696ceed6d5"
+ },
+ "outputs": [],
+ "source": [
+ "shape = (23, 23)\n",
+ "mat = x[rand_mol].reshape(shape)\n",
+ "\n",
+ "plt.figure()\n",
+ "plt.figure(figsize = (6,6))\n",
+ "plt.imshow(mat, origin=\"upper\", cmap='rainbow', vmin=-15, vmax=75, interpolation='nearest')\n",
+ "plt.colorbar(fraction=0.046, pad=0.04).ax.tick_params(labelsize=20)\n",
+ "plt.axis('off')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "n434DnNSwrmi"
+ },
+ "source": [
+ "Note that many of the outputs are zero in the Coulomb matrix. This is because the number of non-zero outputs depends on the size of the molecule, and all smaller molecules contain zero padding.\n",
+ "\n",
+ "Visualize the target data by plotting the distribution of HOMO energies in the dataset:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 510
+ },
+ "id": "YxjygJgJwrmj",
+ "outputId": "c0683fc7-7a3d-490a-e012-bc6e7210c3a5"
+ },
+ "outputs": [],
+ "source": [
+ "plt.hist(y, bins=20, density=False, facecolor='blue')\n",
+ "plt.xlabel(\"Energy [eV]\")\n",
+ "plt.ylabel(\"Number of molecules\")\n",
+ "plt.title(\"Distribution of HOMO energies\")\n",
+ "plt.show()\n",
+ "\n",
+ "## mean value of distribution\n",
+ "print(\"Mean value of HOMO energies in QM7 dataset: %0.2f eV\" %np.mean(y))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "QpgK9ky-wrmk"
+ },
+ "source": [
+ "Before dividing the dataset into training and test set, we shuffle the data. This is because data are often stored in a logical order (e.g., certain types of molecules grouped one after each other). Simply taking the first part for training and the second for testing would not result in a well trained model, since the training set would not represent the test data well (and vice versa)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "6luJyqCKwrmk"
+ },
+ "outputs": [],
+ "source": [
+ "## shuffle the data\n",
+ "\n",
+ "c = list(zip(x, y))\n",
+ "random.shuffle(c)\n",
+ "\n",
+ "x, y = zip(*c)\n",
+ "\n",
+ "x = np.array(x)\n",
+ "y = np.array(y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "XQh-MAv5wrml"
+ },
+ "source": [
+ "Now, we divide the data into training and test set."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "wmnl8ZzAwrmm"
+ },
+ "outputs": [],
+ "source": [
+ "# decide how many samples to take from the database for training and testing\n",
+ "n_train = 1000\n",
+ "n_test = 1000\n",
+ "\n",
+ "# split data in training and test\n",
+ "# take first n_train molecules for training\n",
+ "x_train = x[0:n_train]\n",
+ "y_train = y[0:n_train]\n",
+ "\n",
+ "# take the next n_test data for testing\n",
+ "x_test = x[n_train:n_train + n_test]\n",
+ "y_test = y[n_train:n_train + n_test]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "uEP7iSchwrmm"
+ },
+ "source": [
+ "Check that the training data resemble the test data well by plotting the distribution of HOMO energies for both sets. The distributions should be centered around the same mean value and have the same shape."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 504
+ },
+ "id": "w_H7kOBTwrmn",
+ "outputId": "14677f4e-42d1-4de3-8360-dbadedd01a3d"
+ },
+ "outputs": [],
+ "source": [
+ "plt.hist(y_test, bins=20, density=False, alpha=0.5, facecolor='red', label='test set')\n",
+ "plt.hist(y_train, bins=20, density=False, alpha=0.5, facecolor='gray', label='training set')\n",
+ "plt.xlabel(\"Energy [eV]\")\n",
+ "plt.ylabel(\"Number of molecules\")\n",
+ "plt.legend()\n",
+ "plt.show()\n",
+ "\n",
+ "## mean value of distributions\n",
+ "print(\"Mean value of HOMO energies in training set: %0.2f eV\" %np.mean(y_train))\n",
+ "print(\"Mean value of HOMO energies in test set: %0.2f eV\" %np.mean(y_test))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "0FxS6Kazwrmo"
+ },
+ "source": [
+ "### Training\n",
+ "\n",
+ "In the training phase we use a kernel function to measure the distance between all pairs of molecules (represented by their Coulomb matrices) in the training set. We here employ one of two kernels, the Gaussian kernel or the Laplacian kernel. Both kernels have two hyperparameters: $\\alpha$ controls the penalty term and $\\gamma$ the kernel width.\n",
+ "\n",
+ "To find the optimal hyperparameters, we will do a grid search, i.e. we test the performance of a model trained with values of the hyperparameters that are spaced out on a grid in search space and choose the hyperparameters that yield the best performance.\n",
+ "\n",
+ "The model performance is quantified by splitting off part of the training set as validation set. We validate the model by making predictions on this validation set. This check of the model performance can be further refined with cross-validation, where the roles of training and validation sets alternate. The ratio can be varied, for example in 5-fold cross-validation, the training set is split in 5 equal parts. The model is trained on 80% of the data and validated on the other 20%. Then the roles of training and validation set rotate until each part has served as validation set exactly once.\n",
+ "\n",
+ "For our implementation, we will use the scikit-learn module.\n",
+ "\n",
+ "A more detailed optional explanation, which is not required for the solution of the exercise, can be found in the hidden cell below."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "IqRlAI8983CM"
+ },
+ "source": [
+ "The Gaussian kernel is given by\n",
+ "\n",
+ "\\begin{equation}\n",
+ "k_{Gaussian}(\\boldsymbol{x},\\boldsymbol{x}')=e^{-\\frac{||{\\boldsymbol{x}-\\boldsymbol{x}'}||_2^2}{2\\gamma^2}},\n",
+ "\\end{equation}\n",
+ "\n",
+ "which employs the Euclidean distance as similarity measure. The parameter $\\gamma$ is defined as $\\frac{1}{2\\sigma^2}$, where $\\sigma$ is the standard deviation of the Gaussian kernel (kernel width). The Laplacian kernel is given by\n",
+ "\n",
+ "\\begin{equation}\n",
+ " k_{Laplacian}(\\boldsymbol{x},\\boldsymbol{x}')=e^{-\\frac{||{\\boldsymbol{x}-\\boldsymbol{x}'}||_1}{\\gamma}},\n",
+ "\\end{equation}\n",
+ "\n",
+ "which uses the 1-norm as similarity measure. Here, $\\gamma$ is defined as $\\frac{1}{\\sigma}$, where $\\sigma$ is the kernel width of the Laplacian kernel.\n",
+ "\n",
+ "In the KRR training phase with $N$ training molecules, the machine learns the relationship between the molecules (represented by their Coulomb matrix) and their corresponding (pre-computed) HOMO energies. It does so by employing a function $f(\\boldsymbol{x})$ that maps a training molecule $\\boldsymbol{x}$ to its reference HOMO energy:\n",
+ "\n",
+ "\\begin{equation}\n",
+ "f(\\boldsymbol{x}) = \\sum_{i=1}^N \\omega_i k(\\boldsymbol{x}, \\boldsymbol{x}_i) = HOMO^{ref},\n",
+ "\\end{equation}\n",
+ "\n",
+ "For a given training molecule $\\boldsymbol{x}$, the distance to each molecule in the training set is computed by employing the kernel function $k$ (either Gaussian or Laplacian). Each kernel contribution (distance) is then weighted by a regression weight $\\omega_i$. The above function is thus given by the weighted sum of kernel contributions (sum over $N$ training molecules). The purpose of training is to fit the regression weight $\\omega_i$ so that HOMO$_{ref}$ is matched for each training molecule. In practice, the machine solves the minimization problem\n",
+ "\n",
+ "\n",
+ "\\begin{equation}\n",
+ " \\underset{\\omega}{min} \\sum_{i=1}^N (f(\\boldsymbol{x}_i) - HOMO^{ref}_i)^2 + \\alpha \\boldsymbol{\\omega}^T \\mathbf{K} \\boldsymbol{\\omega}.\n",
+ "\\end{equation}\n",
+ "\n",
+ "for a vector $\\boldsymbol{\\omega} \\in \\mathbb{R}^N = (\\omega_1, \\omega_2, ..., \\omega_N)$ of regression weights. In KRR, the penalty term $ \\alpha \\boldsymbol{\\omega}^T \\mathbf{K} \\boldsymbol{\\omega}$ is added to the minimization problem in order to avoid over- and underfitting. Overfitting occurs when the model learns the training data too well, even the noise and other unimportant details. The model is unable to generalize on unseen data and therefore yields high prediction errors on the test data. Underfitting occurs when the model is too simple and does not learn the training data at all, and therefore is not able to predict test data well either. Both behaviours can be avoided by tuning the parameter $\\alpha \\in \\left[0,1\\right]$ to a reasonable value. This has do be done separately from training. Both the regularization parameter $\\alpha$ and the kernel width $\\gamma$ are so called hyperparameters. Hyperparameters cannot be learned during training and have to be selected beforehand. However, it is not always obvious how to choose these hyperparameters and it often requires intuition or rules of thumb. We here employ a cross-validated grid search in order to find the best values for these two hyperparameters.\n",
+ "\n",
+ "In grid search, a part of the training set is split off as validation set. We set up a grid of pre-defined hyperparameter values and train the machine on the remaining training set, for each possible combination of $\\alpha$ and $\\gamma$ values. We validate each possible combination by making predictions on the validation set. The two hyperparameter values that yield the best performance (lowest error) are then selected for the final model to make predictions on the test set.\n",
+ "\n",
+ "In cross-validation, the roles of training and validation sets alternate. As described above, a part from the training set is split off as validation set. After training one combination of hyperparameters on the remaining training set and validating on the validation set, the validation set becomes the training set and vice versa, and the model is trained on the new training set and validated on the new validation set for the same combination of hyperparameters. The ratio can be varied, for example in 5-fold cross-validation, the training set is split in 5 equal parts. For each combination of hyperparameters, the model is trained on 80% of the data and validated on the other 20%. Then the roles of training and validation set rotate until each part has served as validation set exactly once. The final validation error for one particular combination of hyperparameters is computed as the mean from all 5 errors on the 5 validation sets. The combination with lowest average error is chosen for the final model.\n",
+ "\n",
+ "The cross-validated grid search routine is implemented in scikit-learn."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "O5e2EytL8aDY"
+ },
+ "source": [
+ "### KRR Code"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 778
+ },
+ "id": "hbcOapWDwrmp",
+ "outputId": "1d28b29e-0eb8-4473-fa33-4ae2bd9ee3af"
+ },
+ "outputs": [],
+ "source": [
+ "# set up grids for alpha and gamma hyperparameters.\n",
+ "# first value: lower bound; second value: upper bound;\n",
+ "# third value: number of points to evaluate (here set to '3' --> '-2', '-1' and '0' are evaluated)\n",
+ "# --> make sure to change third value as well when changing the bounds!\n",
+ "alpha = np.logspace(-4, -2, 3)\n",
+ "gamma = np.logspace(-4, -2, 3)\n",
+ "\n",
+ "cv_number = 2 ## choose into how many parts training set is divided for cross-validation\n",
+ "kernel = 'laplacian' # select kernel function here ('rbf': Gaussian kernel, 'laplacian': Laplacian kernel)\n",
+ "scoring_function = 'neg_mean_absolute_error' # it is called \"negative\" because scikit-learn interprets\n",
+ " # highest scoring value as best, but we want small errors\n",
+ "\n",
+ "## define settings for grid search routine in scikit-learn with above defined grids as input\n",
+ "\n",
+ "grid_search = GridSearchCV(KernelRidge(), #machine learning method (KRR here)\n",
+ " [{'kernel':[kernel],'alpha': alpha, 'gamma': gamma}],\n",
+ " cv = cv_number,\n",
+ " scoring = scoring_function,\n",
+ " verbose=1000) ## produces detailed output statements of grid search\n",
+ " # routine so we can see what is computed\n",
+ "\n",
+ "# call the fit function in scikit-learn which fits the Coulomb matrices in the training set\n",
+ "# to their corresponding HOMO energies.\n",
+ "grid_search.fit(x_train, y_train)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mTrweQ1Mwrmr"
+ },
+ "source": [
+ "### Grid search results\n",
+ "\n",
+ "Print out the average validation errors and corresponding hyperparameter combinations"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "066IjiWawrmr",
+ "outputId": "de299c89-9c65-437c-c798-72bb63bd89e0"
+ },
+ "outputs": [],
+ "source": [
+ "means = grid_search.cv_results_['mean_test_score']\n",
+ "stds = grid_search.cv_results_['std_test_score']\n",
+ "for mean, std, params in zip(-means, stds, grid_search.cv_results_['params']):\n",
+ " print(\"%0.3f (+/-%0.03f) for %r\" % (mean, std * 2, params))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SrfhK2fTwrms"
+ },
+ "source": [
+ "Next, we visualize the grid search results by plotting a heatmap."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 488
+ },
+ "id": "T2swGFTLwrmt",
+ "outputId": "e8b282a6-56fb-4a2e-93c0-bacf613f594e"
+ },
+ "outputs": [],
+ "source": [
+ "results = pd.DataFrame(grid_search.cv_results_)\n",
+ "#pd.DataFrame(grid_search.cv_results_)\n",
+ "\n",
+ "pvt = pd.pivot_table(results, values='mean_test_score',\n",
+ " index='param_gamma', columns='param_alpha')\n",
+ "heatmap = sns.heatmap(-pvt, annot=True, cmap='viridis', cbar_kws={'label': \"Mean absolute error [eV]\"})\n",
+ "figure = heatmap.get_figure()\n",
+ "plt.show()\n",
+ "\n",
+ "\n",
+ "print(\"The best combinations of parameters are %s with a score of %0.3f eV on the validation set.\"\n",
+ " % (grid_search.best_params_, -grid_search.best_score_))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-tv1CciKwrmu"
+ },
+ "source": [
+ "### Testing\n",
+ "\n",
+ "With the best combination of hyperparameters, the model is once again trained on the entire training set (this is done automatically in scikit-learn). Then, with the best combination of hyperparameters, predictions are made on the test set to evaluate the final model, which we will use for predictions.\n",
+ "\n",
+ "The mean absolute error of the predicted from the reference HOMO energies and $R^2$ score will be our measure for the quality of the fit.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 504
+ },
+ "id": "k8e9G1iNwrmv",
+ "outputId": "42be6aaa-f279-436b-97d1-b4a137005585"
+ },
+ "outputs": [],
+ "source": [
+ "# predicted HOMO energies for all test molecules\n",
+ "\n",
+ "y_pred = grid_search.predict(x_test) # scikit-learn automatically takes the best combination\n",
+ " # of hyperparameters from grid search\n",
+ "\n",
+ "print(\"Mean absolute error on test set: %0.3f eV\" %(np.abs(y_pred-y_test)).mean())\n",
+ "\n",
+ "# do the regression plot\n",
+ "plt.plot(y_test, y_pred, 'o')\n",
+ "plt.plot([np.min(y_test),np.max(y_test)], [np.min(y_test),np.max(y_test)], '-')\n",
+ "plt.xlabel('reference HOMO energy [eV]')\n",
+ "plt.ylabel('predicted HOMO energy [eV]')\n",
+ "plt.show()\n",
+ "print(\"R^2 score on test set: %.3f\" % r2_score(y_test, y_pred))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "pYf2oHVbwrmw"
+ },
+ "source": [
+ "The $R^2$ score indicates how close the predicted energies in the test set are to the reference energies. The closer the points in the above figure are located to the diagonal, the better the predictions. Points on the diagonal (\"predicted energy\"=\"reference energy\") correspond to $R^2=1$. Therefore, $R^2$ values close to 1 indicate good model performance."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true,
+ "id": "zcdZdkThwrmw",
+ "jupyter": {
+ "outputs_hidden": true
}
+ },
+ "source": [
+ "###**Exercises**\n",
+ "\n",
+ "#### a. Grid search\n",
+ "\n",
+ "Increase the number and range of grid points used for grid search. Which combination of $\\alpha$ and $\\gamma$ works best? How does the computational time increase? Choose a reasonable number of grid points that don't take too long to evaluate."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "mqN4J1xVwrmx"
+ },
+ "outputs": [],
+ "source": [
+ "#Choose alpha and gamma on a finer logarithmically spaced grid. The other parameters can be chosen as above.\n",
+ "alpha = None\n",
+ "gamma = None\n",
+ "cv_number=None\n",
+ "kernel = None\n",
+ "scoring_function = None\n",
+ "\n",
+ "\n",
+ "grid_search = GridSearchCV(KernelRidge(), #machine learning method (KRR here)\n",
+ " [{'kernel':[kernel],'alpha': alpha, 'gamma': gamma}],\n",
+ " cv = cv_number,\n",
+ " scoring = scoring_function,\n",
+ " verbose=1000) ## produces detailed output statements of grid search routine\n",
+ "\n",
+ "grid_search.fit(x_train, y_train)\n",
+ "\n",
+ "results = pd.DataFrame(grid_search.cv_results_)\n",
+ "\n",
+ "pvt = pd.pivot_table(results, values='mean_test_score',\n",
+ " index='param_gamma', columns='param_alpha')\n",
+ "heatmap = sns.heatmap(-pvt, annot=True, cmap='viridis', cbar_kws={'label': \"Mean absolute error [eV]\"})\n",
+ "figure = heatmap.get_figure()\n",
+ "plt.show()\n",
+ "\n",
+ "print(\"The best combinations of parameters are %s with a score of %0.3f eV on the validation set.\"\n",
+ " % (grid_search.best_params_, -grid_search.best_score_))\n",
+ "\n",
+ "y_pred = grid_search.predict(x_test) # scikit-learn automatically takes the best combination of hyperparameters from grid search\n",
+ "\n",
+ "print(\"Mean absolute error on test set: %0.3f eV\" %(np.abs(y_pred-y_test)).mean())\n",
+ "\n",
+ "plt.plot(y_test, y_pred, 'o')\n",
+ "plt.plot([np.min(y_test),np.max(y_test)], [np.min(y_test),np.max(y_test)], '-')\n",
+ "plt.xlabel('reference HOMO energy [eV]')\n",
+ "plt.ylabel('predicted HOMO energy [eV]')\n",
+ "plt.show()\n",
+ "print(\"R^2 score on test set: %.3f\" % r2_score(y_test, y_pred))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Gsy36Rzowrmy"
+ },
+ "source": [
+ "#### b. Kernel function\n",
+ "\n",
+ "Use the Gaussian kernel instead of the Laplacian kernel. Which kernel leads to better model performance?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "srWUmjKapUMD"
+ },
+ "outputs": [],
+ "source": [
+ "#Repeat the above calculation with the Gaussian kernel. The Gaussian kernel is identified with the string 'rbf'.\n",
+ "alpha = None\n",
+ "gamma = None\n",
+ "cv_number=None\n",
+ "kernel = None\n",
+ "scoring_function = None\n",
+ "\n",
+ "grid_search = GridSearchCV(KernelRidge(), #machine learning method (KRR here)\n",
+ " [{'kernel':[kernel],'alpha': alpha, 'gamma': gamma}],\n",
+ " cv = cv_number,\n",
+ " scoring = scoring_function,\n",
+ " verbose=1000) ## produces detailed output statements of grid search routine\n",
+ "\n",
+ "grid_search.fit(x_train, y_train)\n",
+ "\n",
+ "results = pd.DataFrame(grid_search.cv_results_)\n",
+ "\n",
+ "pvt = pd.pivot_table(results, values='mean_test_score',\n",
+ " index='param_gamma', columns='param_alpha')\n",
+ "heatmap = sns.heatmap(-pvt, annot=True, cmap='viridis', cbar_kws={'label': \"Mean absolute error [eV]\"})\n",
+ "figure = heatmap.get_figure()\n",
+ "plt.show()\n",
+ "\n",
+ "print(\"The best combinations of parameters are %s with a score of %0.3f eV on the validation set.\"\n",
+ " % (grid_search.best_params_, -grid_search.best_score_))\n",
+ "\n",
+ "y_pred = grid_search.predict(x_test) # scikit-learn automatically takes the best combination of hyperparameters from grid search\n",
+ "\n",
+ "print(\"Mean absolute error on test set: %0.3f eV\" %(np.abs(y_pred-y_test)).mean())\n",
+ "\n",
+ "plt.plot(y_test, y_pred, 'o')\n",
+ "plt.plot([np.min(y_test),np.max(y_test)], [np.min(y_test),np.max(y_test)], '-')\n",
+ "plt.xlabel('reference HOMO energy [eV]')\n",
+ "plt.ylabel('predicted HOMO energy [eV]')\n",
+ "plt.show()\n",
+ "print(\"R^2 score on test set: %.3f\" % r2_score(y_test, y_pred))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5yJUdzhewrmx"
+ },
+ "source": [
+ "#### c. Cross-validation\n",
+ "\n",
+ "For this exercise, choose the kernel that performed better in the previous exercise. Increase the number of folds used for cross-validation. Does the quality of the model increase? Take note as well of the increasing computational time and choose a number of folds that does not require too much computational time."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Jqy3rfT9wrmy"
+ },
+ "outputs": [],
+ "source": [
+ "#Repeat the above calculation with a higher number of cross-validation folds. Use the kernel that showed better performance.\n",
+ "alpha = None\n",
+ "gamma = None\n",
+ "cv_number=None\n",
+ "kernel = None\n",
+ "scoring_function = None\n",
+ "\n",
+ "grid_search = GridSearchCV(KernelRidge(), #machine learning method (KRR here)\n",
+ " [{'kernel':[kernel],'alpha': alpha, 'gamma': gamma}],\n",
+ " cv = cv_number,\n",
+ " scoring = scoring_function,\n",
+ " verbose=1000) ## produces detailed output statements of grid search routine\n",
+ "\n",
+ "grid_search.fit(x_train, y_train)\n",
+ "\n",
+ "results = pd.DataFrame(grid_search.cv_results_)\n",
+ "#pd.DataFrame(grid_search.cv_results_)\n",
+ "\n",
+ "pvt = pd.pivot_table(results, values='mean_test_score',\n",
+ " index='param_gamma', columns='param_alpha')\n",
+ "heatmap = sns.heatmap(-pvt, annot=True, cmap='viridis', cbar_kws={'label': \"Mean absolute error [eV]\"})\n",
+ "figure = heatmap.get_figure()\n",
+ "plt.show()\n",
+ "\n",
+ "print(\"The best combinations of parameters are %s with a score of %0.3f eV on the validation set.\"\n",
+ " % (grid_search.best_params_, -grid_search.best_score_))\n",
+ "\n",
+ "y_pred = grid_search.predict(x_test) # scikit-learn automatically takes the best combination of hyperparameters from grid search\n",
+ "\n",
+ "print(\"Mean absolute error on test set: %0.3f eV\" %(np.abs(y_pred-y_test)).mean())\n",
+ "\n",
+ "plt.plot(y_test, y_pred, 'o')\n",
+ "plt.plot([np.min(y_test),np.max(y_test)], [np.min(y_test),np.max(y_test)], '-')\n",
+ "plt.xlabel('reference HOMO energy [eV]')\n",
+ "plt.ylabel('predicted HOMO energy [eV]')\n",
+ "plt.show()\n",
+ "print(\"R^2 score on test set: %.3f\" % r2_score(y_test, y_pred))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jjoUSLWuwrmz"
+ },
+ "source": [
+ "#### d. Training set size\n",
+ "\n",
+ "Increase the size of the training set and plot the mean absolute error and $R^2$ score and training time on the test set as a function of the training set size (e.g. use 1000, 2000, 3000 etc. as training set size). For the MAE and $R^2$ values use a logarithmic time axis. Compare with the previous exercises.\n",
+ "\n",
+ "The optimal hyperparameter values for $\\alpha$ and $\\gamma$ can change throughout varying training set sizes. Therefore, when increasing the training set size, it is recommended to perform a cross-validated grid search for each training set size. For the sake of this exercise, we will limit ourselves to taking the optimal hyperparameters from the previous exercise as an approximation and not perform a grid search. Furthermore, we will not cross-validate our model in this exercise in the interest of time."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "0o6HAqu4bYPb"
+ },
+ "outputs": [],
+ "source": [
+ "#We keep the size of the test set constant in this exercise\n",
+ "n_test = 1000\n",
+ "\n",
+ "#We want to iterate over different training set sizes n_train.\n",
+ "#Prepare an iterator with reasonable choices of n_train.\n",
+ "n_train_iterator=range(None,None,None)\n",
+ "\n",
+ "#These lists we want to fill during the iteration with the mean absolute error,\n",
+ "#R^2 score and elapsed training time.\n",
+ "mae_list=[]\n",
+ "r2_list=[]\n",
+ "time_list=[]\n",
+ "\n",
+ "#In alpha and gamma, we want to save the best choice of parameters from\n",
+ "#the previous exercise. Compare with the exercise above, if you are unsure\n",
+ "#how to access them from grid_search.\n",
+ "alpha=None\n",
+ "gamma=None\n",
+ "\n",
+ "#Use the kernel that has shown better performance.\n",
+ "kernel=None\n",
+ "\n",
+ "for n_train in n_train_iterator:\n",
+ " x_train = x[0:n_train]\n",
+ " y_train = y[0:n_train]\n",
+ "\n",
+ " x_test = x[n_train:n_train + n_test]\n",
+ " y_test = y[n_train:n_train + n_test]\n",
+ "\n",
+ " #Here we save the starting time of the kernel ridge training.\n",
+ " start = time.time()\n",
+ "\n",
+ " #In the following we will perform the kernel ridge training without\n",
+ " #cross-validation. For this we use the Object KernelRidge, which\n",
+ " #has previously been used as the estimator in the cross-validation.\n",
+ " #We need to set alpha, gamma and the kernel. For documentation, see:\n",
+ " #https://scikit-learn.org/stable/modules/generated/sklearn.kernel_ridge.KernelRidge.html\n",
+ " kernel_ridge=None\n",
+ "\n",
+ " #After settung up the kernel_ridge object, we need to train with our\n",
+ " #training data using the method fit and make predictions on the test set\n",
+ " #using the method predict in analogy to the cross-validation example.\n",
+ "\n",
+ " #Fill in your answer here!\n",
+ "\n",
+ " #Here we save the ending time of the kernel ridge training.\n",
+ " end = time.time()\n",
+ "\n",
+ " #Compute from the starting and ending time, the elapsed training time and\n",
+ " #append it to the list\n",
+ " time_list.append(None)\n",
+ " #Append the mean absolute error to the following list. Confer with the\n",
+ " #previous exercise, if you are unsure how to compute it.\n",
+ " mae_list.append(None)\n",
+ " r2_list.append(r2_score(y_test, y_pred))\n",
+ "\n",
+ "#In the following plot the size of the training set versus the elapsed time,\n",
+ "#mean absolute error and R^2 score in three separate plots.\n",
+ "\n",
+ "#Fill in your answer here!\n",
+ "\n",
+ "#Here we plot the exact vs. the predicted HOMO energies for our largest training\n",
+ "#set size. Can you see the reduced error from this plot in comparison with\n",
+ "#the previous exercises?\n",
+ "plt.plot(y_test, y_pred, 'o')\n",
+ "plt.plot([np.min(y_test),np.max(y_test)], [np.min(y_test),np.max(y_test)], '-')\n",
+ "plt.xlabel('reference HOMO energy [eV]')\n",
+ "plt.ylabel('predicted HOMO energy [eV]')\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "HNAUxo7IYwkX"
+ },
+ "source": [
+ "#####If you did everything correctly, the plots should look something like the following. Note that because of the random data shuffling in the beginning, they will not look exactly the same.\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ ""
+ ]
+ }
+ ],
+ "metadata": {
+ "anaconda-cloud": {},
+ "colab": {
+ "collapsed_sections": [
+ "0FxS6Kazwrmo"
+ ],
+ "provenance": [],
+ "toc_visible": true
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.9.18"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/AI4Spec_Tutorial2.ipynb b/AI4Spec_Tutorial2.ipynb
index ad52a2b..1450ba5 100644
--- a/AI4Spec_Tutorial2.ipynb
+++ b/AI4Spec_Tutorial2.ipynb
@@ -36,7 +36,7 @@
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": null,
"metadata": {
"id": "qdphsPPRwwNF"
},
@@ -66,7 +66,7 @@
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -74,36 +74,7 @@
"id": "jTZ_1hpqLBK5",
"outputId": "efa921b4-fb04-4fe9-dc78-8e039f748f37"
},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "--2024-03-04 21:34:38-- https://zenodo.org/records/10069732/files/coulomb_7000.npz\n",
- "Resolving zenodo.org (zenodo.org)... 188.184.103.159, 188.184.98.238, 188.185.79.172, ...\n",
- "Connecting to zenodo.org (zenodo.org)|188.184.103.159|:443... connected.\n",
- "HTTP request sent, awaiting response... 200 OK\n",
- "Length: 47096264 (45M) [application/octet-stream]\n",
- "Saving to: ‘coulomb_7000.npz.1’\n",
- "\n",
- "coulomb_7000.npz.1 100%[===================>] 44.91M 46.3MB/s in 1.0s \n",
- "\n",
- "2024-03-04 21:34:39 (46.3 MB/s) - ‘coulomb_7000.npz.1’ saved [47096264/47096264]\n",
- "\n",
- "--2024-03-04 21:34:40-- https://zenodo.org/records/10069732/files/energies_7000.npz\n",
- "Resolving zenodo.org (zenodo.org)... 188.184.98.238, 188.185.79.172, 188.184.103.159, ...\n",
- "Connecting to zenodo.org (zenodo.org)|188.184.98.238|:443... connected.\n",
- "HTTP request sent, awaiting response... 200 OK\n",
- "Length: 896264 (875K) [application/octet-stream]\n",
- "Saving to: ‘energies_7000.npz.1’\n",
- "\n",
- "energies_7000.npz.1 100%[===================>] 875.26K 4.13MB/s in 0.2s \n",
- "\n",
- "2024-03-04 21:34:40 (4.13 MB/s) - ‘energies_7000.npz.1’ saved [896264/896264]\n",
- "\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"!wget https://zenodo.org/records/10069732/files/coulomb_7000.npz\n",
"!wget https://zenodo.org/records/10069732/files/energies_7000.npz"
@@ -111,7 +82,7 @@
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -119,15 +90,7 @@
"id": "6tTcc8Mtwrmd",
"outputId": "e96ae7fe-8e7b-49f5-96d8-f54f9980ddb5"
},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Number of molecules: 7000\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"# Here we load all the data\n",
"x = np.abs(np.load(\"coulomb_7000.npz\")['arr_0'])\n",
@@ -154,7 +117,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@@ -162,102 +125,7 @@
"id": "7CovKOElwrmg",
"outputId": "63b5843a-2a4e-4ae5-92d3-69efb8bf4ce9"
},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "(29, 29)\n",
- "[[36.858 23.545 19.643 14.298 12.615 8.577 10.93 14.165 5.46 5.451\n",
- " 5.466 2.245 2.124 2.203 1.135 2.263 2.071 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [23.545 36.858 33.353 23.482 19.429 13.106 16.645 23.444 2.74 2.752\n",
- " 2.774 3.039 2.741 2.759 1.571 2.759 2.788 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [19.643 33.353 73.517 19.681 21.95 15.243 18.479 20.284 2.35 2.933\n",
- " 2.975 8.241 2.959 2.359 1.972 2.415 3.243 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [14.298 23.482 19.681 36.858 33.39 15.238 15.014 14.537 2.16 2.15\n",
- " 1.729 2.339 5.456 5.44 1.819 2.07 1.759 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [12.615 19.429 21.95 33.39 73.517 35.114 23.509 16.879 1.95 1.931\n",
- " 1.732 2.55 3.934 3.857 3.966 2.322 2.135 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 8.577 13.106 15.243 15.238 35.114 36.858 33.114 15.338 1.343 1.247\n",
- " 1.253 1.617 1.854 2.131 5.488 2.008 1.916 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [10.93 16.645 18.479 15.014 23.509 33.114 53.359 28.812 1.713 1.496\n",
- " 1.667 1.825 1.835 2.231 3.402 3.348 3.366 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [14.165 23.444 20.284 14.537 16.879 15.338 28.812 36.858 2.122 1.714\n",
- " 2.161 1.863 1.738 2.159 1.784 5.445 5.453 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 5.46 2.74 2.35 2.16 1.95 1.343 1.713 2.122 0.5 0.562\n",
- " 0.561 0.273 0.313 0.396 0.18 0.4 0.298 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 5.451 2.752 2.933 2.15 1.931 1.247 1.496 1.714 0.562 0.5\n",
- " 0.563 0.392 0.377 0.324 0.171 0.27 0.259 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 5.466 2.774 2.975 1.729 1.732 1.253 1.667 2.161 0.561 0.563\n",
- " 0.5 0.328 0.262 0.268 0.17 0.346 0.369 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 2.245 3.039 8.241 2.339 2.55 1.617 1.825 1.863 0.273 0.392\n",
- " 0.328 0.5 0.415 0.28 0.22 0.245 0.293 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 2.124 2.741 2.959 5.456 3.934 1.854 1.835 1.738 0.313 0.377\n",
- " 0.262 0.415 0.5 0.561 0.247 0.256 0.234 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 2.203 2.759 2.359 5.44 3.857 2.131 2.231 2.159 0.396 0.324\n",
- " 0.268 0.28 0.561 0.5 0.27 0.361 0.262 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 1.135 1.571 1.972 1.819 3.966 5.488 3.402 1.784 0.18 0.171\n",
- " 0.17 0.22 0.247 0.27 0.5 0.253 0.248 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 2.263 2.759 2.415 2.07 2.322 2.008 3.348 5.445 0.4 0.27\n",
- " 0.346 0.245 0.256 0.361 0.253 0.5 0.567 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 2.071 2.788 3.243 1.759 2.135 1.916 3.366 5.453 0.298 0.259\n",
- " 0.369 0.293 0.234 0.262 0.248 0.567 0.5 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]\n",
- " [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
- " 0. 0. 0. 0. 0. 0. 0. 0. 0. ]]\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"rand_mol = random.randint(0, len(y))\n",
"\n",
@@ -279,7 +147,7 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
@@ -288,27 +156,7 @@
"id": "GJu2RtUhwrmh",
"outputId": "0d52abe6-0262-4ac4-d53b-4f5e4d71ef3b"
},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "