diff --git a/ME-390-Exercises/ME-390-06-neural-nets/neural_nets_pytorch.ipynb b/ME-390-Exercises/ME-390-06-neural-nets/neural_nets_pytorch.ipynb index 17b47df..2c12a25 100644 --- a/ME-390-Exercises/ME-390-06-neural-nets/neural_nets_pytorch.ipynb +++ b/ME-390-Exercises/ME-390-06-neural-nets/neural_nets_pytorch.ipynb @@ -1,1015 +1,1015 @@ { "cells": [ { "cell_type": "markdown", "id": "instant-harmony", "metadata": {}, "source": [ - "# Neural networks with PyTorch - Solutions" + "# Neural networks with PyTorch" ] }, { "cell_type": "markdown", "id": "joined-resident", "metadata": {}, "source": [ "
\n", "\n", "This notebook is part of a series of exercises for the CIVIL-226 Introduction to Machine Learning for Engineers course at EPFL and adapted for the ME-390. Copyright (c) 2021 [VITA](https://www.epfl.ch/labs/vita/) lab at EPFL \n", "Use of this source code is governed by an MIT-style license that can be found in the LICENSE file or at https://www.opensource.org/licenses/MIT\n", "\n", "**Author(s):** David Mizrahi\n", "
\n" ] }, { "cell_type": "markdown", "id": "placed-verification", "metadata": {}, "source": [ "In this exercise, we'll cover the basics of the [PyTorch](https://pytorch.org) package and use it to implement a simple neural network." ] }, { "cell_type": "markdown", "id": "racial-preference", "metadata": {}, "source": [ "## Installation" ] }, { "cell_type": "code", "execution_count": null, "id": "cooked-possible", "metadata": {}, "outputs": [], "source": [ "# Verify that PyTorch and torchvision are installed correctly\n", "# Note: You may need to restart your kernel and re-run this cell before running the following cells\n", "\n", "import torch\n", "import torchvision" ] }, { "cell_type": "code", "execution_count": null, "id": "future-contact", "metadata": {}, "outputs": [], "source": [ "torch.__version__" ] }, { "cell_type": "code", "execution_count": null, "id": "compliant-inspector", "metadata": {}, "outputs": [], "source": [ "torchvision.__version__" ] }, { "cell_type": "markdown", "id": "backed-paint", "metadata": {}, "source": [ "#### For Google Colab\n", "You can run this notebook in Google Colab using the following link: https://colab.research.google.com/github/SYCAMORE-Lab/ME-390-2022/blob/master/ME-390-Exercises/ME-390-06-neural-nets/neural_nets_pytorch.ipynb" ] }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "id": "simple-lounge", "metadata": {}, "outputs": [], "source": [ "try:\n", " import google.colab\n", " IN_COLAB = True\n", "except:\n", " IN_COLAB = False\n", "if IN_COLAB:\n", " # Clone the entire repo to access the files\n", " !git clone -l -s https://github.com/SYCAMORE-Lab/ME-390-2022.git cloned-repo\n", " %cd cloned-repo/ME-390-Exercises/ME-390-06-neural-nets/" ] }, { "cell_type": "markdown", "id": "surrounded-comparison", "metadata": {}, "source": [ "# Exercise 1: Deep Learning with PyTorch" ] }, { "cell_type": "markdown", "id": "muslim-colony", "metadata": {}, "source": [ "PyTorch's website offers a tutorial on the basics of PyTorch, which can be found at the following link: https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html\n", "\n", "We recommend completing this tutorial before moving on to the second exercise." ] }, { "cell_type": "markdown", "id": "structured-monte", "metadata": {}, "source": [ "# Exercise 2: Simple neural nets with PyTorch" ] }, { "cell_type": "markdown", "id": "finite-inquiry", "metadata": {}, "source": [ "In this exercise, we'll implement classifiers with PyTorch.\n", "More specifically, we'll compare a simple logistic regression model to a simple neural network on the MNIST dataset.\n", "\n", "Here is the general pipeline used to train neural networks with PyTorch:\n", "\n", "```\n", "1. Load the dataset\n", "2. Initialize a dataloader, define data transforms\n", "3. Define and instantiate network architecture\n", "4. Choose a loss function\n", "5. Choose an optimizer\n", "6. Define the training loop & number of epochs\n", "7. Train the model\n", "8. Check validation accuracy, visualize results, adjust hyper-parameters (not done in this exercise)\n", "9. Repeat steps 2-9 until satisfied with validation accuracy (not done in this exercise)\n", "10. Check test accuracy\n", "```\n", "\n", "*Recall, Epoch: number of times that the learning algorithm will work through the entire training set*\n", "\n", "In this exercise, we'll follow these steps to implement our classifiers. Let's get started!" ] }, { "cell_type": "markdown", "id": "hourly-veteran", "metadata": {}, "source": [ "## 1. Imports" ] }, { "cell_type": "code", "execution_count": null, "id": "naughty-broadway", "metadata": {}, "outputs": [], "source": [ "# PyTorch & torchvision\n", "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "import torch.optim as optim\n", "\n", "import torchvision\n", "import torchvision.transforms as transforms\n", "from torchvision.datasets import MNIST, FashionMNIST\n", "\n", "# Progress bar\n", "from tqdm.auto import tqdm\n", "\n", "# Helper files\n", "import helpers\n", "import metrics" ] }, { "cell_type": "markdown", "id": "private-sessions", "metadata": {}, "source": [ "Here is a brief description of these imported packages:\n", "\n", "**PyTorch:**\n", "- `torch.nn` Contains the basic building blocks to implement neural nets (incl. different types of layers and loss functions) | [Documentation](https://pytorch.org/docs/stable/nn.html)\n", "- `torch.nn.functional` A functional (stateless) approach to torch.nn, often used for stateless objects (e.g. ReLU) | [Documentation](https://pytorch.org/docs/stable/nn.functional.html) | [More info](https://discuss.pytorch.org/t/what-is-the-difference-between-torch-nn-and-torch-nn-functional/33597/2)\n", "- `torch.optim` A package implementing various optimization algorithms, such as SGD and Adam | [Documentation](https://pytorch.org/docs/stable/optim.html)\n", "\n", "**torchvision:**\n", "- `torchvision.transforms` Common image transformations\n", "- `torchvision.datasets` Popular image datasets\n", "\n", "**`tqdm`:** Popular package used to show progress bars | [Documentation](https://tqdm.github.io/)\n", "\n", "**`helpers`**: Contains functions to help visualize data and predictions\n", "\n", "**`metrics`:** Contains two simple classes that help keep track and compute the loss and accuracy over a training epoch" ] }, { "cell_type": "markdown", "id": "subsequent-salmon", "metadata": {}, "source": [ "## 2. Loading and visualizing the data" ] }, { "cell_type": "markdown", "id": "unlimited-underwear", "metadata": { "tags": [] }, "source": [ "### 2.1. Dataset\n", "\n", "Here, we will use the [MNIST](http://yann.lecun.com/exdb/mnist/) dataset, which consists of black-white images of hand-drawn digits. The training set has 60000 examples and the test set has 10000 examples. \n", "The `torchvision` package provides an easy way to load this data set. \n" ] }, { "cell_type": "code", "execution_count": null, "id": "configured-continent", "metadata": {}, "outputs": [], "source": [ "# Save dataset in a folder called \"data\"\n", "root = \"data\"\n", "\n", "# transforms.ToTensor() is used to convert the downloaded PIL Image to a torch Tensor\n", "train_data = MNIST(root, train=True, transform=transforms.ToTensor(), download=True)\n", "test_data = MNIST(root, train=False, transform=transforms.ToTensor(), download=True)" ] }, { "cell_type": "code", "execution_count": null, "id": "three-county", "metadata": {}, "outputs": [], "source": [ "print(f\"Images in training data: {len(train_data)}\")\n", "print(f\"Images in test data: {len(test_data)}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "sweet-tonight", "metadata": {}, "outputs": [], "source": [ "# Show the mapping from target value to class name (if you're using MNIST, you won't be too surprised)\n", "{i: class_name for i, class_name in enumerate(train_data.classes)}" ] }, { "cell_type": "markdown", "id": "perfect-dating", "metadata": {}, "source": [ "### 2.2. Dataloader" ] }, { "cell_type": "markdown", "id": "confident-albert", "metadata": {}, "source": [ "When training neural networks, we usually use mini-batches of data for each forward + backward pass. \n", "\n", "In order to obtain those mini-batches, we must pass our dataset through `torch.utils.DataLoader`, which combines the dataset and a sampler, and returns an iterable over the data and labels of our dataset.\n", "\n", "In this exercise, we'll pick a batch size of 32." ] }, { "cell_type": "code", "execution_count": null, "id": "thirty-glenn", "metadata": {}, "outputs": [], "source": [ "batch_size = 32\n", "# Reshuffle training data at every epoch, but not the test data \n", "train_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size, shuffle=True)\n", "test_loader = torch.utils.data.DataLoader(test_data, batch_size=batch_size, shuffle=False)" ] }, { "cell_type": "markdown", "id": "baking-algorithm", "metadata": {}, "source": [ "Let's take a look at one of our batches." ] }, { "cell_type": "code", "execution_count": null, "id": "compliant-option", "metadata": { "tags": [] }, "outputs": [], "source": [ "images, targets = iter(train_loader).next()" ] }, { "cell_type": "markdown", "id": "5aa42d2a-78e7-4619-bbef-2e3919ed200a", "metadata": {}, "source": [ "To understand how the images are represented, let us look at one of the images: " ] }, { "cell_type": "code", "execution_count": null, "id": "7f3cd428-a515-4c26-ada1-849c73d9d4fa", "metadata": {}, "outputs": [], "source": [ "print(images[0,0,:,:])" ] }, { "cell_type": "code", "execution_count": null, "id": "identical-slope", "metadata": {}, "outputs": [], "source": [ "helpers.imshow(torchvision.utils.make_grid(images, nrow=8))\n", "print(targets.reshape(-1, 8))" ] }, { "cell_type": "markdown", "id": "potential-gravity", "metadata": {}, "source": [ "## 3. Simple model" ] }, { "cell_type": "markdown", "id": "crazy-shift", "metadata": {}, "source": [ "First, we'll build a simple classifier (akin to logistic regression)." ] }, { "cell_type": "markdown", "id": "underlying-rough", "metadata": {}, "source": [ "### 3.1 Network architecture" ] }, { "cell_type": "markdown", "id": "wooden-norwegian", "metadata": {}, "source": [ "#### Short primer on `nn.Module`\n", "\n", "In PyTorch, each neural net architecture is a subclass of `nn.Module` ([Documentation](https://pytorch.org/docs/stable/generated/torch.nn.Module.html))\n", "\n", "To quote an [official tutorial](https://pytorch.org/tutorials/beginner/nlp/deep_learning_tutorial.html?highlight=module):\n", "\n", "> All network components should inherit from `nn.Module` and override the `forward()` method. That is about it, as far as the boilerplate is concerned. Inheriting from `nn.Module` provides functionality to your component. For example, it makes it keep track of its trainable parameters, you can swap it between CPU and GPU with the `.to(device)` method, where device can be a CPU device `torch.device(\"cpu\")` or CUDA device `torch.device(\"cuda:0\")`.\n", "\n", "\n", "In order to implement your model, you'll therefore need to fill in two methods:\n", "\n", "**`__init__()`**: \n", "\n", "- Initialize your layers here, so that the module can keep track of these layers' parameters. \n", "- It is not necessary to initialize layers with no learnable parameters (e.g. `ReLU`), as you can use the`nn.functional` API for those if you want.\n", "\n", "**`forward()`**:\n", "\n", "- Define your model architecture here (i.e. call your layers in the desired order). You can use any of the Tensor operations in the `forward` function.\n", "- This function defines the computation performed at every call. The backward function (where gradients are computed) is automatically defined for you using autograd. \n", "\n", "The learnable parameters of a model are returned by `model.parameters()`." ] }, { "cell_type": "markdown", "id": "bottom-blank", "metadata": {}, "source": [ "#### One layer neural net\n", "\n", "You should implement a one-layer network as follows:\n", "- First, flatten the image so that rather than matrices they are seen as a vector of dimension 28x28=784.\n", "- A fully-connected layer from the flattened input of shape (784,) to the output layer of shape (10,) as we have 10 different classes.\n", "\n", "Note that there is no hidden layer here. \n", "\n", "Furthermore, the output layer provides a real value corresponding to each of the 10 classes for the digits 0, 1, 2, ..., 9. Hence, to use this network for classification, we can use a cross-entropy loss:: [`CrossEntropyLoss()` module](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html). Hence, we can obtain the probability of each of the 10 classes after training the network.\n", "\n", "As a result, we also added a `predict()` method that adds this softmax layer, it'll be useful later on.\n", "\n", "Notice that we use `class` from python to define the neural network. For a brief introduction to this, go back to Section 8 of the exercise 01-python." ] }, { "cell_type": "markdown", "id": "streaming-screening", "metadata": {}, "source": [ "**One layer neural net:**\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "id": "agreed-sailing", "metadata": {}, "outputs": [], "source": [ "class OneLayerNet(nn.Module):\n", " \"\"\"1-Layer MNIST classifier\"\"\"\n", " \n", " def __init__(self):\n", " super().__init__()\n", " self.fc = nn.Linear(784, 10)\n", "\n", " def forward(self, x: torch.Tensor) -> torch.Tensor:\n", " # Flatten to get tensor of shape (batch_size, 784)\n", " x = x.flatten(start_dim=1)\n", " out = self.fc(x)\n", " return out\n", "\n", " def predict(self, x: torch.Tensor) -> torch.Tensor:\n", " \"\"\"Predicts classes by calculating the softmax\"\"\"\n", " logits = self.forward(x)\n", " return F.softmax(logits, dim=1)\n", "\n", "model = OneLayerNet()" ] }, { "cell_type": "markdown", "id": "geographic-compression", "metadata": { "tags": [] }, "source": [ "**Question:** How many trainable parameters (weights) does this network have?" ] }, { "cell_type": "markdown", "id": "compliant-cleaning", "metadata": {}, "source": [ "**Answer:**\n", "YOUR ANSWER HERE" ] }, { "cell_type": "markdown", "id": "undefined-triple", "metadata": {}, "source": [ "### 3.2. Loss function and optimizer" ] }, { "cell_type": "markdown", "id": "happy-stock", "metadata": {}, "source": [ "As mentioned, we choose Cross Entropy loss as our loss function. For training, we use Stochastic Gradient Descent (SGD) with a learning rate of 0.05 as our optimizer." ] }, { "cell_type": "code", "execution_count": null, "id": "comprehensive-banana", "metadata": {}, "outputs": [], "source": [ "loss_fn = nn.CrossEntropyLoss()\n", "optimizer = optim.SGD(model.parameters(), lr=0.05)" ] }, { "cell_type": "markdown", "id": "packed-trigger", "metadata": {}, "source": [ "### 3.3. Training loop" ] }, { "cell_type": "markdown", "id": "european-mozambique", "metadata": {}, "source": [ "The training loop can be defined as follows:\n", "\n", "```\n", "For each batch in the dataset:\n", " 1. Load the batch\n", " 2. Zero-out the accumulated gradients (PyTorch accumulates gradients during each iteration of an epoch. this is useful for some other neural network training tasks but not for our exercise).\n", " 3. Run the forward pass through your model.\n", " 4. Compute the loss.\n", " 5. Run the backward pass, i.e. compute gradients of the loss w.r.t. to the weights.\n", " 6. Update the weights using the optimizer.\n", "```\n", "\n", "Take a look at the `train()` function written for you below and predict how many iterations of the inner and outer for loops will be done by calling the train function on our dataset:" ] }, { "cell_type": "code", "execution_count": null, "id": "treated-vietnam", "metadata": {}, "outputs": [], "source": [ "def train(model: torch.nn.Module, train_loader: torch.utils.data.DataLoader, loss_fn: torch.nn.Module, optimizer: torch.optim.Optimizer, epochs: int):\n", " \n", " # Initialize metrics for loss and accuracy\n", " loss_metric = metrics.LossMetric()\n", " acc_metric = metrics.AccuracyMetric(k=1)\n", " \n", " # Sets the module in training mode (doesn't have any effect here, but good habit to take)\n", " model.train()\n", " \n", " for epoch in range(1, epochs + 1):\n", " \n", " # Progress bar set-up\n", " pbar = tqdm(total=len(train_loader), leave=True)\n", " pbar.set_description(f\"Epoch {epoch}\")\n", " \n", " # Iterate through data\n", " for data, target in train_loader:\n", " \n", " # Zero-out the gradients\n", " optimizer.zero_grad()\n", " \n", " # Forward pass\n", " out = model(data)\n", " \n", " # Compute loss\n", " loss = loss_fn(out, target)\n", " \n", " # Backward pass\n", " loss.backward()\n", " \n", " # Optimizer step\n", " optimizer.step()\n", " \n", " \n", " # Update metrics & progress bar\n", " loss_metric.update(loss.item(), data.shape[0])\n", " acc_metric.update(out, target)\n", " pbar.update()\n", " \n", " # End of epoch, show loss and acc\n", " pbar.set_postfix_str(f\"Train loss: {loss_metric.compute():.3f} | Train acc: {acc_metric.compute() * 100:.2f}%\")\n", " loss_metric.reset()\n", " acc_metric.reset()" ] }, { "cell_type": "markdown", "id": "similar-personal", "metadata": {}, "source": [ "Now that the train function is defined, it's time to actually train your model." ] }, { "cell_type": "code", "execution_count": null, "id": "latest-utility", "metadata": {}, "outputs": [], "source": [ "# Train model for 10 epochs\n", "train(model, train_loader, loss_fn, optimizer, epochs=10)" ] }, { "cell_type": "markdown", "id": "bbf47f13-d97c-4ec5-baff-7c68728c29f6", "metadata": { "tags": [] }, "source": [ "**Question:** How many iterations are in each epoch? " ] }, { "cell_type": "markdown", "id": "a296aae4-026c-405d-9504-68965c41b5da", "metadata": { "tags": [] }, "source": [ "**Answer:** \n", "YOUR ANSWER HERE" ] }, { "cell_type": "markdown", "id": "driven-chance", "metadata": { "tags": [] }, "source": [ "### 3.4. Test accuracy\n", "\n", "You should now have a model with > 90% train accuracy, but this doesn't tell the whole story. In order to actually estimate how good our model is, we need to check its accuracy on the test set.\n", "\n", "Look through the function `test()` which computes the test accuracy of a given model on a specific dataset.\n", "\n", "This function iterates through a dataset once (using a DataLoader) and displays the accuracy." ] }, { "cell_type": "code", "execution_count": null, "id": "unlikely-interval", "metadata": {}, "outputs": [], "source": [ "def test(model: torch.nn.Module, dataloader: torch.utils.data.DataLoader):\n", " \n", " # Initialize accuracy metric\n", " acc_metric = metrics.AccuracyMetric(k=1)\n", " \n", " # Progress bar set-up\n", " pbar = tqdm(total=len(test_loader), leave=True)\n", " \n", " # Sets the module in eval mode (doesn't have any effect here, but good habit to take)\n", " model.eval()\n", " \n", " with torch.no_grad():\n", " \n", " # Iterate through data\n", " for data, target in dataloader:\n", " \n", " # Forward pass\n", " out = model(data)\n", " \n", " # Update accuracy metric\n", " acc_metric.update(out, target)\n", "\n", " # Update progress bar\n", " pbar.update()\n", " \n", " # End of epoch, show loss and acc\n", " test_acc = acc_metric.compute() * 100\n", " pbar.set_postfix_str(f\"Acc: {test_acc:.2f}%\")\n", " print(f\"Accuracy is {test_acc:.2f}%\")\n" ] }, { "cell_type": "code", "execution_count": null, "id": "direct-triumph", "metadata": {}, "outputs": [], "source": [ "test(model, test_loader)" ] }, { "cell_type": "markdown", "id": "virtual-lighting", "metadata": {}, "source": [ "**Expected result:** >90% test accuracy on MNIST" ] }, { "cell_type": "markdown", "id": "reflected-outside", "metadata": {}, "source": [ "We obtain an accuracy of over 90% with this simple softmax regression model. A random classifier would only get an accuracy of ~10% (why?), so this is quite an improvement!\n", "\n", "But let's not stop there, we can do even better with just a few small changes. Let's see how!" ] }, { "cell_type": "markdown", "id": "concrete-contractor", "metadata": {}, "source": [ "## 4. Simple neural net (with hidden layers)" ] }, { "cell_type": "markdown", "id": "respective-orchestra", "metadata": {}, "source": [ "Time to implement (actual) neural networks. As you'll soon see, the entire pipeline is very similar, only the network architecture changes." ] }, { "cell_type": "markdown", "id": "saving-topic", "metadata": {}, "source": [ "#### Network architecture\n", "\n", "You should implement a three-layer fully-connected neural net (2 hidden layers) composed of the following:\n", "- First, flatten the image (as done previously)\n", "- A fully-connected layer from the flattened input of shape (784,) to the first hidden layer of shape (100, ), with ReLU as an activation.\n", "- A fully-connected layer from the first hidden layer of shape (100,) to the second hidden layer of shape (100, ), with ReLU as an activation.\n", "- A fully-connected layer from the second hidden layer of shape (100,) to the output layer of shape (10,).\n", "\n", "Fully connected neural networks (FCNNs) are sometimes also referred to as multilayer perceptrons (MLP) in ML literature.\n", "\n", "**Note:** You can use the OneLayerNet function above and simply fill in additional information for the layers below" ] }, { "cell_type": "markdown", "id": "published-prize", "metadata": {}, "source": [ "**Three-layer neural net:**\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "id": "biblical-corps", "metadata": {}, "outputs": [], "source": [ "class ThreeLayerNet(nn.Module):\n", " \"\"\"3-Layer neural net\"\"\"\n", " \n", " def __init__(self):\n", " super().__init__()\n", "\n", " self.fc1 = nn.Linear(784, 100)\n", " ### START CODE HERE ###\n", " ### Hint: you need two more linear layers here of appropriate dimension\n", "\n", " ### END CODE HERE ###\n", "\n", " def forward(self, x: torch.Tensor) -> torch.Tensor:\n", " # Flatten to get tensor of shape (batch_size, 784)\n", " x = x.flatten(start_dim=1)\n", " x = F.relu(self.fc1(x))\n", " ### START CODE HERE ###\n", " ### Hint: you need one more relu layer and the output layer \n", "\n", " return ...\n", " ### END CODE HERE ###\n", "\n", " def predict(self, x: torch.Tensor) -> torch.Tensor:\n", " \"\"\"Predicts classes by calculating the softmax\"\"\"\n", " logits = self.forward(x)\n", " return F.softmax(logits, dim=1)\n", "\n", "model = ThreeLayerNet()" ] }, { "cell_type": "markdown", "id": "greek-harassment", "metadata": {}, "source": [ "**Question:** How many trainable parameters (weights) does this network have?" ] }, { "cell_type": "markdown", "id": "worst-exhibit", "metadata": {}, "source": [ "**Answer:**\n", "YOUR ANSWER HERE" ] }, { "cell_type": "markdown", "id": "valuable-trigger", "metadata": {}, "source": [ "#### Loss & optimizer\n", "\n", "Use the same loss function and optimizer as in the previous section. Namely, the Cross-Entropy loss and SGD with lr=0.05." ] }, { "cell_type": "code", "execution_count": null, "id": "framed-recognition", "metadata": {}, "outputs": [], "source": [ "### START CODE HERE ###\n", "loss_fn = ...\n", "optimizer = ...\n", "### END CODE HERE ###" ] }, { "cell_type": "markdown", "id": "partial-collapse", "metadata": {}, "source": [ "#### Training" ] }, { "cell_type": "markdown", "id": "consistent-adaptation", "metadata": {}, "source": [ "Use `train()` to launch the training process, use the same number of epochs as in the previous part." ] }, { "cell_type": "code", "execution_count": null, "id": "intellectual-chess", "metadata": {}, "outputs": [], "source": [ "### START CODE HERE ###\n", "...\n", "### END CODE HERE ###" ] }, { "cell_type": "markdown", "id": "indie-republican", "metadata": {}, "source": [ "#### Test accuracy" ] }, { "cell_type": "markdown", "id": "fourth-geology", "metadata": {}, "source": [ "Compute the test accuracy using `test()`." ] }, { "cell_type": "code", "execution_count": null, "id": "identical-munich", "metadata": {}, "outputs": [], "source": [ "### START CODE HERE ###\n", "...\n", "### END CODE HERE ###" ] }, { "cell_type": "markdown", "id": "local-fashion", "metadata": {}, "source": [ "**Expected result:** >96% test accuracy on MNIST" ] }, { "cell_type": "markdown", "id": "egyptian-timothy", "metadata": {}, "source": [ "## 5. Visualizing predictions" ] }, { "cell_type": "markdown", "id": "educational-macintosh", "metadata": {}, "source": [ "Let's take a look at a few of our model's predictions.\n", "\n", "First, we'll get the predictions for one batch of our test data (this is where the `predict()` method becomes useful)." ] }, { "cell_type": "code", "execution_count": null, "id": "instrumental-acquisition", "metadata": {}, "outputs": [], "source": [ "images, _ = iter(test_loader).next()\n", "preds = model.predict(images)" ] }, { "cell_type": "markdown", "id": "rolled-captain", "metadata": {}, "source": [ "Then, we'll use a function in `helpers` to view the classifier's softmax score next to each image." ] }, { "cell_type": "code", "execution_count": null, "id": "hollywood-packet", "metadata": {}, "outputs": [], "source": [ "# Shows the image next to the classifier's softmax score\n", "# Show for the first 10 images (change value to see more images)\n", "for i in range(10):\n", " helpers.view_prediction(images[i], preds[i], test_data.classes)" ] }, { "cell_type": "markdown", "id": "usual-portsmouth", "metadata": {}, "source": [ "Not bad, right?\n", "\n", "Congratulations on finishing this exercise! If you want to try out your network on a different dataset, take a look at part 6.\n" ] }, { "cell_type": "markdown", "id": "third-jaguar", "metadata": {}, "source": [ "## 6. Training your model on a different dataset\n", "There are many datasets designed to serve as a direct drop-in replacement to MNIST, one of them being [Fashion-MNIST](https://github.com/zalandoresearch/fashion-mnist), by Zalando Research.\n", "\n", "You can try out these two networks on Fashion-MNIST by just replacing the following two lines at the beginning of the notebook and re-running the entire code.\n", "\n", "Replace:\n", "```python\n", "train_data = MNIST(root, train=True, transform=transforms.ToTensor(), download=True)\n", "test_data = MNIST(root, train=False, transform=transforms.ToTensor(), download=True)\n", "```\n", "\n", "with:\n", "```python\n", "train_data = FashionMNIST(root, train=True, transform=transforms.ToTensor(), download=True)\n", "test_data = FashionMNIST(root, train=False, transform=transforms.ToTensor(), download=True)\n", "```\n", "\n", "**Note:** FashionMNIST is a harder dataset than MNIST, so your accuracy will likely be lower." ] }, { "cell_type": "markdown", "id": "behind-endorsement", "metadata": {}, "source": [ "## (Optional) Additional PyTorch resources\n", "- PyTorch cheat sheet: https://pytorch.org/tutorials/beginner/ptcheat.html\n", "- Other PyTorch tutorials: https://pytorch.org/tutorials/index.html\n", "- PyTorch recipes: https://pytorch.org/tutorials/recipes/recipes_index.html (bite-sized code examples on specific PyTorch features)\n", "- PyTorch examples: https://github.com/pytorch/examples" ] } ], "metadata": { "kernelspec": { "display_name": "Python", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.10" } }, "nbformat": 4, "nbformat_minor": 5 }