Low accuracy in Chapter 3 - 118

[YueyangBrian]

I am running PyTorch in jupyter notebook. My train accuracy and test accuracy after each epoch are 10.00% and 9.99%, low and no change. My code is identical to the tutorial. However, in the previous steps, model_2(rand_image_tensor.to(device)) does not show dimensional error since I am using pytorch 2.0 and my RunTimeError is mat1 and mat2 shapes cannot be multiplied (10x49 and 10x10) instead of 1x490 and 10x10 in the tutorial.
But if I run the same code in google colab I got the expected results.
I have no idea what is wrong with my model. Can someone help? Thanks.

class FashionMNISTModelV2(nn.Module):
    def __init__(self, input_shape: int, hidden_units: int, output_shape: int):
        super().__init__()
        self.conv_block_1 = nn.Sequential(
            nn.Conv2d(in_channels=input_shape,
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.Conv2d(in_channels=hidden_units,
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2)
        )
        self.conv_block_2 = nn.Sequential(
            nn.Conv2d(in_channels=hidden_units,
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.Conv2d(in_channels=hidden_units,
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2)
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(in_features=hidden_units*7*7,
                      out_features=output_shape,)
        )
    def forward(self,x):
        x = self.conv_block_1(x)
        x = self.conv_block_2(x)
        x = self.classifier(x)
        return x

torch.manual_seed(42)
model_2 = FashionMNISTModelV2(input_shape = 1,
                              hidden_units = 10,
                              output_shape = len(class_names)
                             ).to(device)

torch.manual_seed(42)
torch.cuda.manual_seed(42)

from timeit import default_timer as timer
train_time_start_model_2 = timer()

epochs = 3

for epoch in tqdm(range(epochs)):
    print(f"Epoch: {epoch}\n")
    train_step(model=model_2,
              data_loader=train_dataloader,
              loss_fn=loss_fn,
              optimizer=optimizer,
              accuracy_fn=accuracy_fn,
              device=device)
    test_step(model=model_2,
             data_loader=test_dataloader,
             loss_fn=loss_fn,
             accuracy_fn=accuracy_fn,
             device=device)

train_time_end_model_2 = timer()

total_train_time_model_2 = print_train_time(start=train_time_start_model_2,
                                            end=train_time_end_model_2,
                                            device=device)
print(f"\nTotal training time on {device}: {total_train_time_model_2:.4f}s")

[LuluW8071]

@YueyangBrian

class FashionMNISTModelV2(nn.Module):
    def __init__(self, input_shape: int, hidden_units: int, output_shape: int):
        super().__init__()
        self.conv_block_1 = nn.Sequential(
            nn.Conv2d(in_channels=input_shape,
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.Conv2d(in_channels=hidden_units,
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2)
        )
        self.conv_block_2 = nn.Sequential(
            nn.Conv2d(in_channels=hidden_units,
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.Conv2d(in_channels=hidden_units,
                      out_channels=hidden_units,
                      kernel_size=3,
                      stride=1,
                      padding=1),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2)
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(in_features=hidden_units*7*7,  
                      out_features=output_shape)
        )

    def forward(self, x):
        x = self.conv_block_1(x)
        x = self.conv_block_2(x)
        x = self.classifier(x)
        return x

Your pasted model class seems fine to me :)

RunTimeError is mat1 and mat2 shapes cannot be multiplied (10x49 and 10x10) instead of 1x490 and 10x10 in the tutorial.

This error is due to mismatch in matrix multiplication most likely when passing from output of self.conv_block_2(x) layer to linear layer of self.classifier(x).

You just need to match column of 1st matrix with row of 2nd matrix to satisfy the multiplication.

If this was ur error in ur local device RunTimeError is mat1 and mat2 shapes cannot be multiplied (10x49 and 10x10)

The error could be in this section
Maybe you forgot to put flatten layer or 7x7 multiplied with hidden_layers

self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(in_features=hidden_units*7*7,
                      out_features=output_shape)
)

Note: U need to re run the optimizer code block cell as it needs to load ur new defined model class. You might have forgot to re run that code block cell so while training, it might be using the correct model for training but loading the old model in the optimizer, which could cause the loss and accuracy values to remain constant across each epoch

[mrdbourke]

Sensational answer, thank you!