Out of wack for plot_loss_curves

[fabriziogiordano]

First of all YouAreAmazing!!

I implemented my first pytorch quickly with a much better understanding of the foundations. But I have so much more to learn... in fact my model is producing this plot_loss_curves:

It seems my model is doing an amazing job in the training and failing miserably in testing.
Am I interpreting the chart correctly?
Any recommendation on how to improve?

[Pouyaexe]

Well, I think you are overfitting your model to the training data. You can try to add more data, change the model architecture or add some dropout layers in order to solve the problem. But before doing any of these, check your optimizer function.

[MHHamdan]

Congrats, @fabriziogiordano good starting, and keep it up.
Indeed as @Pouyaexe mentioned that your model is overfitted on your data since your model memories during the learning phase however ,fail to predict during the test phase; again, as he recommended, you can overcome this issue through augmenting your training data and try to minimize the model architecture like in reducing the number of layers and or several neurons in each layer and adding dropout.

[mrdbourke]

Hi @fabriziogiordano, thank you for the kind words, I really appreciate it!

As @Pouyaexe and @MHHamdan have said, it looks like your model is overfitting.

From 04. PyTorch Custom Datasets:

An overfitting model is one that performs better (often by a considerable margin) on the training set than the validation/test set.

Without knowing what model or data you're using, there are a few general things you can do:

Method to prevent overfitting	What is it?
Get more data	Having more data gives the model more opportunities to learn patterns, patterns which may be more generalizable to new examples.
Simplify your model	If the current model is already overfitting the training data, it may be too complicated of a model. This means it's learning the patterns of the data too well and isn't able to generalize well to unseen data. One way to simplify a model is to reduce the number of layers it uses or to reduce the number of hidden units in each layer.
Use data augmentation	Data augmentation manipulates the training data in a way so that's harder for the model to learn as it artificially adds more variety to the data. If a model is able to learn patterns in augmented data, the model may be able to generalize better to unseen data.
Use transfer learning	Transfer learning involves leveraging the patterns (also called pretrained weights) one model has learned to use as the foundation for your own task. In our case, we could use one computer vision model pretrained on a large variety of images and then tweak it slightly to be more specialized for food images.
Use dropout layers	Dropout layers randomly remove connections between hidden layers in neural networks, effectively simplifying a model but also making the remaining connections better. See torch.nn.Dropout() for more.
Use learning rate decay	The idea here is to slowly decrease the learning rate as a model trains. This is akin to reaching for a coin at the back of a couch. The closer you get, the smaller your steps. The same with the learning rate, the closer you get to convergence, the smaller you'll want your weight updates to be.
Use early stopping	Early stopping stops model training before it begins to overfit. As in, say the model's loss has stopped decreasing for the past 10 epochs (this number is arbitrary), you may want to stop the model training here and go with the model weights that had the lowest loss (10 epochs prior).

These are typically known as "regularization" techniques.

See more here: https://www.learnpytorch.io/04_pytorch_custom_datasets/#8-what-should-an-ideal-loss-curve-look-like