Skip to content

Commit

Permalink
Merge pull request #558 from Pranav-Bobde/patch-1
Browse files Browse the repository at this point in the history
refactor: rephrase text to improve clarity and specificity
  • Loading branch information
merveenoyan authored Nov 22, 2023
2 parents 3d53e6f + de2cb27 commit 632b16e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion chapters/en/chapter1/4.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ By the way, you can evaluate the carbon footprint of your models' training throu

This pretraining is usually done on very large amounts of data. Therefore, it requires a very large corpus of data, and training can take up to several weeks.

*Fine-tuning*, on the other hand, is the training done **after** a model has been pretrained. To perform fine-tuning, you first acquire a pretrained language model, then perform additional training with a dataset specific to your task. Wait -- why not simply train directly for the final task? There are a couple of reasons:
*Fine-tuning*, on the other hand, is the training done **after** a model has been pretrained. To perform fine-tuning, you first acquire a pretrained language model, then perform additional training with a dataset specific to your task. Wait -- why not simply train the model for your final use case from the start (**scratch**)? There are a couple of reasons:

* The pretrained model was already trained on a dataset that has some similarities with the fine-tuning dataset. The fine-tuning process is thus able to take advantage of knowledge acquired by the initial model during pretraining (for instance, with NLP problems, the pretrained model will have some kind of statistical understanding of the language you are using for your task).
* Since the pretrained model was already trained on lots of data, the fine-tuning requires way less data to get decent results.
Expand Down

0 comments on commit 632b16e

Please sign in to comment.