-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How long to fine tune? #43
Comments
Hi, thanks for the kind words. Firstly, having a train-validation-test split is very important. That way, you can monitor for overfitting with the validation set. Second, the number of epochs depends on the learning rate you use. We used a tiny learning rate, hence we just let the model train for as long as possible, and set |
Hi! and thankyou for your excellent contribution to the world of time series!
I am currently using lag llama for finetuning, and was wondering if you had any rules of thumb for fine tuning yet?
I have read that transformers generally require many epochs, and noticed your early stopping patience is fifty. Does this mean we should generally train for many epochs? Or was that early stopping patience set on a very small dataset?
For context, my dataset has about 3 years with of price/energy demand/air temp/solar output data at 5 minute intervals. I have set a long context length to try and capture seasonal effects. Wondering how many epochs I should train for. The base foundation model did not work very on my data
The text was updated successfully, but these errors were encountered: