Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MIDI-160: refactoring configuration #6

Open
wants to merge 47 commits into
base: master
Choose a base branch
from
Open

MIDI-160: refactoring configuration #6

wants to merge 47 commits into from

Conversation

WojciechMat
Copy link
Collaborator

Config refactor

  • removed GPTConfig
  • made vocab_size an argument to GPT
  • removed model and tokenizer from hydra configs for finetuning

@WojciechMat WojciechMat changed the base branch from master to MIDI-157/validation-metrics January 28, 2025 19:44
@WojciechMat
remove model_args, add block_size to hydra configs, make vocab_size a…
6da26c9 
… GPT argument, get rid of GPTConfig, create piano_task_manager only when needed, remove repetitive checkpoint creation
@WojciechMat WojciechMat force-pushed the MIDI-160/config-refactor branch from 790e83c to 6da26c9 Compare January 28, 2025 20:23
@WojciechMat WojciechMat changed the base branch from MIDI-157/validation-metrics to master January 28, 2025 20:23
roszcz
roszcz reviewed Jan 29, 2025
View reviewed changes
gpt2/train.py Outdated Show resolved Hide resolved
gpt2/train.py Outdated Show resolved Hide resolved
gpt2/train.py Outdated Show resolved Hide resolved
gpt2/configs/model/gpt2_xl.yaml Outdated Show resolved Hide resolved
gpt2/train.py Outdated Show resolved Hide resolved
gpt2/train.py Outdated Show resolved Hide resolved
gpt2/train.py Outdated Show resolved Hide resolved
WojciechMat and others added 2 commits January 29, 2025 21:26
@WojciechMat
rename sequence_length and block_size to , do not concatenate dicts i…
f039d8b 
…n checkpoint, use 'name' from tokenizer desc in the checkpoint to choose tokenizer class, remove unnecessary comments and eval file
@roszcz
Minor cleanup
f713204
@roszcz roszcz force-pushed the MIDI-160/config-refactor branch from a767aef to f713204 Compare January 30, 2025 09:26
roszcz and others added 20 commits January 30, 2025 16:31
@roszcz
Move LR scheduler definition out of the training flow
690502f
@WojciechMat
rename tokenizer in checkpoint to tokenizer_desc, remove tokenizer fi…
a33c9bb 
…eld from eval config, make the high_level_piano_eval work with new configs, make model_name a parameter to script, use tmp/checkpoints for model storage
@WojciechMat
make pretraining and finetuning debugging configs, do not concatenate…
eb590f3 
… checkpoints and init_from, but treat init_from as a path to the model
@WojciechMat
eval should work for pretraining too
1545cd8
@WojciechMat
eval should work for pretraining too
6bc7b39
@WojciechMat
bump requirements versions and make them explicit, remove .pt suffix …
7b4b980 
…from metric calculation script for the command to work, add weights_only=False when loading checkpoints with torch, initialize validation dataloaders on master process only
@WojciechMat
make less validation dataloader processes - the same number as traini…
b55fb80 
…ng datakiaders
@WojciechMat
use run stage for evaluation, add tokenized_every dataset config for …
ec87e5a 
…running pretraining with all of our datasets
@SamuelJanas
Change ParametricTaskManager to PianoTaskManager
fe5283f 
change introduced in piano-dataset/pull/2
@SamuelJanas
change roszcz/ -> epr-labs/ for all dataset occurences
df3bf32
@SamuelJanas
change naming: sustain-v2 -> augmented
67ac227
@roszcz
Decrease onboarding friction
b6839f7
@WojciechMat
Training loop and configuration docs, TODO comment regarding gradient…
67ae0b6 
…_accumulation_steps
@WojciechMat
external evaluation documentation
86ebb63
@roszcz
fix eval samplers, control splits used for evaluation
2017129
@roszcz
fix eval script samplers container
969a163
@roszcz
fix wandb for piano metrics
8861e2a 
- using wandb groups to manage distributed sources of metrics (train.py
  and high_level_piano_eval.py)
@roszcz
cleanup eval config
d6c7ae6
@roszcz
start organizing bigger datasets
ff5e733
@roszcz
Switch to private hf repos
8a94b24
@roszcz roszcz force-pushed the MIDI-160/config-refactor branch from f3ba685 to a6d5247 Compare February 9, 2025 07:32
@roszcz roszcz force-pushed the MIDI-160/config-refactor branch from 9ad21ad to 613287b Compare February 10, 2025 20:26
@roszcz roszcz force-pushed the MIDI-160/config-refactor branch from 8896915 to 988057b Compare February 12, 2025 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@roszcz roszcz roszcz left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@WojciechMat @roszcz @SamuelJanas