Experiment steps

fine-w2v2base-bs8-ep300-lr2e-5-non_freeze-lr_cosine-non_freeze-spec_aug

follow TIMIT in wav2vec2.0 paper for 40k steps update (400 epoch) mask_time_prob=0.065, mask_time_length=10, mask_feature_prob=0.012, mask_feature_length=64

Finetune freeze CNN on 95% data
Finetune non-freeze on 95% data
Finetune non-freeze on 100% data with spec augment (TIMIT config in wav2vec2.0)
Let's see :))) maybe edit wav2vec2 loss by calculating CTC Loss on tonal and text also

default: no tonal, non_freeze, lr cosine

Things to test:

The amount of SpecAug
- base from 0.075_10_0.004_64 (Best PER: 0.0908 - 9600)
- reduce to 0.05_10_0.004_20 (Best PER: {} - {})
- increase to 0.075_10_0.012_64 => When watching the loss, increase SpecAug policy make the model works better
Batch size:
- 8 (get from best above)
- 32
LayerNorm / RMSNorm
LR Cosine / LR Linear
Focal loss alpha: 0.99 | 0.75 | 0.5 | 0.25 (3) gamma: 0.5 | 1 | 2 ( 3 | 5)
- to test for focal loss:
  - spec aug policy: 0.05_10_0.004_40
  - speed perturb
  - epoch 100
  - batch size 16
  - lr 2e-5
  - warmup 0.1, cosine decay => best: alpha = 0.99, gamma = 2

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
ablation_study		ablation_study
md_d_metric		md_d_metric
mdd		mdd
.DS_Store		.DS_Store
Design model.ipynb		Design model.ipynb
analyse output.ipynb		analyse output.ipynb
analyst & create data.ipynb		analyst & create data.ipynb
change wav2vec2 tokenizer.ipynb		change wav2vec2 tokenizer.ipynb
finetune_w2v2_focal_ctc_linguistic.py		finetune_w2v2_focal_ctc_linguistic.py
finetune_w2v2_linguistic.py		finetune_w2v2_linguistic.py
finetune_w2v2_only.py		finetune_w2v2_only.py
finetune_w2v2_only_freeze.py		finetune_w2v2_only_freeze.py
finetune_wav2vec2_phoneme_tonal.py		finetune_wav2vec2_phoneme_tonal.py
fix_vi_ftfy.py		fix_vi_ftfy.py
model_architect.md		model_architect.md
n_layer_finetune_w2v2_linguistic.py		n_layer_finetune_w2v2_linguistic.py
readme.md		readme.md
run_focal_ctc.sh		run_focal_ctc.sh
run_md_d.sh		run_md_d.sh
run_n_layer.sh		run_n_layer.sh
run_spec_augment.sh		run_spec_augment.sh
testing.py		testing.py
tokenizer.ipynb		tokenizer.ipynb
tonal extraction.ipynb		tonal extraction.ipynb
train.py		train.py
train_non_tonal.py		train_non_tonal.py
train_papl_cnn_rnn.py		train_papl_cnn_rnn.py
vocab.json		vocab.json

Provide feedback