Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/smaller fixes #40

Merged
merged 24 commits into from
Oct 9, 2023
Merged

Fix/smaller fixes #40

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
f7aadb4
chore: Change configs
saattrupdan Oct 4, 2023
fff3ed9
chore: Change configs
saattrupdan Oct 4, 2023
451db6f
chore: Set `spec_augment=True` (that's the default anyway)
saattrupdan Oct 4, 2023
229752d
fix: Set `ctc_zero_infinity=True`, which might fix the weird loss bug
saattrupdan Oct 4, 2023
b446959
feat: Allow setting first/second momentum of Adamw
saattrupdan Oct 4, 2023
4eb0be6
fix: Create NST-da output folder if it doesn't exist
saattrupdan Oct 4, 2023
c16b9d9
style: Make sample rate more visible as a global variable for NST-da
saattrupdan Oct 4, 2023
b947d7c
fix: Remove all empty or NaN texts
saattrupdan Oct 4, 2023
7ace78f
chore: Reset dataset_probabilities
saattrupdan Oct 4, 2023
05326f7
chore: Update configs
saattrupdan Oct 4, 2023
62e38a6
fix: Directories in the NST script, to enable building it in a dir no…
saattrupdan Oct 4, 2023
98a9029
fix: Remove non-existent audio files from NST-da
saattrupdan Oct 5, 2023
36b50ea
chore: Change hyperparams
saattrupdan Oct 6, 2023
d92caf3
chore: Update lock file
saattrupdan Oct 6, 2023
479cd66
chore: Use ruff
saattrupdan Oct 6, 2023
42a8109
feat: Allow using local datasets
saattrupdan Oct 6, 2023
d8b1bfd
feat: Use `auto_find_batch_size`, remove deprecated arguments in trai…
saattrupdan Oct 6, 2023
e4399f1
chore: Update pre-commits
saattrupdan Oct 6, 2023
25c852d
chore: Update transformers dep
saattrupdan Oct 6, 2023
999cc9d
docs: Update coverage badge
saattrupdan Oct 6, 2023
fc124a5
fix: Use `load_dataset` instead of `DatasetDict.load_from_disk` when …
saattrupdan Oct 7, 2023
f8f0cc7
chore: Change logging_steps to 10
saattrupdan Oct 7, 2023
49032ed
fix: Missing f-string
saattrupdan Oct 9, 2023
c7f032c
docs: Add comment to `max_steps` and `warmup_steps`
saattrupdan Oct 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
defaults:
- model: wav2vec2_alvenir_config
- model: wav2vec2
- datasets:
- nst_da
- common_voice_9_da
Expand Down
6 changes: 3 additions & 3 deletions config/model/wav2vec2_alvenir_config.yaml → config/model/wav2vec2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: wav2vec2_alvenir_config
name: wav2vec2
type: wav2vec2
pretrained_model_id: chcaa/xls-r-300m-danish
freeze_feature_encoder: false
Expand Down Expand Up @@ -33,8 +33,8 @@ decoder:
# Training hyperparameters
batch_size: 8
gradient_accumulation: 32
max_steps: 120_000
max_steps: 13_000
learning_rate: 3e-5
adam_first_momentum: 0.9
adam_second_momentum: 0.98
warmup_steps: 12_000
warmup_steps: 1_300
saattrupdan marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 2 additions & 2 deletions config/model/wav2vec2_no_reg.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ decoder:
# Training hyperparameters
batch_size: 8
gradient_accumulation: 32
max_steps: 120_000
max_steps: 13_000
learning_rate: 3e-5
adam_first_momentum: 0.9
adam_second_momentum: 0.98
warmup_steps: 12_000
warmup_steps: 1_300