Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/smaller fixes #40

Merged
merged 24 commits into from
Oct 9, 2023
Merged

Fix/smaller fixes #40

merged 24 commits into from
Oct 9, 2023

Conversation

saattrupdan
Copy link
Collaborator

@saattrupdan saattrupdan commented Oct 6, 2023

This PR does the following:

  1. Replaces isort and flake8 pre-commit plugins with ruff, which is faster and has more features.
  2. Specifies the common voice version in the dataset, to allow better reproducibility.
  3. Make it possible to train on local datasets, since using streaming with large datasets seem to be quite unstable, as Hugging Face's servers seem to crash all the time.
  4. Add ctc_zero_infinity=True, which might fix the NaN loss issue (see Replicate NST fine-tuning on Alvenir machine to sniff out training or data issues #32).
  5. Add first/second momentum of Adam to the config.
  6. Use the new-ish auto_find_batch_size=True argument.
  7. Fixes an issue which caused bugs when trying to build NST-da in a directory that isn't the current working directory.
  8. Don't include NST-da samples with empty transcriptions anymore.
  9. Change the no_cuda argument in TrainingArguments to the newer equivalent use_cpu argument.

@saattrupdan saattrupdan self-assigned this Oct 6, 2023
Copy link
Collaborator

@sorenmulli sorenmulli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, two small comments

src/coral_models/data.py Outdated Show resolved Hide resolved
config/model/wav2vec2.yaml Outdated Show resolved Hide resolved
@saattrupdan saattrupdan merged commit b3dcf94 into main Oct 9, 2023
7 checks passed
@saattrupdan saattrupdan deleted the fix/smaller-fixes branch October 26, 2023 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants