Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch mosaicml logger to use futures to enable better error handling #2702

Merged
merged 55 commits into from
Nov 14, 2023

Conversation

j316chuck
Copy link
Contributor

@j316chuck j316chuck commented Nov 8, 2023

What does this PR do?

Switch mosaicml logger to use futures to enable better error handling.

Helps close out this PR from Tyler.

What issue(s) does this change relate to?

Tests that if mcli retries fail (currently it'll retry 10x with an exponentail backoff), then the run fails.

Tests

Added unit tests

@j316chuck j316chuck requested a review from siriuslee November 8, 2023 23:40
@j316chuck j316chuck requested review from eracah, dakinggg and a team as code owners November 8, 2023 23:40
@j316chuck j316chuck changed the title Switch mosaicml logger to use futures Switch mosaicml logger to use futures to enable better error handling Nov 8, 2023
Copy link
Contributor

@siriuslee siriuslee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for adding the test

tests/fixtures/autouse_fixtures.py Outdated Show resolved Hide resolved
docs/source/doctest_fixtures.py Outdated Show resolved Hide resolved
Copy link
Contributor

@mvpatel2000 mvpatel2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@j316chuck can we run one final manual test because we've changed a lot

docs/source/doctest_fixtures.py Outdated Show resolved Hide resolved
@j316chuck
Copy link
Contributor Author

test run name: composer-mpt-125m-chinchilla-regression-LqM65R

@j316chuck j316chuck enabled auto-merge (squash) November 14, 2023 23:09
@j316chuck j316chuck disabled auto-merge November 14, 2023 23:10
@j316chuck j316chuck enabled auto-merge (squash) November 14, 2023 23:10
@j316chuck j316chuck merged commit 0f07bf9 into dev Nov 14, 2023
16 checks passed
@j316chuck j316chuck deleted the chuck/protected branch November 14, 2023 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants