Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow sharded grad scaler to cpu offload with FSDP #831

Merged
merged 33 commits into from
Nov 15, 2021
Merged

Allow sharded grad scaler to cpu offload with FSDP #831

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
e51e891
first commit
Oct 19, 2021
c63d1c3
sharded scaler hitting nan assertions
Oct 21, 2021
a39b37b
adding test for sharded grad scaler without cpu offload
Oct 26, 2021
daafd25
ddp grad scaler and fsdp sharded grad scaler test failing
Oct 27, 2021
bc9e244
removing test_output
Oct 27, 2021
917ae0e
fix no cpu offload test
Oct 27, 2021
65c3093
changing optimizer from OSS to SGD
Oct 27, 2021
3404b69
all tests passing, code cleanup pending
Oct 28, 2021
cf7d2e2
code cleanup
Oct 29, 2021
fa18c8e
fix pyproject.toml
Oct 29, 2021
6bb3a71
removing .isort.cfg
Oct 29, 2021
cda515f
resolving merge conflicts
Oct 29, 2021
98d04fd
running isort linter
Nov 1, 2021
41d012e
resolving isort issues
Nov 2, 2021
6346960
resolving black linter issue
Nov 2, 2021
c4e94a5
resolving mypy issues
Nov 2, 2021
8d79f1b
fix import statement
Nov 2, 2021
006db9c
Merge branch 'main' into cpu_gradscaler. Taking in changes from PR 838.
Nov 2, 2021
9ad7d3e
fix mypy error
Nov 2, 2021
bd7c7a9
modifying import statement
Nov 2, 2021
a51b49d
adding pytorch version requirement
Nov 3, 2021
cc63fbd
fixing pytest skip test decorator
Nov 3, 2021
a973fb6
apply version guard for ShardedGradScaler
Nov 3, 2021
d4bb7c5
removing test_fsdp_grad_scaler
Nov 3, 2021
5fb0a77
increasing num_epochs for ShardedGradScaler so that updates are not s…
Nov 4, 2021
345835b
adding support for torch 1.8
Nov 11, 2021
b5cfc86
minor edit
Nov 11, 2021
1ad6277
[skip ci] more torch 1.8 changes
Nov 12, 2021
110e52d
parametrizing the tests
Nov 12, 2021
447b9db
Merge branch 'main' into cpu_gradscaler
Nov 12, 2021
ad5e979
cleanup code with linters
Nov 12, 2021
e693c31
[skip ci] update doc string
Nov 12, 2021
acb4304
[skip ci] addressing some more comments
Nov 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading