Add AWS Inf2 instances support for aws_batch scheduler #977
Conversation
|
Hi @shixianc! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
|
@d4l3k, @kiukchung could you help review? thanks! |
Signed a few mins ago, probably take some time to propagate. |
facebook-github-bot
added
the
CLA Signed
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
|
Hi @d4l3k |
|
@kiukchung @d4l3k rebased. Could you help rerun the checks? |
| return Resource( | ||
| cpu=4, | ||
| gpu=0, | ||
| memMB=32 * GiB, |
There was a problem hiding this comment.
16GB of memory not 32? Pretty sure this refers to the CPU memory not the accelerator memory
There was a problem hiding this comment.
@ashvinnihalani Good catch, thanks for pointing this out.
I was referencing trn1 and thought the field is for GPU memory, but then realize trn1 has same GPU and CPU memory.
| return Resource( | ||
| cpu=32, | ||
| gpu=0, | ||
| memMB=32 * GiB, |
| return Resource( | ||
| cpu=96, | ||
| gpu=0, | ||
| memMB=192 * GiB, |
| ) | ||
|
|
||
|
|
||
| def aws_inf2_48xlarge() -> Resource: |
Add AWS Inf2 instances support for aws_batch scheduler. There're usecases to use torchx to launch data parallel inference jobs on inf2 instances on AWS Batch.
Configurations are referencing https://aws.amazon.com/ec2/instance-types/inf2/
Test plan:
Updated unittest to cover new changes.
=== SCHEDULER REQUEST ===
command:
--nnodes 1 --nproc_per_node 1 --tee 3 --role '' -m abc
image: sha256:3f8f845e25030d9523bf299dee1c3ca6f2b008fc2bf0a2161ed949efe168a3e1
kwargs:
devices:
environment:
LOGLEVEL: WARNING
TORCHX_JOB_ID: local_docker://torchx/abc-kr12qr37093rz
TORCHX_RANK0_HOST: abc-kr12qr37093rz-abc-0
TORCHX_TRACKING_EXPERIMENT_NAME: default-experiment
hostname: abc-kr12qr37093rz-abc-0
labels:
torchx.pytorch.org/app-id: abc-kr12qr37093rz
torchx.pytorch.org/replica-id: '0'
torchx.pytorch.org/role-name: abc
torchx.pytorch.org/version: 0.8.0dev0
mem_limit: 377472m
mounts: []
name: abc-kr12qr37093rz-abc-0
nano_cpus: 192000000000
network: torchx
privileged: false
shm_size: 377472m