Add further instructions for using the accelerator directive and additional options #5488
Conversation
…tional options Signed-off-by: adamrtalbot <[email protected]>
✅ Deploy Preview for nextflow-docs-staging ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
| @@ -232,6 +232,10 @@ The above examples will request 4 GPUs of type `nvidia-tesla-k80`. | |||
| This directive is only used by certain executors. Refer to the {ref}`executor-page` page to see which executors support this directive. | |||
| ::: | |||
|
|
|||
| :::{note} | |||
| Additional options may be required to fully enable the use of accelerators. When using containers with GPUs, you must pass the GPU drivers through to the container. For Docker, this requires the option `--gpus all` in the docker run command. For Apptainer/Singularity, this requires the option `--nv`. The specific implementation details depend on the accelerator and container type being used. | |||
There was a problem hiding this comment.
um, should nextflow takes care of this?
There was a problem hiding this comment.
Not sure - it's tricky. Let's focus on Docker.
If you have more than 1 GPU, you might want to use 1 per-process and add --gpus '"device=1"', --gpus '"device=2"' etc. or give it all of them with --gpus all. On the other hand, some cloud platforms handle this for you and therefore adding anything raises an error. This is assuming you are using an nvidia card and they don't change the syntax. Not sure how it corresponds to AMD or another brand of GPU.
Similarly, what if I'm not using GPUs? For an FGPA this might be something like '--device /dev/dri'.
So it's complicated. I'd like for Nextflow to handle it for you but there's a long list of options.
adamrtalbot
deleted the
document_accelerator_more_instructions_required
branch
… [ci skip] Signed-off-by: adamrtalbot <[email protected]>
Extends the accelerator documentation to clarify that further requirements might be needed.
This wasn't obvious from the existing docs so I felt like it needed to be clarified.