Skip to content

Commit

Permalink
Improve config resolution docs (#4950) [ci skip]
Browse files Browse the repository at this point in the history

Signed-off-by: Ben Sherman <[email protected]>
Signed-off-by: Paolo Di Tommaso <[email protected]>
Co-authored-by: Paolo Di Tommaso <[email protected]>
  • Loading branch information
bentsherman and pditommaso authored Apr 26, 2024
1 parent 0ffcc4c commit 019eb86
Showing 1 changed file with 26 additions and 19 deletions.
45 changes: 26 additions & 19 deletions docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@

## Configuration file

When a pipeline script is launched, Nextflow looks for configuration files in multiple locations. Since each configuration file can contain conflicting settings, the sources are ranked to determine which settings are applied. Possible configuration sources, in order of priority:
When a pipeline script is launched, Nextflow looks for configuration files in multiple locations. Since each configuration file may contain conflicting settings, they are applied in the following order (from lowest to highest priority):

1. Parameters specified on the command line (`--something value`)
2. Parameters provided using the `-params-file` option
3. Config file specified using the `-c my_config` option
4. The config file named `nextflow.config` in the current directory
5. The config file named `nextflow.config` in the workflow project directory
6. The config file `$HOME/.nextflow/config`
7. Values defined within the pipeline script itself (e.g. `main.nf`)
1. Parameters defined in pipeline scripts (e.g. `main.nf`)
2. The config file `$HOME/.nextflow/config`
3. The config file `nextflow.config` in the project directory
4. The config file `nextflow.config` in the launch directory
5. Config file specified using the `-c <config-file>` option
6. Parameters specified in a params file (`-params-file` option)
7. Parameters specified on the command line (`--something value`)

When more than one of these options for specifying configurations are used, they are merged, so that the settings in the first override the same settings appearing in the second, and so on.

:::{tip}
If you want to ignore any default configuration files and use only a custom one, use `-C <config file>`.
You can use the `-C <config-file>` option to use a single configuration file and ignore all other files.
:::

### Config syntax
Expand Down Expand Up @@ -1386,9 +1386,9 @@ process {
}
```

:::{note}
The `withName` selector applies to a process even when it is included from a module under an alias. For example, `withName: hello` will apply to any process originally defined as `hello`, regardless of whether it is included under an alias. Similarly, it will not apply to any process not originally defined as `hello`, even if it is included under the alias `hello`.
:::
The `withName` selector applies both to processes defined with the same name and processes included under the same alias. For example, `withName: hello` will apply to any process originally defined as `hello`, as well as any process included under the alias `hello`.

Furthermore, selectors for the alias of an included process take priority over selectors for the original name of the process. For example, given a process defined as `foo` and included as `bar`, the selectors `withName: foo` and `withName: bar` will both be applied to the process, with the second selector taking priority over the first.

:::{tip}
Label and process names do not need to be enclosed with quotes, provided the name does not include special characters (`-`, `!`, etc) and is not a keyword or a built-in type identifier. When in doubt, you can enclose the label name or process name with single or double quotes.
Expand Down Expand Up @@ -1427,24 +1427,31 @@ The above configuration snippet sets 2 cpus for the processes annotated with the

#### Selector priority

When mixing generic process configuration and selectors the following priority rules are applied (from lower to higher):
Process configuration settings are applied to a process in the following order (from lowest to highest priority):

1. Process generic configuration.
2. Process specific directive defined in the workflow script.
3. `withLabel` selector definition.
4. `withName` selector definition.
1. Process configuration settings (without a selector)
2. Process directives in the process definition
3. `withLabel` selectors matching any of the process labels
4. `withName` selectors matching the process name
5. `withName` selectors matching the process included alias
6. `withName` selectors matching the process fully qualified name

For example:

```groovy
process {
cpus = 4
withLabel: foo { cpus = 8 }
withName: bar { cpus = 32 }
withName: bar { cpus = 16 }
withName: 'baz:bar' { cpus = 32 }
}
```

Using the above configuration snippet, all workflow processes use 4 cpus if not otherwise specified in the workflow script. Moreover processes annotated with the `foo` label use 8 cpus. Finally the process named `bar` uses 32 cpus.
With the above configuration:
- All processes will use 4 cpus (unless otherwise specified in their process definition).
- Processes annotated with the `foo` label will use 8 cpus.
- Any process named `bar` (or imported as `bar`) will use 16 cpus.
- Any process named `bar` (or imported as `bar`) invoked by a workflow named `baz` with use 32 cpus.

(config-report)=

Expand Down

0 comments on commit 019eb86

Please sign in to comment.