Docker container name invalid on Mac when running with '-with-docker' flag

[arielromeo]

Bug report

Expected behavior and actual behavior

Actual behavior when running with '-with-docker' flag is a failure because the docker name, which is stored in the variable 'NXF_BOXID' contains a carry return character '\r'.

The expected behavior is one such docker name is generated, it removes all the non-printable characters making a valid name.

Steps to reproduce the problem

This issue happens on MacOS Monterrey.

  Version: 21.10.5 build 5658
  Created: 08-12-2021 14:39 UTC (11:39 ARST)
  System: Mac OS X 12.0.1
  Runtime: Groovy 3.0.9 on Java HotSpot(TM) 64-Bit Server VM 17.0.1+12-LTS-39
  Encoding: UTF-8 (UTF-8)
  Process: 49432@ariel [192.168.54.143]
  CPUs: 8 - Mem: 16 GB (25.2 MB) - Swap: 6 GB (1.3 GB)

1. Create a nf.cfg file with the following content:

docker {
    enabled = true
}

process {
    container = 'ubuntu:20.04'
    errorStrategy = 'retry'
    maxRetries = 1
}

2. Create a file calle test1.nf with the following content:

prot1_ch = Channel.from(1)

prot2_ch = Channel.from(2)


process foo {

  input:
    val x from prot1_ch

  echo = true

  script:
    "echo $x && uname -a"
}


process bar {

  input:
    val x from prot2_ch

  echo = true

  script:
    "echo $x && uname -a "
}

3. Execute that with this command:

nextflow run -c nf.cfg test1.nf -with-docker

Program output

dic.-27 17:49:00.248 [main] DEBUG nextflow.cli.Launcher - $> nextflow run -c nf.cfg test1.nf -with-docker
dic.-27 17:49:00.461 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 21.10.5
dic.-27 17:49:00.519 [main] INFO  nextflow.cli.CmdRun - Launching `test1.nf` [special_ride] - revision: 354557fae3
dic.-27 17:49:00.543 [main] DEBUG nextflow.config.ConfigBuilder - User config file: /Users/ariel/airflow/nf.cfg
dic.-27 17:49:00.544 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /Users/ariel/airflow/nf.cfg
dic.-27 17:49:00.582 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
dic.-27 17:49:01.240 [main] DEBUG nextflow.config.ConfigBuilder - Enabling execution in Docker container as requested by cli option `-with-docker -`
dic.-27 17:49:01.256 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/Users/ariel/.nextflow/plugins
dic.-27 17:49:01.258 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
dic.-27 17:49:01.269 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
dic.-27 17:49:01.270 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
dic.-27 17:49:01.274 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
dic.-27 17:49:01.321 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
dic.-27 17:49:01.403 [main] DEBUG nextflow.Session - Session uuid: ba8875c5-5f0e-45f8-9001-67e750280077
dic.-27 17:49:01.404 [main] DEBUG nextflow.Session - Run name: special_ride
dic.-27 17:49:01.404 [main] DEBUG nextflow.Session - Executor pool size: 8
dic.-27 17:49:01.447 [main] DEBUG nextflow.cli.CmdRun -
  Version: 21.10.5 build 5658
  Created: 08-12-2021 14:39 UTC (11:39 ARST)
  System: Mac OS X 12.0.1
  Runtime: Groovy 3.0.9 on Java HotSpot(TM) 64-Bit Server VM 17.0.1+12-LTS-39
  Encoding: UTF-8 (UTF-8)
  Process: 49936@ariel [192.168.54.143]
  CPUs: 8 - Mem: 16 GB (69.3 MB) - Swap: 6 GB (1.1 GB)
dic.-27 17:49:01.466 [main] DEBUG nextflow.Session - Work-dir: /Users/ariel/airflow/work [Mac OS X]
dic.-27 17:49:01.467 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /Users/ariel/airflow/bin
dic.-27 17:49:01.481 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
dic.-27 17:49:01.496 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
dic.-27 17:49:01.546 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 9; maxThreads: 1000
dic.-27 17:49:01.718 [main] DEBUG nextflow.Session - Session start invoked
dic.-27 17:49:02.120 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
dic.-27 17:49:02.228 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
dic.-27 17:49:02.229 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic.-27 17:49:02.237 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
dic.-27 17:49:02.243 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=8; memory=16 GB; capacity=8; pollInterval=100ms; dumpInterval=5m
dic.-27 17:49:02.264 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 9; maxThreads: 1000
dic.-27 17:49:02.342 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
dic.-27 17:49:02.342 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic.-27 17:49:02.346 [main] DEBUG nextflow.Session - Workflow process names [dsl1]: bar, foo
dic.-27 17:49:02.346 [main] DEBUG nextflow.script.ScriptRunner - > Await termination
dic.-27 17:49:02.347 [main] DEBUG nextflow.Session - Session await
dic.-27 17:49:02.577 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
dic.-27 17:49:02.585 [Task submitter] INFO  nextflow.Session - [6f/6582af] Submitted process > bar (1)
dic.-27 17:49:02.628 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
dic.-27 17:49:02.629 [Task submitter] INFO  nextflow.Session - [f2/f0ff77] Submitted process > foo (1)
dic.-27 17:49:03.590 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: foo (1); status: COMPLETED; exit: 125; error: -; workDir: /Users/ariel/airflow/work/f2/f0ff7782cfbb732008f1c94226c6bc]
dic.-27 17:49:03.608 [Task monitor] INFO  nextflow.processor.TaskProcessor - [f2/f0ff77] NOTE: Process `foo (1)` terminated with an error exit status (125) -- Execution is retried (1)
dic.-27 17:49:03.626 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
dic.-27 17:49:03.627 [Task submitter] INFO  nextflow.Session - [92/20dac8] Re-submitted process > foo (1)
dic.-27 17:49:03.683 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: bar (1); status: COMPLETED; exit: 125; error: -; workDir: /Users/ariel/airflow/work/6f/6582af2817c051eab3a4909dd08f69]
dic.-27 17:49:03.686 [Task monitor] INFO  nextflow.processor.TaskProcessor - [6f/6582af] NOTE: Process `bar (1)` terminated with an error exit status (125) -- Execution is retried (1)
dic.-27 17:49:03.729 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
dic.-27 17:49:03.730 [Task submitter] INFO  nextflow.Session - [e2/0a30d4] Re-submitted process > bar (1)
dic.-27 17:49:04.578 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: foo (1); status: COMPLETED; exit: 125; error: -; workDir: /Users/ariel/airflow/work/92/20dac83add4bfccb8aabf015318290]
dic.-27 17:49:04.616 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'foo (1)'

Caused by:
  Process `foo (1)` terminated with an error exit status (125)

Command executed:

  echo 1 && uname -a

Command exit status:
  125

Command output:
  (empty)

Command error:
  docker: Error response from daemon: Invalid container name (nxf-hla0yvzPAGnYSt9TXaIbhzkI
  ), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed.
  See 'docker run --help'.

Work dir:
  /Users/ariel/airflow/work/92/20dac83add4bfccb8aabf015318290

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
dic.-27 17:49:04.626 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `foo (1)` terminated with an error exit status (125)
dic.-27 17:49:04.653 [main] DEBUG nextflow.Session - Session await > all process finished
dic.-27 17:49:04.654 [main] DEBUG nextflow.Session - Session await > all barriers passed
dic.-27 17:49:04.655 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: bar (1); status: COMPLETED; exit: 125; error: -; workDir: /Users/ariel/airflow/work/e2/0a30d4c9777c7005bd071d63495cad]
dic.-27 17:49:04.664 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=4; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=2; abortedCount=0; succeedDuration=0ms; failedDuration=3.7s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=2; peakCpus=2; peakMemory=0; ]
dic.-27 17:49:04.847 [main] DEBUG nextflow.CacheDB - Closing CacheDB done
dic.-27 17:49:04.885 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Environment

• Nextflow version: 21.10.5
• Java version: 17.0.1
• Operating system: macOS Monterrey
• Bash version: zsh 5.8 (x86_64-apple-darwin21.0)

Additional context

I've chased this down to

nextflow/modules/nextflow/src/main/groovy/nextflow/executor/BashWrapperBuilder.groovy

Line 220 in ba22bdb

binding.container_boxid = 'export NXF_BOXID="nxf-$(dd bs=18 count=1 if=/dev/urandom 2>/dev/null | base64 | tr +/ 0A)"'

where the variable NXF_BOXID gets appended with a '\r' character which breaks the docker run command.
The presence of that char, can be seen in the output copied above:

Command error:
  docker: Error response from daemon: Invalid container name (nxf-hla0yvzPAGnYSt9TXaIbhzkI
  ), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed.
  See 'docker run --help'.

[mrtamm]

I can confirm this issue is also present in macOS Ventura.

As correctly hinted by @arielromeo, the problem can be fixed by deleting line-break symbols from the generated base64-string using tr -d '\r\n':

binding.container_boxid = 'export NXF_BOXID="nxf-$(dd bs=18 count=1 if=/dev/urandom 2>/dev/null | base64 | tr +/ 0A | tr -d \'\\r\\n\')"'

(I could test this only by directly modifying the generated but failing .command.run script.)

The line-break symbols are added by the base64 command, which in my case is following version:

base64 1.5
Last revised: 10th June 2007
The latest version is always available
at http://www.fourmilab.ch/webtools/base64

For the context of this error, here is also the output from nextflow info:

  Version: 23.04.3 build 5875
  Created: 11-08-2023 18:37 UTC (21:37 EEST)
  System: Mac OS X 13.4.1
  Runtime: Groovy 3.0.16 on OpenJDK 64-Bit Server VM 20.0.1
  Encoding: UTF-8 (UTF-8)

And bash --version:

GNU bash, version 5.2.15(1)-release (aarch64-apple-darwin22.1.0)

[bentsherman]

I created a PR #4334 based on the above suggestion. I can't verify it because I don't use macOS, so if someone on this thread could test the PR that would be appreciated.

[mrtamm]

I built the nextflow software based on the branch 2528-fix-docker-invalid-name-macos and here is the output from ./launch.sh info:

  Version: 23.09.1-edge build 5883
  Created: 22-09-2023 06:16 UTC (09:16 EEST)
  System: Mac OS X 13.4.1
  Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 17.0.8.1+0
  Encoding: UTF-8 (UTF-8)

Now I was able to successfully execute a simple workflow using Docker. 👍

---

nextflow.config:

process.container = 'ubuntu:latest'
docker.enabled = true

script.nf:

process TEST {
   script:
   """
   hostname
   """
}

workflow {
    TEST()
}

Execution:

$ ./launch.sh run script.nf 
N E X T F L O W  ~  version 23.09.1-edge
Launching `script.nf` [voluminous_joliot] DSL2 - revision: 2488ac0d0b
executor >  local (1)
[26/31a7b9] process > TEST [100%] 1 of 1 ✔

⚠️ I assume this change did not break Docker-based flow execution in the Linux environment.

Thank you! 👏