-
Notifications
You must be signed in to change notification settings - Fork 0
Linux: Docker vs. Host System speed test
Is ANTsR slower when running in docker compared to the host system itself? Let's test it on a linux machine
The test was conducted on a dual-CPU workstation with CentOS 7.
The host system used an installation of R3.5.0 with RStudio Desktop v1.1.456.
The docker container has R3.6.1 installed with RStudio Server accessed through the host's Firefox browser.
10 runs of the same LESYMAP analysis were run, and the elapsed time was recorded for each run:
lesydata = file.path(find.package('LESYMAP'),'extdata')
filenames = Sys.glob(file.path(lesydata, 'lesions', 'Subject*.nii.gz'))
behavior = Sys.glob(file.path(lesydata, 'behavior', 'behavior.txt'))
runtime = rep(NA,10)
for (i in 1:10) runtime[i] = system.time(lesymap(filenames, behavior, method = 'BMfast'))['elapsed']
After the test finished, the runtime values were exported with dput(runtime).
Values indicate seconds.
# host system
runtimeHOST = c(59.5020000000004, 63.585, 62.1729999999998, 57.6509999999998, 61.9340000000002, 58.413, 59.5209999999997, 64.0099999999998, 58.297, 58.3209999999999)
# docker container
runtimeDOCKER = c(55.996, 62.935, 56.132, 63.205, 55.942, 57.584, 55.4350000000001, 61.788, 55.835, 54.139)
> data = data.frame(docker=runtimeDOCKER, host=runtimeHOST)
> summary(data)
docker host
Min. :54.14 Min. :57.65
1st Qu.:55.86 1st Qu.:58.34
Median :56.06 Median :59.51
Mean :57.90 Mean :60.34
3rd Qu.:60.74 3rd Qu.:62.11
Max. :63.20 Max. :64.01
> slower = mean(data$docker - data$host)/mean(data$host) * 100
> print(paste0('Docker is ', round(slower,0), '% slower than the host system.'))
[1] "Docker is -4% slower than the host system solution."
The host and docker runs have almost identical performance. Let's run a t-test to check the difference statistically:
# running t-test to see if the two solutions have different performances.
> t.test(data$docker, data$host)
Welch Two Sample t-test
data: data$docker and data$host
t = -1.865, df = 16.079, p-value = 0.08053
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-5.2157586 0.3325586
sample estimates:
mean of x mean of y
57.8991 60.3407
Docker is as fast as a linux host system when running ANTsR. Docker even shows a slight performance advantage, but this difference is marginal and might be related to the different R versions (3.5.0 in host and 3.6.1 in docker). These results encourage the use of docker images routinely in linux to run ANTsR instances. Similar findings of equivalent performance are reported online: https://stackoverflow.com/questions/21889053/what-is-the-runtime-performance-cost-of-a-docker-container
Docker version 19.03.4-rc1, build 4ba2b1c606
R v3.6.1
package * version date lib source
ANTsR 0.5.4.1 2019-11-07 [1] Github (ANTsX/ANTsR@68123f3)
ANTsRCore 0.7.3 2019-11-08 [1] Github (ANTsX/ANTsRCore@43d6341)
ITKR 0.5.2 2019-11-07 [1] Github (stnava/ITKR@fe97efa)
LESYMAP 0.0.0.9221 2019-11-08 [1] Github (dorianps/LESYMAP@ac22136)
Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz
CentOS Linux release 7.7.1908 (Core)
R v3.5.0
package * version date lib source
ANTsR 0.5.4.1 2019-11-07 [1] Github (ANTsX/ANTsR@68123f3)
ANTsRCore 0.7.3 2019-11-08 [1] Github (ANTsX/ANTsRCore@43d6341)
ITKR 0.5.2 2019-11-07 [1] Github (stnava/ITKR@fe97efa)
LESYMAP 0.0.0.9221 2019-11-08 [1] Github (dorianps/LESYMAP@ac22136)