Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Cannot locate specified Dockerfile when using FromDockerfile #763

Closed
L3AZH opened this issue Jan 16, 2023 · 7 comments · Fixed by #814
Closed

[Bug]: Cannot locate specified Dockerfile when using FromDockerfile #763

L3AZH opened this issue Jan 16, 2023 · 7 comments · Fixed by #814
Labels
bug An issue with the library

Comments

@L3AZH
Copy link

L3AZH commented Jan 16, 2023

Testcontainers version

0.17.0

Using the latest Testcontainers version?

Yes

Host OS

Windows

Host arch

x64

Go version

1.19.4

Docker version

$ docker version
Client:
 Cloud integration: v1.0.29
 Version:           20.10.21
 API version:       1.41
 Go version:        go1.18.7
 Git commit:        baeda1f
 Built:             Tue Oct 25 18:08:16 2022
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.15.0 (93002)
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 18:00:19 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.10
  GitCommit:        770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.13.0)
  dev: Docker Dev Environments (Docker Inc., v0.0.5)
  extension: Manages Docker extensions (Docker Inc., v0.2.16)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.22.0)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 27
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.16.3-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 12.35GiB
 Name: docker-desktop
 ID: XT4C:W237:JTIN:MZ6Z:QKJQ:2PM2:VHQJ:E2KN:H4DU:M6LQ:2UBY:THFF
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

What happened?

Describe the bug

I'm creating a go-service with testcontainer by using FromDockerfile in a test function, but when I run a test, it could not locate the dockerfile
I already run the test function go func Test_BuildContainerFromDockerfile(t *testing.T) in file docker_test.go (in the testcontainer src) but the result still the same as my test

My Test function

package main_test

import (
	"context"
	"go-specs-greet/go_specs_greet"
	"go-specs-greet/specifications"
	"log"
	"os/exec"
	"testing"

	"github.com/alecthomas/assert/v2"
	"github.com/testcontainers/testcontainers-go"
	"github.com/testcontainers/testcontainers-go/wait"
)
func TestGreeterServer(t *testing.T) {

	ctx := context.Background()
	cmd := exec.Command("pwd")
	stdout, err := cmd.Output()
	if err != nil {
		log.Fatalln(err)
	}
	path := string(stdout)
	log.Println(path)
	req := testcontainers.ContainerRequest {
		FromDockerfile: testcontainers.FromDockerfile{
			Context: "./docker",
			Dockerfile: "Dockerfile",
			PrintBuildLog: true,
		},
		ExposedPorts: []string{"8080:8080"},
		WaitingFor: wait.ForHTTP("/").WithPort("8080"),
	}

	log.Println(req.FromDockerfile.Context)
	log.Println(req.FromDockerfile.Dockerfile)

	container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
		ContainerRequest: req,
		Started: true,
	})

	assert.NoError(t, err)
	t.Cleanup(func() {
		assert.NoError(t, container.Terminate(ctx))
	})

	driver := go_specs_greet.Driver{BaseURL: "http://localhost:8080"}
	specifications.GreetSpecification(t, driver)
}

The output of my test

$ go test greeter_server_test.go 
2023/01/17 02:37:01 /d/CODE/<my_proj_path>/Scaling acceptance tests/cmd/httpserver

2023/01/17 02:37:01 ./docker
2023/01/17 02:37:01 Dockerfile
2023/01/17 02:37:01 github.com/testcontainers/testcontainers-go - Connected to docker:
  Server Version: 20.10.21
  API Version: 1.41
  Operating System: Docker Desktop
  Total Memory: 12648 MB
2023/01/17 02:37:01 Starting container id: b015bbb76941 image: docker.io/testcontainers/ryuk:0.3.4
2023/01/17 02:37:02 Waiting for container id b015bbb76941 image: docker.io/testcontainers/ryuk:0.3.4
2023/01/17 02:37:02 Container is ready id: b015bbb76941 image: docker.io/testcontainers/ryuk:0.3.4
&{0xc00008cae0}
Error Happended
--- FAIL: TestGreeterServer (0.80s)
    greeter_server_test.go:43: Did not expect an error but got:
        Error response from daemon: Cannot locate specified Dockerfile: Dockerfile: failed to create container
FAIL
FAIL    command-line-arguments  1.182s
FAIL

The test function go func Test_BuildContainerFromDockerfile(t *testing.T) in docker_test.go

func Test_BuildContainerFromDockerfile(t *testing.T) {
	t.Log("getting context")
	ctx := context.Background()
	t.Log("got context, creating container request")
	req := ContainerRequest{
		FromDockerfile: FromDockerfile{
			Context: "./testresources",
		},
		ExposedPorts: []string{"6379/tcp"},
		WaitingFor:   wait.ForLog("Ready to accept connections"),
	}

	redisC, err := prepareRedisImage(ctx, req, t)
	require.NoError(t, err)
	terminateContainerOnEnd(t, ctx, redisC)
}

The output when I run a test function in docker_test.go

Running tool: C:\Program Files\Go\bin\go.exe test -timeout 30s -run ^Test_BuildContainerFromDockerfile$ github.com/testcontainers/testcontainers-go

2023/01/17 02:55:17 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 20.10.21
  API Version: 1.41
  Operating System: Docker Desktop
  Total Memory: 12648 MB
2023/01/17 02:55:17 Starting container id: ff64537c3741 image: docker.io/testcontainers/ryuk:0.3.4
2023/01/17 02:55:18 Waiting for container id ff64537c3741 image: docker.io/testcontainers/ryuk:0.3.4
2023/01/17 02:55:18 Container is ready id: ff64537c3741 image: docker.io/testcontainers/ryuk:0.3.4
&{0xc0000d48a0}
Error Happended
--- FAIL: Test_BuildContainerFromDockerfile (0.81s)
    c:\Users\<user_name>\go\pkg\mod\github.com\testcontainers\[email protected]\docker_test.go:1066: getting context
    c:\Users\<user_name>\go\pkg\mod\github.com\testcontainers\[email protected]\docker_test.go:1068: got context, creating container request
    c:\Users\<user_name>\go\pkg\mod\github.com\testcontainers\[email protected]\docker_test.go:1201: creating redis container
    c:\Users\<user_name>\go\pkg\mod\github.com\testcontainers\[email protected]\docker_test.go:1205: created redis container
    c:\Users\<user_name>\go\pkg\mod\github.com\testcontainers\[email protected]\docker_test.go:1078: 
        	Error Trace:	c:\Users\<user_name>\go\pkg\mod\github.com\testcontainers\[email protected]\docker_test.go:1078
        	Error:      	Received unexpected error:
        	            	Error response from daemon: Cannot locate specified Dockerfile: Dockerfile: failed to create container
        	Test:       	Test_BuildContainerFromDockerfile
FAIL
FAIL	github.com/testcontainers/testcontainers-go	1.199s
FAIL

Relevant log output

No response

Additional information

This is my project
Scaling acceptance tests.zip

@L3AZH L3AZH added the bug An issue with the library label Jan 16, 2023
@L3AZH L3AZH changed the title [Bug]: Cannot locate specified Dockerfile when [Bug]: Cannot locate specified Dockerfile when using FromDockerfile Jan 16, 2023
@mdelapenya
Copy link
Member

Hi @L3AZH I'm able to reproduce the error you describe in this issue, but I think it's not caused by the library itself: in the ZIP file provided above, the Dockerfile lives in a directory with no access to the go.mod file (see line 3 in the Dockerfile).

If from the cmd/httpserver dir you run docker build -t image . then it will also fail, so my first guess is that it tries to build the image, but fails because of the Dockerfile and Docker build context is incorrect.

I'd suggest trying a structured format that allows building the image.

@L3AZH
Copy link
Author

L3AZH commented Feb 11, 2023

Hi @mdelapenya,
Turn out it happened as you said but it still having the same problem
So I moved the Dockerfile to the same level dir with go.mod and edited some lines in.
My structure is like this:

\Scaling acceptance tests
   \cmd\httpserver
       greeter_server_test.go
   \go_specs_greet
       ....
   \specifications
      ....
   Dockerfile
   go.mod
   ...
   main.go

I'm able to run docker build -t testimage . && docker container run -p 8080 -d --name testcontainer testimage with out error.
But when I run the cmd go test cmd/httpserver/greeter_server_test.go . the log still has the same message i describe in this issue.
Do you have any idea about this? i also attached the zip file of my project below:
Scaling acceptance tests.zip

My Dockerfile after edit:

FROM golang:1.19-alpine
RUN mkdir /root/scaling-acceptance-test
WORKDIR /root/scaling-acceptance-test
COPY . .
RUN go mod download
RUN go build -o  /root/scaling-acceptance-test/svr
EXPOSE 8080

CMD ["/root/scaling-acceptance-test/svr"]

My greeter_server_test.go

package main_test

import (
	"context"
	"go-specs-greet/go_specs_greet"
	"go-specs-greet/specifications"
	"log"
	"os/exec"
	"testing"

	"github.com/alecthomas/assert/v2"
	"github.com/testcontainers/testcontainers-go"
	"github.com/testcontainers/testcontainers-go/wait"
)
func TestGreeterServer(t *testing.T) {

	ctx := context.Background()
	cmd := exec.Command("pwd")
	stdout, err := cmd.Output()
	if err != nil {
		log.Fatalln(err)
	}
	path := string(stdout)
	log.Println(path)
	req := testcontainers.ContainerRequest {
		FromDockerfile: testcontainers.FromDockerfile{
			Context: "../../.",
			Dockerfile: "Dockerfile",
			PrintBuildLog: true,
		},
		ExposedPorts: []string{"8080:8080"},
		WaitingFor: wait.ForHTTP("/").WithPort("8080"),
	}

	log.Println(req.FromDockerfile.Context)
	log.Println(req.FromDockerfile.Dockerfile)

	container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
		ContainerRequest: req,
		Started: true,
	})

	assert.NoError(t, err)
	t.Cleanup(func() {
		assert.NoError(t, container.Terminate(ctx))
	})

	driver := go_specs_greet.Driver{BaseURL: "http://localhost:8080"}
	specifications.GreetSpecification(t, driver)
}

@mdelapenya
Copy link
Member

Simply downloading your ZIP file and running the tests I get this:

go test ./... -v -count=1      
=== RUN   TestGreeterServer2
2023/02/11 19:26:35 /tmp/scaling

2023/02/11 19:26:35 .
2023/02/11 19:26:35 ./Dockerfile
2023/02/11 19:26:35 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 20.10.21
  API Version: 1.41
  Operating System: Docker Desktop
  Total Memory: 7851 MB
2023/02/11 19:26:35 Starting container id: b0b24dfb0864 image: docker.io/testcontainers/ryuk:0.3.4
2023/02/11 19:26:35 Waiting for container id b0b24dfb0864 image: docker.io/testcontainers/ryuk:0.3.4
2023/02/11 19:26:35 Container is ready id: b0b24dfb0864 image: docker.io/testcontainers/ryuk:0.3.4
Step 1/8 : FROM golang:1.19-alpine
 ---> 25741ae7796d
Step 2/8 : RUN mkdir /root/scaling-acceptance-test
 ---> Running in 62c541db4153
Removing intermediate container 62c541db4153
 ---> 7ca8cb5b2f8b
Step 3/8 : WORKDIR /root/scaling-acceptance-test
 ---> Running in 84d66839342b
Removing intermediate container 84d66839342b
 ---> 2dc6447e79ba
Step 4/8 : COPY . .
 ---> cc353304809f
Step 5/8 : RUN go mod download
 ---> Running in 188b33c1e7c4
Removing intermediate container 188b33c1e7c4
 ---> dfdc647d4f17
Step 6/8 : RUN go build -o  /root/scaling-acceptance-test/svr
 ---> Running in f80f48fa06ae
Removing intermediate container f80f48fa06ae
 ---> b9971c42dc69
Step 7/8 : EXPOSE 8080
 ---> Running in 8e178a088bc2
Removing intermediate container 8e178a088bc2
 ---> 1eb6fd30b488
Step 8/8 : CMD ["/root/scaling-acceptance-test/svr"]
 ---> Running in 6a1ab429a581
Removing intermediate container 6a1ab429a581
 ---> bf4f4f40c648
Successfully built bf4f4f40c648
Successfully tagged 34e5d424-5493-44a0-aa93-6204cd5591f9:1c581795-b2e3-4c05-a724-a0ba341a78e7
2023/02/11 19:26:53 Starting container id: ec82634027e5 image: 34e5d424-5493-44a0-aa93-6204cd5591f9:1c581795-b2e3-4c05-a724-a0ba341a78e7
    filte_test.go:44: Did not expect an error but got:
        Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:8080 -> 0.0.0.0:0: listen tcp 0.0.0.0:8080: bind: address already in use: failed to start container
--- FAIL: TestGreeterServer2 (18.35s)
FAIL
FAIL    go-specs-greet  18.788s
=== RUN   TestGreeterServer
2023/02/11 19:26:35 /tmp/scaling/cmd/httpserver

2023/02/11 19:26:35 ../../.
2023/02/11 19:26:35 Dockerfile
2023/02/11 19:26:35 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 20.10.21
  API Version: 1.41
  Operating System: Docker Desktop
  Total Memory: 7851 MB
2023/02/11 19:26:35 Starting container id: 23cfd892bef4 image: docker.io/testcontainers/ryuk:0.3.4
2023/02/11 19:26:35 Waiting for container id 23cfd892bef4 image: docker.io/testcontainers/ryuk:0.3.4
2023/02/11 19:26:35 Container is ready id: 23cfd892bef4 image: docker.io/testcontainers/ryuk:0.3.4
Step 1/8 : FROM golang:1.19-alpine
 ---> 25741ae7796d
Step 2/8 : RUN mkdir /root/scaling-acceptance-test
 ---> Running in 590fa988fecd
Removing intermediate container 590fa988fecd
 ---> 912e179af0a7
Step 3/8 : WORKDIR /root/scaling-acceptance-test
 ---> Running in 951cf31907c2
Removing intermediate container 951cf31907c2
 ---> ae3ea9b6c540
Step 4/8 : COPY . .
 ---> 7c4022306f9b
Step 5/8 : RUN go mod download
 ---> Running in 969d11ce09ef
Removing intermediate container 969d11ce09ef
 ---> 3b471543ed5a
Step 6/8 : RUN go build -o  /root/scaling-acceptance-test/svr
 ---> Running in c0189fb30e83
Removing intermediate container c0189fb30e83
 ---> 0e048bc4915f
Step 7/8 : EXPOSE 8080
 ---> Running in affe2cd1c14a
Removing intermediate container affe2cd1c14a
 ---> f036c5257443
Step 8/8 : CMD ["/root/scaling-acceptance-test/svr"]
 ---> Running in 566943157c6a
Removing intermediate container 566943157c6a
 ---> 657237f87306
Successfully built 657237f87306
Successfully tagged d7f8e4f7-0c41-4c22-acd9-bd0a129e471b:3d8e869a-7a8e-4469-a6c5-db184dc6d742
2023/02/11 19:26:53 Starting container id: 5737240a31d5 image: d7f8e4f7-0c41-4c22-acd9-bd0a129e471b:3d8e869a-7a8e-4469-a6c5-db184dc6d742
2023/02/11 19:26:53 Waiting for container id 5737240a31d5 image: d7f8e4f7-0c41-4c22-acd9-bd0a129e471b:3d8e869a-7a8e-4469-a6c5-db184dc6d742
2023/02/11 19:26:53 Container is ready id: 5737240a31d5 image: d7f8e4f7-0c41-4c22-acd9-bd0a129e471b:3d8e869a-7a8e-4469-a6c5-db184dc6d742
    greet.go:16: Expected values to be equal:
        +Hello, World
        \ No newline at end of file
--- FAIL: TestGreeterServer (18.96s)
FAIL
FAIL    go-specs-greet/cmd/httpserver   19.248s
?       go-specs-greet/go_specs_greet   [no test files]
?       go-specs-greet/specifications   [no test files]
FAIL

So it seems to build the Docker image but the assertions fail.

@L3AZH
Copy link
Author

L3AZH commented Feb 11, 2023

So I just do the same as you,
I downloaded the zip file to another machine with Linux os and run the test and the log print out the same as you, so I guess the docker service in my window os has some problem.
Thanks man

@L3AZH L3AZH closed this as completed Feb 11, 2023
@mdelapenya
Copy link
Member

So I just do the same as you, I downloaded the zip file to another machine with Linux os and run the test and the log print out the same as you, so I guess the docker service in my window os has some problem. Thanks man

Mmm, so it happens on a Windows machine? Let me check on that platform, but it will be next Monday 🙏 I'm reopening the issue just until we confirm it's not a Windows issue

@mdelapenya
Copy link
Member

@L3AZH good news! We discovered a weird behaviour in how Docker handles absolute Vs relative paths when tarring string representations for paths on Windows. We have fixed it in #814, always passing the absolute path to Docker.

On other hand, we had a hard time understanding the layout of the project, as it's not very common to see the Dockerfile copying files from . but then locating it in a subdir. I know this is not related to the original issue, but having a more or less standard layout will help you in discarding layout issues from library issues. In this case, we moved the Dockerfile to the root of the project so that it builds the HTTP server living the in the main.go file. Then the tests should live next to the code it tests, so probably you need to move the _test.go file to the root dir too.

In any case, thanks for reaching out with this issue, it helped us to trigger an internal discussion on how Docker works under the hood.

Cheers!

@L3AZH
Copy link
Author

L3AZH commented Feb 15, 2023

Hi @mdelapeny, it’s good to know that, thank you for supporting me with the problem 👍
Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug An issue with the library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants