Fix dockerfile for druid image #15264

georgew5656 · 2023-10-26T21:23:10Z

Fixes docker image build issues with apache/druid.

Description

Currently, trying to build the docker image fails with the following error when trying to grab the static bash utils.

#16 0.263 wget: note: TLS certificate validation not implemented
#16 0.297 wget: TLS error from peer (alert code 80): 80
#16 0.297 wget: error getting response: Connection reset by peer

This seems to be caused by a SSL validation issue in busybox (it seems to happen for github.com but not google.com). I thought this might be fixed by moving to busybox 1.35.0, but I got the same error.

To get this working, I moved the wget commands to another setup layer and copied the file into the distroless build.

It seems like it would be better to switch to busybox 1.35.0 anyways (there is a critical CVE: https://nvd.nist.gov/vuln/detail/CVE-2022-48174), although we technically just use the image for the setup tools, not sure if the CVE actually affects us), but the issue with switching to busybox 1.35.0 is that we get the build errors that were described in this pr: #14518

This can be remediated by also upgrading the distroless image to debian12, but there is no java11-debian12 distroless image, only a java17-debian12 image.

As far as I know druid supports java17 as a target now so maybe this would be okay? I can make that change if people think it makes sense.

I tested the new image and it ran fine.

Release note

Fix druid docker image

Key changed/added classes in this PR

distribution/docker/Dockerfile

This PR has:

abhishekagarwal87 · 2023-10-30T07:30:27Z

@hardikbajaj - FYI

distribution/docker/Dockerfile

abhishekagarwal87 · 2023-10-30T07:35:09Z

I am ok moving to Java 17 based image.

petermarshallio

QQ on the need for --platform

petermarshallio · 2023-10-31T12:38:51Z

distribution/docker/Dockerfile

@@ -17,7 +17,7 @@
 # under the License.
 #

-ARG JDK_VERSION=11
+ARG JDK_VERSION=17

 # The platform is explicitly specified as x64 to build the Druid distribution.


Thank you @georgew5656! I was having real issues getting this to work on my M2 ;)

Is --platform still required here?

i don't think this druid image actually supports multiarch images right now, when I go to the druid dockerhub i only see a amd64 image which would explain why it doesn't work on a arm based mac

@abhishekagarwal87 do you know how this image gets built? is it a pipeline somewhere?

it's a bit of black box for me as well. Apache infra jobs builds and pushes the image as far as I know.

What I shoulda said (!) was I managed to build an arm Docker image using your fix - but yeah the official image would be nicer :)

I checked the dockerhub and the official image is being built right now from a dockerhub automated build on the tags in github:

I don't see a way to pass in a target platform for the automated build, but it seems like it'd be possible to have another Dockerfile configured for arm64, and then we could add a new automated build rule that reads from that instead

e.g. what this repo is showing https://github.com/cgiraldo/docker-hello-multiarch

Another option might be to move the docker image building to a github action on tags only, have some action that calls the appropriate buildx command, and pushes a multi-arch image to dockerhub

The automated dockerhub builds seem nice because then we don't have to worry about dockerhub credentials in our GHA stuff.

gianm · 2023-10-31T16:16:40Z

distribution/docker/Dockerfile

-RUN ["/busybox/busybox", "--install", "/bin"]
-
-# Predefined builtin arg, see: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
+FROM alpine as bash-static


What's the new usage of alpine for? Would it make sense to pin this to a specific version so the build is more reproducible?

i used alpine as the layer to copy the files instead of busybox because wget is broken in busybox and alpine is a simple image to run a wget command, i can pin the image tag to something

since busybox is based on alpine i figured this was the best image to get this working without introducing a bunch of other vuln scan stuff

ok, TY for the explanation.

gianm · 2023-10-31T16:18:24Z

distribution/docker/Dockerfile

@@ -17,7 +17,7 @@
 # under the License.
 #

-ARG JDK_VERSION=11
+ARG JDK_VERSION=17

 # The platform is explicitly specified as x64 to build the Druid distribution.


I think we can raise an Infra ticket, or ask in ASF Slack, to adjust how the images get built. (Or at least ask how they get built.) If it's possible to publish ARM images too that would be nice.

gianm · 2023-10-31T16:18:38Z

distribution/docker/Dockerfile

@@ -17,7 +17,7 @@
 # under the License.
 #

-ARG JDK_VERSION=11
+ARG JDK_VERSION=17


I am good with this, and in fact I prefer it 🙂

abhishekagarwal87 · 2023-11-01T04:26:39Z

@LakshSingla - We may need this for 28 to build docker images.

Fixes docker image build issues with apache/druid.

Fix dockerfile

e16da90

georgew5656 requested review from abhishekagarwal87, FrankChen021, kfaraz and suneet-s October 26, 2023 21:23

abhishekagarwal87 reviewed Oct 30, 2023

View reviewed changes

distribution/docker/Dockerfile Show resolved Hide resolved

Move to java17

16ddc1c

georgew5656 requested a review from abhishekagarwal87 October 30, 2023 17:31

petermarshallio reviewed Oct 31, 2023

View reviewed changes

abhishekagarwal87 approved these changes Oct 31, 2023

View reviewed changes

gianm reviewed Oct 31, 2023

View reviewed changes

tag alpine

22e28b3

gianm approved these changes Oct 31, 2023

View reviewed changes

abhishekagarwal87 added the Release Notes label Nov 1, 2023

abhishekagarwal87 merged commit 49e0cba into apache:master Nov 1, 2023
82 checks passed

LakshSingla mentioned this pull request Nov 6, 2023

[Backport] Fix dockerfile for druid image #15331

Merged

LakshSingla pushed a commit to LakshSingla/druid that referenced this pull request Nov 6, 2023

Fix dockerfile for druid image (apache#15264)

d60f7cf

Fixes docker image build issues with apache/druid.

LakshSingla pushed a commit that referenced this pull request Nov 6, 2023

Fix dockerfile for druid image (#15264)

1953aa2

Fixes docker image build issues with apache/druid.

CaseyPan pushed a commit to CaseyPan/druid that referenced this pull request Nov 17, 2023

Fix dockerfile for druid image (apache#15264)

eaf649b

Fixes docker image build issues with apache/druid.

FrankChen021 mentioned this pull request Dec 5, 2023

Docker images: Support for the Java 17 runtime #15011

Closed

LakshSingla added this to the 29.0.0 milestone Jan 29, 2024

LakshSingla mentioned this pull request Feb 13, 2024

[DRAFT] 29.0.0 release notes #15896

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix dockerfile for druid image #15264

Fix dockerfile for druid image #15264

georgew5656 commented Oct 26, 2023 •

edited

Loading

abhishekagarwal87 commented Oct 30, 2023

abhishekagarwal87 commented Oct 30, 2023

petermarshallio left a comment

petermarshallio Oct 31, 2023

georgew5656 Oct 31, 2023 •

edited

Loading

georgew5656 Oct 31, 2023

abhishekagarwal87 Oct 31, 2023

petermarshallio Oct 31, 2023

jon-wei Oct 31, 2023

jon-wei Oct 31, 2023 •

edited

Loading

jon-wei Oct 31, 2023

jon-wei Oct 31, 2023

gianm Oct 31, 2023

gianm Oct 31, 2023

georgew5656 Oct 31, 2023 •

edited

Loading

gianm Oct 31, 2023

gianm Oct 31, 2023

gianm Oct 31, 2023

abhishekagarwal87 commented Nov 1, 2023

Fix dockerfile for druid image #15264

Fix dockerfile for druid image #15264

Conversation

georgew5656 commented Oct 26, 2023 • edited Loading

Description

Release note

Key changed/added classes in this PR

abhishekagarwal87 commented Oct 30, 2023

abhishekagarwal87 commented Oct 30, 2023

petermarshallio left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgew5656 Oct 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jon-wei Oct 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgew5656 Oct 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abhishekagarwal87 commented Nov 1, 2023

georgew5656 commented Oct 26, 2023 •

edited

Loading

georgew5656 Oct 31, 2023 •

edited

Loading

jon-wei Oct 31, 2023 •

edited

Loading

georgew5656 Oct 31, 2023 •

edited

Loading