Poetry: use locked environment in Docker images

[DMRobertson]

Pulled out of #12337.

At present, needs matrix-org/complement#337.

[DMRobertson]

I've transferred over Rich's outstanding comments from #12337.

[DMRobertson]

@richvdh asked:

I think using ENV here will mean we end up with these env vars exported in the final image? Are you sure that's what we want?

My understanding was that these just apply to the builder stage because we haven't yet begun to define the final stage.

[richvdh]

right. I've not investigated in practice how this works for initial stages of Dockerfiles, but what I do know is that if you set an ENV in the final stage, you'll also get that env var in the final container. See https://docs.docker.com/engine/reference/builder/#env:

The environment variables set using ENV will persist when a container is run from the resulting image.

and:

If an environment variable is only needed during build, and not in the final image, consider setting a value for a single command instead

My instinct here, irrespective of whether it does the right thing in practice, is to avoid the ambiguity by avoiding ENV.

[DMRobertson]

I think I was allowing myself to be a little lazy since I was invoking poetry twice.

Now that we're able to export requirements.txt (92c9a60) this isn't needed.

For context: originally poetry export produced a requirements.txt file which omitted hashes for certain dependencies. That's a poetry bug. We were able to workaround this by adding a missing metadata dependency to Synapse (#12384) and tightening a signedjson dependency (matrix-org/python-signedjson#9). We only found the workaround last week, and I hadn't gone back to make use of requirements.txt here (if it ain't broke...).

[DMRobertson]

@richvdh wondered:

/me wonders how many people rely on things being installed in /usr/local.

Certainly this is going to break anyone who builds docker images that install extension modules.

It might be fine to copy the venv to /usr/local, but I wasn't certain; this change is me erring towards caution. See this comment thread.

[DMRobertson]

I hadn't considered that people were building on top of our docker images. A quick search fins a few.

I couldn't find an equivalent to pip install --prefix in poetry last time I looked. But this comment python-poetry/poetry#234 (comment) might be useful.

[DMRobertson]

If we could dump everything back in /usr/local/ then we wouldn't need matrix-org/complement#337.

[richvdh]

I hadn't considered that people were building on top of our docker images. A quick search fins a few.

and I'll bet there are a whole bunch more that aren't on (public) github.

Copying a virtualenv wholesale into /usr/local doesn't sound like a thing that will work to me. If nothing else there will be a bunch of hardcoded paths which would need updating.

Ultimately, I think our best approach here is just going to be to warn people loudly that this is a change we are making. Which sounds to me like we should also get our story straight about how people are meant to install extensions (#9944) to avoid breaking things twice for people.

To decouple this... could we do the same thing as the debian build, and hack around with poetry export and pip install -r ?

[DMRobertson]

Complement CI is now failing because generate_signing_key isn't present in /usr/local/bin for some reason.

Looking at the container build logs, it looks like we end up building synapse version 0.0.0.

[DMRobertson]

Complement isn't happy and I don't know what's gone wrong with the runtime dep checks. I will have to go back to the drawing board.

[richvdh]

lgtm, assuming it gives a working docker image?

(If you're not familiar with the docker image, you can give it a quick once-over with something like:

mkdir tmp
docker run --rm -it -v `pwd`/tmp:/data -e SYNAPSE_SERVER_NAME=localhost -e SYNAPSE_REPORT_STATS=yes matrixdotorg/synapse:v$ver generate
docker run --rm -it -v `pwd`/tmp:/data -p 18008:8008 matrixdotorg/synapse:v$ver

... and then visit http://localhost:18008/. If you've got a running Synapse, chances are that everything else works.)

[DMRobertson]

lgtm, assuming it gives a working docker image?

I was mainly relying on complement here, but I've manually built the container (changing the tag from matrix-org/synapse:v$ver to a local one) and all seems well.