setuptools_scm, include version in omero metadata

[manics]

Follow up from Slack.... an example of how to add omero-cli-zarr version information into the metadata.

Obviously loads of questions like what do we name the field, do we have separate name/version fields, where in the metadata hierarchy does it belong.

Note with setuptools_scm the version is only updated when you run python setup.py ... or pip install .... For example this means if you use pip install -e . and subsequently make a commit the version won't be updated.

[sbesson]

Similarly to ome/ome-model#123, 👍 for enabling version introspection and 1: for starting to introduce some provenance metadata in the generated Zarr.

Only thought is that this information could be moved to the the top-level dictionary as it's not specifically bound to multiscales or omero specification. We might need to dedicate more time and think how to specify this information in which case getting an internal variable is probably a good first step.

[manics]

We might want it at multiple levels? Your image might have one creator, and the masks another.

[manics]

The other thing to be aware of with setuptools_scm is that you can't pip install a plain archive from github e.g. pip install https://github.com/ome/omero-cli-zarr/archive/master.zip won't work, you'll have to use pip install git+https://github.com/ome/omero-cli-zarr

[manics]

$ cat 1051.zarr/.zattrs
{
    "_creator": {
        "name": "omero-zarr",
        "version": "0.0.2.dev31+g83c63af"
    },
    "plate": {
        "columns": 2,
        "rows": 1
    }
}

+g83c63af in the version corresponds to the last commit on this branch 83c63af

[manics]

Rebased

[sbesson]

Successfully tested by converting the first plate of idr0004

[sbesson@ome-zarr-dev1 ~]$ head -n 6  /uod/idr/objectstore/minio/idr/idr0004-thorpe-rad52/1751.zarr/.zattrs 
{
    "_creator": {
        "name": "omero-zarr",
        "version": "0.0.5.dev4+geb0700c"
    },
    "plate": {

From the PR, I was naively expecting the creator field to who up at the image levels as well but I realized that plate_to_zarr does not call image_to_zarr so this is expected (although I could imagine refactoring).

Overall, same feeling as discussed previously about the usage of setuptools_scm, I like the versioning features a lot but am still undecided about its main limitations: no source archive installation, no tags on forks.

For this repository, I think it's acceptable and we can use it as a learning experience but more thoughts might be required for propagating it across the board.

[sbesson]

Actually adding https://github.com/ome/omero-cli-zarr/runs/1321167390 to my list of high-level concerns with setuptools_scm i.e. the publication of master to Test PyPI failed. Definitely less important than failing the publication of tags on production PyPI but the underlying problem is that the versions generated by setuptools_scm are not automatically PEP 440 compatible.

[manics]

Without setuptools scm wouldn't you still have a problem with deploying master since you'll end up overwriting the previous dev version, which is presumably blocked by pypi?

[sbesson]

Absolutely, I realized (remembered) the continuous delivery workflow of the master branch is not possible which was the reason for ec0bebd.

Maybe said another way, with setuptools_scm, you cannot only publish PEP 440 compliant tags. For production releases and pre-releases, it makes complete sense and does not change the current paradigm. It might affect require more infrastructure if we wanted to push development versions to a CI Nexus.