Possibility to set the package version dynamically

[vorner]

Maintainer notes

• For one approach to tracking a marketing version, see Possibility to set the package version dynamically #6583 (comment)
• For a CLI to edit package.version, see https://github.com/killercup/cargo-edit
• For dynamic version design considerations, see Possibility to set the package version dynamically #6583 (comment)
  • Possibility to set the package version dynamically #6583 (comment)

---

Hello

I'm building a rust application using internal teamcity CI job. I want the build number to be part of the version of the binary (so eg. ./app --version knows from which build it came).

The only way I found so far is to let the build first edit the Cargo.toml (sed -i -e 's/^version = .*/version = "%build.version%"/' Cargo.toml), which seems ugly and fragile.

It would be great if I could somehow override the version from Cargo.toml through the command line ‒ either as a parameter or an environment variable.

Would something like that make sense? Are there plans to support it?

[dwijnand]

I think this could be created as a cargo extension using https://github.com/ordian/toml_edit. You could use https://github.com/killercup/cargo-edit as inspiration.

[vorner]

Actually, I don't want to modify the Cargo.toml at all if possible, just use that different value of version during a single cargo build. So something like cargo build --config package.version="0.1.2".

But yes, you're right, there are certainly less fragile ways to edit Cargo.toml.

[dwijnand]

I think there's a ticket for being able to modify options (I guess I'm both Cargo.toml and .cargo/config) via a command, like git config, but I don't know if it extends to temporary overrides.

[mixalturek]

It's very common use case to specify the version from command line while releasing on a CI server that tracks and auto increments the build number. GNU/make, Maven and Gradle natively support it, manual editing of a build-spec file is not practical and error prone.

make target VERSION=%build.version%
mvn release:prepare -DreleaseVersion=%build.version%
./gradlew build -Pversion=%build.version%

[dwijnand]

mvn release:prepare is based on the maven release plugin. The equivalent is https://github.com/sunng87/cargo-release, and I don't know if it supports specifying the version on the command line.

[vorner]

I don't think it is equivalent for my use case. This is for releasing on crates.io and modifying Cargo.toml.

What I want is somewhat completely the opposite. I want to leave Cargo.toml as it is, not touch git (tags, push) at all ‒ that happens externally. I want to just build the binary, but the binary should be fed with a version different from the one in Cargo.toml, because for this use case, Cargo.toml is not the authoritative source.

[dwijnand]

What does

the binary should be fed with a version different from the one in Cargo.toml

mean? Is it just env!("CARGO_PKG_VERSION") (and variants, like CARGO_PKG_VERSION_MAJOR) usage?

[vorner]

I'm not 100% sure there are no other places the version is passed into the compilation, but I guess so. I haven't tried yet (I might), but I guess cargo will set the env var unconditionally and overwrite it even if I pass it from outside.

[dwijnand]

Maybe we could just change that: only set the environment var if unset.

[lukaslueg]

built may help your use-case to some extent.

[vorner]

Looking at built, it seems to be really nice and useful thing and I'll keep this in mind. But I don't think this will help me here ‒ this doesn't help me push the right version into eg. clap or structopt.

[dwijnand]

Can you not define the version in clap/structopt in terms of built_info::PKG_VERSION?

[vorner]

By default, they take the version from CARGO_PKG_VERSION. I could set that manually to built_info::PKG_VERSION but I don't see how that would help me, because built also takes the value from there.

[dwijnand]

You're right, sorry. I think the solution is to not override the environment variables, then.

[Kaiser1989]

Is there any progress? Also looking for the feature to externally manipulate the package version. (CI is setting the version, Cargo.toml should not be changed).

An Env-Variable would be nice:
CARGO_TOML_PACKAGE_VERSION

This could also be done for every other part of the Cargo.toml:

• CARGO_TOML_PACKAGE_NAME
• CARGO_TOML_PACKAGE_AUTHORS
• ...

The place within the code is here:
https://github.com/rust-lang/cargo/blob/master/src/cargo/util/toml/mod.rs#L68
There should be an env-override after parsing the toml and before checking for unused.

This should be analogous to https://doc.rust-lang.org/cargo/reference/config.html#environment-variables

I could add the implementation if we find a good name for env variables...

[mieubrisse]

Adding another +1 - we're setting the version upon release, and it'd be great to not need to edit Cargo.toml each time

[cgranade]

By way of another +1 and showing analogous functionality in other toolchains, the .NET SDK allows for overriding version numbers at the command line with commands like dotnet build /p:Version=1.0.1 (https://andrewlock.net/version-vs-versionsuffix-vs-packageversion-what-do-they-all-mean/). I've often found that overriding package versions that way can be useful in using CI pipelines to control how packages are versioned.

[Ri0n]

absolutely same for me. CI and stuff.
I was going to use "built"'s GIT_VERSION for that, but we build in docker containers not passing .git inside. So this info is lost.

[Humandoodlebug]

This seems to be working on nightly right now: #6699

e.g. cargo build -Z unstable-options --config 'package.version="1.5.2"'

[QAston]

@Humandoodlebug

cargo build -Z unstable-options --config 'package.version="1.5.2"'

Will not work because --config overrides the cargo configuration, not the build definition in Cargo.toml

[colemickens]

Editing the Cargo.toml isn't practical in some scenarios anyway - in Nix we don't really have conventional network access during build phases, and so it's not possible rewrite toml and regenerate lock files to workaround this.

I do see that there is a package macro that makes it easy to override the build version, falling back to the Cargo version, but it wouldn't be fun adding this to lots of Rust projects if Cargo could do it: https://docs.rs/git-version/latest/git_version/macro.git_version.html

[Snazzie]

Very surprising that this is still an open issue in feb 2022. 3 years have passed! probably lost/forgotten in the 1.2k issues.
Theres no good way to programmatically replace the version in cargo.toml.

Could we @ some of the active maintainers to get some visibility on this issue?

+1 to adding version override in CLI like dotnet.

[brandonros]

Is there a way to get cargo build to blend Cargo.toml version with [[bin]] name output? so that the cargo build --release makes a project-name-version.dll for example?

[burdiyan]

+1 here. I don't understand where the practice of statically specifying a version in a checked in manifest file comes from. I just want to release whatever is already checked in without having to commit again to change any manifests. It's very error prone unless you're using some fancy automated tools. It's so easy to overwrite a version that was already published previously if you release forgetting to change the manifest.

Being able to specify the version in the command line and completely omitting the version from the manifest would be my ideal workflow!

[epage]

I suspect this is a large enough semantic change that this would need a major change proposal or an RFC and need a person to champion the proposal and implementation. It is unlikely for someone else to pick up and do all of that leg work on behalf of someone interested in this work. One challenge though is cargo is currently on a soft feature freeze and the cargo team is wanting to avoid distractions, including mentoring / shepherding, to be able to handle basic maintenance, finish currently committed work ,and to reduce technical debt so we can increase our capacity.

Things off the top of my head that would need to be figured out

• Should the version field be made completely optional? See feat(toml): Allow version-less manifests #12786
• What are cargo's semantics without a version field and when no version is specified?
• What all commands should accept a version?
• How do we pass in versions, especially when running against multiple packages?
• How do we help people when they miss specifying the version for one of the multiple packages?
• How do we help workspace users with path dependencies?
  • Currently, to publish a crate with a path dependency, it also needs to have a version requirement specified. That version requirement must match the crate its pointing to. Tools like cargo-release handle updating version requirements of dependents so they don't become stale
• Is there any prior art in other ecosystems that we can learn lessons from?
• EDIT: How would patching a registry dependency with git work as we require the versions to match?
• EDIT: What packages in the tree can have their versions overwritten (workspace member, local package, all packages)?

[burdiyan]

@epage This is a nice summary to figure out a considerate solution to this problem! Thanks for that!

It's a pity that it's unlikely that this effort would come from the core team, but it's understandable.

On the other hand, maybe an easier and good enough solution could be adding a flag (or env var) to pass the desired version to cargo so it could ignore the one specified in Cargo.toml? I'm absolutely unaware of cargo's internals, but if there's only one place where Cargo.toml is read to extract the version, then it seems like a pretty easy change to make.

In my (somewhat biased) opinion: there should be no static file defining the "current" version of a package at all. This should be determined at release time by looking up the VCS info or elsewhere.

[bcyran]

That is your workflow but I've not seen that generally applied in the Rust community. Instead, I've seen a lot of people fear release automation to the point that they don't even want to adopt a local-only tool that is dry-run by default and optionally logs everything it does to the screen.

I feel like this is a chicken and egg problem. Of course Rust community doesn't apply that workflow because it's not supported by Cargo or any other tools. Of course they fear release automation if the only way to implement it is with brittle scripts.

I find it surprising that using a Version Control System as a single source of truth for managing versions of your software is a controversial topic.

[sunhaitao]

I made a draft patch to solve this. It reads the value of a environment variable to decide the default version. If the variable is unset or the value is not a valid SemVer version, it falls back to use 0.0.0. Since the variable is purely suggestive, I named the variable CARGO_SUGGESTED_PKG_VERSION.

This design has a limit: when building many packages in one call of cargo, all unversioned packages will be affected. Since many build tools for other languages also behave like this, I believe this is an acceptable trade-off for a simple design.

If this design is not fundamentally flawed, I'd like to improve it to make it land. Any suggestions?

[epage]

As mentioned in your PR, this is nominated for being closed.

I'd recommend reviewing the thread and seeing if you have anything to add to what has been said.

[sunhaitao]

I should have mentioned that we (my workmates and I) have a policy to increase versions fast. For example, we increase the {MINOR} part everyday. This make versions in files more painful.

Currently we handle this with generating temporary manifests. But this isn't very lightweight.

[sunhaitao]

I propose that we give this a try on workspace manifests only. It will be enough to solve the problems of those who want this feature without creating too much complexities or risks.

[WhyNotHugo]

Can we drop the S-propose-close? This is one of the most annoying issues when using Cargo to publish crates to a registry.

The proposal to close mentions something about "marking version", simply because someone else further up in the comments mentioned such a concept. The concept of "marketing version" is out of scope on this issue. Let's not use the fact that someone mentioned an out-of-scope topic as an excuse to close the issue.

The issue is quite real for the simplest case of a single crate in a git repository with a single version: the version of the crate used by cargo and used to publish to a registry.

What's being proposed here is "let's provide a mechanism so that the version is read from the version control system".

Currently, someone needs to re-write the file each time a new version is created. This is needless churn, meaningless bureaucracy. It's a requirement to duplicate information. And it needs to be done before creating the tag, otherwise fetching the tag from git will result in inconsistent version numbers.

In fact, the very idea of storing the version number inside the file is a design issue: the version for a project is then stored in two places: in the version control system and in Cargo.toml. These can be different, leading to ambiguity.

Yes, this approach was the norm back in the days of CVS. Git has first-class tags, we don't need to imitate design patterns from the days of CVS.

[sunhaitao]

What's being proposed here is "let's provide a mechanism so that the version is read from the version control system".

"A mechanism to pass the version from outside sources to cargo without modifying any file" would be a more accurate description. VCSs are typical such sources. But other sources (such as dates) are also used in various cases.

[epage]

Before I forget, I thought this post about Python's dynamic metadata interesting: https://lucumr.pocoo.org/2024/11/26/python-packaging-metadata/

One difference between Python and Cargo is that Cargo rewrites the static metadata on publish, so we can take something dynamic and make it static, and we have an Index. However, that doesn't completely remove the complexities.

[epage]

@sunhaitao

I should have mentioned that we (my workmates and I) have a policy to increase versions fast. For example, we increase the {MINOR} part everyday. This make versions in files more painful.

Currently we handle this with generating temporary manifests. But this isn't very lightweight.

Could you go into more detail on this? I assume you are bumping the version every day because you are publishing to a registry every day? Is this crates.io and are these meant for people outside your group to use? What problem are you solving by releasing so rapidly?

To be clear, I'm not questioning doing it but wanting to better understand your use case. I tend to publish my packages after every end-user noticeable change.

[epage]

Can we drop the S-propose-close? This is one of the most annoying issues when using Cargo to publish crates to a registry.

As there are use cases beyond what I acknowledged in my comment, I'll remove it. However, this is unlikely to move forward any time soon and there is a good chance it will still end up closed, see below.

The proposal to close mentions something about "marking version", simply because someone else further up in the comments mentioned such a concept. The concept of "marketing version" is out of scope on this issue. Let's not use the fact that someone mentioned an out-of-scope topic as an excuse to close the issue.

When discussing why a feature is needed, it is fully in scope to discuss use cases for why its needed and to see if there are better alternatives. That is not out of scope. In fact, the original issue sounds very much like its about a marketing version and not a registry version.

The issue is quite real for the simplest case of a single crate in a git repository with a single version: the version of the crate used by cargo and used to publish to a registry.

And as discussed here and here, we need to support more than that simple workflow and we'd need to work out the design considerations for those and recognize there are costs and whether we are willing to accept them.

This is not the type of design work that a couple of answers on a thread will do. Due to all of the interactions and potential costs, this likely eventually become an RFC.

This is needless churn, meaningless bureaucracy.

I think it would be a helpful exercise to consider why it works the way it does rather than dismiss it out right. Its much harder to evaluate how to improve a system if you can't acknowledge what problems it is solving.

[WhyNotHugo]

I think it would be a helpful exercise to consider why it works the way it does rather than dismiss it out right.

When I say "meaningless work", I'm referring to the manual effort of a human having to manually copy the version value from one place to another, ensure that they remain in sync, ensure that the lockfile is updated with it, etc.

Once you repeat a manual process of duplicating information enough times it becomes obvious that this would best be done by software.

Its much harder to evaluate how to improve a system if you can't acknowledge what problems it is solving.

I'd definitely appreciate insight on this, I'm sure there are aspect which I'm ignoring on this topic.

[WhyNotHugo]

As an alternative approach, I've considered writing a wrapper script that the above steps for me and then publishes the crate.

he main caveats with this approach are:

1. cloning the git-tag won't produce the same package as the one submitted to crates.io
2. consumers can't use a subsequent git-commit via cargo because the version won't match the expectation.

[weihanglo]

A relevant discussion on Zulip. tl;dr: some build systems need a new version assigned for every build. It would be interesting if they could use build metadata, but there are reasons build metadata doesn't work.

• crates.io doesn't allow two crate versions only differ in build metadata.
• Build metadata, by Semver spec, shouldn't affect version precedence.

This may be similar to @sunhaitao's use case?

[sunhaitao]

Before I forget, I thought this post about Python's dynamic metadata interesting: https://lucumr.pocoo.org/2024/11/26/python-packaging-metadata/

For cases I just verified, pip wheel generates metadata by saving calculated dynamic values for both setup.py-based and pyproject.toml-based packages. So the problem may only exists on other distribution forms.

One difference between Python and Cargo is that Cargo rewrites the static metadata on publish, so we can take something dynamic and make it static, and we have an Index. However, that doesn't completely remove the complexities.

I think the good balance is: The final package should have static metadata. But they don't all have to be hard-coded in the source. Some of them should be able to be decided on publish. Allowing all of them being so dynamic seems unnecessary.

[sunhaitao]

@epage

Could you go into more detail on this?

Actually, we align the {MINOR} part to "How many days have passed since we started working on this {MAJOR} version". If we pause works on a package for a while, the {MINOR} part will jump several numbers on resuming.

And we don't publish all those versions to end users. Many of them only exist on developer machines. Some on build machines. Only few of them are released out.

[sunhaitao]

@weihanglo

It is similar on frequently assigning a version. But on build machines, we only dare to touch the {version core} part on each commit.

Anyway, changing the version a little on each build is a important use case for this feature.