Cosmos SDK semver-compliant versioning proposal

[gsora]

Problem definition

The current versioning scheme Cosmos SDK doesn't represent its true reliability status, communicating instability to the wider developer public — furthermore, it poses a technical issue which cannot be ignored.

During the development iteration of tracelistener, one of the vital components for Emeris, the team identified a significant road blocker in the Cosmos SDK codebase: versioning.

The component being iterated must depend on multiple Cosmos SDK versions in order to meet the expected deadlines and be productive in the context of the Emeris backend.

tracelistener is written in Go and depends directly on the Cosmos SDK to:

• Unmarshal transaction bodies and observed store objects into their concrete representation.
• Use their concrete representations to operate various logic components in the Emeris backend core functionalities.

The Emeris core functionalities include balance and staking tracking, account and sequence number tracking, IBC metadata parsing, and tracking.

Since the store tracking functionality that currently lives in the Cosmos SDK requires data consumers to unmarshal into the associated concrete type — or interface — depending on a specific version of the proto codec is a necessity.

At the same time, a developer might want to support multiple schema versions to prevent errors when a strange data packet gets detected or simply when a new version of the Cosmos SDK application that is being observed gets released.

The current structure of the Cosmos SDK prevents development that depends on more than one version because of the interactions between Go mod and the current Cosmos SDK versioning scheme.

Go mod primer

When designing mod, Go maintainers decided to implement strict dependency management that made semver an integral part of the codebase.

The Go compiler is versioned using semver and the module system requires that packages follow semver to guarantee a healthy ecosystem.

This dependency management scheme means that each and every Go package must be released under the semver versioning scheme to be fully compliant with mod.

The easiest way to be compliant is the following workflow:

• follow semver during the development process
• main branch equals unstable code
• track each release using Git tags
• a new release means the package definition in go.mod gets bumped too, along with all imports in the library itself

As an example, let's imagine a hypothetical types package that has the following module definition in its go.mod:

module github.com/cosmos/types

When version 2.0.0 of types gets released, a v2.0.0 tag is created and the module definition gets bumped up too:

module github.com/cosmos/types/v2

Minor and patch releases don't need to bump up module name version, a git tag is enough.

In Go mod, using the require keyword allows you to define a package as a dependency.

By using the Go mod version with therequire keyword, a package can include both versions of types, without having to do weird replace directives or break anything:

module coolsoftware

go 1.17

require (
	github.com/cosmos/types v1.2.3
	github.com/cosmos/types/v2 v2.3.4
)

coolsoftware can now use both versions of types.

Current Cosmos SDK status

Cosmos SDK currently employs a custom versioning scheme that resembles semver but doesn't look quite like it if observed closely.

With each major version of the SDK — the major version stands for "potentially breaking, non-retro compatible" — but the minor version number gets incremented instead of major.

For example, SDK v0.42.x has been updated to v0.43.0.

While this versioning method seems obvious to the many development teams working on Cosmos SDK, it doesn't make much sense to the outside world.

Developers who want to write software that works with multiple versions of the SDK will find themselves blocked, because of this custom versioning scheme.

Since Go mod — arguably over-zealously — enforces semver on each package, different Cosmos SDK versions cannot be imported at the same time by the same package.

This inability to import multiple Cosmos SDK versions at the same time is because the Go compiler sees the different versions as the same version with increments on only the minor version.

This incongruity ends up with a package update, which reduces rapid development and causes increased code maintenance to upgrade to each new Cosmos SDK version.

All of this code maintenance overhead leads to frustration and a number of potential developer experience issues with required code maintenance to support different versions of SDK objects and interfaces being the first and most common.

Versioning proposal

We propose to move Cosmos SDK versioning to a semver-compliant scheme.

Instead of start version numbering from 1.0, the proposed versioning scheme can start from the latest minor release in the current versioning scheme, so that v0.43.0 becomes v43.0.0, and v0.42.8 becomes v42.8.0.

This proposed versioning scheme is in complete agreement with Go module versioning — https://golang.org/doc/modules/version-numbers — and will guarantee compatibility with the Go ecosystem and developers.

Clear software release strategies have been key for the success and stability of the Cosmos SDK. We believe it's time we stabilize the Cosmos SDK interface on the versioning side of things, too.

Regarding .proto files versioning, there are two alternatives:

1. Version .proto files with Cosmos SDK, so that with each major Cosmos SDK release the v{major version number} proto namespace prefix is also incremented
2. Move data types outside of the Cosmos SDK and version them separately

Both approaches drop the beta1 suffix from the proto namespacing.

While the second alternative is the most correct and provides the most flexibility, the first alternative is easier and quicker to implement.

---

The Cosmos SDK already enjoys high status and trusted value as the most advanced framework for building custom application-specific blockchains today. By adopting this proposed versioning scheme, the Cosmos SDK will grow even stronger, enable more rapid development, and allow for more developer adoption.

Appendix 1: Script to move from v0.X.Y to vX.Y.0

The following script would be run from a cloned Cosmos SDK repository and is able to implement all changes described in this versioning scheme proposal and also implements the first alternative to .proto file versioning.

#!/bin/bash

VERSION=$1
RAW_VERSION=$(echo $VERSION | sed 's/v//g')

DIR=$2

if [ ! -z ${DIR} ]; then
	pushd $DIR
fi

if ! echo "$RAW_VERSION" | grep -Eq '^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(?:-((?:0|[1-9][0-9]*|[0-9]*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9][0-9]*|[0-9]*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$'; then
	echo "version not formatted correctly"
	exit 1
fi

if [[ ! -d .git ]]; then
	echo "This script must be run from the Cosmos SDK directory"
	exit 1
fi

BRANCH_EXISTS=$(git rev-parse --quiet --verify "$VERSION")
if [ -z "${BRANCH_EXISTS}" ]; then 
	echo "Version $VERSION doesn't exists"
	exit 1
fi

MAJ_VER=$(echo $RAW_VERSION |cut -d"." -f2)
MIN_VER=$(echo $RAW_VERSION |cut -d"." -f3)

NEW_VER="v$MAJ_VER.$MIN_VER.0"

echo Old version: $VERSION
echo New version: $NEW_VER

git checkout -b $NEW_VER

if [ $? -ne 0 ]; then
	echo "cannot create branch, exiting"
	exit 1
fi

MOD_NAME=$(grep module go.mod | sed 's/module //g')

go mod edit -module $MOD_NAME/v$MAJ_VER

set -x
# move every go file to vMAJ_VER
find  . -name "*.go" -exec sed -i "s|github.com/cosmos/cosmos-sdk/|github.com/cosmos/cosmos-sdk/v$MAJ_VER/|g" {} +

# TODO: address single linesin protocgen.sh
find  ./scripts -name "protocgen.sh" -exec sed -i "s|any.proto=github.com/cosmos/cosmos-sdk/codec/types|any.proto=github.com/cosmos/cosmos-sdk/v$MAJ_VER/codec/types|g" {} +
find  ./scripts -name "protocgen.sh" -exec sed -i "s|cp -r github.com/cosmos/cosmos-sdk/\*|cp -r github.com/cosmos/cosmos-sdk/v${MAJ_VER}/*|g" {} +
find  . -name "*.proto" -exec sed -i "s|github.com/cosmos/cosmos-sdk/|github.com/cosmos/cosmos-sdk/v$MAJ_VER/|g" {} +
find  . -name "*.proto" -exec sed -i "s|v1beta1|v${MAJ_VER}beta1|g" {} +
find  . -name "*.go" -exec sed -i "s|v1beta1|v${MAJ_VER}beta1|g" {} +
find ./proto -mindepth 1 -type d -name "v1beta1" -print0 | while read -d $'\0' file; do mv $file $(dirname $file)/v${MAJ_VER}beta1; done
sed -i "s|github.com/cosmos/cosmos-sdk/|github.com/cosmos/cosmos-sdk/v$MAJ_VER/|g" Makefile 

make proto-gen

git add -A
git commit -a -m "create semver branch from version $VERSION to $NEW_VER"

if [ -z ${DIR} ]; then
	popd
fi 

Appendix 2: semver-compliant Cosmos SDK versioning

This https://github.com/allinbits/cosmos-sdk-semver link holds a fork of the latest master branch of Cosmos SDK and moves branches v0.42.8 and v0.43.0 to their semver-compliant editions.

Appendix 3: Go project targeting multiple Cosmos SDK versions

The https://github.com/allinbits/multisdk/ link holds a Go project that targets the Cosmos SDK fork that is described in Appendix 2 and uses the proposed versions as PoC of how multiple Cosmos SDK versions can co-exist in the same package.

[alessio]

Thanks @gsora for taking the time and come up with this proposal, which obviously makes sense to me.

We at Tendermint put our clients' needs first, thus we are ready to throw our weight behind this. We need semver more than anything else, and we need it now. Roadmaps are not a valid excuses to delay the adoption of a fully semver-compliant scheme, which as a matter of fact is as costly as running the git tag -a command in a UNIX terminal. Adopting a semver >0 is painless and riskless, and brings a lot of benefits.

[jackzampolin]

Agree here. Being able to import specific sdk versions in go imports would be huge. Relayer needs this too.

[ethanfrey]

Yes, this is a huge win for client side code.

The breaking client APIs between 0.42 and 0.43 for example already cause headaches in CosmJS. And if you are building Go clients where the compiler prevents you from supporting both versions, this is even harder.

[jackzampolin]

We are going to soon be in a world where it is impossible to build go clients for multiple cosmos sdk chains due this and other issues (i.e. the relayer)

[barriebyron]

Yes! We are growing up. Excellent context for benefits are in explained in broad detail in this Tendermint’s Proposal for a New Versioning of Cosmos SDK post

[robert-zaremba]

Thank you for the proposal. It makes a lot of sense. If we want to do now the shift to major semver releases, then tagging v44.0.0 makes sense.

I have few questions:

1. A change which is not API breaking but consensus breaking would require a full version (eg from 44 -> 45) or only point release (eg 44 -> 44.1)?
2. Why you want to bind proto versioning to SDK versioning if we keep it in the same repo? I think it can stay in the same repository (keeping the monorepository methodology). SDK depends on proto but proto doesn't depend on SDK. Changing proto version will signal to the whole ecosystem a change on the transaction level.
   On the other hand, moving proto to other repository will make it more difficult for integrating with the modules (module proto implementation would need to live outside of the module).... that being said - it could be a good idea - we will eat our own dog food and be proto customer as anyone else.
3. beta tag in proto was already discussed and we agree that it doesn't make sense in the way it is used.

[robert-zaremba]

When thinking more about it: moving proto with compiled go code would make it easier for the whole ecosystem.

[robert-zaremba]

hmm, but I'm not sure if we will be able to easily create types - I think some types have additional dependencies on other types.

[gsora]

Hey Robert, thanks for your questions:

A change which is not API breaking but consensus breaking would require a full version (eg from 44 -> 45) or only point release (eg 44 -> 44.1)?

Yes, because it changes how the software behaves.

I think it can stay in the same repository (keeping the monorepository methodology).

And that's fine, but we should then find a way to make a client be able to import multiple, different version of the proto-generated files, which in my mind means versioning modules.

On the other hand, moving proto to other repository will make it more difficult for integrating with the modules (module proto implementation would need to live outside of the module).... that being said - it could be a good idea - we will eat our own dog food and be proto customer as anyone else.

In a perfect world, each module would live in a separate repository IMO, leaving the SDK as minimal as possible.

That said, I understand the burden of maintaining such system then would become too great.

I believe that moving proto and its Go-generated code in a separate repository would benefit the situation greatly, because at that point you:

• version proto separately
• make a given SDK version depend on a given version of the proto specification

On top of that, add proper proto namespacing and you solved all the issues one could have IMO.

[robert-zaremba]

We addressed the proto breaking changes in pre-0.43 release and fixed it (before releasing 0.43). So 0.44 should be compatible with 0.43 and 0.42. So, if you always use the latest version, then you will be able to deserialize transactions and all objects. Am I missing something?

This is just a tip to see if it works in the meantime.

[gsora]

The spirit of this discussion is not just for proto objects, also for plain SDK packages.

For example I need to use some module's store key from two different SDK versions, and without proper versioning I can't do that.

[giansalex]

[amaury1093]

I don't find it that strange. It's as if we were continuing the current release process/cadence, but just adopting the semver standard as a sign of maturity. FWIW, React did a similar transition: https://reactjs.org/blog/2016/02/19/new-versioning-scheme.html

[robert-zaremba]

Exactly, many projects did something similar once they realized that they stuck in 0.x too long.

[amaury1093]

What about release frequency and LTS?

If we adopt semver right now, should we continue our current release cadence: i.e. one new major version every ~6 months, and only maintaining the last 2 majors? There has even been some voice about smaller & more frequent releases, I have a doubt about too much SDK updating for downstream chain devs.

One advantage of committing to a v1.0 is that we can decide that 1.0 has a LTS of say ~2y, which offers some stability for chains.

[robert-zaremba]

But then, most likely we are not able to make breaking (both API and consensus) changes. I think we are still in a rapid development phase, and doing LTS now would mean that chains will use forks from master or maintain their own complex versions.

[zmanian]

I don't think an LTS is very realistic right now.

[alessio]

Agree with Zaki. Plus, a semver scheme does not necessarily force you into any stable maintenance promises but it will make it easier if one day we'll want and have resources to make one.

[alessio]

But then, most likely we are not able to make breaking (both API and consensus) changes. I think we are still in a rapid development phase, and doing LTS now would mean that chains will use forks from master or maintain their own complex versions.

The ability of breaking APIs is not going to be impacted by a semver compliant scheme. On the contrary, semver will put us in the position to communicate better and more consistently what kind of changes any specific release brings around.

[amaury1093]

sgtm. I generally agree with this proposal.

[creachadair]

The Tendermint Core team have been discussing this topic quite a bit as well. We're also working to define what a stable v1 release should mean for the core consensus engine.

I don't have strong opinions about how the SDK should be versioned (what numbering scheme, etc.). You know best how to version for your customers, and we'd like to see versioning for the SDK and the core remain as independent as possible. That said, I do think it's important that the SDK and the consensus engine continue to work smoothly together, and on that basis I have a few recommendations I hope may help you make decisions about SDK versioning:

• The Cosmos SDK depends deeply on the core consensus engine. Any types or functions from the Tendermint core repository the SDK uses in its own API (e.g., definitions from the TM types package, generated code from the TM .proto files), could expose SDK users to (unintended) breaking changes from TM Core development.

  Some of this risk could be averted by carefully auditing the SDK to make sure core types and functions don't escape directly into the SDK's own API, e.g., providing shim types, maintaining separate protobuf generation, etc.

  Before committing to a stable version, I strongly recommend you carefully vet the SDK's API surface for that kind of dependency. Otherwise the SDK's API commitments are likely to block Core development with indirect semantic version commitments—even for Core features the SDK needs.

• There is some signaling value in moving from v0 to v1, but it seems as if the primary technical motivation for major versioning is exemplified by this paragraph from [1]:

  [T]he current structure of the Cosmos SDK prevents development that depends on more than one version due to the interactions between Go mod and the Cosmos SDK versioning scheme.

  That's a valid consideration, and major versions would address it—but that solution would impact all users of the Cosmos SDK, even those who do not need multiple versions. I don't know how common that is among SDK users, but the cost is worth addressing in the proposal.

  For Go specifically, releasing a major version ≥ 1 asks the toolchain to force you and (all) your users to update the import path whenever there's a new release. If each new release really does contain breaking changes that users have to change code for, that's working-as-intended. But if major version bumps are primarily code-neutral, this is an expensive hammer.

  If only a few users need multiple SDK versions, there are other ways to achieve the same effect, without imposing burden on the common case. For example, you can "soft-fork" each release into a separate repository, or a subdirectory of the main SDK repo. In Git, that will share blobs with the mainline (so the repository doesn't grow much), while allowing Go users to explicitly import older releases under a different import path. Tagging a major module release also allows that, but it forces everyone to update their import paths on every release, even if they don't need any other changes to their code.

  I don't know what the right choice is for the SDK, but if that really is the main motivation for tagging major versions, I recommend weighing that cost into the decision.

[1] https://medium.com/tendermint/tendermints-proposal-for-a-new-versioning-of-cosmos-sdk-d52a01976852

[alessio]

we'd like to see versioning for the SDK and the core remain as independent as possible

Likewise.

• make sure core types and functions don't escape directly into the SDK's

We have a made a tool to generate publicly exported API that can be used to detect changes between release and trigger the bumping of major or minor ver accordingly. The tool is called gensymbols (CC @jgimeno).

No worries then. We know what we are doing.

• that solution would impact all users of the Cosmos SDK, even those who do not need multiple versions

The lack of a consistent semver scheme does impact all users of the Cosmos SDK, even those who do need multiple versions, and those who try to build some predictability and risk management into their software development and business processes.

• But if major version bumps are primarily code-neutral, this is an expensive hammer.

Is it?

• Tagging a major module release also allows that, but it forces everyone to update their import paths on every release, even if they don't need any other changes to their code.

Rest assured, we'll tread very, very carefully.

Thank you for sharing your thoughts!

[crodriguezvega]

Last couple of weeks the IBC team was also discussing how to do versioning of ibc-go and we came up with this guideline. We hope this helps; and if somebody has any feedback to improve, we'll be happy to discuss it.

cc: @colin-axner

[ethanfrey]

Nice write-up and I agree with this.

NB: The state machine could be simplified a bit by API-breaking as the first step (which always leads to major version bump)

[ethanfrey]

I agree with this proposal, and think it will give a strong sign of stability, and much easier dev experience. Cosmos SDK is not just for the Cosmos Hub and a few experimental chains, but a very large ecosystem, with many chains in the $100s of millions and 3-4 over $1 billion market cap. We definitely need a dev process that supports this.

The only point I would disagree with the current discussion is the answer to this question:

A change which is not API breaking but consensus breaking would require a full version (eg from 44 -> 45) or only point release (eg 44 -> 44.1

I believe this should be a point release. In no other software, do I expect a major version bump if eg. a function that used to panic when I pass in a 0, now returns a 0 result (simple bug fix, but is different for a consensus node). They may do this fix in a patch release and follow server. Semver traditionally just describes the API.

Until now we always did the largest bump possible, which was minor. Since there was no way to distinguish which patch version was a non-state breaking change and which was state breaking, this was the only option available to signal it. However, with this proposal we will have 3 choices - major, minor, patch.

I will argue that we should define the bumps as:

• major -> API breaking changes (these should be maybe every 6-9 months)
• minor -> Significant non-breaking additions to the API, or state-breaking changes (eg. software that needs 0.44.1 will run differently on 0.44.0)
• patch -> minor non-breaking (logging, client-side, etc) bug fixes

Looking at the recent development history, we have 0.40, 0.41, 0.42 as state-breaking bug fixes. 0.43 was released as a major API change, and now we want to back port a state-breaking fix (malleability issue) to 0.42, but there is no number available. There is also no communication that 0.42->0.43 is any more significant than 0.40->0.41.

With the semver proposal and my guidelines above, this would have been:

Stargate: 40.0.x, 40.1.x, 40.2.x (0.40, 0.41, 0.42)
Fee grants (needs a name!): 41.0.x, 41.1.x (0.43, 0.44)

The back port of the malleability issue would easily fit in as 40.3.x.

This handles the communication issue to differentiate a series of state-breaking bugfixes which are all technically more-or-less the same for app developers, from a major feature release. As a developer, I can more or less automatically update my go.mod for any patch release, but understand that minor releases will require a hard fork (x/upgrade) for the chain that I am building.

As a module developer (x/wasm), this would be nicer as well. We can only support one major version of Cosmos SDK, and force everyone downstream to go along with it. Clearly, we cannot support 0.42 and 0.44 in one codebase, but there is no reason why the codebase is not clearly defined to support 0.40, 0.41, 0.42 and 0.42-maleability-patch at the same time. With the new github.com/cosmos/types/v41 naming proposal, it is even more essential to keep that same name for these non-API minor bumps to avoid major compatibility issues.

[aaronc]

Thanks @ethanfrey I generally agree with this formulation of major, minor and patch releases.

[aaronc]

Nobody would bother if major version bumps happened even more often. Technically speaking, it's totally fine.

Major version bumps do require changing import paths which will be somewhat tedious for users.

I do still hold out the hope that we will achieve a relatively API stable LTS major version soon whether that's called v1.0 or v46.0.

[jgimeno]

Having a timeline let's say 6-9 months allow also better planning when grooming. If there is an issue with a state change we could know that this cannot be added until next major release so it can go directly to appropiate backlog.

[jackzampolin]

Stability has been 6-9 months out for years now. Can we just accept that this is a fast moving piece of software and give developers the tools to handle those changes gracefully?

[alessio]

Nobody would bother if major version bumps happened even more often. Technically speaking, it's totally fine.

Major version bumps do require changing import paths which will be somewhat tedious for users.

It's tedious and confusing for uses to receive API breaking changes tagged as minor version bumps. Change import paths is trivial.

I do still hold out the hope that we will achieve a relatively API stable LTS major version soon whether that's called v1.0 or v46.0.

Sure. End of year, yeah?

[MikeSofaer]

I think staying on 0.x would be a mistake. At this point it's like Gmail being beta in 2009, the Cosmos SDK is mission-critical for massive financial systems. At this point the time for making a product decision about what to call v1 has passed. We should accept the reality of where we are, eat the frog, and move on with our work. I also think we should be strongly inclined to follow Go's standards, which also means changing to semver. The toolchain implications seems like features, not bugs, to me.

[jackzampolin]

Here here!

[alexanderbez]

So what concrete decision(s) are we talking here and when are we making them?

[ethanfrey]

Some new tags coming out.... Cosmos SDK v44.0.0 baby!

[aaronc]

I think I am generally in favor of this proposal.

Pros I can see are:

• allowing import of multiple versions
• clearer meaning of version identifiers (I mostly agree with @ethanfrey's write up here: Cosmos SDK semver-compliant versioning proposal #10162 (comment) although I'd like less frequent major bumps and more frequent minor bumps)
• takes the pressure off about getting to "v1.0"

The biggest potential con is that we don't get to have a big "v1.0" moment... it just never happens. But maybe that's okay. And maybe we can achieve the spirit of v1.0 with an LTS release where we stay at v46 or v47 for a long time and just incrementally add things through minor releases.

We could consider maybe calling Stargate v1.0, v0.44 v2.0, etc. and lose the historical correspondences to v0.39, v0.42, etc... Maybe that's okay. I don't really have too strong of an opinion.

I still am in favor of splitting this code base into multiple go modules as it makes sense to improve maintainability, and I think that's still possible with this proposal as long as we follow the caveats outlined here: https://github.com/golang/go/wiki/Modules#is-it-possible-to-add-a-module-to-a-multi-module-repository.

I do not want to version proto files the same as the SDK as a whole nor do I want to move them out of the SDK. Instead, I propose go modules for the proto types themselves (maybe "github.com/cosmos/cosmos-sdk/proto" or something like that) that correspond with the proto package versions. There is a bit of complexity in that the proto types do need to implement some interfaces themselves - but maybe this will force us to do some beneficial refactoring to make the system more modular.

[alessio]

Caring about a v1.0 big moment is a marketing issue. It does not belong to our domain of expertise.

Unfortunately, I'm not seeing a way that this proposal can actually get us to a major version sooner without major issues.

I disagree vehemently. The clients of the Cosmos SDK already face major issues due to the lack of a consistent, logical, semver fully compliant versioning scheme.

[aaronc]

Caring about a v1.0 big moment is a marketing issue. It does not belong to our domain of expertise.

I don't think I was implying a v1.0 moment should be our priority at all.

Unfortunately, I'm not seeing a way that this proposal can actually get us to a major version sooner without major issues.

I disagree vehemently. The clients of the Cosmos SDK already face major issues due to the lack of a consistent, logical, semver fully compliant versioning scheme.

But how would it work? @colin-axner just explained how the proposed solutions here aren't feasible. I like this proposal, it just doesn't seem like there's a simple and easy solution as implied here

[robert-zaremba]

Let's try to find a solution. We can't break past releases. But we need a solution which will work both for proto and the SDK.

or moving them to a centralized go module will result in a namespace conflict.

@colin-axner - I think having a centralized go module for each proto package is one of the solution. For example: cosmos.bank.v1 (note, I dropped beta1) proto can generate a cosmos-sdk/proto/bank/v1 package, which is living in a separate module and is versioned separately both from SDK and x/bank (it will have same version as the proto directory). This solution is API breaking.

If there is no other solution, then we can leave v0.42 and v0.44 as is. v0.42 is going to sunset in 4 months anyway. If there is a breaking change which blocks other teams, then maybe we find a workaround for that (eg, copying code), and release v1.0 as the next release without adding more things to the roadmap (except the changes related to generated proto files reorganization).

So, as I proposed in the agenda, let's firstly see how desperately we need to re-version the existing releases and if there are workarounds.

[aaronc]

I think a separate github.com/cosmos/cosmos-sdk/proto/bank/v1 module is the right approach. The problem is all the dependencies each of the existing x/.../types packages has on other parts of the SDK. I just was doing a dependency tree analysis in Goland - client, codec, crypto, types, etc. are use extensively by these x/.../types packages. Not to mention other dependencies. It's not so easy... This will require pulling all that code into some sort of "core" package that all the proto modules depend on which in turn the rest of the SDK depdends on. Frankly, it's nothing short of a complete reorganization of the whole code-base - likely one which is sorely needed - but which won't happen over night. Anyone that wants to help, we definitely would welcome that. But we will need a plan and such as massive refactoring is likely not a good idea against master right now given other high priority work we're trying to release.

[robert-zaremba]

So, I think we all agree that this will be a good change.

[alessio]

I think that we have now a clearer view of the problem, thanks @aaronc and @colin-axner

The CORE Tech team from Tendermint Engineering will now start focusing on this issue and drop all other threads of work. Solving this issue is now priority 0 for us.

[robert-zaremba]

We had a meeting to discuss v1.0 .

TL;DR: we agree that tagging v1.0 now doesn't solve the full problem (we will still have a problem with proto). To solve the proto problem we will need to move proto generated code to a new module, and reorganize the repository structure. This proto work is related to the proto code generator.
We want to do v1.0 as next release, but it must not delay the meta transactions feature which is the top priority.

Followups

• Asses the time needed to finish code generator / cosmos proto v2
• Alessio and Aaron to chase ledger to get responses related to reworking amino signing mode.
• Alessio to verify with AiB stakeholders to provide more support to solve the dependencies.

[alessio]

How's the progress on meta txs? Could anyone provide answers to the following questions, please?

1. What has been done so far.
2. What work is undergoing now?
3. Is there any blocker?

Needless to say, ETAs are always greatly appreciated.

Thanks for considering.

[robert-zaremba]

IIRC from the last cal ETA is around 2 weeks (then tests, audit ....)

[robert-zaremba]

In fact we didn't decide about the final number for the major release (1.0 vs 46.0)

[alessio]

In fact we didn't decide about the final number for the major release (1.0 vs 46.0)

Does it really matter? As long as MAJOR is no longer 0, then we all are happy.

[MikeSofaer]

IMO it does matter. 46 is better, because it is easier psychologically to go to 47.0 than 2.0

[ethanfrey]

We want to do v1.0 as next release, but it must not delay the meta transactions feature which is the top priority.

Intriguing. What exactly is that feature? Is it this? #9912

[robert-zaremba]

Yes.
You could respond in the same thread instead of starting a new one ;)

[jackzampolin]

This is becoming a critical issue for clients. Running against the SDK is becoming less and less friendly to clients in go. Working on the relayer I'm quickly going to be in a place where it is impossible to run against multiple chains. This would be a serious regression for the tooling and a usecase that doesn't seem to be discussed in the discussion above.

[alessio]

It's easy to create issues and meta issues. The hard work is to estimate the required work to deliver issues, clarify AC, and more importantly prioritizing them. Instead, we keep running in circles and wasting our time in meetings for the sake of "co-ordination".

The Cosmos SDK lacks a Product Owner.
A Product Owner is not a committee or a team. The PO is one-only-one person that has the responsibility to make decisions on what must be developed first and what comes last. Needless to say, such person must be given the authority to do so.

The most successful FLOSS projects have a firm and authoritative leadership. The Cosmos SDK utterly lacks one.

I am saddened to say it, but the current state of things is very, very concerning.

[robert-zaremba]

True, there is no dedicated Product Owner.
Needless to say, having a clear list in meta-issue would help (especially if it's easy to create).

[jackzampolin]

We have a large number of meta issues and lists. I disagree that this would help.

[aaronc]

I specifically asked to be included in this call and asked multiple times to have the time adjusted so that I could attend. None of those requests were responded to.

@jackzampolin sorry to hear you had this experience. I don't recall hearing this from you directly. I do know, however, that there was a conflict with the IBC call and that there was an agreement that IBC would change their call time once a month. I wasn't involved in those communications directly but I do know that representatives from IBC attended last time.

[aaronc]

We do have milestones with estimated due dates, we have organized things into epics and all of this is visible for those who attend our calls.

We do make decisions in our calls and even though I am not a dedicated full-time product owner, I think it has been clear to everyone for some time that I am the one making the decisions and have the authority to do so. People may not agree with the decisions I make or the processes that we are following, and we definitely welcome constructive feedback. Actually, one of the primary goals of our calls is to make sure that we are listening to community input and taking this into account when making decisions.

In terms of the Cosmos SDK's users, we have made it explicit that the Hub is our number one user and will be prioritized first when needed. It is precisely because of the Hub's priorities that some things other users really want such as semantic versioning and fixing the bech32 globals were deprioritized. It is not that they were cancelled or postponed indefinitely. We made a decision to do A and then B as I have stated before.

There are actually quite a number of things coming down the pipeline for the SDK after v0.45 that I think users beyond the hub (module, chain and client developers) will appreciate that include the bech32 globals issue, semantic versioning and a host of other things. These things are tracked in our working group meta-issue #9058 and in the Zenhub board. We make decisions on them based on the input we receive and are taking the concerns of all the users we hear from into account. It may be that because of the complexity, size and criticalness of this project, that we prioritize certain users over others. Decisions like that need to be made. I'm sorry if you really wanted B first and we chose A, I really am. But we are trying to do the best we can with limited resources given the size of this project and this is where we're at. Input and ideas definitely welcome, but please guys let's try to be constructive and collaborative. It would be great if there were a full-time product owner beyond me and it's definitely something I'm thinking about it, but we are where we are right now and we're trying our best and trying to improve this project and our processes each day.