Replies: 7 comments 23 replies
-
How would this new release process work for a package author in this scenario?
I see two possible outcomes:
The problem with number 1 is those older lines are never going to get security and bug patches. In order for users to update they also are required to take on breaking changes in node. Imagine a slow to update native dep (hard to imagine right :P) along side a package which updates for new node features then has a CVE on all previous versions. In this case a user cannot take on new security patch of one package because the are unable to update node versions because of the broken native dep. The problem with number 2 is that we have greatly increased the workload on package maintainers, with all the rather obvious issues related work offered freely. I don't think the current release process is without these type of issues, but also the edges are more well known in the community and node breaking changes are slow enough to keep up. This new way sounds like it has more risk of these types of issues appearing. |
Beta Was this translation helpful? Give feedback.
-
Is it possible to devise an incremental approach to going from where we are now to something like this proposal? I think the answer is "no, not really", but if I'm wrong about that, I'd be interested to know what the first incremental step or steps might be. |
Beta Was this translation helpful? Give feedback.
-
After some discussion with James's thread on Twitter with the intent to respect his request to fully parse and ask questions before sharing opinions, I... want to share opinions. My perception so far is that this proposal... changes the mental burden on users without meaningfully improving it. Specifically, we're extending a ecosystem standard paradigm with our own special context that they'll need to learn just for us. I... am concerned about putting the burden on our users - who have consistently voiced that the current system is too confusing - rather than finding an avenue to manage this from our side of the release process. I do think that the proposed system could be beneficial in reducing the burden on releasers and help increase the project's velocity. I am just... extremely cautious about the potential putting additional burden on the ecosystem and our users with little tangible benefit to them. |
Beta Was this translation helpful? Give feedback.
-
I have read the proposal through several times and asked a few questions, but as someone who has shouldered a reasonable portion of the release work recently, I’d like to share some $thoughts. While I agree that the release process can be time-consuming/demanding, I don’t believe the biggest pain points are solved by the new proposal, while it has significant impact on end-users. $thoughts:
The key benefits of the proposal read to me as being mostly to the maintainers/Node.js core developers. It allows Node.js core developers to get their breaking changes out in releases quicker and aims to reduce the burden on releasers. (Although, I don’t believe the proposed model would significantly solve the key challenges I see as a releaser.). I agree with @bnb's thoughts that this will add a burden on the ecosystem and our users with little tangible benefit to them.
As an incremental step, we could preserve most of the current model and start by dropping the ‘Active LTS’ phase for Node.js 16. That would give some time to confirm some of the proposed benefits - such as whether it makes maintenance easier, reduces the burden on releasers, etc. But would also allow us and end-users time to determine the benefits/consequences of not receiving new features in LTS/Stable version. |
Beta Was this translation helpful? Give feedback.
-
This proposal feels a bit to me like a solution looking for a problem. A quick summary of what I see as the identified as the problems being solved for and proposed advantages Problems
Proposed Advantages
There are a variety of new problems that I think this proposed process will introduced, and a variety of threads above where folks have pointed them out... so I'm going to focus more on the proposed problems and advantage and why I am not convinced by this proposal (although I am not outright dismissing it) Time for releases While doing releases can be time intensive, I have not heard the release team asking for this level of radical change. The team right now is the healthiest it has ever been and to the best of my knowledge I have not heard community complaints about the projects ability to ship releases. There are a variety of smaller iterations we could explore including shorter Active LTS cycles (we only just switched Node.js 14 to 12 months and haven't even finished our first round of this new timeline), being less aggressive with backports, or even changing the cadence of releases. I am not convinced that we need to make this radical a change to improve the process nor am I convinced the process needs much change at this point as we appear to be adequately staffed with volunteers. Updating V8 is Slow We run into issues keeping V8 up to date on master because integration is hard. If folks want the latest and greatest V8 they can already download a nightly, which doesn't appear to be substantially different than what "canary" is in the proposed model. As @BethGriggs mentioned above we don't see very large adoption of our nightly releases. Perhaps that is a problem with our nightlies. A focused effort on improving adoption of nightly releases seems like it could be worth investing in. Current release streams can be confusing to both collaborators and consumers I'm not convinced that this is as large of an issue as presented, nor am I convinced that the radically different proposal is substantially better. As mentioned above we utilize Semver and the fact that we support multiple "LTS" lines is the source of the extra labor and confusion. Having less "modes", e.g. no active LTS seems like a far less disruptive change that would have very similar desired outcomes to this proposal. With that said I think the active LTS has been extremely important in supporting module authors and would be skeptical about removing it. As for the advantages there will still be backports, just less
|
Beta Was this translation helpful? Give feedback.
-
Had a chat with James to clarify what I think are the key things (at least in my mind) that we need to discuss/agree on in respect to considering changes to the release process. I don't think these are specific to the model proposed as they could be addressed through different models (including tweaks to what we do today) but are fundamental to agreeing what models might work. I have my opinions/comments on these key issues but will leave that to later as I think its important to tease/out agree on the key decisions first. I think there may be a few more as it is a complex area, but these are the two that I can best identify at this point.
I think that our existing process is based on the opinion at the time in terms of the right balance on those two. If we agree that our opinion has changed since then, that will inform what kind of changes would make sense. |
Beta Was this translation helpful? Give feedback.
-
There appears to be a lack of interest on this so I'll withdraw the proposal. |
Beta Was this translation helpful? Give feedback.
-
/cc @nodejs/releasers @nodejs/tsc
Our current Node.js release model is... complex.
Rather than a single Semantic Version stream, we currently have four, one for the
HEAD
, and one each for the current Long Term Support branches. This makes multiple things fairly complicated. For instance, understanding when a new feature was actually introduced -- in the documentation, we end up with metadata saying that a feature was introduced in 15.x, 14.x, 13.x, 12.x, etc ... with the versions reflecting whatever downstream version is happens to have been backported to. This makes it extremely difficult to reason about what Node.js versions has a given feature.Cutting a new release is... complicated. In addition to the main branch, we have Release and Staging branches for each downlevel LTS stream (so, currently, there are 9 separate branches for
HEAD
,15.x-staging
,15.x
,14.x-staging
,14.x
,12.x-staging
,12.x
,10.x-staging
,10.x
, not to mention the private github organization and branches we use for security triage. Backporting changes from one to the other is complex, with significant and inconsistent lag times in when various features make their way back to older Node.js versions. Sometimes useful features never get backported, other times the simplest oddest things easily jump back multiple Node.js version levels. Backporting is made even more complex by dependencies of semver-patch and semver-minor commits on prior semver-major commits, and we've had quite a few cases over the years where documentation bugs have crept up because some doc change was backported when it shouldn't have been because the underlying feature hadn't actually been backported yet or was based on a semver-major change.Cutting a new release is... time consuming. Folks preparing a new release have to curate a set of commits back first to staging, then to a working proposed release branch, then to the actual release branch, then reconcile that back to both the staging branch and the
HEAD
branch, updating release metadata in the process.I could keep listing issues but anyone who has gone through the process of cutting a new Node.js release already knows that the process is difficult and time consuming.
I'd like to propose a change detailed here: https://docs.google.com/presentation/d/13JkM4SLm176XE92pIiDSNtx-MMY2v7eLs9b_qzSc8DA/edit?usp=sharing
I'm sure there are going to be lots of questions and things to discuss. What I ask right at the start is that you please ask questions to clarify before commenting or offering opinions.
The short version is this:
We switch to a Canary / Current / Stable model.
Canary releases are automatic and weekly. Once per week, the current
HEAD
is automatically rolled into an updated Canary release. All semantic versioning happens only in the Canary stream, with semver-major and semver-minor bumps happening as necessary as things land. The Canary releases are tagged as{major}.{minor}.{patch}-{datestamp}-canary
.Current releases involve nothing more than taking an existing candidate canary
tag
and promoting it (with a few additional metadata bits) as a new current tag. There's no selection of commits into a release. We take the existing canary tag, update the metadata with a commit, and tag it with a new current tag{major}.{minor}.{patch}-{datestamp}-current
, where the{major}.{minor}.{patch}
does not change. A new current can be promoted at any time but would happen at least once per month.Similarly, Stable releases involve nothing more than taking an existing candidate current
tag
and promoting it (also with additional metadata bits) as a new stable tag. There's no selection of commits into a release. We take the existing current tag, update the metadata with a commit, and tag it with a new stable tag{major}.{minor}.{patch}-{datestamp}-stable
, where the{major}.{minor}.{patch}
does not change. New stables can be promoted at any time but would happen at least twice per year.Current release tags are never updated. If new features, bug fixes, security vulnerabilities occur, we land those first in the
HEAD
, cut a new canary tag, and promote a new current tag. There is no backport flow to current, ever.Stable release tags only ever receive targeted critical security and bug fixes applied as patch sets that only update the build metadata (the
{datestamp}
). There is no backport flow for semver-minor or semver-patch changes that are not considered critical bug fixes.One Stable release tag per year would be selected for 3 years of Long Term Support effective from the day the tag is promoted. This would be maintenance only subject to the backport rule just described. Specifically, LTS stables would never receive anything more than critical security and bug fixes.
This approach offers a number of important benefits:
Eliminates Active LTS. The Active LTS cycle has been difficult and complex due to the backporting flow. It also increases the risk of taking on breaking changes and increases the complexity of our branch and version management issues. Eliminating Active LTS is, by itself, a net positive.
Eliminates persistent staging and down-level release branches. Patch sets to update maintenance for critical bug and security fixes done in transient working branches with PR back to tag when ready (similar to the way GitHub's new security advisory workflow works... in fact, we'd be better positioned to use that mechanism with this new approach). Importantly, we could eliminate the multi-branch staging-release structure we have now.
Simplifies current release process by eliminating the "backport and cherry-pick a bunch of commits into a staging branch" part of the flow.
Eliminates the multi-stream semver model we use now. Maintenance updates will bump build metadata rather than version. All semver happens on canary stream only. Make version numbers less important for maintenance.
Eliminates backporting of new features. This does mean that users may have to take on breaking updates to get new features but the overall maintenance burden is reduced.
Decouples the release process from any reliance on the specific semver number (semver numbers become far less important overall).
Allows taking on new V8 versions more rapidly, gaining new features and fixes at a faster pace. These would flow naturally through canary and into current releases, promoting into stable approximately twice per year.
It preserves existing Long Term Support timelines. Just like now, there would be one guaranteed stable release per year that would receive three years of guaranteed support.
It simplifies changelog management and reasoning about when features are added. Since features are never backported, they are only ever added to a single specific Node.js version.
It does mean that semver-major bumps could happen as often as every week but this approach eliminates the focus on specific Node.js version numbers and instead focuses on the Stream and the datestamp. Enterprises that want stable Node.js versions to build on would select a Stable tag covered by LTS. Developers that want the newest features frequently would select the latest Current. Developers who want to live on the cutting edge would select the latest Canary. The specific semantic version associated with any specific tag just becomes additional build metadata.
It does mean that users will not receive new features without also taking on potentially breaking changes. This is a key trade off. New features roll out faster in Canary and Current, but never into existing Stables.
The reference google slides (https://docs.google.com/presentation/d/13JkM4SLm176XE92pIiDSNtx-MMY2v7eLs9b_qzSc8DA/edit#slide=id.gbaf0f0fe08_8_20) include a number of other questions and discussion points that have come up from folks. Before weighing in on this proposal, I ask that you please take a look through the slide deck and formulate any questions that you may have about the proposal.
I'd like to keep the discussion focused and moderated as I'm sure there are many Strongly Held Opinions. Let's start first with clarifying questions before we move on to debate.
Beta Was this translation helpful? Give feedback.
All reactions