how shall we plan 4.0? #1016

ctb · 2020-06-07T15:30:03Z

it seems like we're nearing some kind of critical mass on CLI (and maybe API) breaking changes that would need to be in a new major release.

@luizirber really likes the notion of just doing lots of releases, which I'm on board with? but it seems like maybe doing 4.0, 5.0, and 6.0 in rapid succession would just be confusing :).

but we also don't want to wait forever for a 4.0 because I have the attention span of a gnat, etc. etc. and often don't follow through on medium term plans.

so how can we best collate these features and be poised to release a 4.0 that includes a bunch of new stuff, without blocking on too many features?

my naivest thought is to put a pre-4.0 branch in place that has the same branch protections as master - basically, merges must be reviewed, pass all tests, etc. Have a standing PR for this into master.

alternatively, we can switch 3.x over to a maintenance branch and make master the 4.0. I kinda like this more, actually.

a much worse idea seems to be to have a bunch of hanging PRs that wait to be merged until we're ready to release 4.0. let's not do this.

luizirber · 2020-06-08T00:52:51Z

alternatively, we can switch 3.x over to a maintenance branch and make master the 4.0. I kinda like this more, actually.

That was the plan for 2 -> 3, but ended up not being necessarily because we moved quickly. I prefer this one to the other alternatives.

a much worse idea seems to be to have a bunch of hanging PRs that wait to be merged until we're ready to release 4.0. let's not do this.

Somewhat related to this: some current PRs involved a lot of work and may be better to merge them before 4.x (and breaking changes), to avoid large rewrites/rebase/merging changes. #925 comes to mind.

it seems like we're nearing some kind of critical mass on CLI (and maybe API) breaking changes that would need to be in a new major release.
so how can we best collate these features and be poised to release a 4.0 that includes a bunch of new stuff, without blocking on too many features?

I like using labels and milestones for tracking. Not that many of them are marked, maybe do another check on opened issues and label them?

Issues tagged 4.0: https://github.com/dib-lab/sourmash/issues?q=is%3Aopen+is%3Aissue+label%3A4.0
Milestone 4.0: https://github.com/dib-lab/sourmash/issues?q=is%3Aopen+is%3Aissue+milestone%3A4.0

From those, see which ones are actually breaking, and which ones are big chances/refactors that are not necessarily breaking. Some that come to mind:

[EXP] Set up black and pre-commit hooks #680 is a big change (but not breaking)
Drop Python 2 once khmer 3.0 is released. #318 will be breaking for Python 2
Deprecate 'import sourmash_lib' at some point #441 is breaking (remove functionality)

Maybe create a label "breaking" and mark them? And mark the non-breaking ones as "enhancement"?

olgabot · 2020-06-12T18:59:28Z

a much worse idea seems to be to have a bunch of hanging PRs that wait to be merged until we're ready to release 4.0. let's not do this.

Somewhat related to this: some current PRs involved a lot of work and may be better to merge them before 4.x (and breaking changes), to avoid large rewrites/rebase/merging changes. #925 comes to mind.

If 4.0 comes and #925 is not ready, that's okay. I haven't had time to finish it lately but I can take a crack at it this weekend. Will try to get it in before #680!

ctb · 2020-08-03T15:28:21Z

per #1034, we'll go with latest (for 4.x and beeeeeyond) and stable (for 3.x) branches.

ctb · 2020-08-04T20:50:31Z

with #1144, we now have

https://github.com/dib-lab/sourmash/releases/tag/v4.0.0a0

and pip install -e . produces:

% sourmash info

== This is sourmash version 4.0.0a0. ==
== Please cite Brown and Irber (2016), doi:10.21105/joss.00027. ==

sourmash version 4.0.0a0
- loaded from path: /Users/t/dev/sourmash/sourmash/cli

huzzah!

luizirber · 2020-08-05T17:31:40Z

From #1128 (comment)

In any case, we might need a PR derived from this one for the DeprecationWarnings and merge it in stable? (and then release it as 3.4.2 and continue updating latest? =])

Should we start just adding the DeprecationWarnings to stable and release patch versions (3.4.2, 3.4.3...) to indicate what is going to break in 4.x? And maybe make a final 3.5.0 release right before we release 4.0.0?

ctb · 2020-08-05T18:03:44Z

On Wed, Aug 05, 2020 at 10:31:55AM -0700, Luiz Irber wrote: >From #1128 (comment) > In any case, we might need a PR derived from this one for the `DeprecationWarning`s and merge it in stable? (and then release it as 3.5.1 and continue updating latest? =]) Should we start just adding the `DeprecationWarning`s to `stable` and release patch versions (`3.5.1`, `3.5.2`...) to indicate what is going to break in `4.x`?

yes, all sounds good.

ctb · 2020-08-05T22:57:37Z

ok, so the #1128 merge was not done well... I merged into stable and then from there into latest. I think I prefer:

merge new PRs into latest, with full review etc.
backport into stable periodically as a separate PR, with clear indications of what the original PR or PRs were.

luizirber · 2020-08-06T01:01:34Z

Yeah, latest first and then stable. I assumed it was against latest, but later noticed it was against stable 😞

ctb · 2020-08-06T03:09:14Z

On Wed, Aug 05, 2020 at 06:01:47PM -0700, Luiz Irber wrote: Yeah, `latest` first and then `stable`. I assumed it was against `latest`, but later noticed it was against `stable` :disappointed:

I, err, changed it to stable <cringe>

ctb · 2020-08-06T14:49:20Z

OK so I'm slowly starting to figure this out, sigh. I'll release 3.5.0 sometime soon.

ctb · 2020-08-06T16:30:26Z

Note https://github.com/dib-lab/sourmash/releases/tag/v3.5.0a0 (based on stable), which should alert us to all the deprecations.

luizirber · 2020-08-06T16:49:25Z

Note https://github.com/dib-lab/sourmash/releases/tag/v3.5.0a0 (based on stable), which should alert us to all the deprecations.

Let's try to avoid too many a0 versions, I still remember the many 2.0.0a releases...

luizirber · 2020-08-06T16:50:12Z

I think it is OK for 4.0.0a0, but let's try to release 3.5.0 soon =]

ctb · 2020-08-06T16:51:39Z

On Thu, Aug 06, 2020 at 09:50:26AM -0700, Luiz Irber wrote: I think it is OK for `4.0.0a0`, but let's try to release `3.5.0` soon =]

yes, I did it so that I could get the deprecations working in pytest!

luizirber · 2020-08-06T20:43:23Z

From #1150 (comment):

Looks good, overall. Question: do we want to point at stable URLs for documentation, notebooks, etc? I expect we'll keep stable up to date with the last release, in the future...

I thought stable was temporary until 4.0.0 is released, keep syncing branches will be annoying... Maybe turn stable into a tag, and add to the release notes to replace it on every stable release?

ctb · 2020-08-11T18:37:17Z

any comments on the header, @luizirber ? #1162 --

This is the first of several minor releases (v3.5.x) from the new stable branch that focus on preparing for sourmash v4.0 by introducing deprecations and warnings.

ctb · 2021-02-07T17:45:19Z

reupping question:
What do we do about stable vs latest docs, per comment above? #1016 (comment)

Question: do we want to point at stable URLs for documentation, notebooks, etc? I expect we'll keep stable up to date with the last release, in the future...
I thought stable was temporary until 4.0.0 is released, keep syncing branches will be annoying... Maybe turn stable into a tag, and add to the release notes to replace it on every stable release?

ctb · 2021-02-15T15:06:17Z

ok, I finally grappled with read the docs & stable conversation and I think I understood it!

here's what I think we should do, mechanistically:

now: shift default readthedocs to redirect to stable (v3.5.1 docs); visiting sourmash.rtfd.io will go to stable docs.
merge [MRG] Documentation updates for 4.0 release #1283, the big doc update for 4.0. it will show up on sourmash.rtfd.io/en/latest/, but not be the default docs
release 3.5.1 and 4.0.0a4
(do other stuff here)
release 4.0
point 'stable' docs at 'latest' now, spin off 3.5.1 docs to their own URL.

that way stable always points at the docs you get when you run pip install or conda install without a specific version.

luizirber · 2021-02-15T18:16:11Z

here's what I think we should do, mechanistically:

I agree, some additional comments:

now: shift default readthedocs to redirect to stable (v3.5.1 docs); visiting sourmash.rtfd.io will go to stable docs.

merge [MRG] Documentation updates for 4.0 release #1283, the big doc update for 4.0. it will show up on sourmash.rtfd.io/en/latest/, but not be the default docs

release 3.5.1 and 4.0.0a4

I think we can release 4.0.0rc1 instead of another alpha, I think there are no code changes left.

(do other stuff here)

Probably debug the migration docs from #1283 and recommend specifying sourmash versions using ranges (sourmash >=3, <4), like we do for setuptools_scm in pyproject.toml

I think we should announce the availability of the migration docs and 4.0.0rc1 and ask for feedback, and after some time (a week?) promote the rc to the final release.

release 4.0

point 'stable' docs at 'latest' now, spin off 3.5.1 docs to their own URL.

And probably remove the stable branch?

ctb · 2021-02-15T20:02:22Z

here's what I think we should do, mechanistically:

I agree, some additional comments:

now: shift default readthedocs to redirect to stable (v3.5.1 docs); visiting sourmash.rtfd.io will go to stable docs.

done! 🎉

merge [MRG] Documentation updates for 4.0 release #1283, the big doc update for 4.0. it will show up on sourmash.rtfd.io/en/latest/, but not be the default docs

release 3.5.1 and 4.0.0a4

I think we can release 4.0.0rc1 instead of another alpha, I think there are no code changes left.

sure!

(do other stuff here)

Probably debug the migration docs from #1283 and recommend specifying sourmash versions using ranges (sourmash >=3, <4), like we do for setuptools_scm in pyproject.toml

👍 - I take it that's a suggestion for #1283? 😉

I think we should announce the availability of the migration docs and 4.0.0rc1 and ask for feedback, and after some time (a week?) promote the rc to the final release.

👍

release 4.0

point 'stable' docs at 'latest' now, spin off 3.5.1 docs to their own URL.

And probably remove the stable branch?

I'd like to keep a v3.5 maint branch around Just In Case? Although we could just start with the v3.5.1 tag if we needed to release a v3.5.2. So 👍 for removal.

ctb · 2021-02-16T14:40:31Z

OK - I think the next step is merging #1283 and then I can release 4.0.0rc1!

ctb · 2021-02-17T13:52:37Z

next steps:

once [MRG] Deprecations for next 3.5.x release, in support of 4.0 release. #1308 and [MRG] backport github actions to stable branch (3.5.x) #1317 are merged, release 3.5.1 (release notes)
finish docs [MRG] Documentation updates for 4.0 release #1283 provide better docs and/or examples for command line loading of signatures #1092 Document and streamline/refactor signature loading and saving #919 add explicit documentation about database types/formats #1293 document new SBT zip file output. #973 and release notes Draft release notes for v4.0 #1309
merge [MRG] Documentation updates for 4.0 release #1283 and release 4.0.0rc1
update author list Update author list for 4.0 #622 update authors in setup.py etc. #444
release 4.0 (release notes)
adjust conda packaging Disable Python 3.6 builds in conda-forge and bioconda for 4.0 #1163 increment conda packaging requirements for screed #1299
point default readthedocs at latest again
remove stable branch

ctb · 2021-03-03T05:43:27Z

c'est fini!

luizirber added the 4.0 issues to address for a 4.0 release label Jun 8, 2020

ctb mentioned this issue Jun 16, 2020

sourmash 3.x protein ksizes must be divisible by 3 #1019

Closed

5 tasks

luizirber mentioned this issue Aug 6, 2020

evaluate sourmash public API using a nice checklist #586

Open

ctb mentioned this issue Jan 10, 2021

plans for sourmash 4.0 #835

Closed

This comment has been minimized.

Sign in to view

ctb mentioned this issue Feb 15, 2021

[MRG] update the migration guide for 4.0 with version pinning instructions #1330

Merged

5 tasks

ctb closed this as completed Mar 3, 2021

ctb mentioned this issue Aug 17, 2022

moving towards a sourmash 5.0 release - idle thoughts #2221

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

how shall we plan 4.0? #1016

how shall we plan 4.0? #1016

ctb commented Jun 7, 2020

luizirber commented Jun 8, 2020 •

edited

Loading

olgabot commented Jun 12, 2020

ctb commented Aug 3, 2020

ctb commented Aug 4, 2020

luizirber commented Aug 5, 2020 •

edited

Loading

ctb commented Aug 5, 2020 via email

ctb commented Aug 5, 2020

luizirber commented Aug 6, 2020

ctb commented Aug 6, 2020 via email

ctb commented Aug 6, 2020

ctb commented Aug 6, 2020

luizirber commented Aug 6, 2020

luizirber commented Aug 6, 2020

ctb commented Aug 6, 2020 via email

luizirber commented Aug 6, 2020

ctb commented Aug 11, 2020

This comment has been minimized.

ctb commented Feb 7, 2021

ctb commented Feb 15, 2021 •

edited

Loading

luizirber commented Feb 15, 2021

ctb commented Feb 15, 2021

ctb commented Feb 16, 2021

ctb commented Feb 17, 2021 •

edited

Loading

ctb commented Mar 3, 2021

how shall we plan 4.0? #1016

how shall we plan 4.0? #1016

Comments

ctb commented Jun 7, 2020

luizirber commented Jun 8, 2020 • edited Loading

olgabot commented Jun 12, 2020

ctb commented Aug 3, 2020

ctb commented Aug 4, 2020

luizirber commented Aug 5, 2020 • edited Loading

ctb commented Aug 5, 2020 via email

ctb commented Aug 5, 2020

luizirber commented Aug 6, 2020

ctb commented Aug 6, 2020 via email

ctb commented Aug 6, 2020

ctb commented Aug 6, 2020

luizirber commented Aug 6, 2020

luizirber commented Aug 6, 2020

ctb commented Aug 6, 2020 via email

luizirber commented Aug 6, 2020

ctb commented Aug 11, 2020

This comment has been minimized.

ctb commented Feb 7, 2021

ctb commented Feb 15, 2021 • edited Loading

luizirber commented Feb 15, 2021

ctb commented Feb 15, 2021

ctb commented Feb 16, 2021

ctb commented Feb 17, 2021 • edited Loading

ctb commented Mar 3, 2021

luizirber commented Jun 8, 2020 •

edited

Loading

luizirber commented Aug 5, 2020 •

edited

Loading

ctb commented Feb 15, 2021 •

edited

Loading

ctb commented Feb 17, 2021 •

edited

Loading