what's needed for a 2.0 release?

[ctb]

@brooksph and @taylorreiter both think it's time to get the new functionality out there. what's needed for a 2.0 release?

I think we kinda need to wait for a khmer release given our dependency there.

I'd like to do another JOSS paper but that can be 2.0.1 or something.

[brooksph]

• sbt_search error: mismatch in max hash; comparison fail  #146 fix sbt_search bug with max_hash mismatch

[ctb]

• add --protein/etc arguments to sourmash compare #168 (add --protein to compare)
• Add --csv option to sourmash compare  #167 (add --csv to compare)
• Add sourmash version output command #165 (add version output somewhere)
• maybe --dna flag #151 (--rna)
• maybe signature name multifasta input #132 (output names)
• definitely containment vs resemblance? #127 (properly calculate containment)
• Query routines should identify and extract single signatures w/o requiring parameters #105 (load single signatures)
• probably signature of type "mrnaseq" ? #58 (change default signature type away from mrnaseq / omit altogether)

[luizirber]

I created a project to track this: https://github.com/dib-lab/sourmash/projects/1

I would also like to add

• [MRG] Support storages other than local filesystem #166 (support storages other than local fs)

• 'self repairing' SBTs? #176 (self-repairing SBTs, since I think it might need a version bump on SBTs)

• Split SBT into a submodule #177 and [RFC] Rename sourmash_lib to sourmash #164 don't really change the command line, but I think they are good to get in on the Python API side.

[ctb]

Also

• Metadata - Add taxonomic breakdown to sourmash output  #195 add taxonomic info
• reorder sbt gather/search arguments? #231 reorder search/gather arguments
• update documentation with R code for reading matrix #223 update documentation with R code
• make more use of md5sums #239 report md5 hashes in search
• inconsistency in output location #215 results => stdout instead of stderr
• remove YAML parsing #213 remove YAML parsing
• remove remnants of cardinality once new SBTs are posted #212 remove cardinality stuff
• 2017-metagenome branch odd sbt_gather behavior.  #197 verify fix to inclusion reporting
• SBT search functions currently calculate containment #200 fix SBT search --best-only to deal with comparison
• make it easy to merge multiple signature files #196 signature merge
• Add option to ignore reads containing "N" (unknown/any nucleotide)  #192 documentation update
• on sbt_search and _gather, make sure to check ksize at load time #189 check ksize on SBT stuff at load (before search)
• support krona output? #178 krona output format
• what version of setuptools do we require? #173 setuptools version
• Add sourmash version output command #165 sourmash info
• point sourmash docs default at 'stable' with list of previous versions? #159 docs pointer
• add --with-abundance stuff to sbt_gather / sbt_search #180 with abundance/quantification stuff
• Query routines should identify and extract single signatures w/o requiring parameters #105 single sig extract
• signature of type "mrnaseq" ? #58 signature type

[ctb]

also:

• merge in spacegraphcats fixes, [MRG] rename containment to contained_by #199

[luizirber]

Should we try to tick as many boxes as possible from this checklist? http://python.apichecklist.com/
(probably it's better to make it a goal for 3.0?)

[taylorreiter]

sourmash lca output is compatible with krona. I don't think it's useful to have a krona visual for gather output.

[ctb]

(Everything above that was unchecked was transferred into this issue)

Python cleanup/new functionality

• 'self repairing' SBTs? #176 (self-repairing SBTs, since I think it might need a version bump on SBTs)
• Split SBT into a submodule #177 and [MRG] Rename library to sourmash #374 don't really change the command line, but I think they are good to get in on the Python API side.
• update LCA DB format to have additional information #401 add info to LCA DB

Documentation

• update documentation with R code for reading matrix #223 update documentation with R code
• point sourmash docs default at 'stable' with list of previous versions? #159 docs pointer
• [MRG] add some docs on search, gather, and lca methods #393 - documentation/tutorial stuff
• More API docs. #329 - API docs
• document LCA commands and output somewhere #400 - LCA commands
• add blog post to docs #313 - add blog posts
• sourmash compare output/docs update #297 - sourmash compare
• Comparing metagenomes help? #294 - comparing metagenomes help
• document "correct" khmer pre-processing approach for k-mer trimming #273 - preparing data for sourmash with khmer
• Remove dependency on DISPLAY / X #254 - matplotlib DISPLAY issue to known issues
• provide intuitive documentation as to what sourmash --scaled does #238 - document --scaled
• make it easy to merge multiple signature files #196 signature merge -> docs

Minor new functionality/fixes

• [MRG] add --scaled to sbt index to downsample signatures #378 - add --scaled to downsample when doing index
• reporting on number of sequences with 'sourmash compute --merge' is wrong #352 - reporting on sourmash compute
• add clear to end of screen in logging #344 - cleaner logging
• add CITATION info and output as per khmer #259 - add CITATION info
• maybe signature name multifasta input #132 (output names)
• maybe --dna flag #151 (--rna)

Due diligence

• misleading error message when comparing minhash with 'num' to one with 'scaled' #370 - error message when comparing 'num' sig to 'scaled' sig
• check sbt search with num instead of scaled - does it work properly? #363 - check sbt search with num instead of scaled
• sourmash SBT JSON files should not contain absolute paths #312 - check that SBTs are movable
• what version of setuptools do we require? #173 - setuptools version

Punt to 3.x (proposed)

• provide a mechanism for tracking the source of genomes in SBTs #230 track source of genomes in SBTs/signatures
• sourmash-utils repo? #201 sourmash utils hooks

Other release-level work

• update SBT and LCA databases for 2.0 compatibility #402 make new databases for search, LCA, etc.
• remove alpha and pre-release install info upon 2.0 release #425 revisit install language -> "install latest released version"
• revisit jupyter demo notebooks / binder stuff before 2.0 release #430 revisit ipython notebooks

[ctb]

For the 2.0 release, it looks like we need --

• some new notebooks revisit jupyter demo notebooks / binder stuff before 2.0 release #430
• to add citation information add CITATION info and output as per khmer #259

and that's it!

Then finish and merge #592, and then cut a release, I think?

After that I'd like to submit 2.0.0 to JOSS along with all the other stuff we're writing up.

[luizirber]

I think we should merge #615 before 2.0 too

[ctb]

ok, let me know when it's ready!

[ctb]

all done!

now working on final stuff, #622 author list and #623 change logs/release notes.

[ctb]

#607 removes a bunch of outdated files. And we'll need to merge #592 right at the time of 2.0 release.

[ctb]

ok, I think we're down to:

• [MRG] update documentation & add notebooks to docs #631 update documentation, add notebooks, remove old notebooks & files
• Update author list for 4.0 #622 author list
• [MRG] release notes for v2.0 #630 change logs/release notes

and then I think that's that!

[ctb]

I mean, after that I want to submit to JOSS too :)

[ctb]

looks like author list is it!