Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexing in Rust #773

Merged
merged 5 commits into from
Dec 16, 2019
Merged

Indexing in Rust #773

merged 5 commits into from
Dec 16, 2019

Conversation

luizirber
Copy link
Member

@luizirber luizirber commented Nov 13, 2019

The rust parts could read a SBT and search it, but proper indexing was not implemented. Working on it now.

Other changes:

  • Use GitHub Actions for rust checks (including building on wasm32 targets)
  • Replace rust-bio sequence readers with needletail
  • Track coverage on rust code (in CI, not during dev)
  • Clippy fixes
  • Comment out code that uses features not available in crates.io packages yet (this includes MQF and draff sigs...). This is due to cargo only allowing publishing to crates.io without any external (including git) deps. Also added a CI check to see if nothing that impedes publication is added.
  • Bring in the rust side of nodegraph changes from Replacing C++ with Rust #424 (doesn't expose to Python, that stays in Replacing C++ with Rust #424)

Checklist

  • Is it mergeable?
  • make test Did it pass the tests?
  • make coverage Is the new code covered?
  • Did it change the command-line interface? Only additions are allowed
    without a major version increment. Changing file formats also requires a
    major version number increment.
  • Was a spellchecker run on the source code and documentation after
    changes were made?

@luizirber luizirber added the rust label Nov 13, 2019
@luizirber
Copy link
Member Author

Relevant before merging: doesn't change anything in the Python parts.
#772 (comment)

@codecov
Copy link

codecov bot commented Nov 13, 2019

Codecov Report

Merging #773 into master will decrease coverage by 39.6%.
The diff coverage is 42.75%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #773       +/-   ##
=========================================
- Coverage    89.6%    50%   -39.61%     
=========================================
  Files          30     13       -17     
  Lines        4706   1828     -2878     
  Branches       49    450      +401     
=========================================
- Hits         4217    914     -3303     
- Misses        485    615      +130     
- Partials        4    299      +295
Impacted Files Coverage Δ
src/lib.rs 59.74% <ø> (ø)
src/sketch/mod.rs 33.33% <0%> (ø)
src/index/linear.rs 91.66% <100%> (ø)
src/sketch/nodegraph.rs 42.23% <11.62%> (ø)
src/signature.rs 36.05% <19.04%> (ø)
src/sketch/minhash.rs 37.71% <34.42%> (ø)
src/index/sbt/mhbt.rs 41.53% <43.79%> (ø)
src/index/sbt/mod.rs 60.26% <47.72%> (ø)
src/index/storage.rs 72.22% <50%> (ø)
src/index/bigsi.rs 59.55% <57.14%> (ø)
... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ec8a00c...c992f28. Read the comment docs.

@luizirber luizirber force-pushed the rust/index_abc branch 6 times, most recently from b51846c to bb6ab88 Compare November 13, 2019 02:26
@luizirber
Copy link
Member Author

(side benefits: with coverage being tracked now I actually have to write tests to bring the Rust coverage up 🤣)

add coverage CI job

initial index cmd impl

add batch_insert method

use needletail for fastx parsing

fix finch feature

lint

clippy

fix benchmarks

update CI

comment out code that depends on unreleased crates...

another ci check, can we publish it\?

add windows, macos, beta and nightly tests

tests

move ocf to another repo

needletail 0.3.2

update for finch 0.3.0

fix warnings

initial search test

oracle test

oracle tests for search

install sourmash for oracle tests

use better_panic

better_panic does not work on wasm32 yet..

bring in changes from core/rust

update sourmash.h

fix

keep nodegraph working on wasm32

upgrades

hash_functions

use only hash_functions internally

update sourmash.h

from and clippy

bring back oracle tests

test improvements

niffler instead of ocf

test all features for coverage

use released niffler

indexing almost there

compatibility with sourmash-py SBT

note about md5sum and filenames

move tests into specific SBTs

remove unwrap
@luizirber luizirber changed the title [WIP] indexing in Rust Indexing in Rust Dec 16, 2019
@luizirber luizirber merged commit 93b9b90 into master Dec 16, 2019
@luizirber luizirber deleted the rust/index_abc branch December 16, 2019 01:40
@luizirber luizirber mentioned this pull request Dec 17, 2019
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant