Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend benchmarks from basic tutorial #3031

Merged
merged 16 commits into from
May 14, 2024
Merged

Conversation

flying-sheep
Copy link
Member

@flying-sheep flying-sheep commented Apr 26, 2024

  • Release notes not necessary because:

I added a sparse dataset to cover more code paths in most benchmarks.

regress_out seems pretty slow with sparse data, maybe that should be tackled instead of hidden by disabling the benchmark.

@flying-sheep flying-sheep added this to the 1.10.2 milestone Apr 26, 2024
@flying-sheep flying-sheep changed the title Extend-benchmarks-tutorials Extend benchmarks from basic tutorial Apr 26, 2024
Copy link

codecov bot commented Apr 26, 2024

Codecov Report

Attention: Patch coverage is 69.23077% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 76.27%. Comparing base (d2a5368) to head (69f9781).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3031      +/-   ##
==========================================
- Coverage   76.27%   76.27%   -0.01%     
==========================================
  Files         117      117              
  Lines       12799    12803       +4     
==========================================
+ Hits         9763     9766       +3     
- Misses       3036     3037       +1     
Files Coverage Δ
scanpy/preprocessing/_highly_variable_genes.py 95.60% <100.00%> (ø)
scanpy/preprocessing/_scrublet/core.py 92.90% <100.00%> (+0.14%) ⬆️
scanpy/preprocessing/_simple.py 87.28% <100.00%> (ø)
scanpy/preprocessing/_scrublet/__init__.py 96.70% <0.00%> (-1.08%) ⬇️
scanpy/tools/_rank_genes_groups.py 94.33% <50.00%> (ø)

@flying-sheep flying-sheep force-pushed the extend-benchmarks-tutorials branch from 9954c43 to de2ae21 Compare April 29, 2024 15:26
Copy link

scverse-benchmark bot commented May 7, 2024

Benchmark changes

Change Before [d2a5368] After [69f9781] Ratio Benchmark (Parameter)
- 1.24G 1.03G 0.83 preprocessing_counts.peakmem_scrublet('pbmc3k')
- 466M 366M 0.79 preprocessing_counts.peakmem_scrublet('pbmc68k_reduced')
- 7.53±0.3ms 6.76±0.08ms 0.9 preprocessing_counts.time_log1p('pbmc3k')
- 11.2±0.06s 3.22±0.08s 0.29 preprocessing_counts.time_scrublet('pbmc3k')

Comparison: https://github.com/scverse/scanpy/compare/d2a53680e312835b998077b4e25b254e98bcb5ba..69f9781b5ae3970d735c3b68879a791164d6a54d
Last changed: Mon, 13 May 2024 14:51:19 +0000

More details: https://github.com/scverse/scanpy/pull/3031/checks?check_run_id=24903996679

@flying-sheep flying-sheep marked this pull request as ready for review May 7, 2024 16:25
@flying-sheep flying-sheep requested a review from ilan-gold May 7, 2024 16:26
@flying-sheep
Copy link
Member Author

flying-sheep commented May 7, 2024

Oof, half an hour is definitely too long. I wonder why it takes that long.

I can finish this up on friday

@ilan-gold
Copy link
Contributor

@flying-sheep Not sure. Now that you mention it, the rapids benchmark also takes more time than I'd expect as well...The functions themselves are quite fast.

@flying-sheep flying-sheep mentioned this pull request May 8, 2024
benchmarks/benchmarks/preprocessing.py Outdated Show resolved Hide resolved
benchmarks/benchmarks/preprocessing.py Outdated Show resolved Hide resolved
benchmarks/benchmarks/utils.py Outdated Show resolved Hide resolved
scanpy/preprocessing/_scrublet/__init__.py Show resolved Hide resolved
@flying-sheep flying-sheep requested a review from ilan-gold May 13, 2024 15:04
@flying-sheep flying-sheep merged commit c26480e into main May 14, 2024
14 of 15 checks passed
@flying-sheep flying-sheep deleted the extend-benchmarks-tutorials branch May 14, 2024 09:47
meeseeksmachine pushed a commit to meeseeksmachine/scanpy that referenced this pull request May 14, 2024
flying-sheep added a commit that referenced this pull request May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Benchmarks: Add all from basic tutorial
2 participants