Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: Update test to reflect new report generation behavior #3210

Merged
merged 3 commits into from
Jan 25, 2024

Conversation

mgxd
Copy link
Collaborator

@mgxd mgxd commented Jan 18, 2024

Closes #3209

The exit code for --reports-only should be an indicator of report generation status, not whether or not errors were present in the workflow (those should be reported separately, at the time they are encountered during processing).

mgxd added 2 commits January 18, 2024 16:13
The exit code for `--reports-only` should be an indicator of report generation status.
@effigies
Copy link
Member

I think we do still want to exit non-zero if there were errors detected. It's just that the error counting was removed from the report generation. I was about to work on this, but would you rather?

Copy link

codecov bot commented Jan 18, 2024

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (9568ccb) 72.03% compared to head (6fb9f17) 72.11%.
Report is 31 commits behind head on master.

Files Patch % Lines
fmriprep/cli/run.py 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3210      +/-   ##
==========================================
+ Coverage   72.03%   72.11%   +0.07%     
==========================================
  Files          55       55              
  Lines        4101     4145      +44     
==========================================
+ Hits         2954     2989      +35     
- Misses       1147     1156       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mgxd
Copy link
Collaborator Author

mgxd commented Jan 18, 2024

I'm going back and forth on this - now that we're examining it, it feels like there are separate cases.

  1. During the regular workflow processing, absolutely, the return code should be nonzero as something went wrong, regardless of report generation result.

  2. If running --reports-only, (1) had to have been run and the errors made apparent. So asking fmriprep to regenerate reports, which if it does, feels like it should return a "successful" code.

Perhaps we should expand the return code options and add a new one that specifies report generation was successful, but errors were present?

@effigies
Copy link
Member

Ugh. I forgot that we split up the reports-only and the main implementation. Yes, if we're doing reports-only and it's not just a dry run with reports at the end, then sure, that can exit with 0 as long as the reports actually generate.

IMO the CI test is significantly less useful than it was when written, now that we duplicate report generation in two branches. We could just drop it or at least change it to grep for the error text in the HTML, rather than just checking the exit code.

@mgxd mgxd requested a review from effigies January 25, 2024 21:27
@effigies effigies merged commit 9670513 into nipreps:master Jan 25, 2024
16 of 17 checks passed
@mgxd mgxd deleted the tst/report branch January 26, 2024 15:16
effigies added a commit that referenced this pull request Jun 17, 2024
24.0.0 (June 17, 2024)

New feature release in the 24.0.x series.

This release is an incremental improvement on 23.2.x, with some
fixes for bugs discovered in the updated workflow.

New features include separation of HTML reports by session for subjects
with many BOLD runs, a new ``--fs-no-resume`` option to improve interoperability
with less typical FreeSurfer directories, such as those generated by longitudinal
FreeSurfer or FastSurfer, and adoption of DatasetLinks and BIDS-URIs, to follow
the recommendations of recent versions of BIDS.

With thanks to Dimitri Papadopoulos, Basile Pinsard, Celine Provins, Taylor Salo
and Wang Hao-Ting for their contributions!

* FIX: Add "double" type to allowed DisplacementFieldTransform (#3287)
* FIX: Require recent templateflow, select correct aparc dseg.tsv (#3256)
* FIX: Ensure proper templates are retrieved with sloppy (#3251)
* FIX: Delete summary from functional report when separated by sessions (#3223)
* FIX: Support lists in bids filter file containing ``null`` or ``*`` (#3215)
* FIX: Re-enable anat fasttrack for dataset without t1w (#3202)
* ENH: Use BIDSURI in init_ds_boldmask_wf (#3297)
* ENH: Add templateflow to DatasetLinks (#3267)
* ENH: Track proximal sources of functional GIFTIs (#3263)
* ENH: Support named derivative paths (#3264)
* ENH: Track Sources for standard-space outputs (#3262)
* ENH: Add --fs-no-resume option to reuse existing FreeSurfer outputs without resuming (#3142)
* ENH: Use BIDS URIs to track Sources in sidecars (#3255)
* ENH: Ignore unselected subjects in BIDSLayoutIndexer (#3236)
* ENH: Add metadata for motion parameters (#3245)
* ENH: Separate anatomical and functional reports per session for densely sampled dataset (#3191)
* ENH: Leverage T2w if available for BOLD -> anat coregistration (#3208)
* RF: Fix ITK warp conversion to nitransforms format (#3300)
* RF: Load report assembler from nireports (#3177)
* DOC: Clarify ``--dvars-spike-threshold`` uses standardized DVARS (#3205)
* TST: Update test to reflect new report generation behavior (#3210)
* STY: Manual conversions to f-strings (#3241)
* STY: Apply ruff/pyupgrade rule UP031 (#3280)
* STY: Lint and style check full repository (#3221)
* STY: Adopt ruff for linting and formatting (#3206)
* MNT: Pin libitk 5.3 and note dependencies (#3298)
* MNT: Upgrade ruff pre-commit, add fixing checks (#3283)
* MNT: Complete transition from flake8/black to ruff (#3279)
* MNT: Apply Repo-Review suggestions (#3194)
* MNT: Verbatim copy of Apache license 2.0 (#3259)
* MNT: Bump cryptography from 41.0.7 to 42.0.4 (#3234)
* MNT: Drop copyright year, unused dunder fields (#3247)
* MNT: Update environment pins (#3226)
* MNT: Bump codecov/codecov-action from 3 to 4 (#3219)
* DOCKER: Restore mincinfo binary (#3249)
* CI: Move to new circle machine tags (#3248)
* CI: Avoid ruff warning (#3244)
* CI: Pass ruff tests (#3243)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The test for crash file count on ds054 requires fixing.
2 participants