Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add rustdoc and clippy to github workflow #618

Closed
wants to merge 11 commits into from

Conversation

hellow554
Copy link
Contributor

This commit is quite a mouth full and I'm can understand if you want me to split this into smaller chunks.

Let me start explain it commit by commit.

First I created a clippy toml with the msrv set to 1.36.0, which is the version used in the github workflow.

Next I either fixed all the linted lines, or added a allow with a comment to it.
I did the same for all the tests.

Next I saw, that there is currently a rustfmt which skips the complete source base, based on a commit two years ago: 641671e

"Temporarily disable" 😉

I then reformatted the whole codebase, which might break some PRs but they should be able to rebase it but just using cargo fmt easily.

Next, I fixed two warnings. One stray ``` and the other one being changed in rust-lang/rust#96676 where you have to specify a macro link.

Last, but not least, I added rustfmt rustdoc and clippy to the ci.yml to enable them by default on PRs.


Feel free to comment whatever you think. I try to answer or change it :)

Thanks for the awesome crate :)

Copy link
Member

@phimuemue phimuemue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wohoo, there's some really nice improvements in there! Maybe you can rebase against the most recent git, as some of the things should already be fixed.

In addition, may I ask you for a favor that would imho simplify reviewing quite a bit:

  • Taking up your idea: Could you actually split all the lints into separate commits (one commit per fixed lint, possibly even separated for tests/non-tests)? Sure, it'd be more commits, but it avoids the constant context switching. In addition, we could theoretically cherry-pick the ones that are already good-to-go, while leaving others for later.

  • Possibly postpone the cargo fmt-related stuff to an own PR (after all the other fixes arrived) - or at least do it as the last commit so the first steps could be cherry-picked.

    We once had the discussion cargo fmting everything, and we plan to do it eventually, but it would probably break virtually all existing PRs - something we do not want to do lightly.

    In addition, I am still reluctant to reject PRs that do not adhere to cargo fmt: I think it would be unfortunate if a contributor would have to rework stuff because of a x+1 vs x + 1. However, I'd still be happy about an automated solution that would commit the cargo fmt as a PR automatically if it observes that it is required. (See format all rust files #421 for more discussion.)

src/lib.rs Outdated Show resolved Hide resolved
src/ziptuple.rs Show resolved Hide resolved
src/adaptors/mod.rs Outdated Show resolved Hide resolved
src/groupbylazy.rs Outdated Show resolved Hide resolved
src/groupbylazy.rs Outdated Show resolved Hide resolved
tests/quick.rs Outdated Show resolved Hide resolved
.github/workflows/ci.yml Outdated Show resolved Hide resolved
.rustfmt.toml Outdated Show resolved Hide resolved
@hellow554
Copy link
Contributor Author

So this exploded a bit. It went from 5 to 31 commits, but yeah, it may be easier for you to review, I can understand that.

I removed the fmt part completly and will open a different PR, so no worries about that anymore.

@hellow554 hellow554 changed the title add rustfmt, rustdoc and clippy to github workflow add rustdoc and clippy to github workflow Jun 3, 2022
Copy link
Member

@phimuemue phimuemue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! Thanks for your effort, it's appreciated.

I commented the places that jumped to my eye. (All others look good to me.) Maybe you could dispel my concerns about the few remaining bits.

src/lib.rs Outdated Show resolved Hide resolved
src/adaptors/mod.rs Outdated Show resolved Hide resolved
src/multipeek_impl.rs Outdated Show resolved Hide resolved
.unwrap_or_else(|| buffer.len())
.unwrap_or(buffer.len())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity: Was this suggested by clippy? If not, I'd stay with the original

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes: https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_lazy_evaluations

But there's an open issue for that, because I find that strange too: rust-lang/rust-clippy#8109

Copy link
Member

@phimuemue phimuemue Jun 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not find authoritative reference that calling len is guaranteed to only read a field, so please revert this to unwrap_or_else, and possibly add an explicit allow.

tests/quick.rs Outdated
if it.next().is_none() {
panic!("Iterator shouldn't be finished, may not be deterministic");
}
assert!(it.next().is_some(), "Iterator shouldn't be finished, may not be deterministic");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this a second time raises the question: Is it problematic that it.next appears within the assert? (In my ideal world, an assert should be side-effect free.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see your concern. In my world, side effects in assert is ok, in debug_assert it's bad (and there's a lint for that (written by me): https://rust-lang.github.io/rust-clippy/master/index.html#debug_assert_with_mut_call)

But I would understand if you would like to have that different. Please tell me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contrary to my initial giving in: Let's strive to keep asserts side-effect free. (A correct program should work even if someone decides to redefine assert to a no-op.)

Comment on lines -237 to +231
#[derive(PartialEq, Debug)]
#[derive(PartialEq, Eq, Debug)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends. Might Foo be explicitly used to test stuff that only relies on PartialEq - and not Eq?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine for me.

Comment on lines 30 to 31
- run: RUSTDOCFLAGS="-Dwarnings" cargo doc --all-features
- run: cargo clippy --all -- -Dwarnings
Copy link
Member

@phimuemue phimuemue Jun 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you quickly elaborate what would happen if someone pushes something that does not adhere to cargo clippy ? Would this PR be rejected?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it would be rejected. Same goes for cargo doc (and cargo fmt in the future as well).

Isn't that the point of having a CI, that the code is equally good (or bad)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tl;dr

I'd like to have both a low bar for (especially new) contributors and try to acchieve a certain quality in the code base. Thus:

  • I'm in favor of rejecting wrong documentation, as the result is user-facing and should be correct.
  • I'm unsure if rejecting PRs with lint problems is a good idea.
  • I'm very skeptical to reject PRs because of formatting.

cargo clippy

I assume the codebase will quickly converge to a lint-free state (especially after your changes are merged). Once in that state, avoiding new lint problems should be easy.

From my personal experience, I found it demotivating if a PR is rejected because of a clippy-false-positive (or one that does not matter in the circumstances at hand). I'd like to avoid this situation as far as possible.

  • For one-off contributors, requesting and waiting for changes might be just as much effort as just merging and fixing the lint, possibly indicating that clippy should only warn instead of reject the PR.
  • I assume frequent contributors do not introduce many new lints. (They may have clippy running on their local machines, so are aware of lint problems from the start.)

Generally, I think I'd prefer to only issue a warning (thus requiring human interaction to resolve it) instead of plainly rejecting the PR, assuming that it does not generate too much work on the contributors and the maintainers. I would be open to have clippy reject PRs if it's easier/better to implement/maintain/work with/others really want it/etc, as it sometimes gives really useful advice.

cargo fmt

I guess the main advantage of having a cargo fmtd code base is that contributors have an idea how our code is written. However, for this it should be enough to "cargo fmt from time to time" (read: after a successfully merged PR). As such, I'd prefer not to reject PRs because of formatting problems (again to avoid frustration) - and instead have the formatting problems fixed automatically by our CI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on your side with clippy. I pushed the change just now.

As for formatting, I'm somewhat against your idea. The burden of a contributor is to run cargo fmt prior to git push, so that should be somewhat pain free.

I can understand your point of view that some people just do "a line change", open a PR and never come back again, so you have a failing CI.
But, IIRC you, as a maintainer, can still merge that, despite the failing CI and "fix it later", i.e. after the merge. Alternativly you can always push changes to the branch, when the opener has the tick Allow edits and access to secrets by maintainers which is the default when opening a PR.

But let's talk about cargo fmt in #619 as it's no longer part of this PR.

src/unique_impl.rs Outdated Show resolved Hide resolved
@hellow554
Copy link
Contributor Author

I try to resolve the issues.

I see some concerns and I think thats ok. Everything you think is not worth changing or might be bad can be annotated by #[allow(clippy::xxx)] along with a comment.
That's totally fine, really.

I'm not a perfect rust developer, neither is clippy perfect at detecting whether a change is good or bad.
But I think, clippy suggests a lot of useful lints worth checking at least. Let's try to solve this one together :)

@phimuemue phimuemue mentioned this pull request Jun 8, 2022
bors bot added a commit that referenced this pull request Jun 8, 2022
624: Hellow554 lints 1 r=phimuemue a=phimuemue

First batch of #618.

bors r+

Co-authored-by: Marcel Hellwig <[email protected]>
@phimuemue
Copy link
Member

Hi. I took the liberty to merge the first batch of this. (See #624 - sorry if this was a stupid idea, I'm a github noob.)

Could you rebase so that this PR is updated, and it's clear what remains to be reviewed?

@hellow554
Copy link
Contributor Author

I think that was a good idea, so no worries :)

Rebased.

I think at least bf2c114 should be pulled in as well, but we can do that later. Without it you may get a lot of false clippy suggestions.

@phimuemue phimuemue mentioned this pull request Jun 24, 2022
bors bot added a commit that referenced this pull request Jun 24, 2022
627: Hellow554 lints 2 r=phimuemue a=phimuemue

Second batch of #618.

Co-authored-by: Marcel Hellwig <[email protected]>
Copy link
Member

@phimuemue phimuemue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took the freedom to merge a second batch of this.

@hellow554 Thank you! If you find time, could you let us know if you want to further push things forward on this PR?

clippy.toml Outdated
@@ -0,0 +1 @@
msrv = "1.36.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need some remark that this line has to be updated along other files? Apart from that, I'm fine with this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends. I can't find any word about the msrv, apart from the github workflow, so first step would be to mention it in the readme I guess?
Second: there's the rust-version field in the Cargo.toml file that can be used as well. A clippy PR is pending to parse that field in addition to the clippy.toml file: rust-lang/rust-clippy#8774

As soon as that lands, the clippy.toml can be deleted again, so I don't think that that should be documented? 😅

tests/quick.rs Outdated Show resolved Hide resolved
tests/quick.rs Outdated
if it.next().is_none() {
panic!("Iterator shouldn't be finished, may not be deterministic");
}
assert!(it.next().is_some(), "Iterator shouldn't be finished, may not be deterministic");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contrary to my initial giving in: Let's strive to keep asserts side-effect free. (A correct program should work even if someone decides to redefine assert to a no-op.)

@hellow554 hellow554 force-pushed the clippy-lints branch 2 times, most recently from cfeed52 to 8f411a4 Compare June 27, 2022 06:39
@phimuemue
Copy link
Member

Hi again. If you still want to tackle this: Could you rebase and sqash the fixup into the original commits to avoid back-and-forth changes (and thus, simplify review)? And could you leave the unwrap_or_else in place, and just allow the lint?

@hellow554 hellow554 force-pushed the clippy-lints branch 2 times, most recently from 74bcfe4 to 986992a Compare July 7, 2022 09:15
@hellow554
Copy link
Contributor Author

hellow554 commented Jul 7, 2022

Done :)

I very much thank you for your patience and effort you put into this! It's not easy to review this kind of stuff. Many, many thanks!

Btw: I found many false-positives while using clippy, which helps all users, so many thanks for the many test cases ;)

@phimuemue phimuemue mentioned this pull request Jul 7, 2022
bors bot added a commit that referenced this pull request Jul 7, 2022
630: Hellow554 lints 3 r=phimuemue a=phimuemue

Next batch of #618.

bors r+

Co-authored-by: Marcel Hellwig <[email protected]>
@phimuemue
Copy link
Member

Done :)

I very much thank you for your patience and effort you put into this! It's not easy to review this kind of stuff. Many, many thanks!

Btw: I found many false-positives while using clippy, which helps all users, so many thanks for the many test cases ;)

My pleasure. I merged the third batch in #630.

@hellow554 hellow554 force-pushed the clippy-lints branch 2 times, most recently from 2e2dc50 to 071aa68 Compare July 7, 2022 20:46
@hellow554
Copy link
Contributor Author

Rebased again. Take your time, I'm not in a hurry

@hellow554 hellow554 force-pushed the clippy-lints branch 2 times, most recently from afec10a to cf4e2ae Compare January 10, 2024 12:44
@hellow554
Copy link
Contributor Author

I rebased the work to the latest master. I'm very happy with your decision to leave this open and maybe cherry pick the things you like. That's totally fine to me.
Feel free to browse the commits.

This was referenced Jan 10, 2024
@Philippe-Cholet
Copy link
Member

I kept find_position improvement (merged), Eq implementation for MinMaxResult (merged) and two annoying lints (PR sent), I'm gonna stop here though.

The only remaining question for me here before closing is about adding or not RUSTDOCFLAGS="-Dwarnings" cargo doc --all-features to the CI. I'm not familiar with cargo doc much yet. What kind of error would it prevent: wrong links? bad formatting?

I'm in favor of rejecting wrong documentation, as the result is user-facing and should be correct. - @phimuemue

@hellow554
Copy link
Contributor Author

hellow554 commented Jan 10, 2024

What kind of error would it prevent: wrong links?

Correct. For examples of what this denies see https://doc.rust-lang.org/rustdoc/lints.html

Bad formatting, no: that's what rustfmt is for.

@Philippe-Cholet
Copy link
Member

I was thinking wrong markdown text formatting such as This is **bold* text. but fair enough, thanks for the insightful link.

@jswrenn
Copy link
Member

jswrenn commented Jan 10, 2024

We should absolutely deny warnings in CI!

@Philippe-Cholet
Copy link
Member

I don't know to how many PRs this one lead to but it surely was a great base, thank you.

@danieleades
Copy link
Contributor

We should absolutely deny warnings in CI!

Only if the version of the linter in CI is pinned. It isn't currently.

Otherwise you can introduce breakages in contributor PRs that have nothing to do with the changes in the PR.

This should be done regardless of whether warnings are denied, in my opinion.

@Philippe-Cholet
Copy link
Member

Philippe-Cholet commented Jan 11, 2024

@danieleades It sure seems a good idea (even if it should not be a lot of breakages).
In CI, instead of "stable", use a pinned version such as 1.75.0 and update it from time to time?
I guess dependabot should be able to help us. However, #758 wanted us to drop 1.43.1 (our msrv) so it's not that helping yet.

@danieleades
Copy link
Contributor

danieleades commented Jan 11, 2024

@danieleades It sure seems a good idea (even if it should not be a lot of breakages).
In CI, instead of "stable", use a pinned version such as 1.75.0 and update it from time to time?
I guess dependabot should be able to help us. However, #758 wanted us to drop 1.43.1 (our msrv) so it's not that helping yet.

I'd suggest pinning to a recent nightly version of clippy. The 'rust-version' (MSRV) config will prevent clippy from suggesting incompatible changes

unfortunately dependabot won't be able to automatically update a nightly version

@Philippe-Cholet
Copy link
Member

Clippy Doc recommands stable in our case for maximum compatibility. And I personally prefer that contributors can run the same clippy as the CI one on their computer without having to install nightly.

#740 (comment) agrees about possible (rare) breakage.

@jswrenn @phimuemue @mightyiam Should we pin clippy version to a (still stable IMO) recent version of Rust (and therefore maintain it) to avoid eventual breakage with contributor PRs?

@mightyiam
Copy link
Contributor

Once a while, a new stable rust toolchain is released, along with some new clippy lints, usually. Some of them are on by default. And some of those may trigger on our code. Usually, they are easy to resolve. And if they're not, we can allow on the item. Using "stable" rolling clippy means that in those cases, development will be blocked until they are resolved. That should motivate whomever wishes to see their PR merged to resolve the lint in a prior PR. Then, rebasing is easy. There could even be a button for it in PR pages (that's an option in the repo settings). Therefore I vote rolling stable clippy.

@Philippe-Cholet
Copy link
Member

(As a member,) I have access to a "rebase" button.
I also (for now) would choose small rare blocks over manual pin maintenance. We can still change our mind later. And if there was a big block, we can still temporarily pin it in the CI to resolve the block later.

@danieleades
Copy link
Contributor

actually i think if you use the dtolnay toolchain action to pin to a stable version dependabot will create PRs to bump the version.
That gives you the best of both worlds- linting is pinned and consistent across PRs, and dependabot automagically creates PRs which highlight any new lints or changes to be addressed as they are released.

@mightyiam
Copy link
Contributor

That sounds good, as well.

@danieleades
Copy link
Contributor

That sounds good, as well.

see #846

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants