-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make Iterators.peel return nothing when itr empty #39607
Conversation
Fixes JuliaLang#39569. Old behaviour: ```jl try h, t = Iterators.peel(itr) # do something with h and t catch e if e isa BoundsError # itr was probably empty else rethrow() end end ``` New behaviour: ```jl x = Iterators.peel(itr) if isnothing(x) # itr was empty end h, t = x ```
test/iterators.jl
Outdated
@testset "Iterators.peel" begin | ||
@test Iterators.peel([]) == nothing | ||
@test Iterators.peel(1:10)[1] == 1 | ||
@test Iterators.peel(1:10)[2] |> collect == 2:10 | ||
@test Iterators.peel(x^2 for x in 2:4)[1] == 4 | ||
@test Iterators.peel(x^2 for x in 2:4)[2] |> collect == [9, 16] | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Grep didn't find any existing tests of Iterators.peel, so I added a few simple ones. I'm using the arrow syntax so the characters line up better, making it clearer when I'm calling with the same arguments.
You don’t need to use |
That's a good point. My initial motivation for this PR was that isempty is unsafe for stateful iterators: the fallback implementation consumes a value from the iterator while testing if it is empty. I have some other PRs that fix eachline and readeach, but there will be more around. Anyway, I'd forgotten that those things were related by the time I wrote the commit message for this. If we could fix the isempty fallback method in a backwards compatible way (we'd probably need to add a trait), then that would make this PR redundant, I think, but I'm not sure it can be fixed before V2. |
Can triage give an opinion on this, please? I've updated the first post to reflect all discussion on the original issue and this PR up to this point, so that's all that needs to be read. |
After #39607, `Iterators.peel` can return `nothing` and the return type can be more inaccurate than before for certain iterative inputs where inference doesn't understand its length, e.g. `Iterators.peel(::<:AbstractString)`. The following JET analysis shows an example of this type instability: ```julia ═════ 2 possible errors found ═════ ┌ @ coreio.jl:4 Base.println(Core.tuple(Core.typeassert(Base.stdout, Base.IO)), xs...) │┌ @ strings/io.jl:73 Base.print(Core.tuple(io), xs, Core.tuple("\n")...) ││┌ @ strings/io.jl:46 Base.print(io, x) │││┌ @ show.jl:1283 Base.show_unquoted(Base.IOContext(io, Base.=>(:unquote_fallback, false)), ex, 0, -1) ││││┌ @ show.jl:1634 Base.show_unquoted_quote_expr(io, Base.getproperty(ex, :value), indent, prec, 0) │││││┌ @ show.jl:1651 Base.isidentifier(sym) ││││││┌ @ show.jl:1339 Base.isidentifier(Base.string(s)) │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 1) ││││││││┌ @ tuple.jl:92 Base.iterate(I) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing}): Base.iterate(I::Nothing) ││││││││└─────────────── │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 2, _3) ││││││││┌ @ tuple.jl:97 Base.iterate(I, state) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing, Int64}): Base.iterate(I::Nothing, state::Int64) ││││││││└─────────────── Core.Const(nothing) ``` This PR adds some manual annotations and seems to fix some regressions within Base.
This change could lead to type inference inaccuracy for certain iterators that inference can't understand its length, e.g. |
After #39607, `Iterators.peel` can return `nothing` and the return type can be more inaccurate than before for certain iterative inputs where inference doesn't understand its length, e.g. `Iterators.peel(::<:AbstractString)`. The following JET analysis shows an example of this type instability: ```julia ═════ 2 possible errors found ═════ ┌ @ coreio.jl:4 Base.println(Core.tuple(Core.typeassert(Base.stdout, Base.IO)), xs...) │┌ @ strings/io.jl:73 Base.print(Core.tuple(io), xs, Core.tuple("\n")...) ││┌ @ strings/io.jl:46 Base.print(io, x) │││┌ @ show.jl:1283 Base.show_unquoted(Base.IOContext(io, Base.=>(:unquote_fallback, false)), ex, 0, -1) ││││┌ @ show.jl:1634 Base.show_unquoted_quote_expr(io, Base.getproperty(ex, :value), indent, prec, 0) │││││┌ @ show.jl:1651 Base.isidentifier(sym) ││││││┌ @ show.jl:1339 Base.isidentifier(Base.string(s)) │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 1) ││││││││┌ @ tuple.jl:92 Base.iterate(I) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing}): Base.iterate(I::Nothing) ││││││││└─────────────── │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 2, _3) ││││││││┌ @ tuple.jl:97 Base.iterate(I, state) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing, Int64}): Base.iterate(I::Nothing, state::Int64) ││││││││└─────────────── Core.Const(nothing) ``` This PR adds code refactors or manual annotations and seems to fix some of regressions within Base.
After #39607, `Iterators.peel` can return `nothing` and the return type can be more inaccurate than before for certain iterative inputs where inference doesn't understand its length, e.g. `Iterators.peel(::<:AbstractString)`. The following JET analysis shows an example of this type instability: ```julia ═════ 2 possible errors found ═════ ┌ @ coreio.jl:4 Base.println(Core.tuple(Core.typeassert(Base.stdout, Base.IO)), xs...) │┌ @ strings/io.jl:73 Base.print(Core.tuple(io), xs, Core.tuple("\n")...) ││┌ @ strings/io.jl:46 Base.print(io, x) │││┌ @ show.jl:1283 Base.show_unquoted(Base.IOContext(io, Base.=>(:unquote_fallback, false)), ex, 0, -1) ││││┌ @ show.jl:1634 Base.show_unquoted_quote_expr(io, Base.getproperty(ex, :value), indent, prec, 0) │││││┌ @ show.jl:1651 Base.isidentifier(sym) ││││││┌ @ show.jl:1339 Base.isidentifier(Base.string(s)) │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 1) ││││││││┌ @ tuple.jl:92 Base.iterate(I) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing}): Base.iterate(I::Nothing) ││││││││└─────────────── │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 2, _3) ││││││││┌ @ tuple.jl:97 Base.iterate(I, state) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing, Int64}): Base.iterate(I::Nothing, state::Int64) ││││││││└─────────────── Core.Const(nothing) ``` This PR adds code refactors or manual annotations and seems to fix some of regressions within Base.
After #39607, `Iterators.peel` can return `nothing` and the return type can be more inaccurate than before for certain iterative inputs where inference doesn't understand its length, e.g. `Iterators.peel(::<:AbstractString)`. The following JET analysis shows an example of this type instability: ```julia ═════ 2 possible errors found ═════ ┌ @ coreio.jl:4 Base.println(Core.tuple(Core.typeassert(Base.stdout, Base.IO)), xs...) │┌ @ strings/io.jl:73 Base.print(Core.tuple(io), xs, Core.tuple("\n")...) ││┌ @ strings/io.jl:46 Base.print(io, x) │││┌ @ show.jl:1283 Base.show_unquoted(Base.IOContext(io, Base.=>(:unquote_fallback, false)), ex, 0, -1) ││││┌ @ show.jl:1634 Base.show_unquoted_quote_expr(io, Base.getproperty(ex, :value), indent, prec, 0) │││││┌ @ show.jl:1651 Base.isidentifier(sym) ││││││┌ @ show.jl:1339 Base.isidentifier(Base.string(s)) │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 1) ││││││││┌ @ tuple.jl:92 Base.iterate(I) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing}): Base.iterate(I::Nothing) ││││││││└─────────────── │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 2, _3) ││││││││┌ @ tuple.jl:97 Base.iterate(I, state) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing, Int64}): Base.iterate(I::Nothing, state::Int64) ││││││││└─────────────── Core.Const(nothing) ``` This PR adds code refactors or manual annotations and seems to fix some of regressions within Base.
Co-authored-by: Jameson Nash <[email protected]>
…41094) After JuliaLang#39607, `Iterators.peel` can return `nothing` and the return type can be more inaccurate than before for certain iterative inputs where inference doesn't understand its length, e.g. `Iterators.peel(::<:AbstractString)`. The following JET analysis shows an example of this type instability: ```julia ═════ 2 possible errors found ═════ ┌ @ coreio.jl:4 Base.println(Core.tuple(Core.typeassert(Base.stdout, Base.IO)), xs...) │┌ @ strings/io.jl:73 Base.print(Core.tuple(io), xs, Core.tuple("\n")...) ││┌ @ strings/io.jl:46 Base.print(io, x) │││┌ @ show.jl:1283 Base.show_unquoted(Base.IOContext(io, Base.=>(:unquote_fallback, false)), ex, 0, -1) ││││┌ @ show.jl:1634 Base.show_unquoted_quote_expr(io, Base.getproperty(ex, :value), indent, prec, 0) │││││┌ @ show.jl:1651 Base.isidentifier(sym) ││││││┌ @ show.jl:1339 Base.isidentifier(Base.string(s)) │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 1) ││││││││┌ @ tuple.jl:92 Base.iterate(I) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing}): Base.iterate(I::Nothing) ││││││││└─────────────── │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 2, _3) ││││││││┌ @ tuple.jl:97 Base.iterate(I, state) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing, Int64}): Base.iterate(I::Nothing, state::Int64) ││││││││└─────────────── Core.Const(nothing) ``` This PR adds code refactors or manual annotations and seems to fix some of regressions within Base.
Co-authored-by: Jameson Nash <[email protected]>
…41094) After JuliaLang#39607, `Iterators.peel` can return `nothing` and the return type can be more inaccurate than before for certain iterative inputs where inference doesn't understand its length, e.g. `Iterators.peel(::<:AbstractString)`. The following JET analysis shows an example of this type instability: ```julia ═════ 2 possible errors found ═════ ┌ @ coreio.jl:4 Base.println(Core.tuple(Core.typeassert(Base.stdout, Base.IO)), xs...) │┌ @ strings/io.jl:73 Base.print(Core.tuple(io), xs, Core.tuple("\n")...) ││┌ @ strings/io.jl:46 Base.print(io, x) │││┌ @ show.jl:1283 Base.show_unquoted(Base.IOContext(io, Base.=>(:unquote_fallback, false)), ex, 0, -1) ││││┌ @ show.jl:1634 Base.show_unquoted_quote_expr(io, Base.getproperty(ex, :value), indent, prec, 0) │││││┌ @ show.jl:1651 Base.isidentifier(sym) ││││││┌ @ show.jl:1339 Base.isidentifier(Base.string(s)) │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 1) ││││││││┌ @ tuple.jl:92 Base.iterate(I) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing}): Base.iterate(I::Nothing) ││││││││└─────────────── │││││││┌ @ show.jl:1335 Base.indexed_iterate(Base.Iterators.peel(s), 2, _3) ││││││││┌ @ tuple.jl:97 Base.iterate(I, state) │││││││││ no matching method found for call signature (Tuple{typeof(iterate), Nothing, Int64}): Base.iterate(I::Nothing, state::Int64) ││││││││└─────────────── Core.Const(nothing) ``` This PR adds code refactors or manual annotations and seems to fix some of regressions within Base.
Fixes #39569.
Old behaviour:
peel
throwsBoundsError
whenitr
is empty.New behaviour:
peel
returnsnothing
whenitr
is empty.Motivation:
head, tail... = itr
is a fundamental operation for lots of recursive code, and it would be nice if the lazy version of that (peel
) were more ergonomic to use. This PR makes it about as easy to use asiterate
.Impact: this is a bit backwards incompatible, but I did a code search on GitHub, and none of the 140 matches for Iterators.peel would be broken by this change because they all try to iterate the object returned by
peel
immediately, so they will still get a BoundsError ifitr
is empty.Argument against: if
isempty(itr)
works well then this pattern is fine and works right now:The issue is that
isempty
is not safe for all stateful iterators (see discussion at #27412).