Diagonal(RowVector) segfaults

[andreasnoack]

Found by @alanedelman,

julia> Diagonal(randn(4)')
[1]    47132 segmentation fault  julia-dev

[JeffBezanson]

Stack overflow of course. I don't know why that continues to segfault on some platforms. Here's the stack trace:

ERROR: StackOverflowError:
Stacktrace:
 [1] steprange_last(::Int64, ::Int64, ::Int64) at ./range.jl:0
 [2] Type at ./range.jl:93 [inlined]
 [3] _range at ./range.jl:61 [inlined]
 [4] range at ./range.jl:60 [inlined]
 [5] diagind(::Int64, ::Int64, ::Int64) at ./linalg/dense.jl:237
 [6] diag(::RowVector{Float64,Array{Float64,1}}, ::Int64) at ./linalg/dense.jl:279
 [7] Diagonal(::RowVector{Float64,Array{Float64,1}}) at ./linalg/diagonal.jl:28 (repeats 79997 times)

[andreasnoack]

Yes, the segfault was on macOS and I just tried Linux and indeed it gives the StackOverflowError instead of segfaulting. The issue is that StepRange slicing of a RowVector still produces a RowVector so https://github.com/JuliaLang/julia/blob/7e143bb33375357f0b110443d128f30ca09ffb62/base/linalg/diagonal.jl#L28 is called recursively.

@andyferris I assume that we have discussed linear slicing of RowVectors at some point but I don't recall the details. Would it be possible to return a Vector when slicing a RowVector with a StepRange or would it break everything? My understanding is that a RowVector is mainly special when used in linear algebra operations and that it should be more like an AbstractMatrix for something like indexing.

[andyferris]

Yes, the AbstractMatrix interface takes precedence. I thought I only overloaded hcat?

[andyferris]

The problem is these lines are way to greedy. With JuliaLang/julia#23424 it will be easy to recreate another row vector; I can correct the problem there or we can do it separately.

[andreasnoack]

It would be better not to bundle this with JuliaLang/julia#23424 since, if possible, the fix should be backported.