Julia hangs on DataFrame creation

[blakejohnson]

On 9199a9e3f3521946dda82418dd1e041945787f74,

using DataFrames
df = DataFrame()

causes Julia to spin out into an infinite loop in inference.jl.

Anyone else seeing this?

[johnmyleswhite]

I'm seeing this as well. Working on it now.

[johnmyleswhite]

Still trying to figure this out. The problem is pretty endemic. The following also fails:

using DataFrames
DataArray{Int, 1}(Int[1, 2], falses(2))

Sometimes it seems the failure lies in how inference is used to call repl_show, but it's not clear to me yet.

[milktrader]

Hangs for me too

df = read_table(Pkg.dir("TimeSeries/test/data/GSPC.csv"));

I didn't have this problem this weekend so I'm checking out Julia commits to track it down.

[milktrader]

The problem goes away when using the 12c51d7a8da702b4a25c1774b7bebbd2aae2e21d commit.

#before this commit

c3418cb11833480138a5c476568b3eb56ed07ce7 fix #2562


#after this commit

12c51d7a8da702b4a25c1774b7bebbd2aae2e21d Rename known-mean variants to stdm and varm.

[blakejohnson]

So, it doesn't work on JuliaLang/julia@d74ad66?

[milktrader]

Not sure. It does work on 12c51d7a8da702b4a25c1774b7bebbd2aae2e21d.

[blakejohnson]

I see. You are bisecting?

[milktrader]

Brute force

[kmsquire]

As @blakejohnson suggested, git bisect is a much nicer way to track these things down:

JuliaLang/julia#1689 (comment)

[kmsquire]

So, since this is also affecting me, I went through git bisect with julia. It turns out that JuliaLang/julia@bcfc42d is the last julia build that works with DataFrames, and JuliaLang/julia@c3418cb, mentioned by @milktrader, is the commit which broke julia.

This means it could be an upstream bug, but not necessarily.

[kmsquire]

s/upstream/julia/

[johnmyleswhite]

Thanks for looking into this, Kevin. I haven't had time to work out the source of our troubles. I believe it lies in repl_show, but that's going on very little evidence. The fact that even DataArray doesn't work suggests something's gone wrong quite deep in the architecture.

[kmsquire]

This is being caused by one of the DataArray constructor definitions, which seems to be causing julia's inference to go into an infinite loop (see JuliaLang/julia#2607 (comment)).

Simple fix is to comment out this constructor (https://github.com/HarlanH/DataFrames.jl/blob/master/src/dataarray.jl#L74), but someone with more familiarity with the code could probably come up with a better fix, at least until @JeffBezanson comes up with a fix on the julia side.

[johnmyleswhite]

Thanks for tracking this down, Kevin. I guess it's not surprising that the catch-all ::Any constructor is the source of trouble, although I'm not immediately sure how to work around that problem.

[JeffBezanson]

It's only my problem; type inference should terminate for all julia code no matter what.

[kmsquire]

This is fixed in the latest Julia/DataFrames.

[blakejohnson]

Indeed. Closing.

[johnmyleswhite]

Thanks for handling this, everyone!