Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfault with CSV, SQLite, tables (M1) #1842

Closed
danvinci opened this issue Jan 22, 2022 · 5 comments
Closed

Segfault with CSV, SQLite, tables (M1) #1842

danvinci opened this issue Jan 22, 2022 · 5 comments

Comments

@danvinci
Copy link

danvinci commented Jan 22, 2022

Hi,

the title can be improved but right now I don't know exactly what the issue is about - seems related to #1722

Context:

  • Julia 1.7.1 on M1
  • I have a bunch of CSV files
  • I load these files to a newly-created SQLite file
  • I create new tables resulting from the union of these tables
  • weird stuff happens (Pluto disconnects, some tasks/processes go uncompleted, DB gets corrupted)
  • I start from scratch (delete the DB)
>  From worker 2:	signal (11): Segmentation fault: 11
>       From worker 2:	in expression starting at none:1
>       From worker 2:	getvalue at /Users/dan/.julia/packages/SQLite/5DdLp/src/tables.jl:0
>       From worker 2:	getcolumn at /Users/dan/.julia/packages/SQLite/5DdLp/src/tables.jl:75
>       From worker 2:	unknown function (ip: 0x11aacfd1f)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	getcolumn at /Users/dan/.julia/packages/SQLite/5DdLp/src/tables.jl:77
>       From worker 2:	getcolumn at /Users/dan/.julia/packages/SQLite/5DdLp/src/tables.jl:78 [inlined]
>       From worker 2:	eachcolumns at /Users/dan/.julia/packages/Tables/M26tI/src/utils.jl:127 [inlined]
>       From worker 2:	_buildcolumns at /Users/dan/.julia/packages/Tables/M26tI/src/fallbacks.jl:187
>       From worker 2:	unknown function (ip: 0x11aacf66b)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	buildcolumns at /Users/dan/.julia/packages/Tables/M26tI/src/fallbacks.jl:217 [inlined]
>       From worker 2:	columns at /Users/dan/.julia/packages/Tables/M26tI/src/fallbacks.jl:253 [inlined]
>       From worker 2:	columntable at /Users/dan/.julia/packages/Tables/M26tI/src/namedtuples.jl:171
>       From worker 2:	execute at /Users/dan/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:135
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	execute at /Users/dan/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:144
>       From worker 2:	#execute#3 at /Users/dan/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:153 [inlined]
>       From worker 2:	execute at /Users/dan/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:153 [inlined]
>       From worker 2:	tables at /Users/dan/.julia/packages/SQLite/5DdLp/src/SQLite.jl:665 [inlined]
>       From worker 2:	tables at /Users/dan/.julia/packages/SQLite/5DdLp/src/SQLite.jl:665
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	##function_wrapped_cell#343 at /Users/dan/Documents/pluto_nb/VAERS/v1.jl#==#dba7c5d3-8fb7-49df-827a-a51af3ed6f07:5 [inlined]
>       From worker 2:	##function_wrapped_cell#343 at ./none:0
>       From worker 2:	unknown function (ip: 0x11aace843)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_f__call_latest at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_apply at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	#invokelatest#2 at ./essentials.jl:716
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_apply at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	invokelatest at ./essentials.jl:714
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_apply at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	compute at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:385
>       From worker 2:	#27 at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:540
>       From worker 2:	run_inside_trycatch at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:425
>       From worker 2:	#run_expression#25 at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:540
>       From worker 2:	run_expression##kw at /Users/dan/.julia/packages/Pluto/F4y3q/src/runner/PlutoRunner.jl:455
>       From worker 2:	unknown function (ip: 0x11a922937)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_call at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	eval_body at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_interpret_toplevel_thunk at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_toplevel_eval_flex at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	jl_toplevel_eval_in at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	eval at ./boot.jl:373
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	do_apply at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	#103 at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:274
>       From worker 2:	run_work_thunk at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:63
>       From worker 2:	unknown function (ip: 0x11a910327)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	run_work_thunk at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:72
>       From worker 2:	#96 at ./task.jl:423
>       From worker 2:	unknown function (ip: 0x11a91008b)
>       From worker 2:	jl_apply_generic at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	start_task at /Applications/Julia.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
>       From worker 2:	Allocations: 650141462 (Pool: 649895122; Big: 246340); GC: 124
> Worker 2 terminated.
> Unhandled Task ERROR: IOError: read: connection reset by peer (ECONNRESET)
> Stacktrace:
>   [1] wait_readnb(x::Sockets.TCPSocket, nb::Int64)
>     @ Base ./stream.jl:408
>   [2] (::Base.var"#wait_locked#645")(s::Sockets.TCPSocket, buf::IOBuffer, nb::Int64)
>     @ Base ./stream.jl:894
>   [3] unsafe_read(s::Sockets.TCPSocket, p::Ptr{UInt8}, nb::UInt64)
>     @ Base ./stream.jl:900
>   [4] unsafe_read
>     @ ./io.jl:724 [inlined]
>   [5] unsafe_read(s::Sockets.TCPSocket, p::Base.RefValue{NTuple{4, Int64}}, n::Int64)
>     @ Base ./io.jl:723
>   [6] read!
>     @ ./io.jl:725 [inlined]
> 

Zooming out of the specific task, I'm trying to come up with my own workflow for working with reasonable-sized data without having to rely too much on in-memory DataFrames.

Video here (available soon): https://youtu.be/lkeXCOHWfBc

Notebook: https://github.com/fonsp/Pluto.jl/files/7918391/v1.jl.zip

All files used can be retrieved here: https://vaers.hhs.gov/eSubDownload/index.jsp?fn=AllVAERSDataCSVS.zip

@danvinci danvinci changed the title Notebook disconnects (using SQLite, high disk activity) Notebook disconnects on Distributed.ProcessExitedException Jan 22, 2022
@danvinci
Copy link
Author

I tried using Julia 1.7.1 (x86) and none of the above takes place, runs smoothly 👍

@danvinci danvinci changed the title Notebook disconnects on Distributed.ProcessExitedException Notebook disconnects on Distributed.ProcessExitedException (M1) Jan 22, 2022
@fonsp
Copy link
Owner

fonsp commented Jan 24, 2022

Thanks for the issue! Segfaults are a bug of Julia itself, so we should report it to Julia's issues.

The smaller the example can get, the better. For example, we might be able to take Pluto out of the equation by running the notebook in Julia directly, without Pluto. Does that still produce the segfault?

@fonsp
Copy link
Owner

fonsp commented Jan 24, 2022

This might be fixed by JuliaLang/julia#43664 . Can you try building Julia from master and seeing if that fixes it?

@fonsp fonsp changed the title Notebook disconnects on Distributed.ProcessExitedException (M1) Segfault with CSV, SQLite, tables (M1) Jan 25, 2022
@danvinci
Copy link
Author

danvinci commented Jan 25, 2022

An update on this:

  • running the script alone, Julia 1.7.1 (ARM) works
  • running the script within Pluto, Julia 1.7.1 (ARM) -> Segfault (this issue)
  • running the script within Pluto, Julia 1.7.1 (emulated x86) works
  • running the script within Pluto, Julia master (ARM) works

Tried each x2.

As for building Julia from master, I just used 'make' and I assume it has been compiled for ARM.

@fonsp
Copy link
Owner

fonsp commented Jan 28, 2022

It looks like this will be fixed by Julia 1.8, which will be released in a couple of months. Until then, we recommend using Julia x86! The M1 build is still too experimental.

Thanks for opening the issue and for the useful testing!! Let us know if something comes up again!

@fonsp fonsp closed this as completed Jan 28, 2022
pankgeorg added a commit that referenced this issue Feb 8, 2022
> Try this release in your browser! _(Available 30 minutes after the release)_
>
> <a href="https://binder.plutojl.org/v0.18.0/"><img alt="Binder logo" src="https://mybinder.org/badge_logo.svg"></a>

**Generate your own binder links using [pluto-on-binder.glitch.me](https://pluto-on-binder.glitch.me/)!**

[Diff since v0.17.7](v0.17.7...v0.18.0)

**Merged pull requests:**
- Make document currentscript work (#1765) (@dralletje)
- don't count logs messages multiple times for pluto-logs width (#1843) (@Pangoraw)
- Pattern matching system for lezer code (#1849) (@dralletje)
- minor fix: Tables integration caused method ambiguation for Union{}[] (#1851) (@16bitmood)
- Fix #1852, relocatable changes for sysimage usage. (#1853) (@MichaelHatherly)
- Re-enable parseMixed (#1855) (@pankgeorg)
- DOI in REAMDE for citation (#1864) (@rikhuijzer)
- Use pkgdir instead of pathof (#1867) (@rikhuijzer)
- Hide banner in CI (#1868) (@rikhuijzer)
- Server-side events as configuration (#1871) (@ctrekker)
- fix is_stdlib() for nightly and enable nightly tests (#1874) (@Pangoraw)
- Awesome! Line wrapping! (#1876) (@dralletje)
- respect maxlog keyword for logs, fixes #1858 (#1877) (@Pangoraw)
- allow .. as an identifier, fixes #1875 (#1878) (@Pangoraw)
- Server start event (#1882) (@ctrekker)
- Julia 1.6 minimum (#1885) (@fonsp)
- New heuristic to skip intermediate bond values (#1892) (@fonsp)
- Send queued bond changes when last script finishes (#1898) (@fonsp)

**Closed issues:**
- Pkg.jl will corrupt itself on Windows (#671)
- Failure creating JuMP DenseAxisArray (#806)
- Frontend tests fail too often (#1234)
- Open multiple notebooks at start and without browser (#1278)
- Segmentation faults on Julia 1.7 M1 (#1722)
- Table of content overlaps text in static HTML (#1770)
- Pluto hangs when cell contains `(md" ", md"$t")` (#1800)
- Cell drag drop broken in Linux Chrome v97 (#1809)
- Tables integration causes method ambiguities for `pluto_showable` (#1815)
- Segfault with CSV, SQLite, tables (M1) (#1842)
- Use RelocatableFolders.jl instead of project_relative_dir to allow sysimages (#1852)
- Logging doesn't respect `maxlog` (#1858)
- TOC in dark mode is still light when aside (#1860)
- ProgressLogging: suppressed logging in Turing.jl (#1862)
- Cannot update to version 0.17.7 (#1869)
- Stdlib packages marked as not found (#1872)
- EllipsisNotation.jl (`randn((10,10))[1, ..]`) does not work (#1875)
- Too many intermediate bond values (#1891)
pankgeorg added a commit that referenced this issue Feb 8, 2022
> Try this release in your browser! _(Available 30 minutes after the release)_
>
> <a href="https://binder.plutojl.org/v0.18.0/"><img alt="Binder logo" src="https://mybinder.org/badge_logo.svg"></a>

**Generate your own binder links using [pluto-on-binder.glitch.me](https://pluto-on-binder.glitch.me/)!**

[Diff since v0.17.7](v0.17.7...v0.18.0)

**Merged pull requests:**
- Make document currentscript work (#1765) (@dralletje)
- don't count logs messages multiple times for pluto-logs width (#1843) (@Pangoraw)
- Pattern matching system for lezer code (#1849) (@dralletje)
- minor fix: Tables integration caused method ambiguation for Union{}[] (#1851) (@16bitmood)
- Fix #1852, relocatable changes for sysimage usage. (#1853) (@MichaelHatherly)
- Re-enable parseMixed (#1855) (@pankgeorg)
- DOI in REAMDE for citation (#1864) (@rikhuijzer)
- Use pkgdir instead of pathof (#1867) (@rikhuijzer)
- Hide banner in CI (#1868) (@rikhuijzer)
- Server-side events as configuration (#1871) (@ctrekker)
- fix is_stdlib() for nightly and enable nightly tests (#1874) (@Pangoraw)
- Awesome! Line wrapping! (#1876) (@dralletje)
- respect maxlog keyword for logs, fixes #1858 (#1877) (@Pangoraw)
- allow .. as an identifier, fixes #1875 (#1878) (@Pangoraw)
- Server start event (#1882) (@ctrekker)
- Julia 1.6 minimum (#1885) (@fonsp)
- New heuristic to skip intermediate bond values (#1892) (@fonsp)
- Send queued bond changes when last script finishes (#1898) (@fonsp)

**Closed issues:**
- Pkg.jl will corrupt itself on Windows (#671)
- Failure creating JuMP DenseAxisArray (#806)
- Frontend tests fail too often (#1234)
- Open multiple notebooks at start and without browser (#1278)
- Segmentation faults on Julia 1.7 M1 (#1722)
- Table of content overlaps text in static HTML (#1770)
- Pluto hangs when cell contains `(md" ", md"$t")` (#1800)
- Cell drag drop broken in Linux Chrome v97 (#1809)
- Tables integration causes method ambiguities for `pluto_showable` (#1815)
- Segfault with CSV, SQLite, tables (M1) (#1842)
- Use RelocatableFolders.jl instead of project_relative_dir to allow sysimages (#1852)
- Logging doesn't respect `maxlog` (#1858)
- TOC in dark mode is still light when aside (#1860)
- ProgressLogging: suppressed logging in Turing.jl (#1862)
- Cannot update to version 0.17.7 (#1869)
- Stdlib packages marked as not found (#1872)
- EllipsisNotation.jl (`randn((10,10))[1, ..]`) does not work (#1875)
- Too many intermediate bond values (#1891)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants