Backports for 0.6.1 #23670

ararslan · 2017-09-11T23:39:06Z

This is my first time doing this, so it's probably an enormous mess.

ararslan · 2017-09-12T00:42:31Z

I need to redo this from scratch. Things to do differently:

Cherry pick individual commits; don't use merge commits unless it was a squash merge.
Do not include commits from PRs that still have a [needs X] label for any X.

Edit: Redone. Let's see what happens.

ararslan · 2017-09-13T20:49:21Z

Looks like something was backported incorrectly, likely a bad conflict resolution. Not sure what exactly is causing it but it looks like it's failing somewhere around src/ccall.cpp. @vtjnash, would you be willing to take a look at that and make sure that part looks okay?

vtjnash · 2017-09-14T17:41:45Z

Added #22161, dropped PR #22472 (096e13d ("Store global variables and function addresses as 32bit offsets") and 054618e ("Fix function base address lookup on Win32 and FreeBSD")),
rebased on top of #23683, and corrected minor typos. Still getting some failures in linalg/generic, linalg/cholesky, misc, linalg/qr, and linalg/diagonal, and misc which you'll need to resolve.

ararslan · 2017-09-14T17:46:21Z

Ack, I merged #23683 and now there are conflicts. 😑 The linalg stuff may be due to #22138, which Fredrik has since advised against backporting.

removes the global work queue, which allows increasing the precision of cycle detection and resolution, and decreases the need for the threading synchronization lock surrounding inference updates `inInference` flag usage to be merely a hint for `jl_type_infer` to not bother trying to infer a method (helps avoid accidental infinite recursion over inferring type inference), enable inferring inference (cherry picked from commit 5847317)

- move call to `short-form-function-loc` inside `parse-assignment` this is faster and fixes nested cases like `f(x)=g(x)=1` - fix a case where an `unexpected "foo"` message was missing quotes - optimize common case of arglists of comma-separated simple tokens - some other small optimizations - put code back in precedence order - separate parse-Nary and parse-comma so they're easier to read - cleaner code for parse-unary and parse-factor

Ref #21715 (cherry picked from commit 4bcb1bf)

Ref #21818 (cherry picked from commit 0394f47)

Ref #21818 (cherry picked from commit 2272036)

Ref #21818 (cherry picked from commit fa8c4d2)

Ref #21832 Squashed for backporting (cherry picked from commit d86b37c) (cherry picked from commit 7e7393c) (cherry picked from commit 6a4acc0) (cherry picked from commit 93cf6d9) (cherry picked from commit 2379294) (cherry picked from commit 7ff6322) (cherry picked from commit 49abbf3) (cherry picked from commit eadb699) (cherry picked from commit 1ce1d75)

…cache fix #21963 Ref #21965 (cherry picked from commit 8bacb65)

* Return socket correctly in socket_reuse_port * Run reuseport tests only if SO_REUSEPORT is supported. * Test fix - test for successful addition of workers in distributed test (cherry picked from commit d3bc329)

Ref #22022 (cherry picked from commit 915910c)

Ref #22118 (cherry picked from commit 4a470cb)

Ref #22133 (cherry picked from commit 994f42c)

…docs Ref #22150 (cherry picked from commit 95677b0)

* fix spurious newlines in Base.Test printing * add some doctest to test (cherry picked from commit 2796b40)

…dimsarray (#22289) (cherry picked from commit d9d8223)

* Extend definition of StridedReshapedArray * Add test (cherry picked from commit 92ff1bc)

…#22449) (cherry picked from commit 3808de9)

(cherry picked from commit 15ee73a)

or 250 characters for a certain broken OS Ref #22457 (cherry picked from commit 8c1adc8)

(cherry picked from commit b1c2d73)

nanosoldier · 2017-10-06T09:31:33Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

fredrikekre · 2017-10-06T10:25:29Z

repeat regression is real:
v0.6.0:

julia> @btime repeat($A, inner = $[24], outer=$[1]);
  51.796 μs (611 allocations: 56.66 KiB)

This branch:

julia> @btime repeat($A, inner=$[24], outer=$[1]);
  290.690 μs (3811 allocations: 150.41 KiB)

fredrikekre · 2017-10-06T13:00:40Z

base/abstractarraymath.jl

@@ -415,7 +415,7 @@ _reperr(s, n, N) = throw(ArgumentError("number of " * s * " repetitions " *
                n = inner[i]
                inner_indices[i] = (1:n) + ((c[i] - 1) * n)
            end
-            R[inner_indices...] = A[c]
+            fill!(view(R, inner_indices...), A[c])


This must be the cause of the repeat regression, @pabloferz

This is a necessary fix, it's meant for the case when A[c] is itself an array. I can think of a way of trying to preserve the speed for the cases when A[c] is not an array, but I don't know what would be the action to take with regards to the backport.

(cherry picked from commit af5abb3)

ararslan · 2017-10-06T21:47:38Z

Pablo's PR bettered the performance on master, so let's see what effect it had here.

@nanosoldier runbenchmarks("misc", vs=":release-0.6")

mbauman · 2017-10-06T21:51:06Z

Could I kindly request one more patch to make this round? #23681 would be nice to backport. I don't think I see it listed and it's a very trivial fix.

ararslan · 2017-10-06T21:57:17Z

Can do 👍

(cherry picked from commit c23d6bc)

ararslan · 2017-10-06T23:15:23Z

Will merge once CI is green (FreeBSD and Circle failures are expected) unless someone beats me to it. DO NOT SQUASH!

nanosoldier · 2017-10-06T23:30:49Z

Your benchmark job has completed - no performance regressions were detected. A full report can be found here. cc @ararslan

mauro3 · 2017-10-07T05:53:16Z

Thanks! Now we just need the tag v0.6.1.

tkelman · 2017-10-07T08:07:58Z

Where are your pkgeval results and logs? Coverage.jl fails tests on this branch and it's definitely related, the parser changes were breaking and shouldn't be backported.

StefanKarpinski · 2017-10-07T15:52:28Z

Are you talking about these parser changes: #22161. They were non-breaking according to @JeffBezanson (who seems like a fairly reliable character), and tagged for backporting by @vtjnash. It's possible they were actually breaking, but if so it was subtle. There was a PkgEval run done, and I'm sure @ararslan can post the results here.

ViralBShah · 2017-10-07T16:02:49Z

It would be great to hook up PkgEval the way the benchmarks are hooked up - so it can be triggered from here.

StefanKarpinski · 2017-10-07T19:34:06Z

Automating the running of PkgEval is an excellent idea, similar to nanosoldier. I have to confess I have no idea where the PkgEval code even lives. PkgEval.jl doesn’t exist and there seems to be zero documentation of how to run it.

ararslan · 2017-10-07T19:38:55Z

https://github.com/JuliaCI/PackageEvaluator.jl

PkgEval is just what the cool kids are calling it. 😉

ararslan · 2017-10-07T19:44:18Z

Where are your pkgeval results and logs?

On Nanosoldier, where they ran. I didn't realize I was supposed to post them somewhere, but I can do that.

ararslan · 2017-10-07T20:14:46Z

PkgEval for 0.6.1: https://gist.github.com/ararslan/98ef9116c7bb6b5c7dcb041c9f92a2ef

tkelman · 2017-10-07T23:16:25Z

Doesn't make much sense that those logs seem to say Coverage.jl passed on this branch. It definitely fails locally against latest release-0.6. That PR caused a change in coverage measurement that made Coverage start failing on nightly when it was merged. Something may be flawed about how the backport test binary that pkgeval used was built.

Or however the logs were converted to csv may be flawed, and marking some failures as passes. Install jq and use the summary.sh script in the PackageEvaluator repo.

StefanKarpinski · 2017-10-08T01:58:30Z

PkgEval is just what the cool kids are calling it.

Given that everyone refers to it as PkgEval perhaps we should rename the package?

ViralBShah · 2017-10-08T14:42:53Z

To make sure I understand correctly, is it that the parser changes seem to have broken Coverage.jl?

StefanKarpinski · 2017-10-08T15:35:09Z

That's what @tkelman's saying, although more details about how and details of what's failing would be helpful. And of course, then there's the question of why PkgEval didn't catch it.

ararslan · 2017-10-08T20:10:43Z

It appears that this is what changed in Coverage: JuliaCI/Coverage.jl#136. It seems that change resulted in something more correct for Coverage but because of that broke a test that expected the old behavior. The change seems subtle enough that I doubt pretty much any other package would have noticed. It's still odd that Coverage on PkgEval didn't see it though.

ararslan · 2017-10-08T20:24:23Z

Interesting, downloading the binaries I used for running PkgEval and doing Pkg.test("Coverage") manually fails.

ararslan requested review from StefanKarpinski, vtjnash and JeffBezanson September 11, 2017 23:39

ararslan added this to the 0.6.x milestone Sep 11, 2017

ararslan force-pushed the aa/backports-0.6 branch from 29f58bd to 6afd88e Compare September 13, 2017 20:16

vtjnash force-pushed the aa/backports-0.6 branch from 6afd88e to 92900b8 Compare September 14, 2017 17:41

jrevels and others added 20 commits September 14, 2017 10:56

better static printing of typemap entries

d02d734

Ref #21715 (cherry picked from commit 4bcb1bf)

Fix reuse of client port on Linux. Implement for OSX.

c1fe105

Ref #21818 (cherry picked from commit 0394f47)

fix listenany to return correct port number for porthint of 0

a50154e

Ref #21818 (cherry picked from commit 2272036)

added NEWS entry

152f814

Ref #21818 (cherry picked from commit fa8c4d2)

Method overwriting by an ambiguity should also invalidate the method …

4aa8d72

…cache fix #21963 Ref #21965 (cherry picked from commit 8bacb65)

Return socket correctly in socket_reuse_port (#22017)

b8cf285

* Return socket correctly in socket_reuse_port * Run reuseport tests only if SO_REUSEPORT is supported. * Test fix - test for successful addition of workers in distributed test (cherry picked from commit d3bc329)

Add necessary LLVM patches

718828b

Ref #22022 (cherry picked from commit 915910c)

fix extrema(A,dim) when length(dim)>1

acba61a

Ref #22118 (cherry picked from commit 4a470cb)

fix base(b, big(0), 0) == "0"

29cc5b0

Ref #22133 (cherry picked from commit 994f42c)

prevent Docs.apropos from breaking on random stuff interpolated into …

3e6545e

…docs Ref #22150 (cherry picked from commit 95677b0)

add some doctest to test (#22287)

ca44bb3

* fix spurious newlines in Base.Test printing * add some doctest to test (cherry picked from commit 2796b40)

add a few doctests to broadcast, parse, intfuncs, number and permuted…

9e76e65

…dimsarray (#22289) (cherry picked from commit d9d8223)

Extend definition of StridedReshapedArray (#22429)

34e485c

* Extend definition of StridedReshapedArray * Add test (cherry picked from commit 92ff1bc)

update docs for sysimg to show that default_sysimg_path is a function (…

2389523

…#22449) (cherry picked from commit 3808de9)

add missing method for eigvecs(A::HermOrSym) (#22453)

96ddec8

(cherry picked from commit 15ee73a)

fix homedir error handling for 1020+ character usernames

464079a

or 250 characters for a certain broken OS Ref #22457 (cherry picked from commit 8c1adc8)

optimize repeat(string, n) for repeating single ASCII chars (#22462)

e205c0f

(cherry picked from commit b1c2d73)

fredrikekre reviewed Oct 6, 2017

View reviewed changes

pabloferz mentioned this pull request Oct 6, 2017

Restore repeat performance for arrays of scalars #24022

Merged

Restore repeat preformance for arrays of scalars (#24022)

1a1ef7a

(cherry picked from commit af5abb3)

Make print_matrix O(1) again (#23681)

f651277

(cherry picked from commit c23d6bc)

ararslan changed the title ~~WIP: Backports for 0.6.1~~ Backports for 0.6.1 Oct 6, 2017

ararslan merged commit 389b23c into release-0.6 Oct 7, 2017

ararslan deleted the aa/backports-0.6 branch October 7, 2017 01:18

ararslan removed the backport pending 0.6 label Oct 7, 2017

martinholters mentioned this pull request Oct 16, 2017

Segfault on method error #24122

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backports for 0.6.1 #23670

Backports for 0.6.1 #23670

ararslan commented Sep 11, 2017

ararslan commented Sep 12, 2017 •

edited

Loading

ararslan commented Sep 13, 2017

vtjnash commented Sep 14, 2017

ararslan commented Sep 14, 2017

nanosoldier commented Oct 6, 2017

fredrikekre commented Oct 6, 2017

fredrikekre Oct 6, 2017

pabloferz Oct 6, 2017

pabloferz Oct 6, 2017

ararslan commented Oct 6, 2017

mbauman commented Oct 6, 2017

ararslan commented Oct 6, 2017

ararslan commented Oct 6, 2017 •

edited

Loading

nanosoldier commented Oct 6, 2017

mauro3 commented Oct 7, 2017

tkelman commented Oct 7, 2017

StefanKarpinski commented Oct 7, 2017

ViralBShah commented Oct 7, 2017

StefanKarpinski commented Oct 7, 2017

ararslan commented Oct 7, 2017

ararslan commented Oct 7, 2017

ararslan commented Oct 7, 2017

tkelman commented Oct 7, 2017 •

edited

Loading

StefanKarpinski commented Oct 8, 2017

ViralBShah commented Oct 8, 2017

StefanKarpinski commented Oct 8, 2017

ararslan commented Oct 8, 2017

ararslan commented Oct 8, 2017

Backports for 0.6.1 #23670

Backports for 0.6.1 #23670

Conversation

ararslan commented Sep 11, 2017

ararslan commented Sep 12, 2017 • edited Loading

ararslan commented Sep 13, 2017

vtjnash commented Sep 14, 2017

ararslan commented Sep 14, 2017

nanosoldier commented Oct 6, 2017

fredrikekre commented Oct 6, 2017

fredrikekre Oct 6, 2017

Choose a reason for hiding this comment

pabloferz Oct 6, 2017

Choose a reason for hiding this comment

pabloferz Oct 6, 2017

Choose a reason for hiding this comment

ararslan commented Oct 6, 2017

mbauman commented Oct 6, 2017

ararslan commented Oct 6, 2017

ararslan commented Oct 6, 2017 • edited Loading

nanosoldier commented Oct 6, 2017

mauro3 commented Oct 7, 2017

tkelman commented Oct 7, 2017

StefanKarpinski commented Oct 7, 2017

ViralBShah commented Oct 7, 2017

StefanKarpinski commented Oct 7, 2017

ararslan commented Oct 7, 2017

ararslan commented Oct 7, 2017

ararslan commented Oct 7, 2017

tkelman commented Oct 7, 2017 • edited Loading

StefanKarpinski commented Oct 8, 2017

ViralBShah commented Oct 8, 2017

StefanKarpinski commented Oct 8, 2017

ararslan commented Oct 8, 2017

ararslan commented Oct 8, 2017

ararslan commented Sep 12, 2017 •

edited

Loading

ararslan commented Oct 6, 2017 •

edited

Loading

tkelman commented Oct 7, 2017 •

edited

Loading