Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove preprinter, add parenthesizer callback to emit #43652

Merged
merged 1 commit into from
Apr 19, 2021

Conversation

rbuckton
Copy link
Member

@rbuckton rbuckton commented Apr 13, 2021

This reverts the "preprinter" transform from #42676 that was primarily used to ensure correct parenthesization using the node factory and amends the emitter to support providing a parenthesizer rule callback when emitting certain nodes.

Related #43486

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Apr 13, 2021
@rbuckton rbuckton force-pushed the substitutionParenthesizer branch from d24112d to 8d60631 Compare April 13, 2021 03:35
@rbuckton
Copy link
Member Author

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 13, 2021

Heya @rbuckton, I've started to run the perf test suite on this PR at 8d60631. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..43652

Metric master 43652 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 345,025k (± 0.01%) 344,773k (± 0.02%) -252k (- 0.07%) 344,647k 344,873k
Parse Time 1.94s (± 0.55%) 1.94s (± 0.73%) +0.01s (+ 0.26%) 1.92s 1.98s
Bind Time 0.83s (± 0.82%) 0.84s (± 0.44%) +0.00s (+ 0.24%) 0.83s 0.84s
Check Time 5.19s (± 0.35%) 5.20s (± 0.67%) +0.01s (+ 0.29%) 5.14s 5.29s
Emit Time 5.93s (± 0.48%) 5.92s (± 0.84%) -0.00s (- 0.07%) 5.81s 6.07s
Total Time 13.88s (± 0.35%) 13.91s (± 0.66%) +0.03s (+ 0.18%) 13.71s 14.18s
Compiler-Unions - node (v10.16.3, x64)
Memory used 203,356k (± 0.12%) 202,802k (± 0.13%) -554k (- 0.27%) 202,059k 203,073k
Parse Time 0.78s (± 0.67%) 0.78s (± 0.71%) +0.00s (+ 0.13%) 0.77s 0.79s
Bind Time 0.52s (± 1.11%) 0.53s (± 1.14%) +0.00s (+ 0.38%) 0.51s 0.54s
Check Time 7.46s (± 0.53%) 7.51s (± 0.71%) +0.05s (+ 0.70%) 7.38s 7.61s
Emit Time 2.58s (± 0.36%) 2.49s (± 1.26%) 🟩-0.09s (- 3.49%) 2.43s 2.56s
Total Time 11.35s (± 0.39%) 11.31s (± 0.62%) -0.04s (- 0.32%) 11.20s 11.46s
Monaco - node (v10.16.3, x64)
Memory used 342,709k (± 0.02%) 342,545k (± 0.01%) -164k (- 0.05%) 342,439k 342,605k
Parse Time 1.56s (± 0.33%) 1.56s (± 0.93%) -0.00s (- 0.06%) 1.51s 1.58s
Bind Time 0.75s (± 1.20%) 0.74s (± 0.91%) -0.00s (- 0.13%) 0.74s 0.77s
Check Time 5.33s (± 0.35%) 5.31s (± 0.48%) -0.02s (- 0.39%) 5.23s 5.36s
Emit Time 3.13s (± 1.01%) 3.01s (± 0.74%) 🟩-0.12s (- 3.77%) 2.97s 3.05s
Total Time 10.76s (± 0.41%) 10.62s (± 0.37%) -0.14s (- 1.30%) 10.50s 10.69s
TFS - node (v10.16.3, x64)
Memory used 304,376k (± 0.01%) 304,197k (± 0.02%) -179k (- 0.06%) 304,089k 304,334k
Parse Time 1.22s (± 0.83%) 1.21s (± 0.43%) -0.01s (- 0.58%) 1.20s 1.22s
Bind Time 0.70s (± 0.97%) 0.71s (± 0.67%) +0.00s (+ 0.43%) 0.70s 0.72s
Check Time 4.78s (± 0.64%) 4.79s (± 0.35%) +0.01s (+ 0.21%) 4.77s 4.84s
Emit Time 3.26s (± 1.22%) 3.22s (± 1.73%) -0.05s (- 1.38%) 3.15s 3.36s
Total Time 9.97s (± 0.47%) 9.92s (± 0.45%) -0.04s (- 0.44%) 9.85s 10.05s
material-ui - node (v10.16.3, x64)
Memory used 466,095k (± 0.01%) 466,002k (± 0.02%) -92k (- 0.02%) 465,771k 466,266k
Parse Time 2.01s (± 0.38%) 2.02s (± 0.38%) +0.01s (+ 0.50%) 2.00s 2.03s
Bind Time 0.66s (± 1.05%) 0.67s (± 1.60%) +0.01s (+ 1.53%) 0.64s 0.69s
Check Time 14.31s (± 0.42%) 14.28s (± 0.55%) -0.03s (- 0.20%) 14.18s 14.57s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.98s (± 0.35%) 16.96s (± 0.51%) -0.01s (- 0.06%) 16.87s 17.29s
Angular - node (v12.1.0, x64)
Memory used 322,762k (± 0.03%) 322,464k (± 0.02%) -298k (- 0.09%) 322,321k 322,633k
Parse Time 1.93s (± 0.77%) 1.94s (± 0.86%) +0.01s (+ 0.62%) 1.91s 1.97s
Bind Time 0.82s (± 0.63%) 0.82s (± 0.60%) -0.00s (- 0.37%) 0.81s 0.83s
Check Time 5.11s (± 0.35%) 5.10s (± 0.53%) -0.01s (- 0.23%) 5.04s 5.18s
Emit Time 5.98s (± 0.44%) 6.16s (± 0.57%) +0.18s (+ 2.99%) 6.08s 6.24s
Total Time 13.84s (± 0.28%) 14.02s (± 0.39%) +0.18s (+ 1.28%) 13.92s 14.17s
Compiler-Unions - node (v12.1.0, x64)
Memory used 190,329k (± 0.15%) 189,946k (± 0.11%) -383k (- 0.20%) 189,565k 190,432k
Parse Time 0.77s (± 1.06%) 0.77s (± 0.86%) +0.00s (+ 0.00%) 0.76s 0.79s
Bind Time 0.53s (± 0.56%) 0.53s (± 1.10%) -0.00s (- 0.19%) 0.52s 0.54s
Check Time 7.02s (± 0.72%) 7.00s (± 0.52%) -0.02s (- 0.34%) 6.93s 7.08s
Emit Time 2.55s (± 1.41%) 2.51s (± 1.30%) -0.05s (- 1.80%) 2.41s 2.58s
Total Time 10.88s (± 0.68%) 10.80s (± 0.53%) -0.07s (- 0.68%) 10.62s 10.91s
Monaco - node (v12.1.0, x64)
Memory used 325,118k (± 0.01%) 324,935k (± 0.02%) -182k (- 0.06%) 324,718k 325,090k
Parse Time 1.53s (± 0.92%) 1.53s (± 0.38%) -0.00s (- 0.20%) 1.52s 1.54s
Bind Time 0.72s (± 0.72%) 0.72s (± 0.69%) -0.00s (- 0.42%) 0.71s 0.73s
Check Time 5.14s (± 0.23%) 5.16s (± 0.55%) +0.02s (+ 0.35%) 5.11s 5.23s
Emit Time 3.10s (± 0.94%) 3.08s (± 0.75%) -0.02s (- 0.74%) 3.05s 3.15s
Total Time 10.49s (± 0.24%) 10.48s (± 0.36%) -0.01s (- 0.10%) 10.42s 10.58s
TFS - node (v12.1.0, x64)
Memory used 288,812k (± 0.02%) 288,627k (± 0.01%) -185k (- 0.06%) 288,502k 288,693k
Parse Time 1.21s (± 1.08%) 1.21s (± 0.68%) +0.01s (+ 0.41%) 1.19s 1.23s
Bind Time 0.69s (± 0.69%) 0.69s (± 0.98%) +0.00s (+ 0.14%) 0.68s 0.71s
Check Time 4.68s (± 0.33%) 4.68s (± 0.38%) -0.00s (- 0.02%) 4.65s 4.73s
Emit Time 3.17s (± 0.82%) 3.21s (± 1.37%) +0.04s (+ 1.33%) 3.14s 3.34s
Total Time 9.76s (± 0.41%) 9.80s (± 0.47%) +0.04s (+ 0.45%) 9.70s 9.94s
material-ui - node (v12.1.0, x64)
Memory used 444,059k (± 0.06%) 444,083k (± 0.02%) +24k (+ 0.01%) 443,954k 444,266k
Parse Time 2.03s (± 0.52%) 2.03s (± 0.45%) -0.00s (- 0.20%) 2.01s 2.05s
Bind Time 0.64s (± 0.52%) 0.64s (± 1.04%) 0.00s ( 0.00%) 0.63s 0.66s
Check Time 12.96s (± 0.71%) 12.97s (± 0.64%) +0.01s (+ 0.07%) 12.80s 13.13s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.63s (± 0.59%) 15.64s (± 0.53%) +0.01s (+ 0.03%) 15.47s 15.81s
Angular - node (v14.15.1, x64)
Memory used 321,425k (± 0.01%) 321,119k (± 0.00%) -306k (- 0.10%) 321,092k 321,142k
Parse Time 1.93s (± 0.49%) 1.92s (± 0.51%) -0.02s (- 0.78%) 1.89s 1.94s
Bind Time 0.86s (± 0.58%) 0.86s (± 0.69%) -0.00s (- 0.12%) 0.85s 0.88s
Check Time 5.12s (± 0.59%) 5.13s (± 0.41%) +0.01s (+ 0.20%) 5.10s 5.19s
Emit Time 6.36s (± 0.81%) 6.23s (± 0.45%) -0.13s (- 2.11%) 6.15s 6.30s
Total Time 14.28s (± 0.52%) 14.14s (± 0.26%) -0.14s (- 0.95%) 14.09s 14.27s
Compiler-Unions - node (v14.15.1, x64)
Memory used 189,492k (± 0.02%) 191,041k (± 0.62%) +1,549k (+ 0.82%) 189,046k 192,373k
Parse Time 0.80s (± 0.72%) 0.80s (± 0.56%) -0.00s (- 0.12%) 0.79s 0.81s
Bind Time 0.56s (± 1.04%) 0.56s (± 1.03%) +0.00s (+ 0.54%) 0.55s 0.57s
Check Time 7.09s (± 0.33%) 7.12s (± 0.46%) +0.02s (+ 0.32%) 7.07s 7.23s
Emit Time 2.52s (± 0.52%) 2.48s (± 0.49%) -0.04s (- 1.79%) 2.46s 2.51s
Total Time 10.98s (± 0.32%) 10.95s (± 0.36%) -0.02s (- 0.21%) 10.90s 11.08s
Monaco - node (v14.15.1, x64)
Memory used 324,141k (± 0.01%) 324,036k (± 0.01%) -105k (- 0.03%) 323,966k 324,076k
Parse Time 1.58s (± 0.60%) 1.57s (± 0.59%) -0.01s (- 0.69%) 1.55s 1.59s
Bind Time 0.75s (± 0.69%) 0.75s (± 0.53%) 0.00s ( 0.00%) 0.74s 0.76s
Check Time 5.12s (± 0.78%) 5.09s (± 0.52%) -0.03s (- 0.61%) 5.04s 5.15s
Emit Time 3.20s (± 0.47%) 3.14s (± 0.80%) -0.06s (- 2.00%) 3.09s 3.18s
Total Time 10.66s (± 0.51%) 10.55s (± 0.22%) -0.11s (- 1.00%) 10.50s 10.60s
TFS - node (v14.15.1, x64)
Memory used 287,753k (± 0.01%) 287,568k (± 0.01%) -184k (- 0.06%) 287,538k 287,607k
Parse Time 1.26s (± 1.24%) 1.27s (± 0.87%) +0.01s (+ 0.47%) 1.25s 1.30s
Bind Time 0.71s (± 0.70%) 0.71s (± 0.70%) 0.00s ( 0.00%) 0.70s 0.72s
Check Time 4.69s (± 0.31%) 4.71s (± 0.42%) +0.02s (+ 0.41%) 4.66s 4.74s
Emit Time 3.26s (± 0.64%) 3.31s (± 0.81%) +0.05s (+ 1.62%) 3.24s 3.37s
Total Time 9.93s (± 0.33%) 10.01s (± 0.47%) +0.08s (+ 0.77%) 9.92s 10.10s
material-ui - node (v14.15.1, x64)
Memory used 442,322k (± 0.06%) 442,351k (± 0.00%) +29k (+ 0.01%) 442,284k 442,401k
Parse Time 2.10s (± 0.67%) 2.09s (± 0.62%) -0.01s (- 0.43%) 2.06s 2.12s
Bind Time 0.70s (± 0.74%) 0.69s (± 0.72%) -0.00s (- 0.57%) 0.69s 0.71s
Check Time 13.20s (± 0.75%) 13.00s (± 0.50%) -0.20s (- 1.50%) 12.87s 13.13s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.00s (± 0.64%) 15.79s (± 0.45%) -0.21s (- 1.31%) 15.62s 15.94s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-206-generic
Architecturex64
Available Memory16 GB
Available Memory9 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
Benchmark Name Iterations
Current 43652 10
Baseline master 10

Developer Information:

Download Benchmark

@andrewbranch
Copy link
Member

@rbuckton can you point out what code is new, as opposed to a reversion?

@rbuckton
Copy link
Member Author

In emitter.ts, most of the changes are a reversion. I left in some of the changes in comment/sourcemap emit as it improves the number of cases where we can use our binary expression trampoline to avoid exhausting the call stack.

The only real addition is the ability to pass a parenthesizerRule to emit/emitExpression, which accomplishes the same thing I was trying to fix when I added the preprinter transform, but without the extra full-walk of the tree.

Anything in nodeFactory.ts, parenthesizerRules.ts, and types.ts is new and was added to support passing a parenthesizerRule in emitter.ts.

@rbuckton
Copy link
Member Author

This one wasn't as simple as just rolling back the change, as the work to create a reusable binary expression trampoline is something we want to keep so that we're consistent across the binder, checker, and emitter. As a result, we also kept the changes to comment and source map emit so that we could reuse the logic inside the binary expression trampoline.

Comment on lines +6788 to +6789
getParenthesizeLeftSideOfBinaryForOperator(binaryOperator: SyntaxKind): (leftSide: Expression) => Expression;
getParenthesizeRightSideOfBinaryForOperator(binaryOperator: SyntaxKind): (rightSide: Expression) => Expression;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are new, and exist to support passing in a (node: Node) => Node parenthesizer rule for binary expressions. The implementation simply wraps parenthesizeLeftSideOfBinary or parenthesizeRightSideOfBinary and caches the result for reuse.

@@ -562,31 +562,31 @@ namespace ts {
}

export const nullTransformationContext: TransformationContext = {
get factory() { return factory; },
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is simply a reordering of existing properties to match the order in transformNodes so they (hopefully) share the same "map" in V8.

@@ -161,4 +161,6 @@ var _brokenTree = (0, renderer_1.dom)(component_1.MySFC, { x: 1, y: 2 },
(0, renderer_1.dom)(component_1.MyClass, { x: 3, y: 4 }),
(0, renderer_1.dom)(component_1.MyClass, { x: 5, y: 6 }));
// Should fail, nondom isn't allowed as children of dom
var _brokenTree2 = (0, renderer_1.dom)(DOMSFC, { x: 1, y: 2 }, component_1.tree, component_1.tree);
var _brokenTree2 = (0, renderer_1.dom)(DOMSFC, { x: 1, y: 2 },
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This indentation matches the emit in release-4.2.

Comment on lines +1342 to +1258
if (currentParenthesizerRule) {
lastSubstitution = currentParenthesizerRule(lastSubstitution);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (and the code path that sets currentParenthesizerRule) is the only "actual" change from what we previously were doing in 4.2. The point here is to only invoke the parenthesizer rule when substitution results in a different node (which might have a different precedence).

@rbuckton
Copy link
Member Author

While not perfect (as there are some other changes in emitter.ts unrelated to this), you can get a gist of what changed and was reverted by doing the following on the commandline:

git difftool d3b8d5e9c423d2565e9c5122326fbf4a2a2e5d6a^..substitutionParenthesizer -- src/compiler/emitter.ts

@rbuckton rbuckton force-pushed the substitutionParenthesizer branch from 8d60631 to b48a51a Compare April 16, 2021 01:29
@rbuckton
Copy link
Member Author

I've pushed up a change that reorders some of the functions so that they line up with where they were prior to #43676 to make the command-line diff easier to read through.

Copy link
Member

@andrewbranch andrewbranch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not very familiar with this area of code, nor the changes that are being reverted, but the additions you pointed out make sense to me.

@rbuckton
Copy link
Member Author

I don't think this should have broken anything, but just to be on the safe side...

@typescript-bot run dt
@typescript-bot test this
@typescript-bot user test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 16, 2021

Heya @rbuckton, I've started to run the extended test suite on this PR at b48a51a. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 16, 2021

Heya @rbuckton, I've started to run the parallelized community code test suite on this PR at b48a51a. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 16, 2021

Heya @rbuckton, I've started to run the parallelized Definitely Typed test suite on this PR at b48a51a. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

The user suite test run you requested has finished and failed. I've opened a PR with the baseline diff from master.

@rbuckton
Copy link
Member Author

RWC tests failed, but its unclear why. I'm running them locally.
User tests were fine except for fp-ts, which for some reason ran out of heap memory. I will have to investigate that one to see what's causing the crash.

@rbuckton
Copy link
Member Author

RWC failures are mostly unrelated to this change, the few that are are changes in indentation that were likely present before the preprinter transform was added (similar to what I mentioned here), so this is just going back to existing behavior.

@rbuckton
Copy link
Member Author

The fp-ts user tests didn't crash locally. @weswigham did you mention you were you looking into an OOM?

@weswigham
Copy link
Member

weswigham commented Apr 17, 2021

@sandersn did a bit ago, iirc?

@rbuckton
Copy link
Member Author

I cannot repro the OOM locally.

@typescript-bot user test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 19, 2021

Heya @rbuckton, I've started to run the parallelized community code test suite on this PR at b48a51a. You can monitor the build here.

@rbuckton
Copy link
Member Author

Ok. It didn't crash that time. Must have been a glitch in the matrix.

@rbuckton
Copy link
Member Author

Summary of DT failures:

None of the DT failures seem to be related to this change, but warrant further investigation outside of this PR.

@rbuckton rbuckton merged commit e0d5516 into master Apr 19, 2021
@rbuckton rbuckton deleted the substitutionParenthesizer branch April 19, 2021 16:34
@weswigham
Copy link
Member

The mithril/cytoscape crash already has a PR up with a fix~

@sandersn
Copy link
Member

The other crash has the same origin; the electron-* are known failures that shouldn't show up in the final output. They should, I hope, be gone after the additional grep that @weswigham and I stuck onto the end of the diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants