Unencode quote and splice trees #17342

nicolasstucki · 2023-04-25T12:27:01Z

We currently use the Quote and Splice ASTs for untyped quotes and splices. Then when we type them we encode them into scala.quoted.runtime.Expr.{quote,splice,nestedSplice}. This non-semantic representation if fragile and the source of many past bug. In this PR we change the internal representation of quotes and splices to use the Quote and Splice ASTs as typed trees.

The core of this change in the AST representation and how we type them in QuotesAndSplices. Other changes consist in adapting the code from one representation to the other.

- case class Quote(quoted: Tree)(implicit @constructorOnly src: SourceFile) extends TermTree
+ case class Quote[+T <: Untyped] private[ast] (body: Tree[T])(implicit @constructorOnly src: SourceFile) extends TermTree[T] 

- case class Splice(expr: Tree)(implicit @constructorOnly src: SourceFile) extends TermTree 
+ case class Splice[+T <: Untyped] private[ast] (expr: Tree[T])(implicit @constructorOnly src: SourceFile) extends TermTree[T]

We do not change the TASTy (file and reflection) representation of quotes and splices. This is something that we should consider adding in a future PR.
We can use the simpler encoding of splice in all cases. The additional Quotes in nestedSplice is not used.
This does not change binary compatibility.

…uotedType`

This field is no longer used and can be removed. Removing it is backwards compatible with 3.0 and later releases.

Closes scala#6991

smarter · 2023-04-26T16:09:44Z

compiler/src/dotty/tools/dotc/typer/TypeAssigner.scala

@@ -398,6 +398,9 @@ trait TypeAssigner {
        .appliedTo(defn.QuotesClass.typeRef, defn.QuotedExprClass.typeRef.appliedTo(tpt.tpe))
    tree.withType(lambdaType)

+  def assignType(tree: untpd.SplicedExpr, tpt: Tree)(using Context): SplicedExpr =
+    tree.withType(tpt.tpe)


If tpt is always the same as the type of the node, and if tpt is always EmptyTree in untyped trees, then it shouldn't be necessary and can be removed (possibly using promote in TreeChecker to retype it).

I was wondering how the tpt could be removed. I will try that.

We currently use the tpt in the staging/splicing phases to set staged healed types for quotes and splices. I did not manage to remove it without breaking something in those phases.

I did not manage to remove it without breaking something in those phases.

We have other phases where we transform types of trees like https://github.com/lampepfl/dotty/blob/main/compiler/src/dotty/tools/dotc/transform/ElimErasedValueType.scala, does that not work here? Maybe I can have a look if you show me what the issues you ran into were.

I managed to remove the tpt.

compiler/src/dotty/tools/dotc/ast/Trees.scala

smarter · 2023-04-26T16:30:54Z

compiler/src/dotty/tools/dotc/ast/Trees.scala

+  /** A tree representing a quote `'{ expr }
+   *
+   *  @param  expr  The tree that was quoted
+   *  @param  tpt   The type of the tree that was quoted,


How does this differ from expr.tpe?

In the staging phase we use tpt to store the healed version of the type. This cannot be recomputed trivially from expr.tpe.

Now staging uses expr.tpe to store the new version of type.

compiler/src/dotty/tools/dotc/transform/ElimByName.scala

compiler/src/dotty/tools/dotc/typer/QuotesAndSplices.scala

To transform quotes and splices in inline methods.

compiler/src/dotty/tools/dotc/ast/Trees.scala

Co-authored-by: Guillaume Martres <[email protected]>

nicolasstucki · 2023-05-02T06:52:29Z

I updated the check-files in 95c39bc to fix the CI failures.

Closes scala#18059 Probably fixed by scala#17342

Closes #18059 Probably fixed by #17342 [skip mima] [skip docs] [skip community_build]

Closes #18059 Probably fixed by #17342 [Cherry-picked 050f54b]

nicolasstucki added 14 commits April 25, 2023 11:22

Rename Spliced to SplicedExpr

c2f6362

Split Quoted extractor into QuotedExpr and QuotedTypeOf

938d3ae

Split transformQuotation into transformQuotedExpr and `transformQ…

b7ca9b1

…uotedType`

Add QuotedExpr to encode quotes directly in the AST

deb03a7

Add SplicedExpr to encode splices directly in the AST

b7a7227

Remove splice outer quotes context

fe64a3e

This field is no longer used and can be removed. Removing it is backwards compatible with 3.0 and later releases.

Rename SplicedExpr.{spliced=>expr}

161ef49

Use QuotedExpr instead of Quote trees

cf30d30

Use SplicedExpr instead of Splice trees

f0d758b

Move typedQuotedExpr/typedSplicedExpr to QuotesAndSplices

c6adab8

Rename QuotedExpr to Quote and SplicedExpr to Splice

3a045c4

Add documentation

76d0daf

Backwards compatibility support for quotes and splice in reflection API

75bacdb

Add regression test

0e6dc2e

Closes scala#6991

nicolasstucki self-assigned this Apr 25, 2023

Adapt REPL test output

cdd5ffb

nicolasstucki force-pushed the unencode-quote-trees branch from cde31be to cdd5ffb Compare April 26, 2023 09:06

nicolasstucki marked this pull request as ready for review April 26, 2023 11:34

nicolasstucki requested a review from smarter April 26, 2023 11:34

nicolasstucki assigned smarter and unassigned nicolasstucki Apr 26, 2023

smarter requested changes Apr 26, 2023

View reviewed changes

smarter assigned nicolasstucki and unassigned smarter Apr 26, 2023

nicolasstucki added 4 commits April 27, 2023 08:18

Remove Splice.isInBraces

04f020f

Move Quote/Splice retyping into ReTyper

35db4a1

Add documentation

1ef7f59

Transform Quote/Splice in MiniPhase

687d06e

To transform quotes and splices in inline methods.

smarter reviewed Apr 27, 2023

View reviewed changes

compiler/src/dotty/tools/dotc/ast/Trees.scala Outdated Show resolved Hide resolved

smarter reviewed Apr 27, 2023

View reviewed changes

compiler/src/dotty/tools/dotc/ast/Trees.scala Outdated Show resolved Hide resolved

smarter removed their assignment May 1, 2023

nicolasstucki and others added 15 commits May 2, 2023 08:48

Cleanup

95c39bc

Remove redundant case

0ca6066

Refactor

e3735fa

Refactor quote cancellation logic

6956c43

Remove unnecessary quotation level change

859f6e6

Add missing change of staging level

e4ddc88

Simplify macroDependencies using level from context

7e8d2b1

Fix typo

a3c657b

Refactor use of Quote

be3cd48

Do not traverse type trees to find inline method calls

7bd7c92

Remove post-condition that is already checked by Staging

b991b40

Refactor quote pickling case

e611139

Cleanup

37200be

Remove unnecessary widening

e75cafe

Apply suggestions from code review

9214daa

Co-authored-by: Guillaume Martres <[email protected]>

nicolasstucki force-pushed the unencode-quote-trees branch from f4dd16f to 9214daa Compare May 2, 2023 06:50

nicolasstucki assigned smarter and unassigned nicolasstucki May 2, 2023

nicolasstucki requested a review from smarter May 2, 2023 06:52

nicolasstucki linked an issue May 2, 2023 that may be closed by this pull request

Misleading error message when matching on quotes #6991

Closed

smarter merged commit ec3cdad into scala:main May 2, 2023

smarter deleted the unencode-quote-trees branch May 2, 2023 12:36

nicolasstucki added a commit to dotty-staging/dotty that referenced this pull request Jun 27, 2023

Add regression test for scala#18059

050f54b

Closes scala#18059 Probably fixed by scala#17342

nicolasstucki mentioned this pull request Jun 27, 2023

Add regression test for #18059 #18075

Merged

nicolasstucki added a commit that referenced this pull request Jun 27, 2023

Add regression test for #18059 (#18075)

49b685d

Closes #18059 Probably fixed by #17342 [skip mima] [skip docs] [skip community_build]

Kordyjan added this to the 3.3.1 milestone Aug 1, 2023

Kordyjan pushed a commit that referenced this pull request Nov 23, 2023

Add regression test for #18059

dad3c06

Closes #18059 Probably fixed by #17342 [Cherry-picked 050f54b]

Kordyjan pushed a commit that referenced this pull request Nov 29, 2023

Add regression test for #18059

88454a0

Closes #18059 Probably fixed by #17342 [Cherry-picked 050f54b]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unencode quote and splice trees #17342

Unencode quote and splice trees #17342

nicolasstucki commented Apr 25, 2023 •

edited

Loading

smarter Apr 26, 2023

nicolasstucki Apr 27, 2023

nicolasstucki Apr 27, 2023

smarter Apr 27, 2023

nicolasstucki Apr 28, 2023

smarter Apr 26, 2023

nicolasstucki Apr 27, 2023

nicolasstucki Apr 28, 2023

nicolasstucki commented May 2, 2023

Unencode quote and splice trees #17342

Unencode quote and splice trees #17342

Conversation

nicolasstucki commented Apr 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolasstucki commented May 2, 2023

nicolasstucki commented Apr 25, 2023 •

edited

Loading