Changes to a case class don't invalidate sources that depend on its Mirror

[mrdziuban]

Compiler version

The issue happens on Scala 3.3.4, 3.4.3, 3.5.2, and the latest 3.6.2

Minimized code

https://github.com/mrdziuban/scala3-incremental-compilation-cached-mirror

There are instructions in the README for how to reproduce the issue. There are five relevant files:

1. Test.scala -- contains case class Test, which is the type whose changes are not picked up
2. Deps.scala -- contains case class Deps with a single field that refers to Test
3. Labels.scala -- contains code that produces a String for a given type A, recursing into nested case class fields
4. DepsLabels.scala -- defines val labels calling Labels.derived[Deps]
5. Main.scala -- defines an entrypoint that just prints what was generated in DepsLabels.scala

When adding or removing fields in case class Test, I would expect DepsLabels.scala to be invalidated and recompiled, but it's not. Using sbt -debug, I see this during the incremental compilation after the change:

[debug] None of the modified names appears in source file of example.DepsLabels. This dependency is not being considered for invalidation.

Here's the full sbt -debug output from the incremental compilation, in case it's helpful.

expand

[debug] Copy resource mappings:
[debug]
[debug] [zinc] IncrementalCompile -----------
[debug] IncrementalCompile.incrementalCompile
[debug] previous = Stamps for: 11 products, 5 sources, 2 libraries
[debug] current source = Set(${BASE}/src/main/scala/example/Labels.scala, ${BASE}/src/main/scala/example/Main.scala, ${BASE}/src/main/scala/example/Test.scala, ${BASE}/src/main/scala/example/Deps.scala, ${BASE}/src/main/scala/example/DepsLabels.scala)
[debug] > initialChanges = InitialChanges(Changes(added = Set(), removed = Set(), changed = Set(${BASE}/src/main/scala/example/Test.scala), unmodified = ...),Set(),Set(),API Changes: Set())
[debug]
[debug] Initial source changes:
[debug]   removed: Set()
[debug]   added: Set()
[debug]   modified: Set(${BASE}/src/main/scala/example/Test.scala)
[debug] Invalidated products: Set()
[debug] External API changes: API Changes: Set()
[debug] Modified binary dependencies: Set()
[debug] Initial directly invalidated classes: Set(example.Test)
[debug] Sources indirectly invalidated by:
[debug]   product: Set()
[debug]   binary dep: Set()
[debug]   external source: Set()
[debug] All initially invalidated classes: Set(example.Test)
[debug] All initially invalidated sources:Set(${BASE}/src/main/scala/example/Test.scala)
[debug] Created transactional ClassFileManager with tempDir = /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes.bak
[debug] Initial set of included nodes: example.Test
[debug] About to delete class files:
[debug]   Test$.class
[debug]   Test.class
[debug]   Test$.tasty
[debug]   Test.tasty
[debug] We backup class files:
[debug]   Test$.class
[debug]   Test.class
[debug]   Test$.tasty
[debug]   Test.tasty
[debug] compilation cycle 1
[info] compiling 1 Scala source to /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes ...
[debug] Returning already retrieved and compiled bridge: /Users/matt/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-sbt-bridge/3.6.2/scala3-sbt-bridge-3.6.2.jar.
[debug] [zinc] Running cached compiler 3e30ed51 for Scala Compiler version 3.6.2
[debug] [zinc] The Scala compiler is invoked with:
[debug]   -bootclasspath
[debug]   /Users/matt/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.6.2/scala3-library_3-3.6.2.jar:/Users/matt/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.15/scala-library-2.13.15.jar
[debug]   -classpath
[debug]   /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes
[debug] Scala compilation took 0.323506 s
[info] done compiling
[debug] Registering generated classes:
[debug]   Test$.class
[debug]   Test.class
[debug]   Test$.tasty
[debug]   Test.tasty
[debug] Invalidating (transitively) by inheritance from example.Test...
[debug] Initial set of included nodes: example.Test
[debug] Invalidated by transitive inheritance dependency: Set(example.Test)
[debug] None of the modified names appears in source file of example.DepsLabels. This dependency is not being considered for invalidation.
[debug] None of the modified names appears in source file of example.Deps. This dependency is not being considered for invalidation.
[debug] Change NamesChange(example.Test,ModifiedNames(changes = UsedName(copy,[Default]), UsedName(int,[Default]), UsedName(apply,[Default]), UsedName(copy$default$2,[Default]), UsedName(example;Test;init;,[Default]), UsedName(_2,[Default]))) invalidates 1 classes due to The example.Test has the following regular definitions changed:
[debug]   UsedName(copy,[Default]), UsedName(int,[Default]), UsedName(apply,[Default]), UsedName(copy$default$2,[Default]), UsedName(example;Test;init;,[Default]), UsedName(_2,[Default]).
[debug]   > by transitive inheritance: Set(example.Test)
[debug]   >
[debug]   >
[debug]
[debug] New invalidations:
[debug] Initial set of included nodes:
[debug] Previously invalidated, but (transitively) depend on new invalidations:
[debug] Final step, transitive dependencies:
[debug]   Set()
[debug] No classes were invalidated.
[debug] Removing the temporary directory used for backing up class files: /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes.bak
[debug] Packaging /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/scala3-incremental-compilation-cached-mirror_3-0.1.0-SNAPSHOT.jar ...
[debug] Input file mappings:
[debug]   example
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example
[debug]   example/DepsLabels.tasty
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/DepsLabels.tasty
[debug]   example/Deps.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Deps.class
[debug]   example/Main$.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Main$.class
[debug]   example/Test$.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Test$.class
[debug]   example/Test.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Test.class
[debug]   example/Test.tasty
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Test.tasty
[debug]   example/Deps.tasty
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Deps.tasty
[debug]   example/Deps$.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Deps$.class
[debug]   example/DepsLabels.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/DepsLabels.class
[debug]   example/Labels.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Labels.class
[debug]   example/Labels$Inst.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Labels$Inst.class
[debug]   example/Main.tasty
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Main.tasty
[debug]   example/DepsLabels$.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/DepsLabels$.class
[debug]   example/Main.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Main.class
[debug]   example/Labels.tasty
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Labels.tasty
[debug]   example/Labels$.class
[debug]     /Users/matt/scala3-incremental-compilation-cached-mirror/target/scala-3.6.2/classes/example/Labels$.class
[debug] Done packaging.
[info] running example.Main
[debug]   Classpath:
[debug]   /Users/matt/scala3-incremental-compilation-cached-mirror/target/bg-jobs/sbt_4ae4d706/job-2/target/db951596/becebde8/scala3-incremental-compilation-cached-mirror_3-0.1.0-SNAPSHOT.jar
[debug]   /Users/matt/scala3-incremental-compilation-cached-mirror/target/bg-jobs/sbt_4ae4d706/target/f1a04a8e/35a5dca0/scala3-library_3-3.6.2.jar
[debug]   /Users/matt/scala3-incremental-compilation-cached-mirror/target/bg-jobs/sbt_4ae4d706/target/8eaa0d28/507e61bd/scala-library-2.13.15.jar

Output

The test.sh script included in the reproduction repo produces the following output:

Running with fields: `str: String`
***** Fields: test(str)

Running with fields: `str: String, int: Int`
***** Fields: test(str)

Expectation

DepsLabels.scala should have been recompiled, resulting in the second printed ***** Fields line being ***** Fields: test(str, int)

[bishabosha]

@smarter for comment :)

[smarter]

I would have expected

scala3/compiler/src/dotty/tools/dotc/typer/Synthesizer.scala

Lines 483 to 488 in 30b3eb9

	if ctx.runZincPhases then
	// The mirror should be resynthesized if the constructor of the
	// case class `cls` changes. See `sbt-test/source-dependencies/mirror-product`.
	val rec = ctx.compilationUnit.depRecorder
	rec.addClassDependency(cls, DependencyByMemberRef)
	rec.addUsedName(cls.primaryConstructor)

to cover this, will need to be investigated more (e.g. does it work transparent inline?)

[mrdziuban]

does it work transparent inline?

It does indeed work if I add transparent to all of summonLabel, summonLabels and derived in Labels.scala. It doesn't work with any combination of just one or two though