Support Gradle configuration cache on multiple JVMs

[nedtwigg]

Gradle configuration cache requires task state to be round-trip serializable. As outlined at #987, this is a signficant constraint on plugin developers and it slows down the end-user experience of change-test-change-test on a single JVM daemon. We built a workaround (JvmLocalCache) that complies with Gradle's requirement by making our tasks round-trip serializable without requiring much work from plugin developers or the end user's machine, but it only works within a single JVM.

To work on multiple JVMs, we need to

• make every step round-trip serializable
• remove JvmLocalCache

[eskatos]

FYI Gradle 8 will do a full roundtrip on the first invocation with CC enabled.
See work in progress here gradle/gradle#21985 (comment)
The goal is for this to be the default in 8.0.

[nedtwigg]

Full roundtrip will work great on the same JVM, so Spotless will still work and not work under the same conditions it does currently. EDIT: Compatibility with configuration cache within a single JVM on Gradle 8 is confirmed.

[nedtwigg]

I think we now have a way to incrementally build this until we can eventually remove our workaround. See:

• Start making each FormatterStep roundtrip serializable. #1945

In this comment, I'm going to maintain a list of remaining TODOs before we will fully support Gradle's configuration cache model without any workarounds. If you'd like to help: comment in this issue when you start on a given step, follow-up with a PR (draft is fine) and I will update the checklist below

• remove LineEnding.GIT_ATTRIBUTES (at least for Gradle) because very difficult to make it serializable Support Gradle configuration cache on multiple JVMs  #1274
  • we can keep GIT_ATTRIBUTES_FAST_ALLSAME which is the default anyway
• lib(generic.PipeStepPair) Serializable refactor pipe/fence #1954
• lib(generic.EndWithNewlineStep)
• lib(generic.IndentStep) Start making each FormatterStep roundtrip serializable. #1945
• lib(generic.Jsr223Step) Make all steps with JarState serializable #2039
• lib(generic.LicenseHeaderStep) Port all stragglers except npm #2132
• lib(generic.NativeCmdStep) Port all stragglers except npm #2132
• lib(generic.ReplaceRegexStep)
• lib(generic.ReplaceStep)
• lib(generic.TrimTrailingWhitespaceStep)
• lib(groovy.RemoveSemicolonsStep)
• lib(antlr4.Antlr4FormatterStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(cpp.ClangFormatStep) Serializable foreign exe #1955
• extra(cpp.EclipseFormatterStep) Make the Equo-based steps round-trip serializable #1950
• lib(gherkin.GherkinUtilsStep) Make all steps with JarState serializable #2039
• extra(groovy.GrEclipseFormatterStep) Make the Equo-based steps round-trip serializable #1950
• lib(java.GoogleJavaFormatStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(java.ImportOrderStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(java.PalantirJavaFormatStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(java.RemoveUnusedImportsStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• extra(java.EclipseJdtFormatterStep) Make the Equo-based steps round-trip serializable #1950
• lib(java.FormatAnnotationsStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(java.CleanthatJavaStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(json.gson.GsonStep) Make all steps with JarState serializable #2039
• lib(json.JacksonJsonStep) Make all steps with JarState serializable #2039
• lib(json.JsonSimpleStep) Make all steps with JarState serializable #2039
• lib(json.JsonPatchStep) Make all steps with JarState serializable #2039
• lib(kotlin.KtLintStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(kotlin.KtfmtStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(kotlin.DiktatStep) Start making each FormatterStep roundtrip serializable. #1945
• lib(markdown.FreshMarkStep) Make all steps with JarState serializable #2039
• lib(markdown.FlexmarkStep) Make all steps with JarState serializable #2039
• lib(npm.EslintFormatterStep) Make the npm steps roundtrip serializable #2135
• lib(npm.PrettierFormatterStep) Make the npm steps roundtrip serializable #2135
• lib(npm.TsFmtFormatterStep) Make the npm steps roundtrip serializable #2135
• lib(pom.SortPomStepStep) Make all steps with JarState serializable #2039
• lib(protobuf.BufStep) Serializable foreign exe #1955
• lib(python.BlackStep) Serializable foreign exe #1955
• lib(biome.BiomeStep) Make the npm steps roundtrip serializable #2135
• lib(scala.ScalaFmtStep) Serializable Groovy, Java, Kotlin, and Scala steps #2033
• lib(sql.DBeaverSQLFormatterStep) Port all stragglers except npm #2132
• extra(wtp.EclipseWtpFormatterStep) Serializable refactor legacy eclipse steps #1952
• lib(yaml.JacksonYamlStep) Make all steps with JarState serializable #2039

[lehmanju]

I checked npm.EslintFormatterStep, npm.PrettierFormatterStep, npm.TsFmtFormatterStep and sql.DBeaverSQLFormatterStep. They all use StepHarness for testing which has roundtrip serialization testing enabled by default. Since their tests do not fail, it should mean they are already serializable, am I correct?

For NativeCmdStep I created a PR #2108 which adds a test using StepHarness.