Add JSON jackson step, Refactor with Yaml, enable endWithEol, feature…

[blacelle]

This improves the work in #1478 and #1446

[nedtwigg]

Looks good! Minor comments below.

[blacelle]

With #1478, I got good results regarding mvn cache with ./gradlew :plugin-maven:runMavenBuild :plugin-maven:test --tests '*yaml*' --no-build-cache. Sadly, I again encounter weird issues.

e.g., after modifying a file in the plugin-maven, ./gradlew :plugin-maven:runMavenBuild --no-build-cache --info shows:

> Configure project :plugin-maven
Evaluating project ':plugin-maven' using build file '/Users/blacelle/workspace3/spotless/plugin-maven/build.gradle'.
All projects evaluated.
Task path ':plugin-maven:runMavenBuild' matched project ':plugin-maven'
Task name matched 'runMavenBuild'
Selected primary task 'runMavenBuild' from project :plugin-maven
Tasks to be executed: [task ':plugin-maven:copySourceFiles', task ':plugin-maven:copyMvnw', task ':lib:compileJava', task ':lib:processResources', task ':lib:classes', task ':lib:compileCompatKtLintApiJava', task ':lib:processCompatKtLintApiResources', task ':lib:compatKtLintApiClasses', task ':lib:compileCompatKtLint0Dot31Dot0Java', task ':lib:processCompatKtLint0Dot31Dot0Resources', task ':lib:compatKtLint0Dot31Dot0Classes', task ':lib:compileCompatKtLint0Dot32Dot0Java', task ':lib:processCompatKtLint0Dot32Dot0Resources', task ':lib:compatKtLint0Dot32Dot0Classes', task ':lib:compileCompatKtLint0Dot34Dot2Java', task ':lib:processCompatKtLint0Dot34Dot2Resources', task ':lib:compatKtLint0Dot34Dot2Classes', task ':lib:compileCompatKtLint0Dot45Dot2Java', task ':lib:processCompatKtLint0Dot45Dot2Resources', task ':lib:compatKtLint0Dot45Dot2Classes', task ':lib:compileCompatKtLint0Dot46Dot0Java', task ':lib:processCompatKtLint0Dot46Dot0Resources', task ':lib:compatKtLint0Dot46Dot0Classes', task ':lib:compileCompatKtLint0Dot47Dot0Java', task ':lib:processCompatKtLint0Dot47Dot0Resources', task ':lib:compatKtLint0Dot47Dot0Classes', task ':lib:compileCompatKtLint0Dot48Dot0Java', task ':lib:processCompatKtLint0Dot48Dot0Resources', task ':lib:compatKtLint0Dot48Dot0Classes', task ':lib:compileDiktatJava', task ':lib:compileFlexmarkJava', task ':lib:compileJacksonJava', task ':lib:compileKtfmtJava', task ':lib:compileKtlintJava', task ':lib:compilePalantirJavaFormatJava', task ':lib:compileScalafmtJava', task ':lib:compileSortPomJava', task ':lib:processDiktatResources', task ':lib:diktatClasses', task ':lib:processFlexmarkResources', task ':lib:flexmarkClasses', task ':lib:processJacksonResources', task ':lib:jacksonClasses', task ':lib:processKtfmtResources', task ':lib:ktfmtClasses', task ':lib:processKtlintResources', task ':lib:ktlintClasses', task ':lib:processPalantirJavaFormatResources', task ':lib:palantirJavaFormatClasses', task ':lib:processScalafmtResources', task ':lib:scalafmtClasses', task ':lib:processSortPomResources', task ':lib:sortPomClasses', task ':lib:jar', task ':plugin-maven:install_spotless-lib', task ':lib-extra:compileJava', task ':lib-extra:processResources', task ':lib-extra:classes', task ':lib-extra:jar', task ':plugin-maven:install_spotless-lib-extra', task ':testlib:compileJava', task ':testlib:processResources', task ':testlib:classes', task ':testlib:jar', task ':plugin-maven:install_spotless-testlib', task ':plugin-maven:installLocalDependencies', task ':plugin-maven:createPomXml', task ':plugin-maven:runMavenBuild']
Tasks that were excluded: []
Resolve mutations for :plugin-maven:copySourceFiles (Thread[Execution worker,5,main]) started.
:plugin-maven:copySourceFiles (Thread[Execution worker,5,main]) started.

OK.

Then:

> Task :plugin-maven:copySourceFiles
Caching disabled for task ':plugin-maven:copySourceFiles' because:
Build cache is disabled
Task ':plugin-maven:copySourceFiles' is not up-to-date because:
Input property 'rootSpec$1' file /Users/blacelle/workspace3/spotless/plugin-maven/src/main/java/com/diffplug/spotless/maven/json/JacksonJson.java has changed.
Resolve mutations for :plugin-maven:copyMvnw (Thread[Execution worker,5,main]) started.
:plugin-maven:copyMvnw (Thread[Execution worker Thread 14,5,main]) started.

OK.

Then

> Task :plugin-maven:runMavenBuild UP-TO-DATE
Caching disabled for task ':plugin-maven:runMavenBuild' because:
Build cache is disabled
Skipping task ':plugin-maven:runMavenBuild' as it is up-to-date.

KO : it looks like a change in plugin-maven sourceSets is not considered anymore enough to rebuild the plugin. I do not get how this interacts with --no-build-cache, but I now can get this working only through ./gradlew clean :plugin-maven:runMavenBuild :plugin-maven:test --tests '*yaml*' --no-build-cache which is a pain as the initialization oft he local m2 repo is very slow.

[blacelle]

./gradlew :plugin-maven:runMavenBuild --rerun :plugin-maven:test --tests '*yaml*' --no-build-cache does the job. Not being a gradle-friendly user, if this is the legimitate way to test locally a subset of tests from the CLI (as I failed doing so in both Eclipse and IntelliJ), I suppose it should be documented properly.

[blacelle]

Now, without any visible changes, I get with the same command:

[INFO] Installing /Users/blacelle/workspace3/spotless/plugin-maven/build/mavenProject/pom.xml to /Users/blacelle/workspace3/spotless/plugin-maven/build/localMavenRepository/com/diffplug/spotless/spotless-maven-plugin/2.30.0-SNAPSHOT/spotless-maven-plugin-2.30.0-SNAPSHOT.pom

• In plugin 'org.gradle.api.plugins.BasePlugin' type 'org.gradle.api.tasks.bundling.Jar' property '$1' specifies file '/Users/blacelle/workspace3/spotless/plugin-maven/build/mavenProject/target/spotless-maven-plugin-2.31.0-SNAPSHOT.jar' which doesn't exist.

We see something is wrong with the plugin version.

[blacelle]

@nedtwigg Ok. Some (magic for newcomers) is going on. I understand some gradle versions are extrapolated from the CHANGES.MD file. As I added some notes in the ## [Unreleased] block, then versions are shuffled and my command is KO. Reverting the maven-plugin/CHANGES.MD fixes the issue. Is it a bug in the gradle config, or in the code extrapolating from a MD?

[nedtwigg]

My guess is a bug in the maven build. I'm gonna work on #554 to try to solve these at the root.

[nedtwigg]

I would merge origin feat/maven-build into this PR. #1496 will definitely get merged, and it should fix the issues you're having.

[blacelle]

Thanks @nedtwigg . I'm willing to sync by branch as soon as #1496 is merged, and provide some feedback. (If necessary, I can test in before merging, but I suppose we can easily push more improvments regarding to maven-build after the merge).

[blacelle]

This introduces jackson-formatters in Gradle, in order to simplify the developement process (referring to maven cache issues (CLI + IDE)).

[blacelle]

@nedtwigg I would need a hand. I have issues with com.diffplug.gradle.spotless.YamlExtensionTest#testFormatYaml_WithJackson_defaultConfig_separatorComments. It fails around com.diffplug.spotless.yaml.JacksonYamlStep.State#State as the config is null.

* What went wrong:
Could not determine the dependencies of task ':spotlessApply'.
> Could not create task ':spotlessJsonApply'.
> Could not create task ':spotlessJson'.
> ARG

Also why is it referring to spotlessJson?

I've not been able to debug Gradle-based unit-tests through Intellij (I hoped Gradle unit-tests would be easier to test than mvn ones). And even adding --stack-traces and so-on, I can't get a relevant stack-traces (hoping to explain the null JacksonConfig). Thanks

[nedtwigg]

Here's the answer to some of the problems. Hopefully merging main will resolve the rest!

[blacelle]

Thanks @nedtwigg , I resynced my branch.

I still get the unexpected null JacksonConfig issue

com.diffplug.gradle.spotless.YamlExtensionTest#testFormatYaml_WithJackson_defaultConfig_separatorComments. It fails around com.diffplug.spotless.yaml.JacksonYamlStep.State#State as the config is null.

but now with the proper step referred i nthe log:

* What went wrong:
Could not determine the dependencies of task ':spotlessApply'.
> Could not create task ':spotlessYamlApply'.
> Could not create task ':spotlessYaml'.
> JacksonYamlConfig must not be null

[blacelle]

With the new mvn-build setup, running tests in Intellij has a new faulty behavior:

> Task :lib:javadoc FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':lib:javadoc'.
> Javadoc generation failed. Generated Javadoc options file (useful for troubleshooting): '/Users/blacelle/workspace3/spotless/lib/build/tmp/javadoc/javadoc.options'

While it is awkward to have unittests broken by Javadoc, fixes in the Javadoc (necessary to fix the whole build anyway) enables testing of mvn tests from Intellij. Excellent.

[blacelle]

The issues related with null jackson config were due to JacksonYamlGradleConfig extending AJacksonGradleConfig, and createStep was relying a the extended class field. Noob issue, but too bad it was difficult to get a stack (I ended with Throwables.getStackTraceAsString(new RuntimeException())).

Note for gradle tests: one can easily add stacktraces with:

gradleRunner().withArguments("spotlessApply", "--stacktrace").build();

[blacelle]

./gradlew :plugin-maven:test --tests '*json*'

and

./gradlew :plugin-maven:test --tests '*yaml*'

Run smoothly. That's great to facilitate mvn-plugin testing from CLI.

[blacelle]

@nedtwigg This is ready for review.

[blacelle]

We followed suggestion from FasterXML/jackson-dataformats-text#66

[nedtwigg]

This looks great, except the table in the root README.md needs to get updated (new name for Yaml, check gradle, add JacksonJsonStep). I did this locally, but I can't push to your branch. It's not a big deal, but if possible it would be better if you submit PR's from blacelle/spotless, as it seems that solven-eu does not allow me to help with PRs.

[blacelle]

Moved into #1508

[nedtwigg]

Released in plugin-gradle 6.14.0 and plugin-maven 2.31.0.