#14316 Use local repo commands instead of the github api for Release Drop Down population #14834

JudeRV · 2025-06-27T17:49:43Z

Summary

Replaces the GitHub API call with a local Git command to fetch version tags for the Releases drop down. Fixes #14316

Changes

Added a minimumVersion parameter for parsing the tags to limit the amount shown in the dropdown. This avoids the dropdown having all of these values:

Detected Project Version: 7.0.0-M4 and Software Versions: 7.0.0-M4,7.0.0-M3,7.0.0-M1,6.2.3,6.2.2,6.2.1,6.2.0,6.1.3,6.1.2,6.1.1,6.1.0,6.0.0,6.0.0-RC1,6.0.0-M4,6.0.0-M3,6.0.0-M2,6.0.0-M1,5.3.6,5.3.5,5.3.4,5.3.3,5.3.2,5.3.1,5.3.0,5.2.6,5.2.5,5.2.4,5.2.3,5.2.2,5.2.1,5.2.0,5.1.10,5.1.9,5.1.8,5.1.7,5.1.6,5.1.5,5.1.4,5.1.3,5.1.2,5.1.1,5.1.0,5.0.3,5.0.2,5.0.1,5.0.0,5.0.0-RC4,5.0.0-RC3,5.0.0.RC2,5.0.0.RC1,5.0.0-M4,5.0.0-M3,5.0.0-M2,5.0.0-M1,4.1.4,4.1.3,4.1.2,4.1.1,4.1.0,4.1.0.M5,4.1.0.M4,4.1.0.M3,4.1.0.M2,4.1.0.M1,4.0.13,4.0.12,4.0.11,4.0.10,4.0.9,4.0.8,4.0.7,4.0.6,4.0.5,4.0.4,4.0.3,4.0.2,4.0.1,4.0.0,4.0.0.RC2,4.0.0.RC1,4.0.0-M2,4.0.0-M1,3.3.18,3.3.17,3.3.16,3.3.15,3.3.14,3.3.13,3.3.12,3.3.11,3.3.10,3.3.9,3.3.8,3.3.7,3.3.6,3.3.5,3.3.4,3.3.3,3.3.2,3.3.1,3.3.0,3.3.0.RC1,3.3.0.M2,3.3.0.M1,3.2.13,3.2.12,3.2.11,3.2.10,3.2.9,3.2.8,3.2.7,3.2.6,3.2.5,3.2.4,3.2.3,3.2.2,3.2.1,3.2.0,3.2.0.RC2,3.2.0.RC1,3.2.0.M2,3.2.0.M1,3.1.16,3.1.15,3.1.14,3.1.13,3.1.12,3.1.11,3.1.10,3.1.9,3.1.8,3.1.7,3.1.6,3.1.5,3.1.4,3.1.3,3.1.2,3.1.1,3.1.0,3.1.0.RC2,3.1.0.RC1,3.1.0.M3,3.1.0.M2,3.1.0.M1,3.0.17,3.0.16,3.0.15,3.0.14,3.0.13,3.0.12,3.0.11,3.0.10,3.0.9,3.0.8,3.0.7,3.0.6,3.0.5,3.0.4,3.0.3,3.0.2,3.0.1,3.0.0,3.0.0.RC3,3.0.0.RC2,3.0.0-RC1,3.0.0-M2,3.0.0-M1,2.6.1,2.6.0,2.5.6,2.5.5,2.5.4,2.5.3,2.5.2,2.5.1,2.5.0,2.4.5,2.4.4,2.4.3,2.4.2,2.4.1,2.4.0,2.4.0.RC2,2.4.0.RC1,2.4.0.M2,2.4.0.M1,2.3.11,2.3.10,2.3.9,2.3.8,2.3.7,2.3.6,2.3.5,2.3.4,2.3.3,2.3.2,2.3.1,2.3.0,2.3.0.RC2,2.3.0.RC1,2.3.0.M2,2.3.0.M1,2.2.5,2.2.4,2.2.3,2.2.2,2.2.1,2.2.0,2.2.0.RC4,2.2.0-RC3,2.2.0.RC2,2.2.0.RC1,2.2.0.M6,2.2.0.M5,2.2.0.M4,2.1.5,2.1.4,2.1.3,2.1.2,2.1.1,2.1.0,2.1.0-RC4,2.1.0.RC3,2.1.0.RC2,2.1.0.RC1,2.1.0.M4,2.1.0.M3,2.1.0.M2,2.0.4,2.0.3,2.0.2,2.0.1,2.0.0,2.0.0.RC3,2.0.0.RC2,2.0.0.RC1,2.0.0.M2,2.0.0.M1,1.4.0.M1,1.3.9,1.3.8,1.3.7,1.3.6,1.3.5,1.3.4,1.3.3,1.3.2,1.3.1,1.2.5,1.2.4,1.2.3,1.2.2,1.2.1,1.1.2,1.1.1,1.0.4,1.0.3,1.0.2,1.0.1,1.0.RC4,1.0.0.RC1,1.0.0.M6,0.6.RC1,0.5.6,0.5.5,0.5.5.RC1,0.5.RC2,0.4.2,0.4.1,null

Edited visitFile() function inside of Files.walkFileTree() to correctly copy the modified text over to modified-guide/. It seemed to be overwriting the new changes by copying the original file onto the target, but now the files under modified-guide/ show correct HTML.
Added modified-guide/index.html to the list of files for which the dropdown should be generated, which was the behavior prior to a recent change.
Edited SoftwareVersion.groovy to only initialize a patch value when the data type matches. This avoids building versions like these:

Let me know if there are any concerns!

- Rewrote listRepoTags to use Git commands instead of GitHub API - Refactored visitFile to generate output without copying files - Rewrote parseSoftwareVersions to work with new input format & add optional minimum value

- Only assign patch values for integers, prevents tags such as 0.2.RC2 - Fixes error thrown by AddReleaseDropDown.parseSoftwareVersions

- Include single.html and index.html in new corrected paths to add dropdown to

jamesfredley · 2025-06-27T18:35:02Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

@@ -144,34 +141,34 @@ abstract class AddReleaseDropDown extends DefaultTask {

        softwareVersions
                .forEach { softwareVersion ->
-                    final String versionName = softwareVersion.versionText
+                    final String versionName = softwareVersion?.versionText


Avoids a null pointer after moving to git commands, which included a null tag.

jamesfredley · 2025-06-27T18:37:35Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

@@ -118,7 +116,6 @@ abstract class AddReleaseDropDown extends DefaultTask {

                    filesToChange.remove(absolutePath)
                }
-                Files.copy(file, targetFile, StandardCopyOption.REPLACE_EXISTING)


This was overwriting the file which just had the drop down added with the original which did not have the dropdown. The delete code above was also removed, since the only files changed were targetFile's.

The original intent of my changes was to split the guide steps from a single step into many so that the following happens

generate guide to original (input is the adoc files and output is the original folder)

update the guide in modified (input is the original folder and output is the modified folder)

copy modified to end location (input is the modified folder and output is the staging location for the generated docs)

Each of the above steps then has a consistent input/output which means they can be easily cached by Gradle. This also means that whenever the task runs, it should always remove the target directory since it' "owns" those files.

jamesfredley · 2025-06-27T18:44:59Z

grails-doc/build.gradle

@@ -128,7 +128,7 @@ tasks.register('createReleaseDropdown', AddReleaseDropDown).configure { AddRelea
    it.group = 'documentation'
    it.dependsOn 'publishGuide'

-    it.inputFiles = project.layout.buildDirectory.files('original-guide/guide/single.html')
+    it.inputFiles = project.layout.buildDirectory.files('modified-guide/guide/single.html', 'modified-guide/index.html')


grails-core/grails-doc/build.gradle

Line 151 in 3c04692

def inputFiles = [project.layout.buildDirectory.file('manual/guide/single.html').get().asFile, project.layout.buildDirectory.file('manual/index.html').get().asFile]

This historically added the drop down to both files.

Good catch!

jdaugherty

Thank you for the contribution. After reviewing this, it looks like this is a bigger change to ensure we're using task configuration avoidance / lazy properties & cacheing. I've tried to make comments inline, but if you'd like to pair on some of this, I can make time given the scope of the changes required.

jdaugherty · 2025-06-27T19:30:59Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

@@ -118,7 +116,6 @@ abstract class AddReleaseDropDown extends DefaultTask {

                    filesToChange.remove(absolutePath)
                }
-                Files.copy(file, targetFile, StandardCopyOption.REPLACE_EXISTING)


The original intent of my changes was to split the guide steps from a single step into many so that the following happens

generate guide to original (input is the adoc files and output is the original folder)

update the guide in modified (input is the original folder and output is the modified folder)

copy modified to end location (input is the modified folder and output is the staging location for the generated docs)

Each of the above steps then has a consistent input/output which means they can be easily cached by Gradle. This also means that whenever the task runs, it should always remove the target directory since it' "owns" those files.

jdaugherty · 2025-06-27T19:31:42Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

@@ -80,9 +80,10 @@ abstract class AddReleaseDropDown extends DefaultTask {
    @TaskAction
    void addReleaseDropDown() {


There's another task in grails-gradle that seems to also be for adding a drop down. Can we move this file there and eliminate the duplication? Especially because we're making it generic now and not specific to the apache grails-core repo.

jdaugherty · 2025-06-27T19:32:36Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

@@ -80,9 +80,10 @@ abstract class AddReleaseDropDown extends DefaultTask {
    @TaskAction
    void addReleaseDropDown() {
        String projectVersion = version.get()
+        SoftwareVersion minimumVersion = new SoftwareVersion(major: 5)


Can we please change the 5 to a gradle property? To be correctly cacheable, you can't call methods on project from a task action so you'll have to make it an argument to the task and then in the constructor you will have to use a convention to default it from a property.

jdaugherty · 2025-06-27T19:33:01Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

@@ -103,9 +104,6 @@ abstract class AddReleaseDropDown extends DefaultTask {
            @Override
            FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
                Path targetFile = targetOutputDirectory.resolve(guideDirectory.relativize(file))
-                if (Files.exists(targetFile)) {


In order for this task to be correctly cacheable, we need to always purge the target location. It sounds like we overlapped the target files and that was a bug. We need to add this back.

jdaugherty · 2025-06-27T19:33:44Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

@@ -144,34 +141,34 @@ abstract class AddReleaseDropDown extends DefaultTask {

        softwareVersions
                .forEach { softwareVersion ->
-                    final String versionName = softwareVersion.versionText
+                    final String versionName = softwareVersion?.versionText
                    final String href = GRAILS_DOC_BASE_URL + "/" + versionName  + page


As part of removing the duplicate task, we should make this an argument to the task. Then in our build file where we register this task, we'll just default this as an argument to the task.

jdaugherty · 2025-06-27T19:36:09Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

-        if(token) {
-            connection.setRequestProperty("Authorization", "Bearer ${token}")
-        }
+    private List<String> listRepoTags() {


The more I think about this, I think we need to split this task into 2 tasks. 1 task to extract the tags and 1 task to update the guide. The main reason for this is we don't want to run the tag list every time. You can make it cache via inputs/outputs. The inputs/outputs for the tags task should be: input = CI build || 1 day cache, output = file with the tag lists for this task to read from. The inputs to this task then are the tag list & the base generated documentation.

jdaugherty · 2025-06-27T19:36:22Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

-        }
+    private List<String> listRepoTags() {
+        File repoRoot = project.rootProject.projectDir
+        def command = ["git", "-C", repoRoot.absolutePath, "tag", "-l", "--sort=-creatordate"]


In CI, the tags aren't fetched by default. We'll need to change the checkout commands so they're fetched.

jdaugherty · 2025-06-27T19:36:53Z

buildSrc/src/main/groovy/grails/doc/AddReleaseDropDown.groovy

+        File repoRoot = project.rootProject.projectDir
+        def command = ["git", "-C", repoRoot.absolutePath, "tag", "-l", "--sort=-creatordate"]
+
+        def process = new ProcessBuilder(command)


We shouldn't be opening processes directly from Gradle. This is another reason to split the tasks - the tag task should be a JavaExec with a doLast{} that saves the output.

jdaugherty · 2025-06-27T19:37:36Z

grails-doc/build.gradle

@@ -128,7 +128,7 @@ tasks.register('createReleaseDropdown', AddReleaseDropDown).configure { AddRelea
    it.group = 'documentation'
    it.dependsOn 'publishGuide'

-    it.inputFiles = project.layout.buildDirectory.files('original-guide/guide/single.html')
+    it.inputFiles = project.layout.buildDirectory.files('modified-guide/guide/single.html', 'modified-guide/index.html')


Good catch!

grails-gradle/docs-core/src/main/groovy/grails/doc/dropdown/SoftwareVersion.groovy

JudeRV added 5 commits June 27, 2025 01:46

Use Git commands to fetch version tags

6481079

- Rewrote listRepoTags to use Git commands instead of GitHub API - Refactored visitFile to generate output without copying files - Rewrote parseSoftwareVersions to work with new input format & add optional minimum value

Update SoftwareVersion to fix parsing error

b86c0ea

- Only assign patch values for integers, prevents tags such as 0.2.RC2 - Fixes error thrown by AddReleaseDropDown.parseSoftwareVersions

Update gradle file to include correct files

ba73cf2

- Include single.html and index.html in new corrected paths to add dropdown to

Merge branch '7.0.x' of https://github.com/JudeRV/grails-core into 7.0.x

4a4e76b

Rewrite patch check & add comment

11b1765

jamesfredley assigned JudeRV Jun 27, 2025

jamesfredley added this to Apache Grails 7.0.x Jun 27, 2025

jamesfredley moved this to In Progress in Apache Grails 7.0.x Jun 27, 2025

jamesfredley added this to the grails:7.0.0-RC1 milestone Jun 27, 2025

jamesfredley reviewed Jun 27, 2025

View reviewed changes

jamesfredley approved these changes Jun 27, 2025

View reviewed changes

jamesfredley requested a review from jdaugherty June 27, 2025 18:46

jdaugherty requested changes Jun 27, 2025

View reviewed changes

jamesfredley self-requested a review June 27, 2025 20:11

Merge branch 'apache:7.0.x' into 7.0.x

239db31

		@@ -80,9 +80,10 @@ abstract class AddReleaseDropDown extends DefaultTask {
		@TaskAction
		void addReleaseDropDown() {

Uh oh!

#14316 Use local repo commands instead of the github api for Release Drop Down population #14834

Are you sure you want to change the base?

#14316 Use local repo commands instead of the github api for Release Drop Down population #14834

Uh oh!

Conversation

JudeRV commented Jun 27, 2025 • edited by jamesfredley Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Uh oh!

jamesfredley Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jdaugherty left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jdaugherty Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

JudeRV commented Jun 27, 2025 •

edited by jamesfredley

Loading

jamesfredley Jun 27, 2025 •

edited

Loading

jdaugherty Jun 27, 2025 •

edited

Loading