Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitHub workflow testing generated sources #139

Open
sbesson opened this issue Dec 8, 2020 · 1 comment
Open

Add GitHub workflow testing generated sources #139

sbesson opened this issue Dec 8, 2020 · 1 comment

Comments

@sbesson
Copy link
Member

sbesson commented Dec 8, 2020

Regarding CI, I think the current approach is testing a lot of different combinations, but is not as effective as it could be in testing what is important. For example, the important part for the code generation is ensuring that the generated code doesn't differ between python versions (for consistency) and compared with the previous commit (to highlight generated source changes for review). But neither are tested. If you check both of these separately, and fail if there are changes between Python versions, and warn if there is a change compared with the previous commit, then you can build with a single Python version since you know that the code you are building is identical, and you'll be notified there is something additional to review if there are changes against the previous commit.

The above would reduce the job count to n Python jobs and m Java jobs which will test effectively, rather than n × m combinations which aren't giving the same coverage. (Not including the Python module, which is already tested separately.)

You could copy the approach taken here with the script_unix_java_gensrc block, and the diffsrc block below, and do that for each supported Python version. Here it's done with Gradle, but the Maven approach is the same. Now you're on GitHub Actions this probably becomes easier than it was with Travis!

Originally posted by @rleigh-codelibre in #127 (comment)

@sbesson
Copy link
Member Author

sbesson commented Dec 8, 2020

A few additional thoughts:

  • generally, the aim and the strategy described above make complete sense both in terms of optimizing the usage of the testing resources as well as increasing the coverage of the generated sources
  • there is no dedicated infrastructure in this repository and the Gitlab pipelines mentioned above are the best starting point
  • regarding the specifics of GitHub workflows, the actions/upload-artifact and actions/download-artifact actions should provide built-in infrastructure to share artifacts between steps and/or builds. I think the only states allowed are success/failure/cancelled at the moment so more investigation might be necessary to allow notifications when the sources are identical between versions but have changed compared to the base branch. For comparing the state of the pull requests against the base branch, I think the following should checkout the commit :
 - name: Checkout
  uses: actions/checkout@v2
  with:
    ref: ${{ github.event.pull_request.base.sha }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant