Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test] Add regression testing #1363

Closed
sk- opened this issue Feb 22, 2021 · 1 comment · Fixed by #1918
Closed

[Test] Add regression testing #1363

sk- opened this issue Feb 22, 2021 · 1 comment · Fixed by #1918

Comments

@sk-
Copy link
Contributor

sk- commented Feb 22, 2021

As reported by #1360, SGVO should guarantee that at least the default settings produce a lossless optimization. On the other hand for the rest of the plugins, we should still require that the changes do not introduce visual differences that are too noticeable.

In #1357 @TrySound started the work for adding such regression testing using the W3C SVG 1.1 Test Suite. And @sk- has been working on a similar effort using a dataset taken from Wikimedia Commons.

I think we should have both a minimal dataset for doing some regression testing on CI (as the operation is rather expensive) an an extensive collection to be used offline before releasing. (To reduce the chance of errors we could instruct contributors to run that test suite on their machines before sending a PR, it could be a checkbox in the PR template).

In my testing the files from Wikimedia commons are quite a good source as they unveil quite a few errors. Also we should use the Oxygen icon them as at least svgcleaner claims svgo breaks many files.

For comparing the images I think reg-suit is a good alternative, as it integrates with github and has a visual ui, which makes it easy to see the differences (see these screenshots).

@TrySound
Copy link
Member

I found oxygen library is quite huge. 400mb gzipped and 1.2gb unpacked. Not a good case for running on CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants