Skip to content

investigate huge floating point errors in (flaky) tests #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sfindeisen opened this issue May 19, 2022 · 1 comment
Open

investigate huge floating point errors in (flaky) tests #47

sfindeisen opened this issue May 19, 2022 · 1 comment
Labels
help wanted Extra attention is needed

Comments

@sfindeisen
Copy link
Contributor

sfindeisen commented May 19, 2022

Depending on the architecture and/or platform, we're sometimes getting test errors like:

FAIL (31.18s)
      test/common/TestMnistFCNN.hs:132:
      expected: 0.8991
       but got: 0.7424999999999999
      Use -p '/2 artificial 5 4 3 2 1 5 4 3 2 2393 1.0/' to rerun this test only.

This difference is huuge.

Let's try to investigate this and possibly also other similar test cases, understand what is going on, then fix the test case and/or the implementation.

Related issue: #46

@sfindeisen
Copy link
Contributor Author

More failures:

$ cabal test --enable-optimization shortTestForCI
[...]
FAIL (10.29s)
      test/common/TestMnistFCNN.hs:198:
      expected: 0.7756000000000001
       but got: 0.7882
      Use -p '/VV artificial 5 4 3 2 1 5 4 3 2 0 18 2393 1.0/' to rerun this test only.
[...]
FAIL (7.47s)
      test/common/TestMnistFCNN.hs:275:
      expected: 0.8085
       but got: 0.8114
      Use -p '/LL artificial 5 4 3 2 1 5 4 3 2 0 3 3 2393 1.0 0.5/' to rerun this test only.
[...]
FAIL (13.25s)
      test/common/TestMnistFCNN.hs:548:
      expected: 0.8246
       but got: 0.8202
      Use -p '/S artificial 5 4 3 2 1 5 4 3 2 6 2393 1.0 0.5/' to rerun this test only.
[...]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants