Skip to content

Extend fuzz tests #196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 36 additions & 3 deletions s2/encode_fuzz_test.go
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
// Fuzz tests for s2 decoding. Run using
// go test -fuzz=Fuzz github.com/golang/geo/s2
// Fuzz tests for s2 decoding.
package s2

import (
"bytes"
"testing"
)

// go test -fuzz=FuzzDecodeCellUnion github.com/golang/geo/s2
func FuzzDecodeCellUnion(f *testing.F) {
cu := CellUnion([]CellID{
CellID(0x33),
Expand All @@ -28,6 +28,39 @@ func FuzzDecodeCellUnion(f *testing.F) {
if got := c.ApproxArea(); got < 0 {
t.Errorf("ApproxArea() = %v, want >= 0. CellUnion: %v", got, c)
}
// TODO: Test more invariants that should hold for all CellUnion.
buf := new(bytes.Buffer)
if err := c.Encode(buf); err != nil {
// Re-encoding the cell union does not necessarily produce the same bytes, as there could be additional bytes following after n cells were read.
t.Errorf("encode() = %v. got %v, want %v. CellUnion: %v", err, buf.Bytes(), encoded, c)
}
if c.IsValid() {
c.Normalize()
if !c.IsNormalized() {
t.Errorf("IsNormalized() = false, want true. CellUnion: %v", c)
}
}
})
}

// go test -fuzz=FuzzDecodePolygon github.com/golang/geo/s2
func FuzzDecodePolygon(f *testing.F) {
for _, p := range []*Polygon{near0Polygon, near01Polygon, near30Polygon, near23Polygon, far01Polygon, far21Polygon, south0abPolygon} {
buf := new(bytes.Buffer)
if err := p.Encode(buf); err != nil {
f.Errorf("error encoding %v: ", err)
}
f.Add(buf.Bytes())
}

f.Fuzz(func(t *testing.T, encoded []byte) {
p := &Polygon{}
if err := p.Decode(bytes.NewReader(encoded)); err != nil {
// Construction failed, no need to test further.
return
}
if got := p.Area(); got < 0 {
t.Errorf("Area() = %v, want >= 0. Polygon: %v", got, p)
}
// TODO: Test more methods on Polygon.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the rationale for

  1. one file with all fuzz tests (or all decoding related fuzz tests)?
  2. having monolithic FuzzFoo tests rather than testing properties independently?
    ?

http://go/unit-testing-practices?polyglot=go#behavior-testing-examples

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. That's mostly me being oldschool. In the past, I added fuzz tests to a project that supported earlier go versions that did not yet support fuzz testing, so the fuzz tests needed to be in a seperate file, where a minimum go version was enforced. In this project, we might want to move the fuzz tests to the respective type test, e.g. polygon_test.go. Do you have a preference?

  2. Not sure what you mean. The primary thing under test is Decode: Will it always create a valid X? The properties we check on the decoded object feel secondary, hence I feel checking them in FuzzDecodeX is approrpiate.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Seems better to keep all polygon tests together rather than all fuzz tests together.
  2. You're not testing a single property. You're testing forall bytes decode doesn't crash && forall polygons area non-negative && more in the future. This seems partially due to the limited set of types supported by fuzzing. Maybe it's also computationally efficient depending on how hard it is for the fuzzer to find valid polygons.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Done
  2. I think this is correct. With every pass, the fuzzer tries to find new interesting examples that execute more code paths. Having a large variety of code paths executed inside the fuzz test is preferable.

})
}