Feature Request : Support non-deterministic ordering of slice args in Mock.On(...)

[manan-sc]

Assume that a function that we're mocking accepts a slice[] of strings, and it returns a response friendData thereafter.

A typical mock for this would be something like below -

serviceMock.On("GetFriends", friendIDs).Return(friendData, err)

where, friendIDs = []string{"abc", "def", "xyz"}

The mock will work if (and only if)GetFriends is definitely called with friendIDs = []string{"abc", "def", "xyz"}. However, if it's called with say, friendIDs = []string{"xyz", "abc", "def"}, the mock will panic with mock: Unexpected Method Call.

Something like below -

panic: 

mock: Unexpected Method Call
-----------------------------

GetFriends([]string)
                0: []string{"xyz", "abc", "def"}

The closest call I have is: 

GetFriends([]string)
                0: []string{"abc", "def", "xyz"}

Difference found in argument 0:

--- Expected
+++ Actual
@@ -1,4 +1,4 @@
 ([]string) (len=2) {
- (string) (len=3) "abc",
- (string) (len=3) "def",
- (string) (len=3) "xyz",
+ (string) (len=3) "xyz",
+ (string) (len=3) "abc",
+ (string) (len=3) "def",
 }

Diff: 
        0: FAIL:  ([]string{"xyz", "abc", "def"}) != ([]string{"abc", "def", "xyz"}) [recovered]
        panic: 

It could be that the friendIDs are returned by another service (or a DB). Using mock.Anything (although it works) seems a bit misleading - it's not as if we don't care as to what argument was used to call the function, just that the ordering of individual elements inside that slice is immaterial.

Is there an intention to support Any ordering of elements inside a slice, so long as all the elements (and only those elements) are present in the argument?

[brackendawson]

I don't think any built in argument matcher for slices in any order is likely to be added, maybe after generics.

However you can build your own argument matchers using mock.MatchedBy(), here's one using the well known elements match algorithm:

package kata

import (
	"testing"

	"github.com/stretchr/testify/mock"
)

func TestCompare(t *testing.T) {
	m := &myMock{}
	m.Test(t)
	m.On("MyFunc", mock.MatchedBy(func(friends []string) bool {
		return elementsMatch(friends, []string{"barney", "fred", "wilma"})
	})).Return(nil)
	defer m.AssertExpectations(t)

	m.MyFunc([]string{"fred", "wilma", "barney"})
}

func elementsMatch(x, y []string) bool {
	if len(x) != len(y) {
		return false
	}
	diff := make(map[string]int, len(x))
	for _, _x := range x {
		diff[_x]++
	}
	for _, _y := range y {
		if _, ok := diff[_y]; !ok {
			return false
		}
		diff[_y] -= 1
		if diff[_y] == 0 {
			delete(diff, _y)
		}
	}
	return len(diff) == 0
}

type myMock struct {
	mock.Mock
}

func (m *myMock) MyFunc(friends []string) error {
	return m.Called(friends).Error(0)
}

[manan-sc]

I don't think any built in argument matcher for slices in any order is likely to be added, maybe after generics.

However you can build your own argument matchers using mock.MatchedBy(), here's one using the well known elements match algorithm:

package kata

import (
	"testing"

	"github.com/stretchr/testify/mock"
)

func TestCompare(t *testing.T) {
	m := &myMock{}
	m.Test(t)
	m.On("MyFunc", mock.MatchedBy(func(friends []string) bool {
		return elementsMatch(friends, []string{"barney", "fred", "wilma"})
	})).Return(nil)
	defer m.AssertExpectations(t)

	m.MyFunc([]string{"fred", "wilma", "barney"})
}

func elementsMatch(x, y []string) bool {
	if len(x) != len(y) {
		return false
	}
	diff := make(map[string]int, len(x))
	for _, _x := range x {
		diff[_x]++
	}
	for _, _y := range y {
		if _, ok := diff[_y]; !ok {
			return false
		}
		diff[_y] -= 1
		if diff[_y] == 0 {
			delete(diff, _y)
		}
	}
	return len(diff) == 0
}

type myMock struct {
	mock.Mock
}

func (m *myMock) MyFunc(friends []string) error {
	return m.Called(friends).Error(0)
}

Nice, thanks @brackendawson - that should certainly work! I'd like to know why you think that this feature will be more useful until after there's support for generics in go.

[brackendawson]

I'd like to know why you think that this feature will be more useful until after there's support for generics in go.

I'm probably just being a bit thick, adding any generic code to the repo would mean 1.18 being the minimum supported Go version. Thinking some more we could definitely just use reflection and have a function in mock like:

// AnyOrder can be used to match a mock call based the elements of a
// slice or array ignoring the order of the elements.
func AnyOrder(list interface{}) argumentMatcher

(argumentMatcher might not be the best return value but it could certainly work).

Then you can just:

mockedThing.On("FunctionName", mock.AnyOrder([]string{"tom", "dick", "harry"}).Return("things")

I'd ✅ that, I wonder what the maintainers would think?

[manan-sc]

I'd like to know why you think that this feature will be more useful until after there's support for generics in go.

I'm probably just being a bit thick, adding any generic code to the repo would mean 1.18 being the minimum supported Go version. Thinking some more we could definitely just use reflection and have a function in mock like:

// AnyOrder can be used to match a mock call based the elements of a
// slice or array ignoring the order of the elements.
func AnyOrder(list interface{}) argumentMatcher

(argumentMatcher might not be the best return value but it could certainly work).

Then you can just:

mockedThing.On("FunctionName", mock.AnyOrder([]string{"tom", "dick", "harry"}).Return("things")

I'd ✅ that, I wonder what the maintainers would think?

Yeah, that sounds good to me - the semantics around AnyOrder indicating that the argument is to be matched by something that has the same elements (with their individual frequencies), and is agnostic of the relative element ordering.

[yusrilmr]

I am fond of @brackendawson 's idea with mock.AnyOrder().
I hope the maintainers will consider it.

[botkevin]

I hope the maintainers will add this method or at least make mock.argumentMatcher an exported type.
func MatchedBy(fn interface{}) argumentMatcher returns an unexported type, which I am pretty sure is bad convention anyways. Could I get a reason why this is unexported?

I am trying to make a method that can mock a list with the elements in any order:

func ElementsMatch[K comparable](a, b []K) bool {
	...
}

func MockAnyOrder[K comparable](a []K) (m mock.argumentMatcher) {
	m = mock.MatchedBy(func(x []K) bool {
		return ElementsMatch(x, a)
	})
	return
}

myMock.On("MyMethod", MockAnyOrder([]string{"a","b"}).return(nil)

and it is not possible because mock.argumentMatcher is unexported.

[manan-sc]

Well, you could use something like below (taking inspiration from @brackendawson's idea) -

// Test body in case of general bool-type assertions
assert.True(t, ElementsMatch([]string{a, b}, []string{b, a}))

// Test body in case of fn argument matcher assertions
userIDs := []string{a, b}
funcMock.On("GetFoo", mock.MatchedBy(func(someIDs []string) bool {
		return ElementsMatch(someIDs, userIDs)
	})).Return(fooData, err)


// ElementsMatch asserts that the specified listA(array, slice...) is equal to specified
// listB(array, slice...) ignoring the order of the elements. If there are duplicate elements,
// the number of appearances of each of them in both lists should match.
//
// ElementsMatch([1, 3, 2, 3], [1, 3, 3, 2])
func ElementsMatch(listA, listB []string) bool {
	if len(listA) != len(listB) {
		return false
	}
	diff := make(map[string]int, len(listA))
	for _, a := range listA {
		diff[a]++
	}

	for _, b := range listB {
		if _, ok := diff[b]; !ok {
			return false
		}
		diff[b]--
		if diff[b] == 0 {
			delete(diff, b)
		}
	}
	return len(diff) == 0
}

[botkevin]

Yes but it gets really verbose when you have multiple mocks. It would be nice to not have to repeat the code, and it would improve readability.

[manan-sc]

Yes but it gets really verbose when you have multiple mocks. It would be nice to not have to repeat the code, and it would improve readability.

You needn't repeat the code if you put the ElementsMatch function in a helper package, export it, and use it wherever you want it. Perhaps that's what you're after if I'm understanding this correctly?

[brackendawson]

Unless we put something in Testify then any hacks we suggest will cause you to repeat yourself, I'll open a PR for my idea so it's ready next time a maintainer does a drive-by, we see them from time to time.

Here's an especially bad way of hacking it that repeats a little less code:

package main

import (
	"testing"

	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/mock"
)

type myMock struct {
	mock.Mock
}

func (m *myMock) Process(elems []string) error {
	return m.Called(elems).Error(0)
}

func TestIfy(t *testing.T) {
	m := &myMock{}
	m.Test(t)
	m.AssertExpectations(t)
	m.On("Process", mock.MatchedBy(AnyOrder([]string{"a", "b", "c"}))).Return(nil).Once()
	assert.NoError(t, m.Process([]string{"c", "a", "b"}))
}

func AnyOrder[T any](expectation []T) func([]T) bool {
	return func(elems []T) bool {
		return assert.ElementsMatch(&testing.T{}, elems, expectation)
	}
}

[botkevin]

@manan-sc No. I mean repeating

funcMock.On("GetFoo", mock.MatchedBy(func(someIDs []string) bool {
		return ElementsMatch(someIDs, userIDs)
	})).Return(fooData, err)

instead of a more simple

funcMock.On("GetFoo", MockAnyOrder(userIDs)).Return(fooData, err)

@brackendawson's idea seems fine, but it's also pretty verbose for such a simple problem. Hopefully we can just eventually get a canonical provided way to do this.