Pact Standalone not honoring * matchers

[mattermack]

The pact standalone does not appear to support the * in JSON path. It does when we explicitly enumerate the items in an array.

Sent - working
Here is what I send to the stand alone:
{ "description": "General Meetup Categories 4", "providerState": "A GET request to return JSON using Meetups category api under version 2 4", "request": { "method": "GET", "headers": { "Content-Type": "application/json" }, "path": "/2/categories" }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "matchingRules": { "$.body.results[1].name": { "match": "regex", "regex": "Games|Book Clubs" }, "$.body.results[1].sort_name": { "match": "regex", "regex": "Games|Book Clubs" }, "$.body.results[1].id": { "match": "type" }, "$.body.results[1].shortname": { "match": "regex", "regex": "Games|Book Clubs" } }, "body": { "results": [ { "name": "Games", "sort_name": "Games", "id": 11, "shortname": "Games" }, { "name": "Book Clubs", "sort_name": "Book Clubs", "id": 18, "shortname": "Book Clubs" } ] } } }

Received - Working
Here is what I get back
{ "description": "General Meetup Categories 4", "providerState": "A GET request to return JSON using Meetups category api under version 2 4", "request": { "method": "GET", "path": "/2/categories", "headers": { "Content-Type": "application/json" } }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "body": { "results": [ { "name": "Games", "sort_name": "Games", "id": 11, "shortname": "Games" }, { "name": "Book Clubs", "sort_name": "Book Clubs", "id": 18, "shortname": "Book Clubs" } ] }, "matchingRules": { "$.body.results[1].name": { "match": "regex", "regex": "Games|Book Clubs" }, "$.body.results[1].sort_name": { "match": "regex", "regex": "Games|Book Clubs" }, "$.body.results[1].id": { "match": "type" }, "$.body.results[1].shortname": { "match": "regex", "regex": "Games|Book Clubs" } } } }

That works as intended. However, if I add that I want all items in an array to match this pattern [*] instead of [1], matchers are not returned.

Sent - I think with the bug
{ "description": "General Meetup Categories 6", "providerState": "A GET request to return JSON using Meetups category api under version 2 6", "request": { "method": "GET", "headers": { "Content-Type": "application/json" }, "path": "/6/categories" }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "matchingRules": { "$.body.results[*].name": { "match": "regex", "regex": "Games|Book Clubs" }, "$.body.results[*].sort_name": { "match": "regex", "regex": "Games|Book Clubs" }, "$.body.results[*].id": { "match": "type" }, "$.body.results[*].shortname": { "match": "regex", "regex": "Games|Book Clubs" } }, "body": { "results": [ { "name": "Games", "sort_name": "Games", "id": 11, "shortname": "Games" }, { "name": "Book Clubs", "sort_name": "Book Clubs", "id": 18, "shortname": "Book Clubs" } ] } } }

Received - notice no matchers
{ "description": "General Meetup Categories 6", "providerState": "A GET request to return JSON using Meetups category api under version 2 6", "request": { "method": "GET", "path": "/6/categories", "headers": { "Content-Type": "application/json" } }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "body": { "results": [ { "name": "Games", "sort_name": "Games", "id": 11, "shortname": "Games" }, { "name": "Book Clubs", "sort_name": "Book Clubs", "id": 18, "shortname": "Book Clubs" } ] } } }

This is tested completely outside of PHP or Pact-PHP but with the stand alone Ruby.

1. Using Postman, delete -> http://localhost:7200/interactions
2. Using Postman, post-> http://localhost:7200/interactions
3. Using Postman, get http://localhost:7200/2/categories or http://localhost:7200/6/categories
4. Using Postman, post http://localhost:7200/pact

Leveraging v1.26.0 on Windows

[bethesque]

Can you use https://github.com/pact-foundation/pact-ruby-standalone-e2e-example to recreate the issue please?

[mattermack]

You bet. I treated the master branch as the one I'm having a problem with. I used the branch working-for-me as the issue. All logs should be attached and versioned.

Issue
https://github.com/mattermack/pact-ruby-standalone-e2e-example/blob/master/pacts/foo-bar.json
https://github.com/mattermack/pact-ruby-standalone-e2e-example/blob/master/script/consumer-interaction.json
Note that no matchers were added, when using the [*] notation or in the logs. However, this is clearly in the consumer-interaction.json. Do I just have jsonpath poorly defined? I want to say for every object in the array, the name should run this matcher.

Working
https://github.com/mattermack/pact-ruby-standalone-e2e-example/blob/working-for-me/pacts/foo-bar.json
Note that the only change was the [1] for the first element in the array.

Note that I ran this on Ubuntu Linux version 1.26.0. Previous tests were on the Windows versions.

[bethesque]

Is this the logic for sending the interactions to the mock service?

[mattermack]

The logic for sending the interaction to the mock service is from script/consumer-create-pact.sh

# set up interaction curl -X POST -H "X-Pact-Mock-Service: true;Content-Type: \"application/json\"" -d@script/consumer-interaction.json localhost:1234/interactions

Is that what you meant? It is almost like the ruby end being /interactions cannot parse the [*] syntax and ignores those matchers.

[mattermack]

From the specification, if I had to narrow down the cases, I think these two would be the ones that I suspect have an issue:

• Array With Regex Matcher
• Objects in array type matching

I took Array With Regex Matcher and created a simple pact with it. I simulated all this quickly on Windows using Postman. You can see Postman clearly sending over matchers:

• $.body.myDates match type
• $.body.myDates[*] match regex

However, the mock server only sees and writes the $.body.myDates to disk (or responds to Postman).

The postman collection is on https://github.com/mattermack/pact-ruby-standalone-e2e-example/tree/master/Windows-Postman

[bethesque]

To be honest, I use the old Ruby format for interacting with the mock service, because it's just much simpler to generate! So do the other wrapper languages.

Here is the js code as an example:

Each like: https://github.com/pact-foundation/pact-js/blob/69683bf35d99ab2581360ae5bb040c266e49a279/src/dsl/matchers.ts#L195

Something like: https://github.com/pact-foundation/pact-js/blob/69683bf35d99ab2581360ae5bb040c266e49a279/src/dsl/matchers.ts#L215

Term: https://github.com/pact-foundation/pact-js/blob/69683bf35d99ab2581360ae5bb040c266e49a279/src/dsl/matchers.ts#L40

[mattermack]

Is there something I can do to make this easy to replicate? I replicated it on based on a pact-specification test case. It is replicated in e2e and in Postman

[bethesque]

There are no tests for it unfortunately. But the format is so simple, it should be reasonable straightforward to copy. You should be able to play around with the e2e example.

[mattermack]

Does the existing e2e not demostrate the issue? https://github.com/mattermack/pact-ruby-standalone-e2e-example

I can start digging into Ruby code too! I think it is somewhere in the /interactions end point. I bet it is a library or regex that was quickly updated on the Ruby side. That's at least by guess assuming I understand the JSONPath requirements, which I may not

[bethesque]

Sometimes it's easier to set up the examples in Ruby! pact-foundation/pact-ruby-e2e-example@0b23d39

You may well be right that the mock service is ignoring the matching rules, and it's possible that we've just never noticed it because nobody else is using it in that way.

Honestly, I'd recommend you just used the simple JSON syntax like all the other client libraries do to communicate with the mock service. Having implemented both, I can say that the logic is much simpler! Here is the request that is sent to the mock service to do the thing that you want to do: pact-foundation/pact-ruby-e2e-example@0b23d39#diff-6697b25e61ffb252a7c6f1be79bfe292R3

[mattermack]

Oh, that's interesting. The simpled JSON syntax does not actually send the pact syntax like the pact-specification examples. Instead there is some underlying "JSON metadata" like "json_class": "Pact::Term" or "json_class":"Pact::ArrayLike".

To date with the examples, I've been using e2e to highlight direct requests with bash to demonostrate the issue (meaning no PHP at all). I'll explore two scenarios:

• a use cases defined in pact-specification tests in that JSON
• a use case defined in pact-specification with the associated "metadata" for the lack of a better description.

Are these json_classes defined anywhere we can implement? (I'll get out on the documentation after doing the dishes :) )

[mattermack]

Also, with the last explanation, the js examples make more sense. Thanks (and the dishes are done #winning)

[bethesque]

There are only those three classes, and the examples are pretty much all there is. But let me know if you need more.

[gyulavoros]

@mattermack Did you have any luck patching the standalone ruby version, or you have started to use the JSON syntax?

[cfmack]

At the time, I hacked/patched the standalone ruby version. That is likely a very old patch by now.

[bethesque]

@gyulavoros if you have an issue, please create a reproducible example using the https://github.com/pact-foundation/pact-ruby-standalone-e2e-example and raise a new github issue.

[gyulavoros]

@bethesque Here is a fork with a reproducible example: https://github.com/gyulavoros/pact-ruby-standalone-e2e-example

If you run script/consumer-create-pact.sh, you'll see the following log:

INFO  WEBrick 1.3.1
INFO  ruby 2.2.2 (2015-04-13) [x86_64-darwin13]
INFO  WEBrick::HTTPServer#start: pid=36006 port=1234
Cleared interactions
WARN: Ignoring unsupported matching rules {"match"=>"type"} for path $['body']['data'][*]['message']
Registered interactions
{"message":"Hello world"}Interactions matched
{
  "consumer": {
    "name": "Foo"
  },
  "provider": {
    "name": "Bar"
  },
  "interactions": [
    {
      "description": "foo",
      "request": {
        "method": "GET",
        "path": "/foo"
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "message": "Hello world"
        }
      }
    }
  ],
  "metadata": {
    "pactSpecification": {
      "version": "2.0.0"
    }
  }
}
INFO  going to shutdown ...
INFO  WEBrick::HTTPServer#start done.

Please note the line:

WARN: Ignoring unsupported matching rules {"match"=>"type"} for path $['body']['data'][*]

If I'd change this line to $.body.data[0].message it works, but loose matching would only be applied to the first array element.

[gyulavoros]

@bethesque Did you have the time to take a look at the example?

[erik-inkapool]

I also hit this issue when writing custom pact generation code. It feels strange to me that Pact Standalone doesn't follow the pact json specification. I don't think pact should exclusively rely on undocumented json_class keys if there's a clear, documented way to achieve the same result from the specification.