Feat: Eventbridge v2: Add pattern matching

[maxhoheiser]

Motivation

This PR adds pattern matching to distribute incoming events based on event bus specified in the event, rules in the matching event bus and matches the patterns in the rules.
It uses the existing rule matching algorithm from v1, which is limited, e.g. patterns with anything but or exists = True.

Changes

Add parsing for event pattern to a dictionary, and add validating of pattern.
Add correct formatting of events and add additional keys.
Add API method to test event pattern using same matching algorithm as for distributing events to targets.
Remove skip markers from now passing tests.

Testing

Set the env variable PROVIDER_OVERRIDE_EVENTS=v2 and run tests in events folder. All unsupported tests have the appropriate skip markers.

[github-actions]

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 37m 53s ⏱️ -55s
2 926 tests ±0  2 632 ✅  - 2  294 💤 +2  0 ❌ ±0 
2 928 runs  ±0  2 632 ✅  - 2  296 💤 +2  0 ❌ ±0 

Results for commit aae673a. ± Comparison against base commit 0f7d49f.

This pull request removes 5 and adds 5 tests. Note that renamed tests count towards both.

tests.aws.services.events.test_events.TestEvents ‑ test_event_pattern
tests.aws.services.events.test_events.TestEvents ‑ test_put_event_without_source
tests.aws.services.events.test_events.TestEvents ‑ test_put_events_with_nested_event_pattern
tests.aws.services.events.test_events.TestEvents ‑ test_put_events_with_values_in_array
tests.aws.services.events.test_events.TestEvents ‑ test_put_target_id_validation

tests.aws.services.events.test_event_patterns ‑ test_event_pattern_source
tests.aws.services.events.test_events.TestEventPattern ‑ test_put_events_pattern_nested
tests.aws.services.events.test_events.TestEventPattern ‑ test_put_events_pattern_with_values_in_array
tests.aws.services.events.test_events.TestEventTarget ‑ test_put_target_id_validation
tests.aws.services.events.test_events.TestEvents ‑ test_put_events_without_source

This pull request skips 1 test.

tests.aws.services.events.test_events.TestEvents ‑ test_put_event_without_detail

♻️ This comment has been updated with latest results.

[maxhoheiser]

Rename to a more meaningful function name, also include docstring

[joe4dev]

maybe dispatch_events, deliver_events, ...?

The API is called put_entries but that's from a user's perspective. Within the EventBridge service, using active terminology such as dispatch makes more sense. Just reading over the AWS docs and drafting a docstring might help decide upon a better name.

[maxhoheiser]

renamed in: da0f12a

[joe4dev]

LGTM 👍

Can we avoid duplicating the InternalInvalidEventPatternException before merging?
There are also a couple of enabled tests, which still have the unknown marker (see comments).

[joe4dev]

According to the AWS docs, detail should always be a JSON object. Hence, I guess we can safely assume that other strings or base64-encoded content is not expected here, right?

detail — A JSON object that contains information about the event. For more information about what can be included in this field, see Event message detail field.

[joe4dev]

super nit: event_pattern_str or str typing could clarify the type here (especially given we have both versions in our codebase)

[maxhoheiser]

Now for the v2 provider, since we use only the Java matching engine we only ever have event pattern as a string, we don't parse it into a JSON object. In addition, the API string type is called EventPattern, therefor I would keep it without the str suffix.
Also, I think the naming convention of match_rule is misleading since we don't match the rule but the rule_pattern and only pass in the pattern. We should think of updating this in the event_ruler.

[joe4dev]

Now for the v2 provider, since we use only the Java matching engine we only ever have event pattern as a string, we don't parse it into a JSON object. In addition, the API string type is called EventPattern, therefor I would keep it without the str suffix.

It's a good thing we don't need JSON parsing for the event pattern anymore in the new provider, but right one line below we have the counter-example of transforming an event into an event_str 😬 . Looking over the many implementations in LocalStack, this str vs json duality was very confusing, and being explicit can help to clarify the API usage here.

---

How do you think does rule differ from rule_pattern?

the naming convention of match_rule is misleading since we don't match the rule but the rule_pattern and only pass in the pattern.

Background

The terminology Ruler.matchesRule(event, rule) in the event_ruler.py follows the official AWS event-ruler and aims to be more generic than just for EventBridge. Notice that different services use slightly different terminology:

• Amazon EventBridge event patterns
• Amazon EventBridge Pipes filtering, also called FilterCriteria
• Event Source Mapping Lambda event filtering, also called FilterCriteria
• Amazon SNS message filtering, also called filter policy

[maxhoheiser]

json vs strg makes sense - the old fallacy if you build it you know it and think its obvious to everyone else