MG-2142 - Consume Things connect/disconnect event in bootstrap #2192

JeffMboya · 2024-04-19T12:46:39Z

What type of PR is this?

This is a feature because it adds the connectEvent struct to the consumer package.

What does this do?

This PR adds a new struct, connectEvent, to the consumer package. This struct contains the thingID and channelID, and is triggered whenever a Thing connects to the system.

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?

Yes, I have included tests for my changes.

Did you document any new/modified feature?

No

bootstrap/postgres/configs.go

arvindh123

I have question here, The event published by connectThingEvent and disconnectThingEvent are same event published by thing service on disconnect and connect ?

JeffMboya · 2024-04-26T10:08:34Z

I have question here, The event published by connectThingEvent and disconnectThingEvent are same event published by thing service on disconnect and connect ?

They are not the same. The event published by things service is a notification that a thing has been connected or disconnected from a channel. The event published by connectThingEvent and disconnectThingEvent in bootstrap are an acknowledgement that a thing has been connected or disconnected from a channel

bootstrap/postgres/configs.go

arvindh123

@JeffMboya I have suggestion for handling error rollbacks

arvindh123 · 2024-04-26T11:27:49Z

bootstrap/events/consumer/streams.go

+ thingConnect = "policy.create"
 thingDisconnect = "policy.delete"


this should be group.assign and group.unassign
But I guess we could not use this operation, because adding channel to group also use the same operation

arvindh123

While trying to assign parent group for channel , like below request

curl --location 'http://localhost/channels/795b0be8-f8d9-4101-a40f-859ed573c71c/groups/assign' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <Token>' \
--data '{
    "group_ids" : [ "5c6de392-6dc4-46fc-8e2c-e6f4ce4fffab" ]
}'

In events received below logs, which use group.assign operation , the same operation is used for thing connect, so could not different between thing connection event and group assign channel events

[#8] Received on "events.magistrala.things" with reply "_INBOX.7g7jymtBXH0wOSok79kUGo.0LlN1Z1P"
*�{"group_id":"795b0be8-f8d9-4101-a40f-859ed573c71c","member_ids":["5c6de392-6dc4-46fc-8e2c-e6f4ce4fffab"],"occurred_at":1714131210669902557,"operation":"group.assign"}

Additionally,

In the event message group_id filed contains channel id
actually here we should have group id

In member_ids field , it contains
group_id , actually here we should have channel id

This issue might be a blocker for this PR

bootstrap/service.go

rodneyosodo · 2024-05-17T11:44:21Z

bootstrap/postgres/configs_test.go

+ repo := postgres.NewConfigRepository(db, testLog)
+ err := deleteChannels(context.Background(), repo)
+ require.Nil(t, err, "Channels cleanup expected to succeed.")
+


Test this occurrences using table testing:

Connect for an already connected thing

Connect for a disconnected thing

Connect for an invalid thing

Connect for a random thing

Connect for an empty thing

bootstrap/events/producer/events.go

bootstrap/events/consumer/streams.go

bootstrap/configs.go

Signed-off-by: JeffMboya <[email protected]>

Signed-off-by: JeffMboya <[email protected]> Add ConnectThing method to bootstrap Signed-off-by: JeffMboya <[email protected]> Consume ThingConnect in bootstrap Signed-off-by: JeffMboya <[email protected]> Consume ThingConnect in bootstrap Signed-off-by: JeffMboya <[email protected]> Consume ThingConnect in bootstrap Signed-off-by: JeffMboya <[email protected]> Consume ThingConnect event Signed-off-by: JeffMboya <[email protected]> Consume ThingsConnect event Signed-off-by: JeffMboya <[email protected]> Consume ThingsConnect event Signed-off-by: JeffMboya <[email protected]> Consume ThingsConnect event Signed-off-by: JeffMboya <[email protected]> Implement ReadStringSlice Signed-off-by: JeffMboya <[email protected]> Add memberKind and relation checks Signed-off-by: JeffMboya <[email protected]> Add memberKind and relation checks Signed-off-by: JeffMboya <[email protected]> Add memberKind and relation checks Signed-off-by: JeffMboya <[email protected]> Add memberKind and relation checks Signed-off-by: JeffMboya <[email protected]> Add memberKind and relation checks Signed-off-by: JeffMboya <[email protected]> Add TestDisconnectThing to configs_test Signed-off-by: JeffMboya <[email protected]> Update mocks Signed-off-by: JeffMboya <[email protected]>

Signed-off-by: JeffMboya <[email protected]> Refactor: revert to thingID and channelID Signed-off-by: JeffMboya <[email protected]>

Signed-off-by: JeffMboya <[email protected]> Refactor: add memberKind ans relation constants Signed-off-by: JeffMboya <[email protected]> Refactor: add memberKind ans relation constants Signed-off-by: JeffMboya <[email protected]> Refactor: add memberKind ans relation constants Signed-off-by: JeffMboya <[email protected]> Refactor: add memberKind ans relation constants Signed-off-by: JeffMboya <[email protected]>

Signed-off-by: JeffMboya <[email protected]>

bootstrap/configs.go

bootstrap/events/consumer/streams.go

Signed-off-by: JeffMboya <[email protected]>

bootstrap/postgres/configs.go

bootstrap/events/consumer/streams.go

bootstrap/postgres/configs.go

Signed-off-by: JeffMboya <[email protected]>

bootstrap/postgres/configs.go

Signed-off-by: JeffMboya <[email protected]>

rodneyosodo · 2024-05-28T13:21:14Z

bootstrap/events/consumer/streams.go

+ cte := decodeConnectThing(msg)
+ if cte.channelID == "" {
+ return svcerr.ErrMalformedEntity
+ }


Also, check length of cte.thingID is not zero

rodneyosodo · 2024-05-28T13:21:51Z

bootstrap/events/consumer/streams.go

+
+ return connectionEvent{
+ channelID: read(event, "group_id", ""),
+ thingID: ReadStringSlice(event, "member_ids"),


Rename to thingIDs

rodneyosodo · 2024-05-28T13:22:44Z

bootstrap/postgres/configs.go

 if pgErr, ok := err.(*pgconn.PgError); ok && pgErr.Code == pgerrcode.UniqueViolation {
- e = repoerr.ErrConflict
+ return "", repoerr.ErrConflict


Use switch statement

rodneyosodo · 2024-05-28T13:23:18Z

bootstrap/postgres/configs.go

+func (cr configRepository) ConnectThing(ctx context.Context, channelID, thingID string) error {
+ q := `UPDATE configs SET state = $1 WHERE EXISTS (
+ SELECT 1 FROM connections WHERE config_id = $2 AND channel_id = $3)
+ RETURNING *`


No need of returning anything since we don't use it

rodneyosodo · 2024-05-28T13:23:24Z

bootstrap/postgres/configs.go

- SELECT 1 FROM connections WHERE config_id = $2 AND channel_id = $3)`
- if _, err := cr.db.ExecContext(ctx, q, bootstrap.Inactive, thingID, channelID); err != nil {
+ SELECT 1 FROM connections WHERE config_id = $2 AND channel_id = $3)
+ RETURNING *`


rodneyosodo · 2024-05-28T13:25:19Z

bootstrap/postgres/configs.go

@@ -451,12 +454,33 @@ func (cr configRepository) RemoveChannel(ctx context.Context, id string) error {
 return nil
 }

+func (cr configRepository) ConnectThing(ctx context.Context, channelID, thingID string) error {
+ q := `UPDATE configs SET state = $1 WHERE EXISTS (


No need for subquery have a look at https://github.com/absmach/magistrala/blob/31d0e8c4529339a6626c9ff03a65972bf407dd89/pkg/clients/postgres/clients.go#L27C1-L54C2

JeffMboya force-pushed the MG-2142 branch 3 times, most recently from 90b9f69 to 3198bd8 Compare April 23, 2024 11:56

JeffMboya self-assigned this Apr 23, 2024

JeffMboya force-pushed the MG-2142 branch from 16c748a to 8c72bd0 Compare April 24, 2024 12:29

arvindh123 reviewed Apr 25, 2024

View reviewed changes

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

arvindh123 reviewed Apr 25, 2024

View reviewed changes

JeffMboya marked this pull request as ready for review April 26, 2024 10:10

arvindh123 reviewed Apr 26, 2024

View reviewed changes

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

arvindh123 reviewed Apr 26, 2024

View reviewed changes

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

arvindh123 requested changes Apr 26, 2024

View reviewed changes

arvindh123 reviewed Apr 26, 2024

View reviewed changes

arvindh123 requested changes Apr 26, 2024

View reviewed changes

JeffMboya force-pushed the MG-2142 branch 2 times, most recently from 0b87686 to 9380eef Compare May 6, 2024 12:36

JeffMboya force-pushed the MG-2142 branch 3 times, most recently from 15ddd1f to 068d129 Compare May 8, 2024 11:25

JeffMboya changed the title ~~MG-2142 - Consume Things connect event in bootstrap~~ MG-2142 - Consume Things connect/disconnect event in bootstrap May 12, 2024

JeffMboya force-pushed the MG-2142 branch 6 times, most recently from b78c9aa to f6aa2fe Compare May 13, 2024 09:57

dborovcanin force-pushed the MG-2142 branch from f6aa2fe to 87860ef Compare May 15, 2024 10:57

rodneyosodo requested changes May 17, 2024

View reviewed changes

JeffMboya force-pushed the MG-2142 branch 2 times, most recently from 386c14c to 73f7017 Compare May 20, 2024 07:32

JeffMboya force-pushed the MG-2142 branch 2 times, most recently from 8369752 to 2e5c476 Compare May 22, 2024 05:20

rodneyosodo requested a review from nyagamunene May 22, 2024 10:23

JeffMboya added 7 commits May 22, 2024 21:18

Refactor connectionEvent in events.go

65ead8a

Signed-off-by: JeffMboya <[email protected]>

Refactor: revert to thingID and channelID

a3bf741

Signed-off-by: JeffMboya <[email protected]> Refactor: revert to thingID and channelID Signed-off-by: JeffMboya <[email protected]>

Refactor: removed saved method

2c33328

Signed-off-by: JeffMboya <[email protected]>

Refactor: Add tests

5201153

Signed-off-by: JeffMboya <[email protected]>

Refactor: Add tests

0d39d7c

Signed-off-by: JeffMboya <[email protected]>

JeffMboya force-pushed the MG-2142 branch from 2e5c476 to 0d39d7c Compare May 22, 2024 18:18

rodneyosodo requested changes May 23, 2024

View reviewed changes

bootstrap/configs.go Show resolved Hide resolved

bootstrap/events/consumer/streams.go Outdated Show resolved Hide resolved

bootstrap/events/consumer/streams.go Outdated Show resolved Hide resolved

JeffMboya added 5 commits May 23, 2024 12:22

refactor: ConnectThing and DisconnectThing methods

7502ccb

Signed-off-by: JeffMboya <[email protected]>

fix: fix ConnectThing and DisconnectThing comments

ca13f1d

Signed-off-by: JeffMboya <[email protected]>

refactor: ConnectThing and DisconnectThing tests

2c79cc4

Signed-off-by: JeffMboya <[email protected]>

refactor: ConnectThing and DisconnectThing tests

ce2a54c

Signed-off-by: JeffMboya <[email protected]>

refactor: ConnectThing and DisconnectThing tests

0178aac

Signed-off-by: JeffMboya <[email protected]>

rodneyosodo requested changes May 27, 2024

View reviewed changes

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

bootstrap/events/consumer/streams.go Show resolved Hide resolved

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

arvindh123 requested changes May 27, 2024

View reviewed changes

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

arvindh123 requested changes May 27, 2024

View reviewed changes

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

JeffMboya added 2 commits May 27, 2024 14:30

refactor: ConnectThing and DisconnectThing tests

cf22f96

Signed-off-by: JeffMboya <[email protected]>

fix: Add logging for error rollback in Save method

0412dc2

Signed-off-by: JeffMboya <[email protected]>

JeffMboya requested review from rodneyosodo and arvindh123 May 27, 2024 12:29

arvindh123 requested changes May 28, 2024

View reviewed changes

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

arvindh123 requested changes May 28, 2024

View reviewed changes

bootstrap/postgres/configs.go Outdated Show resolved Hide resolved

refactor: defer rollback

a377d0d

Signed-off-by: JeffMboya <[email protected]>

arvindh123 approved these changes May 28, 2024

View reviewed changes

rodneyosodo requested changes May 28, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MG-2142 - Consume Things connect/disconnect event in bootstrap #2192

MG-2142 - Consume Things connect/disconnect event in bootstrap #2192

JeffMboya commented Apr 19, 2024 •

edited

arvindh123 left a comment

JeffMboya commented Apr 26, 2024

arvindh123 left a comment •

edited

arvindh123 Apr 26, 2024 •

edited

arvindh123 left a comment •

edited

rodneyosodo May 17, 2024

rodneyosodo May 28, 2024

rodneyosodo May 28, 2024

rodneyosodo May 28, 2024

rodneyosodo May 28, 2024

rodneyosodo May 28, 2024

rodneyosodo May 28, 2024

		thingConnect = "policy.create"
		thingDisconnect = "policy.delete"

MG-2142 - Consume Things connect/disconnect event in bootstrap #2192

Are you sure you want to change the base?

MG-2142 - Consume Things connect/disconnect event in bootstrap #2192

Conversation

JeffMboya commented Apr 19, 2024 • edited

What type of PR is this?

What does this do?

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?

Did you document any new/modified feature?

arvindh123 left a comment

Choose a reason for hiding this comment

JeffMboya commented Apr 26, 2024

arvindh123 left a comment • edited

Choose a reason for hiding this comment

arvindh123 Apr 26, 2024 • edited

Choose a reason for hiding this comment

arvindh123 left a comment • edited

Choose a reason for hiding this comment

rodneyosodo May 17, 2024

Choose a reason for hiding this comment

rodneyosodo May 28, 2024

Choose a reason for hiding this comment

rodneyosodo May 28, 2024

Choose a reason for hiding this comment

rodneyosodo May 28, 2024

Choose a reason for hiding this comment

rodneyosodo May 28, 2024

Choose a reason for hiding this comment

rodneyosodo May 28, 2024

Choose a reason for hiding this comment

rodneyosodo May 28, 2024

Choose a reason for hiding this comment

JeffMboya commented Apr 19, 2024 •

edited

arvindh123 left a comment •

edited

arvindh123 Apr 26, 2024 •

edited

arvindh123 left a comment •

edited