feat: delete UPF and gNB is propagated to NS #279

patriciareinoso · 2025-01-09T12:59:54Z

Description

This PR improves the DELETE operation over UPFs and gNBs: when a UPF or gNB is deleted, it is also removed from the Network Slices containing them.

To delete the elements from the Network slice we use the patch operation.

UPF: remove the field
gNBs: remove an element from the list of gNBs

DELETE operations on inventory no longer use the configChannel -> reduce complexity.

This PR uses transactions to ensure atomicity in operations. If the second operation fails, the transaction is rolled back. The mongoDB config needs to support transactions. This PR depends on omec-project/util#96

Enforce Transactions in Webconsole

During webconsole initialization we will check that transactions are supported (replica set or sharded config enabled) If not, initialization will fail and the program will exit. One request per minute is made to mongoDB and there is a time out of 3 minutes (total 3 requests). MongoDB refuses connections if too many requests are done, waiting 1 minute before checking again is useful if the container restarts the service automatically after failure.

2025-01-20T10:16:43.655Z [pebble] Service "nms" starting: webconsole [ -cfg /config/webuicfg.yaml ]
2025-01-20T10:16:43.674Z [nms] 2025-01-20T10:16:43.673Z	INFO	webconsole-src/server.go:22	webui	{"component": "WebUI", "category": "App"}
2025-01-20T10:16:43.674Z [nms] 2025-01-20T10:16:43.674Z	INFO	factory/factory.go:68	configPod running in 5G deployment	{"component": "WebUI", "category": "CONFIG"}
2025-01-20T10:16:43.674Z [nms] 2025-01-20T10:16:43.674Z	INFO	webui_service/webui_init.go:95	WebUI Log level is set to [info] level	{"component": "WebUI", "category": "Init"}
2025-01-20T10:16:43.674Z [nms] 2025-01-20T10:16:43.674Z	INFO	logger/logger.go:72	set log level: info	{"component": "WebUI", "category": "Init"}
2025-01-20T10:16:43.674Z [nms] 2025-01-20T10:16:43.674Z	INFO	logger/logger.go:65	set log level: info	{"component": "LIB", "category": "Util"}
2025-01-20T10:16:43.674Z [nms] 2025-01-20T10:16:43.674Z	INFO	dbadapter/db_adapter.go:79	connected to MongoDB	{"component": "WebUI", "category": "DB"}
2025-01-20T10:16:43.674Z [nms] 2025-01-20T10:16:43.674Z	INFO	dbadapter/db_adapter.go:89	checking for replica set or sharded config in MongoDB...	{"component": "WebUI", "category": "DB"}
2025-01-20T10:19:43.676Z [nms] 2025-01-20T10:19:43.676Z	ERROR	webui_service/webui_init.go:158	failed to connect to MongoDB client	{"component": "WebUI", "category": "DB", "free5gc": "error", "error": "timed out while waiting for Replica Set or sharded config to be set in MongoDB"}
2025-01-20T10:19:43.682Z [pebble] Service "nms" stopped unexpectedly with code 0
2025-01-20T10:19:43.682Z [pebble] Service "nms" on-success action is "restart", waiting ~500ms before restart (backoff 1)
2025-01-20T10:19:44.216Z [pebble] Service "nms" starting: webconsole [ -cfg /config/webuicfg.yaml ]
2025-01-20T10:19:44.257Z [nms] 2025-01-20T10:19:44.256Z	INFO	webconsole-src/server.go:22	webui	{"component": "WebUI", "category": "App"}
2025-01-20T10:19:44.257Z [nms] 2025-01-20T10:19:44.257Z	INFO	factory/factory.go:68	configPod running in 5G deployment	{"component": "WebUI", "category": "CONFIG"}
2025-01-20T10:19:44.257Z [nms] 2025-01-20T10:19:44.257Z	INFO	webui_service/webui_init.go:95	WebUI Log level is set to [info] level	{"component": "WebUI", "category": "Init"}
2025-01-20T10:19:44.257Z [nms] 2025-01-20T10:19:44.257Z	INFO	logger/logger.go:72	set log level: info	{"component": "WebUI", "category": "Init"}
2025-01-20T10:19:44.257Z [nms] 2025-01-20T10:19:44.257Z	INFO	logger/logger.go:65	set log level: info	{"component": "LIB", "category": "Util"}
2025-01-20T10:19:44.258Z [nms] 2025-01-20T10:19:44.258Z	INFO	dbadapter/db_adapter.go:79	connected to MongoDB	{"component": "WebUI", "category": "DB"}
2025-01-20T10:19:44.258Z [nms] 2025-01-20T10:19:44.258Z	INFO	dbadapter/db_adapter.go:89	checking for replica set or sharded config in MongoDB...	{"component": "WebUI", "category": "DB"}
2025-01-20T10:22:44.263Z [nms] 2025-01-20T10:22:44.262Z	ERROR	webui_service/webui_init.go:158	failed to connect to MongoDB client	{"component": "WebUI", "category": "DB", "free5gc": "error", "error": "timed out while waiting for Replica Set or sharded config to be set in MongoDB"}
2025-01-20T10:22:44.268Z [pebble] Service "nms" stopped unexpectedly with code 0
2025-01-20T10:22:44.268Z [pebble] Service "nms" on-success action is "restart", waiting ~500ms before restart (backoff 1)
2025-01-20T10:22:44.812Z [pebble] Service "nms" starting: webconsole [ -cfg /config/webuicfg.yaml ]
2025-01-20T10:22:44.853Z [nms] 2025-01-20T10:22:44.853Z	INFO	webconsole-src/server.go:22	webui	{"component": "WebUI", "category": "App"}
2025-01-20T10:22:44.855Z [nms] 2025-01-20T10:22:44.854Z	INFO	factory/factory.go:68	configPod running in 5G deployment	{"component": "WebUI", "category": "CONFIG"}
2025-01-20T10:22:44.855Z [nms] 2025-01-20T10:22:44.854Z	INFO	webui_service/webui_init.go:95	WebUI Log level is set to [info] level	{"component": "WebUI", "category": "Init"}
2025-01-20T10:22:44.855Z [nms] 2025-01-20T10:22:44.854Z	INFO	logger/logger.go:72	set log level: info	{"component": "WebUI", "category": "Init"}
2025-01-20T10:22:44.855Z [nms] 2025-01-20T10:22:44.854Z	INFO	logger/logger.go:65	set log level: info	{"component": "LIB", "category": "Util"}
2025-01-20T10:22:44.855Z [nms] 2025-01-20T10:22:44.855Z	INFO	dbadapter/db_adapter.go:79	connected to MongoDB	{"component": "WebUI", "category": "DB"}
2025-01-20T10:22:44.855Z [nms] 2025-01-20T10:22:44.855Z	INFO	dbadapter/db_adapter.go:89	checking for replica set or sharded config in MongoDB...	{"component": "WebUI", "category": "DB"}

Network Slice before and after deletion

Before

{
    _id: ObjectId('677fc3b095a76cc8d03397df'),
    'slice-id': { sd: '010203', sst: '1' },
    'site-device-group': [ 'cows' ],
    'site-info': {
      gNodeBs: [ { name: 'gnb1', tac: 1 } ],
      plmn: { mcc: '208', mnc: '93' },
      'site-name': 'demo',
      upf: { 'upf-name': 'upf1', 'upf-port': '8805' }
    },
    'slice-name': 'slice1'
  },

After

{
    _id: ObjectId('677fc3b095a76cc8d03397df'),
    'slice-id': { sd: '010203', sst: '1' },
    'site-device-group': [ 'cows' ],
    'site-info': {
      gNodeBs: [ ],
      plmn: { mcc: '208', mnc: '93' },
      'site-name': 'demo'
    },
    'slice-name': 'slice1'
  },

Signed-off-by: Patricia Reinoso <[email protected]>

ghislainbourgeois

Overall looks good to me, I have some questions regarding the CORS header, and I would suggest taking the opportunity to clean up the different err variables to be more idiomatic.

configapi/api_inventory.go

Signed-off-by: Patricia Reinoso <[email protected]>

patriciareinoso · 2025-01-13T17:20:57Z

After exploring transactions in mongoDB, turns out that they can only be used on MongoDB Replica Set or sharded deployments. Both the SD-Core Helm Charts and Charmed Aether SD-Core use Replica Set. But if there is a StandAlone deployment, the Delete operation will fail (it will not be possible to delete a gNB or UPF at all). Right now I handled both cases : Replica Set/sharded and Stand Alone (sequential execution). It needs this PR omec-project/util#96 to work.

Is the use of a MongoDB in replica set /sharded configuration something that could impose in webconsole? it would be a check we do at initialization and it would simplify the delete and update operations as we would assume that transactions are supported.
@gatici @ghislainbourgeois @gab-arrobo @gruyaume

Signed-off-by: Patricia Reinoso <[email protected]>

ghislainbourgeois · 2025-01-14T14:10:23Z

After exploring transactions in mongoDB, turns out that they can only be used on MongoDB Replica Set or sharded deployments. Both the SD-Core Helm Charts and Charmed Aether SD-Core use Replica Set. But if there is a StandAlone deployment, the Delete operation will fail (it will not be possible to delete a gNB or UPF at all). Right now I handled both cases : Replica Set/sharded and Stand Alone (sequential execution). It needs this PR omec-project/util#96 to work.

Is the use of a MongoDB in replica set /sharded configuration something that could impose in webconsole? it would be a check we do at initialization and it would simplify the delete and update operations as we would assume that transactions are supported. @gatici @ghislainbourgeois @gab-arrobo @gruyaume

If it is already what is used in the Helm Charts, I think we can enforce it, as it makes the code more robust. I can ask the question on the TST if there are other types of deployments out there that would not be using a Replica Set.

I looked a bit into it and Mongo does not suggest Standalone as a good production deployment, and it looks relatively simple to convert a standalone deployment to a replicat set.

Signed-off-by: Patricia Reinoso <[email protected]>

configapi/api_inventory.go

Signed-off-by: Patricia Reinoso <[email protected]>

ghislainbourgeois · 2025-01-20T15:20:34Z

LGTM, however I would like the input of @gab-arrobo or @thakurajayL before merging.

gatici

LGTM

gab-arrobo

I am going to try to take a look at this PR tomorrow morning

go.mod

Signed-off-by: Patricia Reinoso <[email protected]>

gab-arrobo

Overall, it looks good to me.

docs/docs.go.license

README.md

gab-arrobo · 2025-01-22T17:45:20Z

dbadapter/db_adapter.go

@@ -22,14 +25,18 @@ type DBInterface interface {
 	RestfulAPIPutOneNotUpdate(collName string, filter bson.M, putData map[string]interface{}) (bool, error)
 	RestfulAPIPutMany(collName string, filterArray []primitive.M, putDataArray []map[string]interface{}) error
 	RestfulAPIDeleteOne(collName string, filter bson.M) error
+	RestfulAPIDeleteOneWithContext(collName string, filter bson.M, context context.Context) error


What about setting the context as first parameter to follow GO's common practices?

Suggested change

RestfulAPIDeleteOneWithContext(collName string, filter bson.M, context context.Context) error

RestfulAPIDeleteOneWithContext(context context.Context, collName string, filter bson.M) error

oh, I did not know about this convention. if I change this here, i would need to make an update also in utils/mongoapi otherwise I get an error about not implementing the correct interface. Can we move forward with this PR? and I will make the changes in utils/mongoapi to align all the methods and create another PR (way smaller) updating the interface here ?

Draft PR for modify order of parameters omec-project/util#109

I just approved and merged omec-project/util#109. It is up to you to make the changes in this PR or creating another small PR for those changes (However, keep in mind that dependabot will automatically open a PR for the dependency update on Sunday night. So, it will be good to have the PR to use the new changes in omec-project/util#109 before Sunday)

dbadapter/db_adapter.go

Signed-off-by: Patricia Reinoso <[email protected]>

gab-arrobo

Looks good to me.
@patriciareinoso, it is up to you on how to proceed about the reordering of parameters (context) in the different functions

Signed-off-by: Patricia Reinoso <[email protected]>

patriciareinoso added 2 commits January 9, 2025 13:50

feat: delete UPF and gNB is propagated to NS

3a7dd9b

Signed-off-by: Patricia Reinoso <[email protected]>

update swagger doc

491c95c

Signed-off-by: Patricia Reinoso <[email protected]>

patriciareinoso mentioned this pull request Jan 9, 2025

feat: implement PUT for inventory UPF #278

Merged

ghislainbourgeois requested changes Jan 9, 2025

View reviewed changes

ghislainbourgeois requested a review from gab-arrobo January 9, 2025 15:35

gatici suggested changes Jan 10, 2025

View reviewed changes

patriciareinoso added 3 commits January 10, 2025 13:33

comments from review: improve err variables and logging

397c185

Signed-off-by: Patricia Reinoso <[email protected]>

use transactions

5573990

Signed-off-by: Patricia Reinoso <[email protected]>

restore CORS header

02e165e

Signed-off-by: Patricia Reinoso <[email protected]>

patriciareinoso requested review from ghislainbourgeois and gatici January 13, 2025 18:59

rename function to SupportsTransactions

c470089

Signed-off-by: Patricia Reinoso <[email protected]>

patriciareinoso force-pushed the TELCO-1550-delete-upf-gnb branch from e924ab3 to c470089 Compare January 14, 2025 10:05

patriciareinoso added 2 commits January 14, 2025 11:08

clean up

11e3026

Signed-off-by: Patricia Reinoso <[email protected]>

Merge branch 'main' into TELCO-1550-delete-upf-gnb

4f74177

This was referenced Jan 14, 2025

feat: support empty UPF field in NS canonical/sdcore-nms#705

Merged

feat: add MongoDB functions to handle transactions omec-project/util#96

Merged

patriciareinoso mentioned this pull request Jan 15, 2025

feat: locally deploy MongoDB as replica set canonical/sdcore-nms#706

Merged

6 tasks

patriciareinoso added 3 commits January 16, 2025 12:36

enforce transactions

4bc7d50

Signed-off-by: Patricia Reinoso <[email protected]>

delete code does not fail silently anymor

356ae2f

Signed-off-by: Patricia Reinoso <[email protected]>

update packages

f7c0f71

Signed-off-by: Patricia Reinoso <[email protected]>

gatici reviewed Jan 17, 2025

View reviewed changes

patriciareinoso added 4 commits January 17, 2025 16:29

enforce rs in mongodb

c965765

Signed-off-by: Patricia Reinoso <[email protected]>

improve logs

ef4380a

Signed-off-by: Patricia Reinoso <[email protected]>

handle error when abort transaction fails

0f223a0

Signed-off-by: Patricia Reinoso <[email protected]>

add doc

2b34d07

Signed-off-by: Patricia Reinoso <[email protected]>

patriciareinoso requested a review from gatici January 20, 2025 11:58

ghislainbourgeois previously approved these changes Jan 20, 2025

View reviewed changes

patriciareinoso mentioned this pull request Jan 20, 2025

feat: Delete DG operation, remove the DG from the NS list #291

Closed

gatici approved these changes Jan 21, 2025

View reviewed changes

gab-arrobo reviewed Jan 22, 2025

View reviewed changes

go.mod Outdated Show resolved Hide resolved

update mongoapi

62ad25a

Signed-off-by: Patricia Reinoso <[email protected]>

patriciareinoso dismissed ghislainbourgeois’s stale review via 62ad25a January 22, 2025 08:53

Merge branch 'main' into TELCO-1550-delete-upf-gnb

85853e8

gab-arrobo reviewed Jan 22, 2025

View reviewed changes

take care of comments

bca5c94

Signed-off-by: Patricia Reinoso <[email protected]>

gab-arrobo previously approved these changes Jan 23, 2025

View reviewed changes

update mongoapi

e5f6880

Signed-off-by: Patricia Reinoso <[email protected]>

patriciareinoso dismissed gab-arrobo’s stale review via e5f6880 January 24, 2025 08:58

patriciareinoso mentioned this pull request Jan 24, 2025

feat: update mongo db interface #294

Closed

ghislainbourgeois approved these changes Jan 24, 2025

View reviewed changes

ghislainbourgeois merged commit a1b39f2 into omec-project:main Jan 24, 2025
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: delete UPF and gNB is propagated to NS #279

feat: delete UPF and gNB is propagated to NS #279

patriciareinoso commented Jan 9, 2025 •

edited

Loading

ghislainbourgeois left a comment

patriciareinoso commented Jan 13, 2025 •

edited

Loading

ghislainbourgeois commented Jan 14, 2025

ghislainbourgeois commented Jan 20, 2025

gatici left a comment

gab-arrobo left a comment

gab-arrobo left a comment

gab-arrobo Jan 22, 2025

patriciareinoso Jan 23, 2025

patriciareinoso Jan 23, 2025

gab-arrobo Jan 23, 2025

gab-arrobo left a comment

	RestfulAPIDeleteOneWithContext(collName string, filter bson.M, context context.Context) error
	RestfulAPIDeleteOneWithContext(context context.Context, collName string, filter bson.M) error

feat: delete UPF and gNB is propagated to NS #279

feat: delete UPF and gNB is propagated to NS #279

Conversation

patriciareinoso commented Jan 9, 2025 • edited Loading

Description

Enforce Transactions in Webconsole

Network Slice before and after deletion

Before

After

ghislainbourgeois left a comment

Choose a reason for hiding this comment

patriciareinoso commented Jan 13, 2025 • edited Loading

ghislainbourgeois commented Jan 14, 2025

ghislainbourgeois commented Jan 20, 2025

gatici left a comment

Choose a reason for hiding this comment

gab-arrobo left a comment

Choose a reason for hiding this comment

gab-arrobo left a comment

Choose a reason for hiding this comment

gab-arrobo Jan 22, 2025

Choose a reason for hiding this comment

patriciareinoso Jan 23, 2025

Choose a reason for hiding this comment

patriciareinoso Jan 23, 2025

Choose a reason for hiding this comment

gab-arrobo Jan 23, 2025

Choose a reason for hiding this comment

gab-arrobo left a comment

Choose a reason for hiding this comment

patriciareinoso commented Jan 9, 2025 •

edited

Loading

patriciareinoso commented Jan 13, 2025 •

edited

Loading