docs: Proposal for a TargetNodeDirectory in Tractus-X.

[bmg13]

WHAT

Decision Record for new proposal for FederatedCatalog TargetNodeDirectory with Tractus-X.

Related with #1585

[github-actions]

This pull request is stale because it has been open for 7 days with no activity.

[florianrusch]

Doesn't such a solution already exists with the BPN/EDC-Discovery? How would your service be different?

[paullatzelsperger]

Doesn't such a solution already exists with the BPN/EDC-Discovery? How would your service be different?

The TargetNodeDirectory is an internal service to the FederatedCatalog, that would source its information from the BPN/EDC-Discovery service.

[rafaelmag110]

Suggested change

	This solution allows the member to choose the relevant data that interests them (offers from specific participants) enabling the reduction of storage cost and less network calls. Additionally, each member has control on how to host and manage this new service, service changes do not affect other parties (unless contract changes) and is able to be independently scalable.
	This solution allows the member to choose precisely the Target Catalog Nodes that interests them reducing storage cost and network calls. Additionally, each member has control on how to host and manage this new service, service changes do not affect other parties (unless contract changes) and is able to be independently scalable.

[bmg13]

Yes, nice touch, added here.

[rafaelmag110]

If you feel the need to put something in simpler terms, why do you keep both forms and not only the simpler one?

[bmg13]

Yeah. Removed duplicated content in here.

[rafaelmag110]

As this is a decision record document, I would avoid asking for a suggestion. That's what discussions are for. I believe you - as a contributor - have the knowledge to decide which option would suit the needs of tractusx and clear state why that decision is being taken.

The reviewer task is then to give its opinion on the decisions you made and possibly add a suggestion.

[bmg13]

Removed the suggestion and focused only on proposed solution. Changed in here.

[rafaelmag110]

Overall this decision record seems to be much more complicated than it actually needs to be. As I said in one of the comments, if you feel the need to use simpler terms, then avoid having repeated text. This is a straight forward decision to explain:
-The FC crawls Target Catalog Nodes, which in essence are connector URLs.
-These nodes are resolved by the FC ExectionManager from the TargetNodeDirectory
-The current directory implementation is not ideal because ...
-The decision is to implement it the following way ...

[rafaelmag110]

IMO I'd avoid making the TargetNodeDirectory extension dependent on the tractusx discovery service. It might come to a point where the association decides the DS is not longer needed and we would need to revisit this again.

A TargetCatalogNode is a connector URL, and representing it as a mapping between BPNL - Connector URL makes it unnecessarily complex.

We can allow the user of this new service API to input various TargetCatalogNodes (as in, a list of connector urls), in a single POST but how those URLs are resolved beforehand isn't the TargetNodeDirectory business.

[bmg13]

IMO I'd avoid making the TargetNodeDirectory extension dependent on the tractusx discovery service. It might come to a point where the association decides the DS is not longer needed and we would need to revisit this again.

A TargetCatalogNode is a connector URL, and representing it as a mapping between BPNL - Connector URL makes it unnecessarily complex.

We can allow the user of this new service API to input various TargetCatalogNodes (as in, a list of connector urls), in a single POST but how those URLs are resolved beforehand isn't the TargetNodeDirectory business.

Yeah, I was aiming at abstracting this action from the user and trying to allow the service to do it on its own, however, after reading that comment, I have to agree with you. Removing an external dependency (with some overhead even) from this service and allowing it to work only as a TargetNodeDirectory while giving the user that control is the best approach, specially in the long term. Changed the doc to reflect it.

[ndr-brt]

What are the reasons to deploy this service separatedly? It could be deployed together with the FederatedCatalog.

I also think it could implement a process to keep the directory in sync with the Discovery Service, and to provide an API to manage a "BPN Whitelist", that will permit to "enable" only specific BPNs.

[ndr-brt]

what do we mean here with "service"? a class? a runtime? an endpoint?
could we give it a name as well?

[bmg13]

the idea would be to have a runnable jar (runtime)
following a suggestion, maybe TargetNodeDirectoryService.
Changed in here.

[ndr-brt]

is this operation meant to be done manually?

[bmg13]

Yes, with the goal of the removing the Discovery Service dependency from this TargetNodeDirectoryService the responsibility of obtaining and saving the the Connectors' URL's lies with the user, as sugggested here.

[bmg13]

What are the reasons to deploy this service separatedly? It could be deployed together with the FederatedCatalog.

The idea is to have this TargetNodeDirectoryService as a valid solution to any use, however if some user intends to define other TND they should not be locked at this Service, I believe.

I also think it could implement a process to keep the directory in sync with the Discovery Service, and to provide an API to manage a "BPN Whitelist", that will permit to "enable" only specific BPNs.

Initially, something similar was proposed, the user inputs the BPNs from which they want the catalogs from (working as a whitelist) and the TargetNodeDirectoryService would collect the respective Connectors' URL's from the Discovery Service and store them. However, and rightly pointed out by @rafaelmag110 in here, there is a way to ensure the user chooses the Connectors' URL's they want, whitelisting them, beforehand (manually by using the Discovery Service, per example) and not create that explicit dependency on TargetNodeDirectoryService.

[ndr-brt]

What are the reasons to deploy this service separatedly? It could be deployed together with the FederatedCatalog.

The idea is to have this TargetNodeDirectoryService as a valid solution to any use, however if some user intends to define other TND they should not be locked at this Service, I believe.

I also think it could implement a process to keep the directory in sync with the Discovery Service, and to provide an API to manage a "BPN Whitelist", that will permit to "enable" only specific BPNs.

Initially, something similar was proposed, the user inputs the BPNs from which they want the catalogs from (working as a whitelist) and the TargetNodeDirectoryService would collect the respective Connectors' URL's from the Discovery Service and store them. However, and rightly pointed out by @rafaelmag110 in here, there is a way to ensure the user chooses the Connectors' URL's they want, whitelisting them, beforehand (manually by using the Discovery Service, per example) and not create that explicit dependency on TargetNodeDirectoryService.

To me looks overkill to have another separated service just for the TargetNodeDirectory, I didn't see in this document a valid reason to do so.

I also don't see why it shouldn't interact with the Discovery Service, I mean, everything in software can change, but that's what we do, we adapt to update things to make them work altogether :) (plus I don't think that it would eventually disappear from one day to another).
One downside that I see by having this process as manual is that, for example, if a participant changes the URL, there will be the need of a manual intervention to update it.
BPNs instead are more stable data (it's, in fact, the current Catena-X participant identifier), shorter and easier to read than an URL.

Machines are made for automation, manual work is boring and prone to error, let's just avoid it.

EDIT: I posted this as a comment but I deleted it and re-proposed it as review :)

[bmg13]

To me looks overkill to have another separated service just for the TargetNodeDirectory, I didn't see in this document a valid reason to do so.

Okay, possible unclear phrasing on my side, but what was intended with this proposal would be to have the TargetNodeDirectoryService as an own runtime, however Tractus-EDC Helm charts would be updated to include this service. Were you suggesting this or are you considering the TargetNodeDirectoryService to be embedded in the FederatedCatalog runtime?

[ndr-brt]

To me looks overkill to have another separated service just for the TargetNodeDirectory, I didn't see in this document a valid reason to do so.

Okay, possible unclear phrasing on my side, but what was intended with this proposal would be to have the TargetNodeDirectoryService as an own runtime, however Tractus-EDC Helm charts would be updated to include this service. Were you suggesting this or are you considering the TargetNodeDirectoryService to be embedded in the FederatedCatalog runtime?

I was suggesting to keep it embedded in the federated catalog, because I don't see why it should deployed in a separated runtime.
I mean, every service could be deployed in a separated runtime, but unless there's a good reason to do so, I would avoid that.

[bmg13]

I was suggesting to keep it embedded in the federated catalog, because I don't see why it should deployed in a separated runtime.

Okay, this make sense, will rewrite to include TND embedded in the federated catalog.

I also don't see why it shouldn't interact with the Discovery Service, I mean, everything in software can change, but that's what we do, we adapt to update things to make them work altogether :) (plus I don't think that it would eventually disappear from one day to another).
One downside that I see by having this process as manual is that, for example, if a participant changes the URL, there will be the need of a manual intervention to update it.
BPNs instead are more stable data (it's, in fact, the current Catena-X participant identifier), shorter and easier to read than an URL.

I see. So we will suggest having a DiscoveryServiceRetriever (DSR) extension included in the Federated Catalog. This will allow to:

• Have and API for a member to add/delete the BPNL's (or connector URL's) of participants from which catalogs are wanted;
• Create a bearer token (after requesting from identity provider) to retrieve the connector url's associated with stored BPNL's;
• Store the data retrieved.

If by any chance a member does not aim at use the Discovery Service, a new extension can be create and replace the DSR.

This will allow to use the SQL (or InMem) store already present in the Federated Catalog.
Changes added here.

[ndr-brt]

at this point my suggestion would be to rephrase this and #1555 in a single DR, because they target the same scope, it will be easier to follow for the community.

[bmg13]

at this point my suggestion would be to rephrase this and #1555 in a single DR, because they target the same scope, it will be easier to follow for the community.

I followed the idea of it being two separated decisions (despite same scope), but will try to merge them in the meantime. Just highlighting that the DR will contain two decisions, one for FC distribution and other for the TND.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[bmg13]

Considering the suggestion, this PR is closed and existing proposal added to this PR.