CMR-10310: Allow the subscription endpoint to take an https URL in the EndPoint element

[jmaeng72]

Overview

What is the feature/fix?

Ingest subscription could only take in SQS ARN as endpoint to send sub messages to

What is the Solution?

Add the ability to send sub message to any HTTP/S endpoint

What areas of the application does this impact?

Ingest

Checklist

• I have updated/added unit and int tests that prove my fix is effective or that my feature works
• New and existing unit and int tests pass locally and remotely
• clj-kondo has been run locally and all errors corrected
• I have removed unnecessary/dead code and imports in files I have changed
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have cleaned up integration tests by doing one or more of the following:
  • migrated any are2 tests to are3 in files I have changed
  • de-duped, consolidated, removed dead int tests
  • transformed applicable int tests into unit tests
  • refactored to reduce number of system state resets by updating fixtures (use-fixtures :each (ingest/reset-fixture {})) to be :once instead of :each

[TylerHeald1]

add newline at end of file

[TylerHeald1]

test looks the same as the one above?

[eereiter]

Metadata DB shouldn't send messages out.

All endpoints should subscribe to the external SNS topic. The lambda should be a CMR lambda that listens to the SNS topic that was subscribed. When an URL endpoint arrives it should send messages to the endpoint.

All metadata db does is check to see if a granule is part of a collection where a subscription exists and then creates enough information to be placed onto an internal subscription queue. This is where the subscription workers will pick it up where they can do ACL checks based on the data that is in the message. If the message can be sent on, then the workers will create the proper notification message and place it on the external SNS Topic, that is where the CMR Lambda will pick up the message and forward it to the URL in the message.

[eereiter]

Metadata DB should not be sending out notifications to subscribers, It needs to put enough information into the notification message so that the subscription workers can do more processing and then it needs to place the messages onto the internal subscription notification topic. The sending of the notification to the end point URL needs to happen in the lambda that is listening to the external CMR topic.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 52.38095% with 20 lines in your changes missing coverage. Please review.

Project coverage is 58.25%. Comparing base (1063bb2) to head (261d144).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...app/src/cmr/metadata_db/services/subscriptions.clj	44.11%	17 Missing and 2 partials ⚠️
ingest-app/src/cmr/ingest/api/subscriptions.clj	87.50%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2209      +/-   ##
==========================================
- Coverage   58.26%   58.25%   -0.02%     
==========================================
  Files        1056     1056              
  Lines       71097    71134      +37     
  Branches     2035     2034       -1     
==========================================
+ Hits        41426    41436      +10     
- Misses      27778    27798      +20     
- Partials     1893     1900       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.