Ecarton/cumulus 3751 s3 task #3910
Conversation
* Update migration to remove vacuum statements that can time out in a Lambda env * Remove additional vacuum statements * Add CL
…umulus into CUMULUS-3757-move-granule
…umulus into CUMULUS-3757-move-granule
…umulus into CUMULUS-3757-move-granule
…s to launch templates (#3880) * CUMULUS-3759:Migrated ECS Autoscaling group from launch configurations to launch templates * update tf resources * add ec2 container service policy * update userdata format * add cumulus-std deployment config * update autoscaling_cf_template * delete userdata file * rename * instance refresh lifecycle * no need AmazonEC2ContainerServiceforEC2Role? * migrate fakeprovider to launch templates * refactor * remove variable from cumulus-std
…1-using-api-endpoint
| await recursivelyDeleteS3Bucket(t.context.systemBucket); | ||
| }); | ||
|
|
||
| test.serial('Should move files to final location with cmr xml file', async (t) => { |
There was a problem hiding this comment.
Nit: test titles should probably include thing being tested (e.g. changeGranuleCollectionS3 should....)
| ); | ||
| }); | ||
|
|
||
| test.afterEach.always(async (t) => { |
There was a problem hiding this comment.
We should probably also kill the topic created in the beforeEach
| (urlObject) => urlObject.URL | ||
| ); | ||
|
|
||
| t.true(onlineAccessURLs.includes( |
There was a problem hiding this comment.
Nit/Suggestion: Use t.assert(assertion, text) so we avoid output like "false is not true"
| 'MOD11A1.A2017200.h19v04.006.2017201090724.cmr.xml' | ||
| ); | ||
|
|
||
| const CollectionInformation = UMM.Granule.Collection; |
There was a problem hiding this comment.
It's probably a minor quibble, but we're not directly testing that it only changed this value. Thoughts?
| @@ -0,0 +1,1000 @@ | |||
| 'use strict'; | |||
There was a problem hiding this comment.
The tests as exist look good, but there are enough methods abstracted we should probably more directly test them (happy path concerns) beyond 'did the files move', particularly if we're asserting just that an object exists and the like.
| @@ -0,0 +1,45 @@ | |||
| # @cumulus/change-granule-collection-s3 | |||
|
|
|||
| This lambda function moves granules between collections in s3 and in postgres | |||
There was a problem hiding this comment.
Suggestions: Should probably not read 'move', and updates CMR metadata
Co-authored-by: Jonathan Kovarik <[email protected]>
…to ecarton/cumulus-3751-s3-task
| cmrGranuleUrlType = 'both', | ||
| distributionBucketMap, | ||
| }) { | ||
| const updatemetadataObject = cloneDeep(metadataObject); |
| ); | ||
| granuleId = inputPayload.granules[0].granuleId; | ||
|
|
||
| ingestExecutionArn = await buildAndStartWorkflow( |
There was a problem hiding this comment.
I think we could probably make this go faster and/or reduce the potential for random failures if we just setup the granule directly/via API rather than run a workflow, particularly given this is a single Lambda invocation. I'm going to take a stab at that.
| } | ||
| }); | ||
| beforeAll(async () => { | ||
| const inputPayloadFilename = './spec/parallel/ingestGranule/IngestGranule.input.payload.json'; |
There was a problem hiding this comment.
We shouldn't use a payload from another test, replication is fine here, or moving it to test-data.
Summary: 3751 just the s3 copy part
Addresses CUMULUS-3751: Move granules across collections
Changes
PR Checklist