Fix: Race between layer and Lambda update (#5927) #6259

dsotirho-ucsc · 2024-05-14T18:17:02Z

Connected issues: #5927

Checklist

Author

PR is a draft
Target branch is develop
Name of PR branch matches issues/<GitHub handle of author>/<issue#>-<slug>
On ZenHub, PR is connected to all issues it (partially) resolves
PR description links to connected issues
PR title matches¹ that of a connected issue _{or comment in PR explains why they're different}
PR title references all connected issues
For each connected issue, there is at least one commit whose title references that issue

¹ when the issue title describes a problem, the corresponding PR
title is Fix: followed by the issue title

Author (partiality)

Added p tag to titles of partial commits
This PR is labeled partial _{or completely resolves all connected issues}
This PR partially resolves each of the connected issues _{or does not have the partial label}

Author (chains)

This PR is blocked by previous PR in the chain _{or is not chained to another PR}
The blocking PR is labeled base _{or this PR is not chained to another PR}
This PR is labeled chained _{or is not chained to another PR}

Author (reindex, API changes)

Added r tag to commit title _{or the changes introduced by this PR will not require reindexing of any deployment}
This PR is labeled reindex:dev _{or the changes introduced by it will not require reindexing of dev}
This PR is labeled reindex:anvildev _{or the changes introduced by it will not require reindexing of anvildev}
This PR is labeled reindex:anvilprod _{or the changes introduced by it will not require reindexing of anvilprod}
This PR is labeled reindex:prod _{or the changes introduced by it will not require reindexing of prod}
This PR is labeled reindex:partial and its description documents the specific reindexing procedure for dev, anvildev, anvilprod and prod _{or requires a full reindex or carries none of the labels reindex:dev, reindex:anvildev, reindex:anvilprod and reindex:prod}
This PR and its connected issues are labeled API _{or this PR does not modify a REST API}
Added a (A) tag to commit title for backwards (in)compatible changes _{or this PR does not modify a REST API}
Updated REST API version number in app.py _{or this PR does not modify a REST API}

Author (upgrading deployments)

Ran make image_manifests.json and committed the resulting changes _{or this PR does not modify azul_docker_images, or any other variables referenced in the definition of that variable}
Documented upgrading of deployments in UPGRADING.rst _{or this PR does not require upgrading deployments}
Added u tag to commit title _{or this PR does not require upgrading deployments}
This PR is labeled upgrade _{or does not require upgrading deployments}
This PR is labeled deploy:shared _{or does not modify image_manifests.json, and does not require deploying the shared component for any other reason}
This PR is labeled deploy:gitlab _{or does not require deploying the gitlab component}
This PR is labeled deploy:runner _{or does not require deploying the runner image}

Author (hotfixes)

Added F tag to main commit title _{or this PR does not include permanent fix for a temporary hotfix}
Reverted the temporary hotfixes for any connected issues _{or the none of the stable branches (anvilprod and prod) have temporary hotfixes for any of the issues connected to this PR}

Author (before every review)

Rebased PR branch on develop, squashed old fixups
Ran make requirements_update _{or this PR does not modify requirements*.txt, common.mk, Makefile and Dockerfile}
Added R tag to commit title _{or this PR does not modify requirements*.txt}
This PR is labeled reqs _{or does not modify requirements*.txt}
make integration_test passes in personal deployment _{or this PR does not modify functionality that could affect the IT outcome}

Peer reviewer (after approval)

PR is not a draft
Ticket is in Review requested column
PR is awaiting requested review from system administrator
PR is assigned to only the system administrator

System administrator (after approval)

Actually approved the PR
Labeled connected issues as demo or no demo
Commented on connected issues about demo expectations _{or all connected issues are labeled no demo}
Decided if PR can be labeled no sandbox
A comment to this PR details the completed security design review
PR title is appropriate as title of merge commit
N reviews label is accurate
Moved ticket to Approved column
PR is assigned to only the operator

Operator (before pushing merge the commit)

System administrator

Background migrations for dev.gitlab are complete _{or this PR is not labeled deploy:gitlab}
Background migrations for anvildev.gitlab are complete _{or this PR is not labeled deploy:gitlab}
PR is assigned to only the operator

Operator (before pushing merge the commit)

Operator (chain shortening)

Changed the target branch of the blocked PR to develop _{or this PR is not labeled base}
Removed the chained label from the blocked PR _{or this PR is not labeled base}
Removed the blocking relationship from the blocked PR _{or this PR is not labeled base}
Removed the base label from this PR _{or this PR is not labeled base}

Operator (after pushing the merge commit)

Operator (reindex)

Operator

Propagated the deploy:shared, deploy:gitlab, deploy:runner, reindex:partial, reindex:anvilprod and reindex:prod labels to the next promotion PRs _{or this PR carries none of these labels}
Propagated any specific instructions related to the deploy:shared, deploy:gitlab, deploy:runner, reindex:partial, reindex:anvilprod and reindex:prod labels, from the description of this PR to that of the next promotion PRs _{or this PR carries none of these labels}
PR is assigned to no one

Shorthand for review comments

L line is too long
W line wrapping is wrong
Q bad quotes
F other formatting problem

codecov · 2024-05-14T18:38:04Z

Codecov Report

Attention: Patch coverage is 18.75000% with 13 lines in your changes missing coverage. Please review.

Project coverage is 85.35%. Comparing base (6e66a5b) to head (f612650).
Report is 449 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/azul/lambdas.py	20.00%	12 Missing ⚠️
src/azul/terraform.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #6259      +/-   ##
===========================================
- Coverage    85.38%   85.35%   -0.03%     
===========================================
  Files          155      155              
  Lines        20735    20743       +8     
===========================================
+ Hits         17704    17706       +2     
- Misses        3031     3037       +6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

coveralls · 2024-05-14T18:38:40Z

coverage: 85.376% (-0.02%) from 85.399%
when pulling f612650 on issues/dsotirho-ucsc/5927-layer-and-lambda-race
into 6e66a5b on develop.

dsotirho-ucsc · 2024-05-16T15:49:10Z

6259_IT_2024-05-16.txt

Apply complete! Resources: 7 added, 10 changed, 7 destroyed.
python /Users/daniel/repo/azul2/scripts/delete_lambda_versions.py
2024-05-15 22:27:05,758    INFO MainThread botocore.credentials: Found credentials in shared credentials file: ~/.aws/credentials
2024-05-15 22:27:05,832    INFO MainThread azul.deployment: Allocated new Boto3 client for 'lambda' with ID 4406124240
2024-05-15 22:27:07,575    INFO MainThread __main__: Deleting published version 8 of azul-indexer-daniel-aggregate_retry
2024-05-15 22:27:08,081    INFO MainThread __main__: Deleting published version 8 of azul-service-daniel
2024-05-15 22:27:09,137    INFO MainThread __main__: Deleting published version 8 of azul-indexer-daniel-aggregate
2024-05-15 22:27:10,433    INFO MainThread __main__: Deleting published version 8 of azul-indexer-daniel-indexercachehealth
2024-05-15 22:27:14,468    INFO MainThread __main__: Deleting published version 8 of azul-service-daniel-servicecachehealth
2024-05-15 22:27:15,399    INFO MainThread __main__: Deleting published version 8 of azul-service-daniel-manifest
2024-05-15 22:27:16,776    INFO MainThread __main__: Deleting published version 8 of azul-indexer-daniel
2024-05-15 22:27:19,294    INFO MainThread __main__: Deleting published version 8 of azul-indexer-daniel-contribute_retry
2024-05-15 22:27:20,534    INFO MainThread __main__: Deleting published version 8 of azul-indexer-daniel-contribute
python /Users/daniel/repo/azul2/scripts/post_deploy_tdr.py
…

dsotirho-ucsc · 2024-05-16T16:05:29Z

It seems AWS keeps an internal counter of the published version of each Lambda, so even if the published version(s) are deleted before the deploy, the next published version will be +1 larger than the last. Since this is the case, there seems no benefit in running the delete prior to the deploy as opposed to after, so I opted for the latter.

nadove-ucsc · 2024-05-16T19:22:55Z

src/azul/terraform.py

+            # race conditions when an update to the function's configuration and
+            # code rely on the update of each other in order to work correctly.


Suggested change

# race conditions when an update to the function's configuration and

# code rely on the update of each other in order to work correctly.

# race conditions when there's a cyclic dependency between an update

# to the function's configuration and an update to its code

I think my way is easier to read, assuming I've correctly parsed the intent here

nadove-ucsc · 2024-05-16T19:25:17Z

scripts/delete_lambda_versions.py

+            if version['Version'] == '$LATEST':
+                pass
+            else:
+                version_number = version['Version']
+                log.info('Deleting published version %s of %s', version_number, lambda_.name)
+                aws.lambda_.delete_function(
+                    FunctionName=lambda_.name,
+                    Qualifier=version_number
+                )


Suggested change

if version['Version'] == '$LATEST':

pass

else:

version_number = version['Version']

log.info('Deleting published version %s of %s', version_number, lambda_.name)

aws.lambda_.delete_function(

FunctionName=lambda_.name,

Qualifier=version_number

)

version = version['Version']

if version == '$LATEST':

pass

else:

log.info('Deleting published version %s of %s', version, lambda_.name)

aws.lambda_.delete_function(

FunctionName=lambda_.name,

Qualifier=version

)

I think it's better to reuse the version variable instead of introducing a new one (especially since I'm confused about the difference in their names). Also, we can assign to it earlier to avoid repeating the dictionary lookup.

nadove-ucsc · 2024-05-16T19:26:51Z

scripts/delete_lambda_versions.py

+        response = aws.lambda_.list_versions_by_function(
+            FunctionName=lambda_.name
+        )


Fits on one line

Suggested change

response = aws.lambda_.list_versions_by_function(

FunctionName=lambda_.name

)

response = aws.lambda_.list_versions_by_function(FunctionName=lambda_.name)

dsotirho-ucsc · 2024-05-17T00:07:10Z

6259_IT_2024-05-16.txt

Apply complete! Resources: 7 added, 10 changed, 7 destroyed.
python /Users/daniel/repo/azul2/scripts/delete_lambda_versions.py
2024-05-16 16:35:52,794    INFO MainThread botocore.credentials: Found credentials in shared credentials file: ~/.aws/credentials
2024-05-16 16:35:52,870    INFO MainThread azul.deployment: Allocated new Boto3 client for 'lambda' with ID 4483204624
2024-05-16 16:35:55,007    INFO MainThread __main__: Fetching the published versions of azul-indexer-daniel-aggregate_retry
2024-05-16 16:35:55,148    INFO MainThread __main__: Deleting published version 11 of azul-indexer-daniel-aggregate_retry
2024-05-16 16:35:55,406    INFO MainThread __main__: Fetching the published versions of azul-service-daniel
2024-05-16 16:35:55,542    INFO MainThread __main__: Deleting published version 10 of azul-service-daniel
2024-05-16 16:35:55,810    INFO MainThread __main__: Fetching the published versions of azul-indexer-daniel-aggregate
2024-05-16 16:35:55,959    INFO MainThread __main__: Deleting published version 11 of azul-indexer-daniel-aggregate
2024-05-16 16:35:56,210    INFO MainThread __main__: Fetching the published versions of azul-indexer-daniel-indexercachehealth
2024-05-16 16:35:56,343    INFO MainThread __main__: Deleting published version 11 of azul-indexer-daniel-indexercachehealth
2024-05-16 16:35:56,598    INFO MainThread __main__: Fetching the published versions of azul-service-daniel-servicecachehealth
2024-05-16 16:35:56,733    INFO MainThread __main__: Deleting published version 10 of azul-service-daniel-servicecachehealth
2024-05-16 16:35:57,024    INFO MainThread __main__: Fetching the published versions of azul-service-daniel-manifest
2024-05-16 16:35:57,172    INFO MainThread __main__: Deleting published version 10 of azul-service-daniel-manifest
2024-05-16 16:35:57,456    INFO MainThread __main__: Fetching the published versions of azul-indexer-daniel
2024-05-16 16:35:57,611    INFO MainThread __main__: Deleting published version 11 of azul-indexer-daniel
2024-05-16 16:35:57,892    INFO MainThread __main__: Fetching the published versions of azul-indexer-daniel-contribute_retry
2024-05-16 16:35:58,020    INFO MainThread __main__: Deleting published version 11 of azul-indexer-daniel-contribute_retry
2024-05-16 16:35:58,279    INFO MainThread __main__: Fetching the published versions of azul-indexer-daniel-contribute
2024-05-16 16:35:58,431    INFO MainThread __main__: Deleting published version 11 of azul-indexer-daniel-contribute
python /Users/daniel/repo/azul2/scripts/post_deploy_tdr.py
…

src/azul/lambdas.py

dsotirho-ucsc · 2024-05-17T18:59:07Z

6259_IT_2024-05-17.txt

nadove-ucsc · 2024-05-17T20:17:19Z

src/azul/lambdas.py

        return [
            Lambda.from_response(function)
            for response in self._lambda.get_paginator('list_functions').paginate()
            for function in response['Functions']
+            if deployment_only is False or deployment(function) == config.deployment_stage


Suggested change

if deployment_only is False or deployment(function) == config.deployment_stage

if not deployment_only or deployment(function) == config.deployment_stage

dsotirho-ucsc · 2024-05-17T22:18:55Z

6259_IT_2024-05-17.txt

hannes-ucsc

I did not ask for any code changes in my previous review, only for additional evidence, yet the code was changed. Was this intentional, and if so, why?

dsotirho-ucsc · 2024-08-23T00:56:59Z

Yes, a fixup commit was added to re-combine the two list_lambdas() methods into one.

Background: When I refactored the code out of manage_lambdas() that listed the Lambda functions. I did this as a separate new method list_deployment_lambdas() instead of combining with the existing list_lambdas(). I didn't combine these at the time due to the conflicting behavior (from all deployments or not, & for all Lambda versions or not) being required by different call sites.

When testing manage_lambdas.py I discovered a bug introduced by this PR where now that Lambda functions have multiple versions (latest & a numbered version), manage_lambdas.py was unnecessarily fetching all versions of my Lambda functions and process each function twice as a result.

❯ python scripts/manage_lambdas.py --disable
2024-08-22 17:29:46,024    INFO MainThread botocore.credentials: Found credentials in shared credentials file: ~/.aws/credentials
2024-08-22 17:29:46,070    INFO MainThread azul.deployment: Allocated new Boto3 client for 'lambda' with ID 4371164368
2024-08-22 17:29:47,774    INFO MainThread azul.lambdas: Setting concurrency limit for azul-indexer-daniel-aggregate_retry to zero.
2024-08-22 17:29:48,270    INFO MainThread azul.lambdas: Setting concurrency limit for azul-service-daniel to zero.
2024-08-22 17:29:48,771    INFO MainThread azul.lambdas: Setting concurrency limit for azul-indexer-daniel-aggregate to zero.
2024-08-22 17:29:49,272    INFO MainThread azul.lambdas: Setting concurrency limit for azul-indexer-daniel-indexercachehealth to zero.
2024-08-22 17:29:49,826    INFO MainThread azul.lambdas: Setting concurrency limit for azul-service-daniel-servicecachehealth to zero.
2024-08-22 17:29:50,341    INFO MainThread azul.lambdas: Setting concurrency limit for azul-service-daniel-manifest to zero.
2024-08-22 17:29:50,848    INFO MainThread azul.lambdas: Setting concurrency limit for azul-indexer-daniel to zero.
2024-08-22 17:29:51,362    INFO MainThread azul.lambdas: Setting concurrency limit for azul-indexer-daniel-contribute_retry to zero.
2024-08-22 17:29:51,852    INFO MainThread azul.lambdas: Setting concurrency limit for azul-indexer-daniel-contribute to zero.
2024-08-22 17:29:52,352 WARNING MainThread azul.lambdas: azul-indexer-daniel is already disabled.
2024-08-22 17:29:52,597 WARNING MainThread azul.lambdas: azul-indexer-daniel-contribute is already disabled.
2024-08-22 17:29:52,833 WARNING MainThread azul.lambdas: azul-indexer-daniel-aggregate_retry is already disabled.
2024-08-22 17:29:53,091 WARNING MainThread azul.lambdas: azul-indexer-daniel-contribute_retry is already disabled.
2024-08-22 17:29:53,333 WARNING MainThread azul.lambdas: azul-indexer-daniel-aggregate is already disabled.
2024-08-22 17:29:53,573 WARNING MainThread azul.lambdas: azul-indexer-daniel-indexercachehealth is already disabled.
2024-08-22 17:29:53,818 WARNING MainThread azul.lambdas: azul-service-daniel-servicecachehealth is already disabled.
2024-08-22 17:29:54,067 WARNING MainThread azul.lambdas: azul-service-daniel-manifest is already disabled.
2024-08-22 17:29:54,307 WARNING MainThread azul.lambdas: azul-service-daniel is already disabled.

This prompted me to take another look at the two list_lambdas() methods and to combine them

hannes-ucsc · 2024-08-24T02:15:11Z

src/azul/lambdas.py

+                     all_versions: bool = False
+                     ) -> list[Lambda]:
+        """
+        Return a list of AWS Lambda functions


Suggested change

Return a list of AWS Lambda functions

Return a list of all AWS Lambda functions (or function versions) in the current account, or the given deployment.

hannes-ucsc · 2024-08-24T02:17:12Z

src/azul/lambdas.py

-            for lambda_name in [metadata['FunctionName'] for metadata in lambda_page['Functions']]:
-                if any(lambda_name.startswith(prefix) for prefix in lambda_prefixes):
-                    self.manage_lambda(lambda_name, enabled)
+        for lambda_ in self.list_lambdas(deployment=config.deployment_stage):


Suggested change

for lambda_ in self.list_lambdas(deployment=config.deployment_stage):

for function in self.list_lambdas(deployment=config.deployment_stage):

As noted before, we are phasing this term out. You don't have to remove all usages but just don't add new ones.

hannes-ucsc · 2024-08-24T02:18:06Z

src/azul/lambdas.py

+
+    def delete_stale_function_versions(self):
+        """
+        Delete all but the latest published version of every AWS Lambda function


Suggested change

Delete all but the latest published version of every AWS Lambda function

Delete all but the latest published version of every AWS Lambda function

Can there be more than one published version? If not, "the latest" qualifier seems redundant.

Can there be more than one published version?

Yes, however I was using the term "published" incorrectly. In AWS terminology a published version is a copy of the unpublished ($LATEST) version. There can be many published versions (each with a unique version number), but only one unpublished version.

https://docs.aws.amazon.com/lambda/latest/dg/configuration-versions.html

In my latest fixup commit I've changed the name and docstring of this method accordingly.

#6259 (comment)

Then I believe either the deletion logic is wrong or the publish TF functionality doesn't work. One of the published versions must be live so the logic below deletes it, or the latest version is live which raises the question as to why we published an immutable version.

I might of misunderstood your original question. There can be multiple published versions of a function in the sense that AWS allows it, however with this PR there will only ever be at most one published version, along with the unpublished version.

Both the published and unpublished versions are live, however only the unpublished version is used since our code does not qualify the function ARN with a version. https://docs.aws.amazon.com/lambda/latest/dg/configuration-versions.html#versioning-versions-using

Still don't understand it. Please raise in PL.

hannes-ucsc · 2024-08-24T02:22:06Z

src/azul/lambdas.py

+        paginator = self._lambda.get_paginator('list_functions')
+        lambda_prefixes = [
+            config.qualified_resource_name(lambda_infix, stage=deployment)
+            for lambda_infix in config.lambda_names()


Suggested change

for lambda_infix in config.lambda_names()

for lambda_name in config.lambda_names()

hannes-ucsc · 2024-08-24T02:24:26Z

src/azul/lambdas.py

+        lambda_prefixes = [
+            config.qualified_resource_name(lambda_infix, stage=deployment)
+            for lambda_infix in config.lambda_names()
+            if deployment is not None


In larger constructs I see the point of a constant comprehension guard, but here no.

hannes-ucsc · 2024-08-24T02:25:17Z

src/azul/lambdas.py

            for function in response['Functions']
+            if deployment is None or any(


Suggested change

if deployment is None or any(

if lambda_prefixes is None or any(

dsotirho-ucsc · 2024-08-27T20:56:40Z

6259_IT_2024-08-27.txt

hannes-ucsc · 2024-08-28T17:36:28Z

src/azul/lambdas.py

+
+    def delete_stale_function_versions(self):
+        """
+        Delete all but the latest published version of every AWS Lambda function


#6259 (comment)

Then I believe either the deletion logic is wrong or the publish TF functionality doesn't work. One of the published versions must be live so the logic below deletes it, or the latest version is live which raises the question as to why we published an immutable version.

dsotirho-ucsc · 2024-09-11T17:41:17Z

6259_IT_2024-09-11.txt

#6259 (comment)

hannes-ucsc

#6259 (comment)

github-actions bot added the orange [process] Done by the Azul team label May 14, 2024

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5927-layer-and-lambda-race branch 2 times, most recently from 1de4271 to 6275fec Compare May 15, 2024 22:44

dsotirho-ucsc assigned nadove-ucsc May 16, 2024

dsotirho-ucsc requested review from nadove-ucsc May 16, 2024 16:03

nadove-ucsc requested changes May 16, 2024

View reviewed changes

nadove-ucsc removed their assignment May 16, 2024

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5927-layer-and-lambda-race branch from 6275fec to e5d5938 Compare May 16, 2024 23:24

dsotirho-ucsc assigned nadove-ucsc May 17, 2024

nadove-ucsc requested changes May 17, 2024

View reviewed changes

src/azul/lambdas.py Outdated Show resolved Hide resolved

src/azul/lambdas.py Outdated Show resolved Hide resolved

nadove-ucsc removed their assignment May 17, 2024

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5927-layer-and-lambda-race branch from e5d5938 to d126563 Compare May 17, 2024 17:20

dsotirho-ucsc assigned nadove-ucsc May 17, 2024

nadove-ucsc requested changes May 17, 2024

View reviewed changes

nadove-ucsc removed their assignment May 17, 2024

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5927-layer-and-lambda-race branch from d126563 to 184a0e0 Compare May 17, 2024 21:39

dsotirho-ucsc requested a review from nadove-ucsc May 17, 2024 22:19

dsotirho-ucsc assigned nadove-ucsc May 17, 2024

nadove-ucsc previously approved these changes May 17, 2024

View reviewed changes

nadove-ucsc removed their assignment May 17, 2024

nadove-ucsc marked this pull request as ready for review May 17, 2024 22:38

nadove-ucsc requested a review from hannes-ucsc as a code owner May 17, 2024 22:38

dsotirho-ucsc requested a review from hannes-ucsc August 21, 2024 21:59

dsotirho-ucsc assigned hannes-ucsc Aug 21, 2024

hannes-ucsc reviewed Aug 22, 2024

View reviewed changes

hannes-ucsc removed their assignment Aug 22, 2024

dsotirho-ucsc requested a review from hannes-ucsc August 23, 2024 17:20

dsotirho-ucsc assigned hannes-ucsc Aug 23, 2024

hannes-ucsc requested changes Aug 24, 2024

View reviewed changes

hannes-ucsc removed their assignment Aug 24, 2024

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5927-layer-and-lambda-race branch from fd98952 to d21db43 Compare August 27, 2024 20:23

dsotirho-ucsc requested a review from hannes-ucsc August 27, 2024 20:57

dsotirho-ucsc assigned hannes-ucsc Aug 27, 2024

hannes-ucsc requested changes Aug 28, 2024

View reviewed changes

hannes-ucsc removed their assignment Aug 28, 2024

dsotirho-ucsc added 2 commits September 10, 2024 15:02

Add check_python prereq to Makefile targets that run Python

615743e

Add a version attribute to the Lambda class

87a0893

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5927-layer-and-lambda-race branch from d21db43 to 2567ecb Compare September 10, 2024 23:07

dsotirho-ucsc added 2 commits September 11, 2024 09:00

Add a deployment-specific option to list_lambdas()

a578c47

Fix: Race between layer and Lambda update (#5927)

fa91f1e

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5927-layer-and-lambda-race branch from 2567ecb to 0ba2a94 Compare September 11, 2024 16:25

fixup! Fix: Race between layer and Lambda update (#5927)

f612650

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5927-layer-and-lambda-race branch from 0ba2a94 to f612650 Compare September 11, 2024 16:29

dsotirho-ucsc requested a review from hannes-ucsc September 11, 2024 17:54

dsotirho-ucsc assigned hannes-ucsc Sep 11, 2024

hannes-ucsc reviewed Sep 11, 2024

View reviewed changes

hannes-ucsc removed their assignment Sep 11, 2024

hannes-ucsc force-pushed the develop branch from 33a7a95 to 118ade8 Compare February 7, 2025 02:20

dsotirho-ucsc mentioned this pull request May 15, 2025

Race between layer and Lambda update #5927

Open

		# race conditions when an update to the function's configuration and
		# code rely on the update of each other in order to work correctly.

	if deployment_only is False or deployment(function) == config.deployment_stage
	if not deployment_only or deployment(function) == config.deployment_stage

	Return a list of AWS Lambda functions
	Return a list of all AWS Lambda functions (or function versions) in the current account, or the given deployment.

	for lambda_ in self.list_lambdas(deployment=config.deployment_stage):
	for function in self.list_lambdas(deployment=config.deployment_stage):

	Delete all but the latest published version of every AWS Lambda function
	Delete all but the latest published version of every AWS Lambda function

	for lambda_infix in config.lambda_names()
	for lambda_name in config.lambda_names()

		for function in response['Functions']
		if deployment is None or any(

	if deployment is None or any(
	if lambda_prefixes is None or any(

Fix: Race between layer and Lambda update (#5927) #6259

Are you sure you want to change the base?

Fix: Race between layer and Lambda update (#5927) #6259

Uh oh!

Conversation

dsotirho-ucsc commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Author

Author (partiality)

Author (chains)

Author (reindex, API changes)

Author (upgrading deployments)

Author (hotfixes)

Author (before every review)

Peer reviewer (after approval)

System administrator (after approval)

Operator (before pushing merge the commit)

System administrator

Operator (before pushing merge the commit)

Operator (chain shortening)

Operator (after pushing the merge commit)

Operator (reindex)

Operator

Shorthand for review comments

Uh oh!

codecov bot commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

coveralls commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dsotirho-ucsc commented May 16, 2024

Uh oh!

dsotirho-ucsc commented May 16, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsotirho-ucsc commented May 17, 2024

Uh oh!

Uh oh!

Uh oh!

dsotirho-ucsc commented May 17, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsotirho-ucsc commented May 17, 2024

Uh oh!

hannes-ucsc left a comment

Choose a reason for hiding this comment

Uh oh!

dsotirho-ucsc commented Aug 23, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsotirho-ucsc Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hannes-ucsc Aug 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsotirho-ucsc commented May 14, 2024 •

edited

Loading

codecov bot commented May 14, 2024 •

edited

Loading

coveralls commented May 14, 2024 •

edited

Loading

dsotirho-ucsc Aug 27, 2024 •

edited

Loading

hannes-ucsc Aug 28, 2024 •

edited

Loading

hannes-ucsc Aug 28, 2024 •

edited

Loading

dsotirho-ucsc commented Sep 11, 2024 •

edited

Loading