bug: API Gateway v2 HTTP_PROXY override path is not working

[MrZoidberg]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

I have the following construct created using cdklocal:

API GW v2 1 -> API GW v2 2 -> Lambda function.

API GW 1 has the following route:
awslocal apigatewayv2 get-routes --api-id 7d29c397

{
    "Items": [
        {
            "ApiKeyRequired": false,
            "AuthorizationType": "CUSTOM",
            "AuthorizerId": "821dc6ce",
            "RouteId": "61865a14",
            "RouteKey": "ANY /es-batch/{proxy+}",
            "Target": "c895c89a"
        }
    ]
}

Here is a target for with route:
awslocal apigatewayv2 get-integrations --api-id 7d29c397

{
    "Items": [
        {
            "ConnectionType": "INTERNET",
            "Description": "Integration for es-batch module",
            "IntegrationId": "c895c89a",
            "IntegrationMethod": "ANY",
            "IntegrationType": "HTTP_PROXY",
            "IntegrationUri": "http://154f05a5.execute-api.amazonaws.com:4566/",
            "PayloadFormatVersion": "2.0",
            "RequestParameters": {
                "overwrite:path": "${request.path}"
            },
            "TimeoutInMillis": 30000
        }
    ]
}

API GW 2 has the following route:
awslocal apigatewayv2 get-routes --api-id 154f05a5

{
    "Items": [
        {
            "AuthorizationType": "NONE",
            "OperationName": "Proxy route",
            "RouteId": "96b06ec3",
            "RouteKey": "ANY /{proxy+}",
            "Target": "eadfa35c"
        }
    ]
}

The target for that route is the Lambda function. Here is how I can it using API GW 2:

curl --location 'http://154f05a5.execute-api.localhost.localstack.cloud:4566/es-batch/v1/data-pipelines?tenantId=tenant-local'  --header 'Accept: application/json' response is 200.

However, when I use API GW 1 I receive 404 error:

curl --location 'http://7d29c397.execute-api.localhost.localstack.cloud:4566/es-batch/v1/data-pipelines?tenantId=tenant-local' --header 'Accept: application/json'

< HTTP/1.1 404
< Content-Type: text/plain; charset=utf-8
< Content-Length: 9
< Connection: close
< date: Tue, 09 Apr 2024 15:10:17 GMT
< server: hypercorn-h11
<
* Closing connection
Not Found

Here are the logs:

Start of execution environment 4faf06b69dc3b57cc4222f4242632aa0 for function arn:aws:lambda:us-east-1:000000000000:function:AuthorizerLambda:$LATEST took 758.54ms
2024-04-09 18:12:06 2024-04-09T15:12:06.858 DEBUG --- [ asgi_gw_1] l.s.l.i.docker_runtime_exe : Sending invoke-payload '{"invoke-id": "101e705b-133c-4e67-a446-51883c76212f", "invoked-function-arn": "arn:aws:lambda:us-east-1:000000000000:function:AuthorizerLambda", "payload": "{"version": "2.0", "type": "REQUEST", "routeArn": "arn:aws:execute-api:us-east-1:000000000000:7d29c397/$default/GET/es-batch/v1/data-pipelines", "identitySource": ["Bearer XXXX"], "routeKey": "ANY /es-batch/v1/data-pipelines", "rawPath": "/es-batch/v1/data-pipelines", "rawQueryString": "tenantId=tenant-local", "cookies": null, "headers": {"Accept": "application/json", "bd-host": "tenant-local.api.us1.bdconnected.com", "Authorization": "Bearer XXXX", "User-Agent": "PostmanRuntime/7.37.0", "Postman-Token": "6dc408cc-09a2-48bc-a410-2041b6ee6e5a", "Host": "7d29c397.execute-api.localhost.localstack.cloud:4566", "Accept-Encoding": "gzip, deflate, br", "Connection": "keep-alive", "X-Forwarded-For": "172.17.0.1, 7d29c397.execute-api.localhost.localstack.cloud:4566", "x-localstack-edge": "http://7d29c397.execute-api.localhost.localstack.cloud:4566\"}, "queryStringParameters": {"tenantId": "tenant-local"}, "pathParameters": {"proxy": "v1/data-pipelines"}, "stageVariables": null, "requestContext": {"accountId": "000000000000", "apiId": "7d29c397", "authentication": null, "domainName": "7d29c397.execute-api.localhost.localstack.cloud", "...' to executor '4faf06b69dc3b57cc4222f4242632aa0'
2024-04-09 18:12:06 2024-04-09T15:12:06.862 INFO --- [ asgi_gw_2] localstack.request.http : POST /_localstack_lambda/4faf06b69dc3b57cc4222f4242632aa0/invocations/101e705b-133c-4e67-a446-51883c76212f/logs => 202
2024-04-09 18:12:06 2024-04-09T15:12:06.864 INFO --- [ asgi_gw_3] localstack.request.http : POST /localstack_lambda/4faf06b69dc3b57cc4222f4242632aa0/invocations/101e705b-133c-4e67-a446-51883c76212f/response => 202
2024-04-09 18:12:06 2024-04-09T15:12:06.878 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : Got logs for invocation '101e705b-133c-4e67-a446-51883c76212f'
2024-04-09 18:12:06 2024-04-09T15:12:06.878 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : [AuthorizerLambda-101e705b-133c-4e67-a446-51883c76212f] START RequestId: 101e705b-133c-4e67-a446-51883c76212f Version: $LATEST
2024-04-09 18:12:06 2024-04-09T15:12:06.880 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : [AuthorizerLambda-101e705b-133c-4e67-a446-51883c76212f] 2024/04/09 15:12:06 token: XXXX
2024-04-09 18:12:06 2024-04-09T15:12:06.880 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : [AuthorizerLambda-101e705b-133c-4e67-a446-51883c76212f] END RequestId: 101e705b-133c-4e67-a446-51883c76212f
2024-04-09 18:12:06 2024-04-09T15:12:06.880 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : [AuthorizerLambda-101e705b-133c-4e67-a446-51883c76212f] REPORT RequestId: 101e705b-133c-4e67-a446-51883c76212f Duration: 1.35 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 128 MB
2024-04-09 18:12:06 2024-04-09T15:12:06.880 DEBUG --- [ asgi_gw_1] l.s.lambda.provider : Lambda invocation duration: 783.58ms
2024-04-09 18:12:06 2024-04-09T15:12:06.888 INFO --- [ asgi_gw_5] l.s.apigateway.authorizers : Received authorizer result: {"isAuthorized":true}
2024-04-09 18:12:06 2024-04-09T15:12:06.889 DEBUG --- [ asgi_gw_5] l.s.apigateway.authorizers : Updating identity context: {'cognitoIdentityId': None, 'cognitoIdentityPoolId': None}
2024-04-09 18:12:06 2024-04-09T15:12:06.899 DEBUG --- [ asgi_gw_5] l.s.a.integrations : Unable to extract message value from expression '${request.path}', falling back to using verbatim string: Error on line 1, col 1: Unexpected character: {
2024-04-09 18:12:06 2024-04-09T15:12:06.900 DEBUG --- [ asgi_gw_5] l.s.a.integrations : Sending GET request to http://154f05a5.execute-api.amazonaws.com:4566?tenantId=tenant-local in response to invocation message {'version': '2.0', 'routeKey': 'GET /es-batch/v1/data-pipelines', 'rawPath': '/es-batch/v1/data-pipelines', 'rawQueryString': 'tenantId=tenant-local', 'headers': {'accept': 'application/json', 'bd-host': 'tenant-local.api.us1.bdconnected.com', 'Authorization': 'Bearer XXXXXXXX', 'User-Agent': 'PostmanRuntime/7.37.0', 'Postman-Token': '6dc408cc-09a2-48bc-a410-2041b6ee6e5a', 'Host': '7d29c397.execute-api.localhost.localstack.cloud:4566', 'accept-encoding': 'gzip, deflate, br', 'Connection': 'keep-alive', 'x-localstack-edge': 'http://7d29c397.execute-api.localhost.localstack.cloud:4566', 'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate, br', 'Forwarded': 'by=172.17.0.1, 7d29c397.execute-api.localhost.localstack.cloud:4566'}, 'body': b'', 'requestContext': {'accountId': '4566/', 'apiId': '7d29c397', 'authorizer': {'lambda': {'isAuthorized': True}}, 'domainName': '7d29c397.execute-api.localhost.localstack.cloud', 'domainPrefix': '7d29c397', 'http': {'method': 'GET', 'path': '/es-batch/v1/data-pipelines', 'protocol': 'HTTP/1.1', 'sourceIp': '172.17.0.1', 'userAgent': 'PostmanRuntime/7.37.0'}, 'requestId': '39f9e5c6-3880-4398-befe-5c50148662da', 'routeKey': 'ANY /es-batch/v1/data-pipelines', 'stage': '$default', 'time': '09/Apr/2024:15:12:06 +0000', 'timeEpoch': 1712675526900}, 'isBase64Encoded': False, 'queryStringParameters': {'tenantId': 'tenant-local'}, 'stageVariables': {}, 'pathParameters': {'proxy': 'v1/data-pipelines'}}
2024-04-09 18:12:06 2024-04-09T15:12:06.908 DEBUG --- [ asgi_gw_3] l.s.a.apigateway_utils : No matching stage (None) for invocation path (falling back to $default stage): /
2024-04-09 18:12:06 2024-04-09T15:12:06.908 ERROR --- [ asgi_gw_3] l.s.a.apigateway_utils : Unable to find a matching route for GET /
2024-04-09 18:12:06 2024-04-09T15:12:06.908 DEBUG --- [ asgi_gw_3] l.s.apigateway.authorizers : No authorizer configured for resource: GET /?tenantId=tenant-local
2024-04-09 18:12:06 2024-04-09T15:12:06.909 DEBUG --- [ asgi_gw_3] l.s.apigateway.authorizers : Authorization failed: Not Found - Traceback (most recent call last):
2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/authorizers.py.enc", line 345, in is_request_authorized
2024-04-09 18:12:06 try:AuthorizerService().check_request_authorization(invocation_context)
2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/authorizers.py.enc", line 315, in check_request_authorization
2024-04-09 18:12:06 try:J=get_target_resource_method_or_route(A)
2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/apigateway_utils.py.enc", line 228, in get_target_resource_method_or_route
2024-04-09 18:12:06 return find_matching_route(A)
2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^
2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/apigateway_utils.py.enc", line 201, in find_matching_route
2024-04-09 18:12:06 else:return find_matching_http_route(B,A)
2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/apigateway_utils.py.enc", line 193, in find_matching_http_route
2024-04-09 18:12:06 if not A:LOG.error(f"Unable to find a matching route for {F} {E}");raise NotFoundException(_N)
2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-09 18:12:06 localstack_ext.aws.api.apigatewayv2.NotFoundException: Not Found
2024-04-09 18:12:06
2024-04-09 18:12:06 2024-04-09T15:12:06.910 INFO --- [ asgi_gw_3] localstack.request.http : GET / => 404

To summarize:

It seems "overwrite:path": "${request.path}" does not work in Localstack as the same CDK code works in AWS account.

Expected Behavior

API GW 1 integration should replace the path in the request to API GW 2 with the original path using next configuration of the HTTP_PROXY integration: "overwrite:path": "${request.path}"

How are you starting LocalStack?

With a docker-compose file

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

localstack start -d

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

cio, err := input.ApigwClient.CreateIntegration(ctx, &apigatewayv2.CreateIntegrationInput{
	ApiId:           awsClient.String(input.APIGatewayID),
	IntegrationType: gwTypes.IntegrationTypeHttpProxy,
	ConnectionType:  gwTypes.ConnectionTypeInternet,

	Description:          awsClient.String(fmt.Sprintf("Integration for %s module", input.Slug)),
	IntegrationMethod:    awsClient.String("ANY"),
	IntegrationUri:       awsClient.String(input.InternalAPIGatewayURL),
	PayloadFormatVersion: awsClient.String("1.0"),
	RequestParameters: map[string]string{
		"overwrite:path": "$request.path",
	},
}, func(_ *apigatewayv2.Options) {})
if err != nil {
	return nil, err //nolint:wrapcheck // no need to wrap here
}

authorizationType := gwTypes.AuthorizationTypeNone
if input.AuthorizerID != "" {
	authorizationType = gwTypes.AuthorizationTypeCustom
}
route, err := input.ApigwClient.CreateRoute(ctx, &apigatewayv2.CreateRouteInput{
	ApiId:             awsClient.String(input.APIGatewayID),
	RouteKey:          awsClient.String(fmt.Sprintf("%s %s", input.Method, input.Route)),
	AuthorizationType: authorizationType,
	ApiKeyRequired:    optional.Option(false),
	AuthorizerId:      optional.Option(input.AuthorizerID),
	Target:            awsClient.String(fmt.Sprintf("integrations/%s", *cio.IntegrationId)),
}, func(_ *apigatewayv2.Options) {})
if err != nil {
	return nil, err //nolint:wrapcheck // no need to wrap here
}

Environment

- OS: OSX 14.4.1
- LocalStack: 3.3.0 Pro

Anything else?

localstack-logs.txt

[localstack-bot]

Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our Slack Community for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our contributing guidelines and our contributing guide.