Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Region isn't read by S3 PUT helper (AWS S3 Uploads keep failing with PermanentRedirect) #1446

Open
kwongtn98 opened this issue Oct 30, 2023 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@kwongtn98
Copy link

kwongtn98 commented Oct 30, 2023

Describe the bug
On upload to S3, PermanentRedirect error keeps happening as follows:

[ip-10-0-0-142.ap-southeast-1.compute.internal-0:ERROR] (hook=onItemCaptured) Error: operation error S3: PutObject, https response error StatusCode: 301, RequestID: 50...S, HostID: Cm...=, api error PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
[ip-10-0-0-75.ap-southeast-1.compute.internal-0:ERROR] (hook=onItemCaptured) Error: operation error S3: PutObject, https response error StatusCode: 301, RequestID: HH...E, HostID: tB...=, api error PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
[ip-10-0-0-142.ap-southeast-1.compute.internal-0:ERROR] (hook=onItemCaptured) Error: operation error S3: PutObject, https response error StatusCode: 301, RequestID: K8...X, HostID: Rd...=, api error PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
[ip-10-0-0-75.ap-southeast-1.compute.internal-0:ERROR] (hook=onItemCaptured) Error: operation error S3: PutObject, https response error StatusCode: 301, RequestID: PD...4, HostID: 1z...=, api error PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
[ip-10-0-0-142.ap-southeast-1.compute.internal-0:ERROR] (hook=onItemCaptured) Error: operation error S3: PutObject, https response error StatusCode: 301, RequestID: AY...G, HostID: 9C...=, api error PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.

Provide more information
AWS EKS, Istio

config.yaml as follows

scripting:
  env:
    AWS_ACCESS_KEY_ID: AK...6
    AWS_REGION: ap-southeast-1
    AWS_SECRET_ACCESS_KEY: mj...e
    RECORDING_KFL: "http or dns"
    S3_BUCKET: ma...k
  source: /home/kwongtn/ms...s/kubeshark
  watchscripts: true

To Reproduce
Steps to reproduce the behavior:

  1. Run kubeshark tap
  2. Go to web UI to start monitoring
  3. Open scripts window
  4. Click on example Upload PCAP File of a Stream to an AWS S3 Bucket If HTTP Status Code is 500 or Upload a PCAP Snapshot to an AWS S3 Bucket If HTTP Status Code is 500
  5. Generate a 500 error (or remove && data.response.status === 500 in line 4)
  6. See errors

Expected behavior
Stuff should upload successfully to the bucket.

Desktop (please complete the following information):

  • Windows 10 20H2
  • Google Chrome

Additional context
Logs
kubeshark_logs_2023_10_30__18_01_40.zip

S3 permissions

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::ma..k/*",
                "arn:aws:s3:::ma..k/",
            ]
        }
    ]
}

Since I don't have access to the s3 endpoint that kubeshark is attempting to reach, I'm also suspecting that kubeshark is using the old S3 endpoint format:
https://my-bucket-123.s3-us-west-2.amazonaws.com/test/123

Instead of the new one:
https://my-bucket-123.s3.us-west-2.amazonaws.com/test/123

Take note of the "." after s3

@alongir alongir self-assigned this Oct 31, 2023
@alongir alongir added the bug Something isn't working label Oct 31, 2023
@alongir
Copy link
Member

alongir commented Oct 31, 2023

HI @kwongtn98. I was able to reproduce this error locally. Apparently we override the region with us-east-2. To bypass this issue, you can open a bucket on us-east-2 as opposed to ap-southeast-1.
Also, please use the new version of the script.

@alongir alongir changed the title AWS S3 Uploads keep failing with PermanentRedirect AWS Region isn't read by S3 PUT helper (AWS S3 Uploads keep failing with PermanentRedirect) Oct 31, 2023
@alongir alongir assigned mertyildiran and unassigned alongir Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants