Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

luarocks-admin add AWS S3 #1669

Open
3 tasks
ochaton opened this issue May 6, 2024 · 1 comment
Open
3 tasks

luarocks-admin add AWS S3 #1669

ochaton opened this issue May 6, 2024 · 1 comment

Comments

@ochaton
Copy link

ochaton commented May 6, 2024

Hello, everyone!

It is very convenient to host personal luarocks server in ordinary S3 bucket behind AWS Cloudfront or S3 Website or anything else.

We use AWS S3 + AWS Cloudfront to host https://moonlibs.org but we have to do lot's of abnormal hacks (download content, rebuild manifest locally, upload bundle back to AWS S3) to make it work.

Almost anyone, I think, is able to use S3 bucket to host there personal libraries.

To support this feature, all we need is:

  • Allow specifying server with scheme "s3://"
  • Allow to specify S3_BUCKET, AWS_ACCEESS_KEY_ID, AWS_ACCESS_SECRET_KEY, AWS_REGION and S3_ENDPOINT_URL variables
  • Download manifests as usual (using wget/curl etc.) but upload contents using aws cli.

So it should look something like that:

when S3_ENDPOINT_URL is defined:

aws --profile ${AWS_PROFILE} --region ${AWS_REGION} s3 --endpoint-url ${S3_ENDPOINT_URL} cp ./manifest s3://${S3_BUCKET}

when uploading to AWS:

aws --profile ${AWS_PROFILE} --region ${AWS_REGION} s3 cp ./manifest s3://${S3_BUCKET}

and so on. Also, it could be nice to provide specification of Cache-Control and default ACL, all of them depend on configuration of S3 Bucket and provided S3 storage (no automatics needed, just the possibility of specifying these things).

It should work well with AWS S3, Google Cloud Storage and locally hosted Minio.

What do you think?

@daurnimator
Copy link
Member

Hello, everyone!

It is very convenient to host personal luarocks server in ordinary S3 bucket behind AWS Cloudfront or S3 Website or anything else.

We use AWS S3 + AWS Cloudfront to host https://moonlibs.org but we have to do lot's of abnormal hacks (download content, rebuild manifest locally, upload bundle back to AWS S3) to make it work.

Almost anyone, I think, is able to use S3 bucket to host there personal libraries.

To support this feature, all we need is:

* [ ]  Allow specifying server with scheme "s3://"

* [ ]  Allow to specify S3_BUCKET, AWS_ACCEESS_KEY_ID, AWS_ACCESS_SECRET_KEY, AWS_REGION and S3_ENDPOINT_URL variables

* [ ]  Download manifests as usual (using wget/curl etc.) but upload contents using aws cli.

So it should look something like that:

when S3_ENDPOINT_URL is defined:

aws --profile ${AWS_PROFILE} --region ${AWS_REGION} s3 --endpoint-url ${S3_ENDPOINT_URL} cp ./manifest s3://${S3_BUCKET}

when uploading to AWS:

aws --profile ${AWS_PROFILE} --region ${AWS_REGION} s3 cp ./manifest s3://${S3_BUCKET}

and so on. Also, it could be nice to provide specification of Cache-Control and default ACL, all of them depend on configuration of S3 Bucket and provided S3 storage (no automatics needed, just the possibility of specifying these things).

It should work well with AWS S3, Google Cloud Storage and locally hosted Minio.

What do you think?

There is far more to an AWS credential chain; I would be very hesitant to implement that ourselves.


Can't you just run aws s3 sync or similar to upload a luarocks dir?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants