diff --git a/README.md b/README.md index 921b473cb..ef3d6fc82 100644 --- a/README.md +++ b/README.md @@ -64,16 +64,136 @@ docker-compose -f environment/docker-compose.yaml up -d ``` 3. Make sure allocation.txt file exist in the default folder ``~/.zcn`` -## Test MinIO Client +4. Now you can interact with the clint API follow this [doc](/client-api/README.md) -MinIO client `mc` provides a modern alternative to UNIX commands such as ls, cat, cp, mirror, diff, etc. It supports filesystems and Amazon S3-compatible cloud storage services. To interact with the client API follow this [doc](/client-api/README.md). You can also interact with the log search API by following this [doc.](/logsearchapi/README.md) +5. You can also interact with the logsearch API by following this [doc](/logsearchapi/README.md) + + +## Test using AWS Client `aws` +### Installation +Install from here: https://aws.amazon.com/cli/ + +### Configuration +Fetch the access key and secret from your deployed zs3server. To configure `aws cli`, type `aws configure` and +specify the zs3server key information like below: +``` +aws configure +AWS Access Key ID [None]: miniouser +AWS Secret Access Key [None]: miniopassword +Default region name [None]: us-east-1 +Default output format [None]: ENTER +``` + +Additionally enable AWS Signature Version ‘4’ for zs3server. + +`aws configure set default.s3.signature_version s3v4` + +### Examples +#### To list your buckets +``` +aws --endpoint-url https://localhost:9000 s3 ls +2016-03-27 02:06:30 deebucket +2016-03-28 21:53:49 guestbucket +2016-03-29 13:34:34 mbtest +2016-03-26 22:01:36 mybucket +2016-03-26 15:37:02 testbucket +``` + +#### To list contents inside bucket +``` +aws --endpoint-url https://localhost:9000 s3 ls s3://mybucket +2016-03-30 00:26:53 69297 argparse-1.2.1.tar.gz +2016-03-30 00:35:37 67250 simplejson-3.3.0.tar.gz +``` + +#### To make a bucket +``` +aws --endpoint-url https://localhost:9000 s3 mb s3://mybucket +make_bucket: s3://mybucket/ +``` + +#### To add an object to a bucket +``` +aws --endpoint-url https://localhost:9000 s3 cp simplejson-3.3.0.tar.gz s3://mybucket +upload: ./simplejson-3.3.0.tar.gz to s3://mybucket/simplejson-3.3.0.tar.gz +``` + +#### To delete an object from a bucket +``` +aws --endpoint-url https://localhost:9000 s3 rm s3://mybucket/argparse-1.2.1.tar.gz +delete: s3://mybucket/argparse-1.2.1.tar.gz +``` + +#### To remove a bucket +``` +aws --endpoint-url https://localhost:9000 s3 rb s3://mybucket +remove_bucket: s3://mybucket/ +``` + + + +## Test using MinIO Client `mc` +`mc` provides a modern alternative to UNIX commands such as ls, cat, cp, mirror, diff etc. It supports filesystems +and Amazon S3 compatible cloud storage services. + +### Installation +Install from here for your os: https://min.io/docs/minio/macos/index.html ## Configure MinIO Client ``` mc config host add zcn http://localhost:9000 miniouser miniopassword -mc ls zcn //List your buckets +``` + +### Examples +#### To list your buckets +``` +mc ls zcn/ +2016-03-27 02:06:30 deebucket +2016-03-28 21:53:49 guestbucket +2016-03-29 13:34:34 mbtest +2016-03-26 22:01:36 mybucket +2016-03-26 15:37:02 testbucket +``` + +#### To list contents inside bucket +``` +mc ls zcn/mybucket +2016-03-30 00:26:53 69297 argparse-1.2.1.tar.gz +2016-03-30 00:35:37 67250 simplejson-3.3.0.tar.gz +``` + +#### To make a bucket +``` +mc mb zcn/mybucket +make_bucket: zcn/mybucket +``` + +#### To add an object to a bucket +``` +mc cp simplejson-3.3.0.tar.gz zcn/mybucket +upload: ./simplejson-3.3.0.tar.gz to zcn/mybucket/simplejson-3.3.0.tar.gz +``` + +#### To delete an object from a bucket +``` +mc rm zcn/mybucket/argparse-1.2.1.tar.gz +delete: zcn/mybucket/argparse-1.2.1.tar.gz +``` -[2017-02-22 01:50:43 PST] 0B user/ -[2017-02-26 21:43:51 PST] 0B datasets/ -[2017-02-26 22:10:11 PST] 0B assets/ +#### To remove a bucket +``` +mc rb zcn/mybucket +remove_bucket: zcn/mybucket/ ``` + +Check `mc --help` for the exhaustive list of cmds available. + + +## Test using Postman Using REST APIs +- Add the following authorization settings +![](./assets/postman-auth.png "Authorization settings") +- The `AccessKey` would be the MINIO_ROOT_USER which you set earlier during zs3server deployment and `SecretKey` would be the MINIO_ROOT_PASSWORD. +- If you do not want to share the MINIO_ROOT_USER and MINIO_ROOT_PASSWORD, you can also create a user from minio console and share their access key and secret instead. +- Use the REST APIs to interact with the server. +- Postman collection for the same is provided below: +[Postman Collection](./assets/Zs3ServerCollection.postman_collection.json) \ No newline at end of file diff --git a/assets/Zs3ServerCollection.postman_collection.json b/assets/Zs3ServerCollection.postman_collection.json new file mode 100644 index 000000000..cb5b69f84 --- /dev/null +++ b/assets/Zs3ServerCollection.postman_collection.json @@ -0,0 +1,2025 @@ +{ + "info": { + "_postman_id": "99233ff4-f095-4b0d-9935-ef0fc700948b", + "name": "Zs3Server Collection", + "description": "The S3 Compatible API implements the most commonly used actions from the AWS S3 API, and easily integrates with your existing applications, data management tools, and S3 gateways.\n\n## Prerequisites\n\n- Open the minio UI locally by navigating to localhost:9000.\n- Click on Account settings and create a user with an access key and secret.\n \n\nOnce you have your application key and secret, navigate to the API's collection variables (Click on **S3 Compatible API** in the tree on the left then click the **Variables** tab). Copy your application key, application key id into the current values of the three variables:\n\n## Test your settings\n\n- Go to [S3 Get Bucket Location](https://).\n- Enter the name of a bucket to which your application key has access.\n- Click **Send**.\n \n\nYou should see an XML response with the location of the bucket:\n\n\n\n## Troubleshooting\n\nIf you don't see an XML response with the bucket location, check that:\n\n- The application key has access to the bucket that you specified.\n- You didn't inadvertently swap the application key and application key ID.\n- You saved the collection variables.\n \n\n## Authentication\n\nThe API supports [AWS Signature Version 4 (SigV4)](https://) for authentication and does not support V2 signatures at this time. This collection uses Postman's built-in implementation of AWS SigV4.\n\n## Requests\n\nCommunicate with the API by sending HTTP requests to resource endpoints. The HTTP method determines the action taken.\n\n| Method | Usage |\n| --- | --- |\n| DELETE | Use the DELETE method to destroy a resource such as a bucket, object or bucket setting. If it is not found, the operation will return a 4xx error and an appropriate message. |\n| GET | To retrieve object data or configuration, use the GET method. Object data is returned in the format it was written; configuration is returned as XML. GET methods are read-only and do not modify any resources. |\n| PUT | Use the PUT method to write object data or configuration. Configuration must be formatted as XML according to the schema defined by the Amazon S3 API. |\n| POST | Use the POST method with the S3 Delete Object, S3 Complete Multipart Upload, and S3 Create Multipart Upload operations. |\n\n## Response Codes\n\nThe API uses standard HTTP response codes to show the outcome of requests. Response codes in the 2xx range indicate success, while codes in the 4xx range indicate a client-side error, such as an authorization failure or a malformed request. 4xx errors return an XML response object explaining the problem. For example:\n\n``` xml\n\n NoSuchBucket\n The specified bucket does not exist: NonExistentBucket\n NonExistentBucket\n\n\n ```\n\nCodes in the 5xx range indicate a server-side problem preventing B2 from fulfilling your request, and include JSON diagnostics:\n\n``` json\n{\n \"code\": \"internal_error\",\n \"message\": \"incident id 2f0e432cc922-9c2c28e96a16711b\",\n \"status\": 500\n}\n\n ```\n\n| Response | Description |\n| --- | --- |\n| 200 OK | The response contains your requested information. |\n| 201 Created | Your request was accepted. The resource was created. |\n| 202 Accepted | Your request was accepted. The resource was created or updated. |\n| 204 No Content | Your request succeeded, there is no additional information returned. |\n| 400 Bad Request | Your request was malformed. |\n| 401 Unauthorized | You did not supply valid authentication credentials. |\n| 403 Forbidden | You are not allowed to perform that action. |\n| 404 Not Found | No results were found for your request. |\n| 429 Too Many Requests | Your request exceeded the API rate limit. |\n| 500 Internal Server Error | We were unable to perform the request due to server-side problems. |", + "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json", + "_exporter_id": "5200574" + }, + "item": [ + { + "name": "Bucket Operations", + "item": [ + { + "name": "S3 Create Bucket", + "request": { + "method": "PUT", + "header": [ + { + "key": "x-amz-acl", + "value": "", + "description": "Optional. Allowed values are \"private\" or \"public-read\". Defaults to \"private\".", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-bucket-object-lock-enabled", + "value": "", + "description": "Optional. Specifies whether object lock should be appied to the bucket.", + "type": "text", + "disabled": true + } + ], + "body": { + "mode": "raw", + "raw": "\n {{region}}\n", + "options": { + "raw": { + "language": "xml" + } + } + }, + "url": { + "raw": "{{s3baseurl}}/:bucket", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket" + ], + "variable": [ + { + "key": "bucket", + "value": "2home", + "description": "Required. The name of the bucket to create." + } + ] + }, + "description": "Creates a new bucket.\n\nThe bucket name must be globally unique. For restrictions on bucket names such as minimum and maximum length see [Bucket Names](https://www.backblaze.com/b2/docs/buckets.html#bucketNames).\n\nYou can optionally specify a canned ACL of `private` (the default) or `public-read` via the `x-amz-acl` header." + }, + "response": [ + { + "name": "S3 Create Bucket", + "originalRequest": { + "method": "PUT", + "header": [ + { + "key": "x-amz-acl", + "value": "private", + "description": "Optional. Allowed values are \"private\" or \"public-read\". Defaults to \"private\".", + "type": "text" + }, + { + "key": "x-amz-bucket-object-lock-enabled", + "value": "", + "description": "Optional. Specifies whether object lock should be appied to the bucket.", + "type": "text", + "disabled": true + } + ], + "body": { + "mode": "raw", + "raw": "\n \n", + "options": { + "raw": { + "language": "xml" + } + } + }, + "url": { + "raw": "s3..backblazeb2.com/:bucket", + "host": [ + "s3", + "", + "backblazeb2", + "com" + ], + "path": [ + ":bucket" + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. The name of the bucket to create." + } + ] + } + }, + "status": "OK", + "code": 200, + "_postman_previewlanguage": "plain", + "header": [ + { + "key": "x-amz-request-id", + "value": "699c79d678ac85eb" + }, + { + "key": "x-amz-id-2", + "value": "aMb41pGbJOWYzrTUIY0JmFjSgZMljS2L4" + }, + { + "key": "Location", + "value": "/my-bucket-name" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Type", + "value": "" + }, + { + "key": "Content-Length", + "value": "0" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 22:46:26 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "" + } + ] + }, + { + "name": "S3 Delete Bucket", + "request": { + "method": "DELETE", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "{{s3baseurl}}/:bucket", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket" + ], + "variable": [ + { + "key": "bucket", + "value": "", + "description": "Required. Specifies the bucket to delete." + } + ] + }, + "description": "Deletes the bucket specified. Only buckets that contain no versions of any files can be deleted." + }, + "response": [ + { + "name": "S3 Delete Bucket", + "originalRequest": { + "method": "DELETE", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "s3.{{region}}.backblazeb2.com/:bucket", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + ":bucket" + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. Specifies the bucket to delete." + } + ] + } + }, + "status": "No Content", + "code": 204, + "_postman_previewlanguage": "plain", + "header": [ + { + "key": "x-amz-request-id", + "value": "a4c7d533ccb3871b" + }, + { + "key": "x-amz-id-2", + "value": "aMeE1bGaLOXwzCTWSYxtmozRWZGRjvmJy" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Type", + "value": "" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 23:46:27 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "" + } + ] + }, + { + "name": "S3 Delete Objects", + "event": [ + { + "listen": "prerequest", + "script": { + "exec": [ + "// Set Content-MD5 HTTP header with the MD5 hash of the body", + "pm.request.headers.add({", + " key: 'Content-MD5', ", + " value: CryptoJS.MD5(pm.request.body.toString()).toString(CryptoJS.enc.Base64)", + "});" + ], + "type": "text/javascript" + } + } + ], + "request": { + "method": "POST", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "type": "text", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "disabled": true + }, + { + "key": "x-amz-bypass-governance-retention", + "value": "", + "type": "text", + "description": "Optional. Specifies whether the object should be deleted even if there is a governance object lock in place.", + "disabled": true + } + ], + "body": { + "mode": "raw", + "raw": "\n \n key1\n \n \n key2\n versionId\n \n true\n", + "options": { + "raw": { + "language": "xml" + } + } + }, + "url": { + "raw": "{{s3baseurl}}/:bucket/?delete", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket", + "" + ], + "query": [ + { + "key": "delete", + "value": null, + "description": "Must be present to delete multiple objects." + } + ], + "variable": [ + { + "key": "bucket", + "value": "", + "description": "Required. The name of the bucket" + } + ] + }, + "description": "Deletes multiple objects from a bucket in a single request. In the XML body, provide the objects keys and, optionally, version IDs if you want to delete a specific object version." + }, + "response": [ + { + "name": "S3 Delete Objects", + "originalRequest": { + "method": "POST", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "type": "text", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "disabled": true + }, + { + "key": "x-amz-bypass-governance-retention", + "value": "", + "type": "text", + "description": "Optional. Specifies whether the object should be deleted even if there is a governance object lock in place.", + "disabled": true + } + ], + "body": { + "mode": "raw", + "raw": "\n \n object-1\n \n \n object-2\n \n", + "options": { + "raw": { + "language": "xml" + } + } + }, + "url": { + "raw": "s3.{{region}}.backblazeb2.com/:bucket/?delete", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + ":bucket", + "" + ], + "query": [ + { + "key": "delete", + "value": null, + "description": "Must be present to delete multiple objects." + } + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. The name of the bucket" + } + ] + } + }, + "status": "OK", + "code": 200, + "_postman_previewlanguage": "xml", + "header": [ + { + "key": "x-amz-request-id", + "value": "d149d18cdac3d9ca" + }, + { + "key": "x-amz-id-2", + "value": "aMec1XWZmOTAzoTWJY0pmxDTDZDNjxWI4" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Type", + "value": "application/xml" + }, + { + "key": "Content-Length", + "value": "611" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 23:45:22 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "\n\n \n true\n 4_z6145af89f355ac2f74ed0c1b_f401a918cd72e68af_d20220128_m234522_c004_v0402003_t0046\n object-1\n \n \n true\n 4_z6145af89f355ac2f74ed0c1b_f401a918cd72e68b0_d20220128_m234522_c004_v0402002_t0049\n object-2\n \n" + } + ] + }, + { + "name": "S3 List Buckets", + "request": { + "method": "GET", + "header": [], + "url": { + "raw": "{{s3baseurl}}/", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + "" + ] + }, + "description": "Lists buckets in your account in alphabetical order by bucket name." + }, + "response": [ + { + "name": "S3 List Buckets", + "originalRequest": { + "method": "GET", + "header": [], + "url": { + "raw": "s3.{{region}}.backblazeb2.com/", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + "" + ] + } + }, + "status": "OK", + "code": 200, + "_postman_previewlanguage": "xml", + "header": [ + { + "key": "x-amz-request-id", + "value": "51aa803fb55f6d2a" + }, + { + "key": "x-amz-id-2", + "value": "aMYs1j2ZdOU4zaDVuY5Vm5zQWZMljb2LY" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Type", + "value": "application/xml" + }, + { + "key": "Content-Length", + "value": "1414" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 20:43:20 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "\n\n \n 15f935cf4dcb\n \n \n \n \n metadaddy-test\n 2022-01-04T22:52:35.669Z\n \n \n metadaddy-synology\n 2022-01-25T04:23:07.343Z\n \n \n" + } + ] + }, + { + "name": "S3 List Object Versions", + "request": { + "method": "GET", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "{{s3baseurl}}/:bucket/?versions", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket", + "" + ], + "query": [ + { + "key": "versions", + "value": null, + "description": "Must be present to retrieve object versions" + }, + { + "key": "delimiter", + "value": "", + "description": "Optional. The delimiter is used to group keys.", + "disabled": true + }, + { + "key": "encoding-type", + "value": "", + "description": "Optional. Specifies an encoding method to use for characters that are not supported in XML 1.0.", + "disabled": true + }, + { + "key": "key-marker", + "value": "", + "description": "Optional. The key that you want to start listing from.", + "disabled": true + }, + { + "key": "max-keys", + "value": "", + "description": "Optional. The maximum number of keys that should be returned.", + "disabled": true + }, + { + "key": "prefix", + "value": "", + "description": "Optional. Limit response to keys with this prefix.", + "disabled": true + }, + { + "key": "version-id-marker", + "value": null, + "description": "Optional. The object version id that you want to start listing from.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "", + "description": "Required. The name of the bucket" + } + ] + }, + "description": "Returns metadata of the versions of objects in the bucket. Use the query parameters to filter the search." + }, + "response": [ + { + "name": "S3 List Object Versions", + "originalRequest": { + "method": "GET", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "s3.{{region}}.backblazeb2.com/:bucket/?versions", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + ":bucket", + "" + ], + "query": [ + { + "key": "delimiter", + "value": "", + "description": "Optional. The delimiter is used to group keys.", + "disabled": true + }, + { + "key": "encoding-type", + "value": "", + "description": "Optional. Specifies an encoding method to use for characters that are not supported in XML 1.0.", + "disabled": true + }, + { + "key": "key-marker", + "value": "", + "description": "Optional. The key that you want to start listing from.", + "disabled": true + }, + { + "key": "max-keys", + "value": "", + "description": "Optional. The maximum number of keys that should be returned.", + "disabled": true + }, + { + "key": "prefix", + "value": "", + "description": "Optional. Limit response to keys with this prefix.", + "disabled": true + }, + { + "key": "version-id-marker", + "value": null, + "description": "Optional. The object version id that you want to start listing from.", + "disabled": true + }, + { + "key": "versions", + "value": null + } + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. The name of the bucket" + } + ] + } + }, + "status": "OK", + "code": 200, + "_postman_previewlanguage": "xml", + "header": [ + { + "key": "x-amz-request-id", + "value": "3075a9560f9d6f1d" + }, + { + "key": "x-amz-id-2", + "value": "aMWk1vGZLOdYz9jV4Y5lmcjRnZCVjVWIE" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Type", + "value": "application/xml" + }, + { + "key": "Content-Length", + "value": "2646" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 23:41:05 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "\n\n \n true\n object-2\n 2022-01-28T23:40:31.000Z\n \n 15f935cf4dcb\n \n \n 4_z6145af89f355ac2f74ed0c1b_f40057de773908ada_d20220128_m234031_c004_v0402002_t0050\n \n false\n 1000\n my-bucket-name\n \n \"85f30635602dc09bd85957a6e82a2c21\"\n true\n object-1\n 2022-01-28T23:24:07.000Z\n \n 15f935cf4dcb\n \n \n 11\n STANDARD\n 4_z6145af89f355ac2f74ed0c1b_f4030346ce9e208f0_d20220128_m232407_c004_v0402005_t0021\n \n \n \"85f30635602dc09bd85957a6e82a2c21\"\n false\n object-1\n 2022-01-28T23:12:33.000Z\n \n 15f935cf4dcb\n \n \n 11\n STANDARD\n 4_z6145af89f355ac2f74ed0c1b_f416a807037a7ee2a_d20220128_m231233_c004_v0402000_t0056\n \n" + } + ] + }, + { + "name": "S3 List Objects", + "request": { + "method": "GET", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "{{s3baseurl}}/:bucket/", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket", + "" + ], + "query": [ + { + "key": "delimiter", + "value": "", + "description": "Optional. The delimiter is used to group keys.", + "disabled": true + }, + { + "key": "encoding-type", + "value": "", + "description": "Optional. Specifies an encoding method to use for characters that are not supported in XML 1.0.", + "disabled": true + }, + { + "key": "marker", + "value": "", + "description": "Optional. The key that you want to start listing from.", + "disabled": true + }, + { + "key": "max-keys", + "value": "", + "description": "Optional. The maximum number of keys that should be returned.", + "disabled": true + }, + { + "key": "prefix", + "value": "", + "description": "Optional. Limit response to keys with this prefix.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "home", + "description": "Required. The name of the bucket" + } + ] + }, + "description": "Returns a list of up to 1,000 objects in the bucket, sorted alphabetically by key. Use the query parameters to filter the search.\n\nNote - **S3 List Objects V2** (below) should be used in preference to this operation, since S3 List Objects has no mechanism to page through more than 1,000 results." + }, + "response": [ + { + "name": "S3 List Objects", + "originalRequest": { + "method": "GET", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "s3.{{region}}.backblazeb2.com/:bucket/", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + ":bucket", + "" + ], + "query": [ + { + "key": "delimiter", + "value": "", + "description": "Optional. The delimiter is used to group keys.", + "disabled": true + }, + { + "key": "encoding-type", + "value": "", + "description": "Optional. Specifies an encoding method to use for characters that are not supported in XML 1.0.", + "disabled": true + }, + { + "key": "marker", + "value": "", + "description": "Optional. The key that you want to start listing from.", + "disabled": true + }, + { + "key": "max-keys", + "value": "", + "description": "Optional. The maximum number of keys that should be returned.", + "disabled": true + }, + { + "key": "prefix", + "value": "", + "description": "Optional. Limit response to keys with this prefix.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. The name of the bucket" + } + ] + } + }, + "status": "OK", + "code": 200, + "_postman_previewlanguage": "xml", + "header": [ + { + "key": "x-amz-request-id", + "value": "87e06e4dfe6ac590" + }, + { + "key": "x-amz-id-2", + "value": "aMaw1+2ZLOZwzhTU/Y9VmpzQHZHxjKWK7" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Type", + "value": "application/xml" + }, + { + "key": "Content-Length", + "value": "1358" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 23:42:28 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "\n\n \n \"85f30635602dc09bd85957a6e82a2c21\"\n object-1\n 2022-01-28T23:24:07.000Z\n \n 15f935cf4dcb\n \n \n 11\n STANDARD\n \n \n \"85f30635602dc09bd85957a6e82a2c21\"\n object-2\n 2022-01-28T23:25:45.000Z\n \n 15f935cf4dcb\n \n \n 11\n STANDARD\n \n false\n 1000\n my-bucket-name\n \n \n" + } + ] + }, + { + "name": "S3 List Objects V2", + "request": { + "method": "GET", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "{{s3baseurl}}/:bucket/?list-type=2", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket", + "" + ], + "query": [ + { + "key": "list-type", + "value": "2", + "description": "Must be set to 2 for V2 object list" + }, + { + "key": "continuation-token", + "value": "", + "description": "Optional. Continuation token returned by a previous request.", + "disabled": true + }, + { + "key": "delimiter", + "value": "", + "description": "Optional. The delimiter is used to group keys.", + "disabled": true + }, + { + "key": "encoding-type", + "value": "", + "description": "Optional. Specifies an encoding method to use for characters that are not supported in XML 1.0.", + "disabled": true + }, + { + "key": "fetch-owner", + "value": "", + "description": "Optional. Set to 'true' to return the owner with each key.", + "disabled": true + }, + { + "key": "max-keys", + "value": "", + "description": "Optional. The maximum number of keys that should be returned.", + "disabled": true + }, + { + "key": "prefix", + "value": "", + "description": "Optional. Limit response to keys with this prefix.", + "disabled": true + }, + { + "key": "start-after", + "value": "", + "description": "Optional. B2 will start listing after this key.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "home", + "description": "Required. The name of the bucket" + } + ] + }, + "description": "Returns a list of up to 1,000 objects in the bucket, sorted alphabetically by key.\n\nUse the query parameters to filter the search.\n\nBy default, sets of up to 1,000 results are returned. You can set the `max-keys` to set the maximum result set size to less than 1,000. If the search results in more than the maximum result set size, then the first set is returned in the initial response, the `` response element contains the value `true` and the `` element contains a token to retrieve the next set of results. Use this token as the `continuation-token` query parameter in another request to retrieve the next set of results." + }, + "response": [ + { + "name": "S3 List Objects V2", + "originalRequest": { + "method": "GET", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "s3.{{region}}.backblazeb2.com/:bucket/?list-type=2", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + ":bucket", + "" + ], + "query": [ + { + "key": "list-type", + "value": "2", + "description": "Must be set to 2 for V2 object list" + }, + { + "key": "continuation-token", + "value": "", + "description": "Optional. Continuation token returned by a previous request.", + "disabled": true + }, + { + "key": "delimiter", + "value": "", + "description": "Optional. The delimiter is used to group keys.", + "disabled": true + }, + { + "key": "encoding-type", + "value": "", + "description": "Optional. Specifies an encoding method to use for characters that are not supported in XML 1.0.", + "disabled": true + }, + { + "key": "fetch-owner", + "value": "", + "description": "Optional. Set to 'true' to return the owner with each key.", + "disabled": true + }, + { + "key": "max-keys", + "value": "", + "description": "Optional. The maximum number of keys that should be returned.", + "disabled": true + }, + { + "key": "prefix", + "value": "", + "description": "Optional. Limit response to keys with this prefix.", + "disabled": true + }, + { + "key": "start-after", + "value": "", + "description": "Optional. B2 will start listing after this key.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. The name of the bucket" + } + ] + } + }, + "status": "OK", + "code": 200, + "_postman_previewlanguage": "xml", + "header": [ + { + "key": "x-amz-request-id", + "value": "fbaa4aa26f4ad37b" + }, + { + "key": "x-amz-id-2", + "value": "aMSI102bnOUEz/zU1Y9lmtTR4ZCpj02IQ" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Type", + "value": "application/xml" + }, + { + "key": "Content-Length", + "value": "1042" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 23:43:55 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "\n\n \n \"85f30635602dc09bd85957a6e82a2c21\"\n object-1\n 2022-01-28T23:24:07.000Z\n 11\n STANDARD\n \n \n \"85f30635602dc09bd85957a6e82a2c21\"\n object-2\n 2022-01-28T23:25:45.000Z\n 11\n STANDARD\n \n \n \"3729bf569924c845dfe90bfe6281a9ff-1\"\n object-3\n 2022-01-28T23:27:20.000Z\n 11\n STANDARD\n \n false\n 1000\n my-bucket-name\n \n 3\n" + } + ] + } + ], + "description": "This set of operations allows you to manipulate B2 buckets.\n\nThere is no limit to the number of files in a bucket, but there is a limit of 100 buckets per account.\n\nAside from S3 List Buckets, all bucket operations use an endpoint of the form `\nhttps://s3..backblazeb2.com/`." + }, + { + "name": "Object Operations", + "item": [ + { + "name": "S3 Delete Object", + "event": [ + { + "listen": "prerequest", + "script": { + "exec": [ + "" + ], + "type": "text/javascript" + } + } + ], + "request": { + "method": "DELETE", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-bypass-governance-retention", + "value": "", + "description": "Optional. Specifies whether the object should be deleted even if there is a governance object lock in place.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "{{s3baseurl}}/:bucket/:key", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket", + ":key" + ], + "query": [ + { + "key": "versionId", + "value": "", + "description": "VersionId to delete.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "", + "description": "Required. The name of the bucket" + }, + { + "key": "key", + "value": "", + "description": "Required. Key of the object to read." + } + ] + } + }, + "response": [ + { + "name": "S3 Delete Object", + "originalRequest": { + "method": "DELETE", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-bypass-governance-retention", + "value": "", + "description": "Optional. Specifies whether the object should be deleted even if there is a governance object lock in place.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "s3.{{region}}.backblazeb2.com/:bucket/:key", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + ":bucket", + ":key" + ], + "query": [ + { + "key": "versionId", + "value": "", + "description": "VersionId to delete.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. The name of the bucket" + }, + { + "key": "key", + "value": "object-1", + "description": "Required. Key of the object to read." + } + ] + } + }, + "status": "No Content", + "code": 204, + "_postman_previewlanguage": "plain", + "header": [ + { + "key": "x-amz-request-id", + "value": "462500c9d699ed92" + }, + { + "key": "x-amz-id-2", + "value": "aMR01LmYuOegzLTUBY5hmJjQOZKdjfGJC" + }, + { + "key": "x-amz-delete-marker", + "value": "true" + }, + { + "key": "x-amz-version-id", + "value": "4_z6145af89f355ac2f74ed0c1b_f40057de773908ada_d20220128_m234031_c004_v0402002_t0050" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Type", + "value": "" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 23:40:31 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "" + } + ] + }, + { + "name": "S3 Get Object", + "event": [ + { + "listen": "prerequest", + "script": { + "exec": [ + "// Set keyMd5 HTTP header with the MD5 hash of the key", + "var key = pm.request.headers.get('x-amz-server-side-encryption-customer-key');", + "if (key) {", + " pm.request.headers.add({", + " key: 'keyMd5', ", + " value: CryptoJS.MD5(CryptoJS.enc.Base64.parse(key)).toString(CryptoJS.enc.Base64)", + " });", + "}" + ], + "type": "text/javascript" + } + } + ], + "request": { + "method": "GET", + "header": [ + { + "key": "Range", + "value": "", + "description": "Optional. The HTTP range of bytes required, for example, bytes=0-999 ", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-algorithm", + "value": "", + "description": "Optional. Encryption algorithm to be used for SSE-C.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-key", + "value": "", + "description": "Optional. Base64-encoded customer key for SSE-C.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "{{s3baseurl}}/:bucket/:key", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket", + ":key" + ], + "query": [ + { + "key": "partNumber", + "value": "", + "description": "Part number to read, between 1 and 10,000.", + "disabled": true + }, + { + "key": "response-cache-control", + "value": "", + "description": "Specifies Cache-Control header for the response.", + "disabled": true + }, + { + "key": "response-content-encoding", + "value": "", + "description": "Specifies Content-Encoding header for the response.", + "disabled": true + }, + { + "key": "response-content-language", + "value": "", + "description": "Specifies Content-Language header for the response.", + "disabled": true + }, + { + "key": "response-content-type", + "value": "", + "description": "Specifies Content-Type header for the response.", + "disabled": true + }, + { + "key": "response-expires", + "value": "", + "description": "Specifies Expires header for the response.", + "disabled": true + }, + { + "key": "versionId", + "value": "", + "description": "VersionId to read.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "2home", + "description": "Required. The name of the bucket" + }, + { + "key": "key", + "value": "license2", + "description": "Required. Key of the object to read." + } + ] + } + }, + "response": [ + { + "name": "S3 Get Object", + "originalRequest": { + "method": "GET", + "header": [ + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-algorithm", + "value": "AES256", + "description": "Optional. Encryption algorithm to be used for SSE-C.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-key", + "value": "kmxlnaH92TEckb+8giJF8qPR5lPY5NDBsIpQQZwAn7A=", + "description": "Optional. Base64-encoded customer key for SSE-C.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-key-MD5", + "value": "{{keyMd5}}", + "description": "Optional. Base64-encoded MD5 hash of the customer key for SSE-C. Calculated in pre-request script.", + "type": "text", + "disabled": true + } + ], + "url": { + "raw": "s3.{{region}}.backblazeb2.com/:bucket/:key", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + ":bucket", + ":key" + ], + "query": [ + { + "key": "partNumber", + "value": "", + "description": "Part number to read, between 1 and 10,000.", + "disabled": true + }, + { + "key": "response-cache-control", + "value": "", + "description": "Specifies Cache-Control header for the response.", + "disabled": true + }, + { + "key": "response-content-encoding", + "value": "", + "description": "Specifies Content-Encoding header for the response.", + "disabled": true + }, + { + "key": "response-content-language", + "value": "", + "description": "Specifies Content-Language header for the response.", + "disabled": true + }, + { + "key": "response-content-type", + "value": "", + "description": "Specifies Content-Type header for the response.", + "disabled": true + }, + { + "key": "response-expires", + "value": "", + "description": "Specifies Expires header for the response.", + "disabled": true + }, + { + "key": "versionId", + "value": "", + "description": "VersionId to read.", + "disabled": true + } + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. The name of the bucket" + }, + { + "key": "key", + "value": "object-1", + "description": "Required. Key of the object to read." + } + ] + } + }, + "status": "OK", + "code": 200, + "_postman_previewlanguage": "plain", + "header": [ + { + "key": "Accept-Ranges", + "value": "bytes" + }, + { + "key": "Last-Modified", + "value": "Fri, 28 Jan 2022 23:12:33 GMT" + }, + { + "key": "ETag", + "value": "\"85f30635602dc09bd85957a6e82a2c21\"" + }, + { + "key": "Cache-Control", + "value": "no-cache" + }, + { + "key": "x-amz-request-id", + "value": "22f4a1772746da99" + }, + { + "key": "x-amz-id-2", + "value": "aMWA1NmaNOcQz7zWKY2lmZzT6ZHdjpGJT" + }, + { + "key": "x-amz-version-id", + "value": "4_z6145af89f355ac2f74ed0c1b_f416a807037a7ee2a_d20220128_m231233_c004_v0402000_t0056" + }, + { + "key": "Content-Type", + "value": "text/plain" + }, + { + "key": "Content-Length", + "value": "11" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 23:14:25 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "object data" + } + ] + }, + { + "name": "S3 Put Object", + "event": [ + { + "listen": "prerequest", + "script": { + "exec": [ + "// // Set keyMd5 HTTP header with the MD5 hash of the key", + "// var key = pm.request.headers.get('x-amz-server-side-encryption-customer-key');", + "// if (key) {", + "// pm.request.headers.add({", + "// key: 'x-amz-server-side-encryption-customer-key-MD5', ", + "// value: CryptoJS.MD5(CryptoJS.enc.Base64.parse(key)).toString(CryptoJS.enc.Base64)", + "// });", + "// }", + "", + "// // Set contentMd5 HTTP header with the MD5 hash of the body if object lock is set", + "// var retainUntil = pm.request.headers.get('x-amz-object-lock-retain-until-date');", + "// if (retainUntil) {", + "// pm.request.headers.add({", + "// key: 'Content-MD5', ", + "// value: CryptoJS.MD5(pm.request.body.toString()).toString(CryptoJS.enc.Base64)", + "// });", + "// }" + ], + "type": "text/javascript" + } + } + ], + "request": { + "method": "PUT", + "header": [ + { + "key": "Cache-Control", + "value": "", + "description": "Optional. Specifies Cache-Control header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Disposition", + "value": "", + "description": "Optional. Specifies Content-Disposition header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Encoding", + "value": "", + "description": "Optional. Specifies Content-Encoding header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Language", + "value": "", + "description": "Optional. Specifies Content-Encoding header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Length", + "value": "", + "description": "Optional. Specifies Content-Length header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Type", + "value": "", + "description": "Optional. Specifies Content-Type header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Expires", + "value": "", + "description": "Optional. Specifies Expires header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-acl", + "value": "", + "type": "text", + "description": "Optional. If supplied, must be the same as the bucket ACL.", + "disabled": true + }, + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "type": "text", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "disabled": true + }, + { + "key": "x-amz-object-lock-legal-hold", + "value": "", + "type": "text", + "description": "Optional. Specifies whether a legal hold will be applied. Allowed values are 'ON' or 'OFF'.", + "disabled": true + }, + { + "key": "x-amz-object-lock-mode", + "value": "", + "type": "text", + "description": "Optional. The lock mode to apply. Allowed values are 'GOVERNANCE' or 'COMPLIANCE'.", + "disabled": true + }, + { + "key": "x-amz-object-lock-retain-until-date", + "value": "", + "type": "text", + "description": "Optional. Timestamp for expiration of object lock.", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-algorithm", + "value": "", + "type": "text", + "description": "Optional. Encryption algorithm to be used for SSE-C.", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-key", + "value": "", + "type": "text", + "description": "Optional. Base64-encoded customer key for SSE-C.", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption", + "value": "", + "description": "Optional. Set to AES256 to use SSE-B2.", + "type": "text", + "disabled": true + } + ], + "body": { + "mode": "file", + "file": { + "src": "/Users/lakshmi/Documents/work/mc/staticcheck.conf" + } + }, + "url": { + "raw": "{{s3baseurl}}/:bucket/:key?multiupload=true", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket", + ":key" + ], + "query": [ + { + "key": "multiupload", + "value": "true" + } + ], + "variable": [ + { + "key": "bucket", + "value": "buck1" + }, + { + "key": "key", + "value": "nest1/myfile.conf" + } + ] + } + }, + "response": [ + { + "name": "S3 Put Object", + "originalRequest": { + "method": "PUT", + "header": [ + { + "key": "Cache-Control", + "value": "", + "description": "Optional. Specifies Cache-Control header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Disposition", + "value": "", + "description": "Optional. Specifies Content-Disposition header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Encoding", + "value": "", + "description": "Optional. Specifies Content-Encoding header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Language", + "value": "", + "description": "Optional. Specifies Content-Encoding header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Length", + "value": "", + "description": "Optional. Specifies Content-Length header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Content-Type", + "value": "", + "description": "Optional. Specifies Content-Type header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "Expires", + "value": "", + "description": "Optional. Specifies Expires header on retrieval.", + "type": "text", + "disabled": true + }, + { + "key": "x-amz-acl", + "value": "", + "type": "text", + "description": "Optional. If supplied, must be the same as the bucket ACL.", + "disabled": true + }, + { + "key": "x-amz-expected-bucket-owner", + "value": "", + "type": "text", + "description": "Optional. Account ID of the expected bucket owner. If the bucket is owned by a different owner, a 403 Access Denied error will be returned.", + "disabled": true + }, + { + "key": "x-amz-object-lock-legal-hold", + "value": "", + "type": "text", + "description": "Optional. Specifies whether a legal hold will be applied. Allowed values are 'ON' or 'OFF'.", + "disabled": true + }, + { + "key": "x-amz-object-lock-mode", + "value": "", + "type": "text", + "description": "Optional. The lock mode to apply. Allowed values are 'GOVERNANCE' or 'COMPLIANCE'.", + "disabled": true + }, + { + "key": "x-amz-object-lock-retain-until-date", + "value": "", + "type": "text", + "description": "Optional. Timestamp for expiration of object lock.", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-algorithm", + "value": "", + "type": "text", + "description": "Optional. Encryption algorithm to be used for SSE-C.", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption-customer-key", + "value": "", + "type": "text", + "description": "Optional. Base64-encoded customer key for SSE-C.", + "disabled": true + }, + { + "key": "x-amz-server-side-encryption", + "value": "", + "description": "Optional. Set to AES256 to use SSE-B2.", + "type": "text", + "disabled": true + } + ], + "body": { + "mode": "raw", + "raw": "object data" + }, + "url": { + "raw": "s3.{{region}}.backblazeb2.com/:bucket/:key", + "host": [ + "s3", + "{{region}}", + "backblazeb2", + "com" + ], + "path": [ + ":bucket", + ":key" + ], + "variable": [ + { + "key": "bucket", + "value": "my-bucket-name", + "description": "Required. The name of the bucket." + }, + { + "key": "key", + "value": "object-1", + "description": "Required. Key of the object to be written." + } + ] + } + }, + "status": "OK", + "code": 200, + "_postman_previewlanguage": "plain", + "header": [ + { + "key": "x-amz-request-id", + "value": "4d38f7d0978e0ed5" + }, + { + "key": "x-amz-id-2", + "value": "aMaM1sGZVOSAzPDVLYy9mJDTIZFVjImI7" + }, + { + "key": "ETag", + "value": "\"85f30635602dc09bd85957a6e82a2c21\"" + }, + { + "key": "x-amz-version-id", + "value": "4_z6145af89f355ac2f74ed0c1b_f416a807037a7ee2a_d20220128_m231233_c004_v0402000_t0056" + }, + { + "key": "Cache-Control", + "value": "max-age=0, no-cache, no-store" + }, + { + "key": "Content-Length", + "value": "0" + }, + { + "key": "Date", + "value": "Fri, 28 Jan 2022 23:12:33 GMT" + }, + { + "key": "Keep-Alive", + "value": "timeout=5" + }, + { + "key": "Connection", + "value": "keep-alive" + } + ], + "cookie": [], + "body": "" + } + ] + }, + { + "name": "S3 PutMultipleObject", + "event": [ + { + "listen": "prerequest", + "script": { + "exec": [ + "// // Set keyMd5 HTTP header with the MD5 hash of the key", + "// var key = pm.request.headers.get('x-amz-server-side-encryption-customer-key');", + "// if (key) {", + "// pm.request.headers.add({", + "// key: 'x-amz-server-side-encryption-customer-key-MD5', ", + "// value: CryptoJS.MD5(CryptoJS.enc.Base64.parse(key)).toString(CryptoJS.enc.Base64)", + "// });", + "// }", + "", + "// // Set contentMd5 HTTP header with the MD5 hash of the body if object lock is set", + "// var retainUntil = pm.request.headers.get('x-amz-object-lock-retain-until-date');", + "// if (retainUntil) {", + "// pm.request.headers.add({", + "// key: 'Content-MD5', ", + "// value: CryptoJS.MD5(pm.request.body.toString()).toString(CryptoJS.enc.Base64)", + "// });", + "// }" + ], + "type": "text/javascript" + } + } + ], + "request": { + "method": "PUT", + "header": [], + "body": { + "mode": "formdata", + "formdata": [ + { + "key": "doublefile", + "type": "file", + "src": "/Users/lakshmi/Documents/work/zboxcli/main.go" + }, + { + "key": "doublefile2", + "type": "file", + "src": "/Users/lakshmi/Documents/work/zboxcli/README.md" + } + ] + }, + "url": { + "raw": "{{s3baseurl}}/:bucket?multiupload=true", + "host": [ + "{{s3baseurl}}" + ], + "path": [ + ":bucket" + ], + "query": [ + { + "key": "multiupload", + "value": "true" + } + ], + "variable": [ + { + "key": "bucket", + "value": "wm6" + } + ] + } + }, + "response": [] + } + ] + } + ], + "auth": { + "type": "awsv4", + "awsv4": [ + { + "key": "service", + "value": "s3", + "type": "string" + }, + { + "key": "region", + "value": "", + "type": "string" + }, + { + "key": "secretKey", + "value": "{{applicationKey}}", + "type": "string" + }, + { + "key": "accessKey", + "value": "{{applicationKeyId}}", + "type": "string" + } + ] + }, + "event": [ + { + "listen": "prerequest", + "script": { + "type": "text/javascript", + "exec": [ + "" + ] + } + }, + { + "listen": "test", + "script": { + "type": "text/javascript", + "exec": [ + "" + ] + } + } + ], + "variable": [ + { + "key": "applicationKeyId", + "value": "", + "type": "string" + }, + { + "key": "applicationKey", + "value": "", + "type": "string" + }, + { + "key": "s3baseurl", + "value": "", + "type": "string" + } + ] +} \ No newline at end of file diff --git a/assets/postman-auth.png b/assets/postman-auth.png new file mode 100644 index 000000000..5a89ec612 Binary files /dev/null and b/assets/postman-auth.png differ diff --git a/client-api/s3api/put_object.go b/client-api/s3api/put_object.go index 326a0f0c8..4cdf636fc 100644 --- a/client-api/s3api/put_object.go +++ b/client-api/s3api/put_object.go @@ -40,7 +40,9 @@ func putObject(bucketName string, objectName string, minioCredentials MinioCrede return putObjectResponse, err } - uploadInfo, err := minioClient.PutObject(ctx, bucketName, objectName, file, fileStat.Size(), minio.PutObjectOptions{ContentType: "application/octet-stream"}) + // TODO: Enable multi part once it is supported. + // Disabling multi part as of now because multipart APIs are not implemented for zcn gateway. + uploadInfo, err := minioClient.PutObject(ctx, bucketName, objectName, file, fileStat.Size(), minio.PutObjectOptions{DisableMultipart: true,ContentType: "application/octet-stream"}) if err != nil { fmt.Println("error from PutObject", err) return putObjectResponse, err diff --git a/cmd/gateway/zcn/dStorage.go b/cmd/gateway/zcn/dStorage.go index 75ff45f14..238957b83 100644 --- a/cmd/gateway/zcn/dStorage.go +++ b/cmd/gateway/zcn/dStorage.go @@ -11,8 +11,8 @@ import ( "time" zerror "github.com/0chain/errors" + "github.com/0chain/gosdk/constants" "github.com/0chain/gosdk/zboxcore/sdk" - "github.com/0chain/gosdk/zboxcore/zboxutil" "github.com/minio/minio/internal/logger" "github.com/mitchellh/go-homedir" ) @@ -200,11 +200,6 @@ func getFileReader(ctx context.Context, alloc *sdk.Allocation, remotePath string func putFile(ctx context.Context, alloc *sdk.Allocation, remotePath, contentType string, r io.Reader, size int64, isUpdate, shouldEncrypt bool) (err error) { logger.Info("started PutFile") - cb := &statusCB{ - doneCh: make(chan struct{}, 1), - errCh: make(chan error, 1), - } - _, fileName := filepath.Split(remotePath) fileMeta := sdk.FileMeta{ Path: "", @@ -220,28 +215,24 @@ func putFile(ctx context.Context, alloc *sdk.Allocation, remotePath, contentType return err } - logger.Info("creating chunked upload") - chunkUpload, err := sdk.CreateChunkedUpload(workDir, alloc, fileMeta, newMinioReader(r), isUpdate, false, false, zboxutil.NewConnectionId(), - sdk.WithStatusCallback(cb), - ) - - if err != nil { - logger.Error(err.Error()) - return + logger.Info("starting chunked upload") + opRequest := sdk.OperationRequest{ + OperationType: constants.FileOperationInsert, + FileReader: newMinioReader(r), + Workdir: workDir, + RemotePath: remotePath, + FileMeta: fileMeta, + Opts: []sdk.ChunkedUploadOption{ + sdk.WithChunkNumber(250), + }, } - err = chunkUpload.Start() + err = alloc.DoMultiOperation([]sdk.OperationRequest{opRequest}) if err != nil { - logger.Info("error from PutFile") logger.Error(err.Error()) return } - select { - case <-cb.doneCh: - case err = <-cb.errCh: - } - return } diff --git a/cmd/gateway/zcn/gateway-zcn.go b/cmd/gateway/zcn/gateway-zcn.go index 176c50744..8171efbc6 100644 --- a/cmd/gateway/zcn/gateway-zcn.go +++ b/cmd/gateway/zcn/gateway-zcn.go @@ -4,8 +4,6 @@ import ( "context" "errors" "fmt" - "github.com/0chain/gosdk/constants" - "github.com/minio/minio/internal/logger" "io" "net/http" "os" @@ -14,6 +12,9 @@ import ( "sync" "time" + "github.com/0chain/gosdk/constants" + "github.com/minio/minio/internal/logger" + "github.com/0chain/gosdk/zboxcore/sdk" "github.com/minio/cli" "github.com/minio/madmin-go" @@ -493,7 +494,13 @@ func getRelativePathOfObj(refPath, bucketName string) string { func (zob *zcnObjects) MakeBucketWithLocation(ctx context.Context, bucket string, opts minio.BucketOptions) error { // Create a directory; ignore opts remotePath := filepath.Join(rootPath, bucket) - return zob.alloc.CreateDir(remotePath) + createDirOp := sdk.OperationRequest{ + OperationType: constants.FileOperationCreateDir, + RemotePath: remotePath, + } + return zob.alloc.DoMultiOperation([]sdk.OperationRequest{ + createDirOp, + }) } func (zob *zcnObjects) PutObject(ctx context.Context, bucket, object string, r *minio.PutObjReader, opts minio.ObjectOptions) (objInfo minio.ObjectInfo, err error) { @@ -592,6 +599,7 @@ func (zob *zcnObjects) PutMultipleObjects( options := []sdk.ChunkedUploadOption{ sdk.WithEncrypt(false), + sdk.WithChunkNumber(200), } operationRequests[idx] = sdk.OperationRequest{ FileMeta: fileMeta, @@ -640,8 +648,14 @@ func (zob *zcnObjects) CopyObject(ctx context.Context, srcBucket, srcObject, des } else { dstRemotePath = filepath.Join(rootPath, destBucket, destObject) } - - err = zob.alloc.CopyObject(srcRemotePath, dstRemotePath) + copyOp := sdk.OperationRequest{ + OperationType: constants.FileOperationCopy, + RemotePath: srcRemotePath, + DestPath: dstRemotePath, + } + err = zob.alloc.DoMultiOperation([]sdk.OperationRequest{ + copyOp, + }) if err != nil { return } diff --git a/docker.local/base.Dockerfile b/docker.local/base.Dockerfile index 11d27cc7b..72f35dacf 100644 --- a/docker.local/base.Dockerfile +++ b/docker.local/base.Dockerfile @@ -3,6 +3,19 @@ FROM golang:1.20.4-alpine3.18 as zbox_base RUN apk add --update --no-cache linux-headers build-base git cmake bash perl grep py3-pip python3-dev curl RUN apk upgrade -# Install Herumi's cryptography RUN apk del libstdc++ gmp-dev openssl-dev vips-dev RUN apk add --update --no-cache --repository http://dl-cdn.alpinelinux.org/alpine/edge/main libstdc++ gmp-dev openssl-dev vips-dev + +# Install Herumi's cryptography +RUN cd /tmp && \ + wget -O - https://github.com/herumi/mcl/archive/refs/tags/v1.81.tar.gz | tar xz && \ + wget -O - https://github.com/herumi/bls/archive/refs/tags/v1.35.tar.gz | tar xz && \ + mv mcl* mcl && \ + mv bls* bls + +RUN cd /tmp && \ + make -C mcl -j $(nproc) lib/libmclbn256.so install && \ + cp mcl/lib/libmclbn256.so /usr/local/lib && \ + make MCL_DIR=$(pwd)/mcl -C bls -j $(nproc) install && \ + rm -R /tmp/mcl && \ + rm -R /tmp/bls \ No newline at end of file diff --git a/go.mod b/go.mod index fabe02ac4..2a165f7e9 100644 --- a/go.mod +++ b/go.mod @@ -5,7 +5,7 @@ go 1.20 require ( cloud.google.com/go/storage v1.27.0 github.com/0chain/errors v1.0.3 - github.com/0chain/gosdk v1.8.18-0.20230901213317-53d640a9b7f9 + github.com/0chain/gosdk v1.10.1-0.20231202122557-ee87ed91fcf2 github.com/Azure/azure-pipeline-go v0.2.2 github.com/Azure/azure-storage-blob-go v0.10.0 github.com/Shopify/sarama v1.28.0 @@ -43,7 +43,7 @@ require ( github.com/klauspost/cpuid/v2 v2.2.4 github.com/klauspost/pgzip v1.2.5 github.com/klauspost/readahead v1.4.0 - github.com/klauspost/reedsolomon v1.11.7 + github.com/klauspost/reedsolomon v1.11.8 github.com/lib/pq v1.9.0 github.com/miekg/dns v1.1.43 github.com/minio/cli v1.22.0 @@ -56,7 +56,7 @@ require ( github.com/minio/parquet-go v1.1.0 github.com/minio/pkg v1.1.14 github.com/minio/selfupdate v0.4.0 - github.com/minio/sha256-simd v1.0.0 + github.com/minio/sha256-simd v1.0.1 github.com/minio/simdjson-go v0.2.1 github.com/minio/sio v0.3.0 github.com/minio/zipindex v0.2.1 @@ -208,6 +208,7 @@ require ( github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021 // indirect github.com/prometheus/common v0.39.0 // indirect github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect + github.com/remeh/sizedwaitgroup v1.0.0 // indirect github.com/rivo/uniseg v0.4.3 // indirect github.com/rjeczalik/notify v0.9.2 // indirect github.com/rs/xid v1.3.0 // indirect @@ -250,6 +251,7 @@ require ( gopkg.in/h2non/filetype.v1 v1.0.5 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/ini.v1 v1.67.0 // indirect + gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce // indirect gopkg.in/square/go-jose.v2 v2.3.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect diff --git a/go.sum b/go.sum index 0ea79d120..941dfe7f6 100644 --- a/go.sum +++ b/go.sum @@ -62,8 +62,8 @@ github.com/0chain/common v0.0.6-0.20230127095721-8df4d1d72565 h1:z+DtCR8mBsjPnEs github.com/0chain/common v0.0.6-0.20230127095721-8df4d1d72565/go.mod h1:UyDC8Qyl5z9lGkCnf9RHJPMektnFX8XtCJZHXCCVj8E= github.com/0chain/errors v1.0.3 h1:QQZPFxTfnMcRdt32DXbzRQIfGWmBsKoEdszKQDb0rRM= github.com/0chain/errors v1.0.3/go.mod h1:xymD6nVgrbgttWwkpSCfLLEJbFO6iHGQwk/yeSuYkIc= -github.com/0chain/gosdk v1.8.18-0.20230901213317-53d640a9b7f9 h1:GHTdYTmhNY9genBkNWLXdn3Z1yCtcbSNkcIFaKrqBRU= -github.com/0chain/gosdk v1.8.18-0.20230901213317-53d640a9b7f9/go.mod h1:3NKNYzmnMIYqZwwwOgZwMmTW1DT1ZUAmKyVPmYQOiT4= +github.com/0chain/gosdk v1.10.1-0.20231202122557-ee87ed91fcf2 h1:pkePFrvGORhJE7jI25yBb+ytBYkfKHmw8uMT98vySv4= +github.com/0chain/gosdk v1.10.1-0.20231202122557-ee87ed91fcf2/go.mod h1:DAg/de6vodjEa7CM1/LjElOwntRtNV5lb9rMRaR7fzU= github.com/Azure/azure-amqp-common-go/v2 v2.1.0/go.mod h1:R8rea+gJRuJR6QxTir/XuEd+YuKoUiazDC/N96FiDEU= github.com/Azure/azure-pipeline-go v0.2.1/go.mod h1:UGSo8XybXnIGZ3epmeBw7Jdz+HiUVpqIlpz/HKHylF4= github.com/Azure/azure-pipeline-go v0.2.2 h1:6oiIS9yaG6XCCzhgAgKFfIWyo4LLCiDhZot6ltoThhY= @@ -1030,8 +1030,8 @@ github.com/klauspost/pgzip v1.2.5 h1:qnWYvvKqedOF2ulHpMG72XQol4ILEJ8k2wwRl/Km8oE github.com/klauspost/pgzip v1.2.5/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs= github.com/klauspost/readahead v1.4.0 h1:w4hQ3BpdLjBnRQkZyNi+nwdHU7eGP9buTexWK9lU7gY= github.com/klauspost/readahead v1.4.0/go.mod h1:7bolpMKhT5LKskLwYXGSDOyA2TYtMFgdgV0Y8gy7QhA= -github.com/klauspost/reedsolomon v1.11.7 h1:9uaHU0slncktTEEg4+7Vl7q7XUNMBUOK4R9gnKhMjAU= -github.com/klauspost/reedsolomon v1.11.7/go.mod h1:4bXRN+cVzMdml6ti7qLouuYi32KHJ5MGv0Qd8a47h6A= +github.com/klauspost/reedsolomon v1.11.8 h1:s8RpUW5TK4hjr+djiOpbZJB4ksx+TdYbRH7vHQpwPOY= +github.com/klauspost/reedsolomon v1.11.8/go.mod h1:4bXRN+cVzMdml6ti7qLouuYi32KHJ5MGv0Qd8a47h6A= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= @@ -1203,8 +1203,9 @@ github.com/minio/selfupdate v0.3.1/go.mod h1:b8ThJzzH7u2MkF6PcIra7KaXO9Khf6alWPv github.com/minio/selfupdate v0.4.0 h1:A7t07pN4Ch1tBTIRStW0KhUVyykz+2muCqFsITQeEW8= github.com/minio/selfupdate v0.4.0/go.mod h1:mcDkzMgq8PRcpCRJo/NlPY7U45O5dfYl2Y0Rg7IustY= github.com/minio/sha256-simd v0.1.1/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM= -github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g= github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM= +github.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM= +github.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8= github.com/minio/simdjson-go v0.2.1 h1:nxYlp4Qd0w2pwLlif00l5vTFL6PcNAKpyHq27/pageg= github.com/minio/simdjson-go v0.2.1/go.mod h1:JPUSkRykfSPS+AhO0YPA1h0l5vY7NqrF4zel2b12wxc= github.com/minio/sio v0.2.1/go.mod h1:8b0yPp2avGThviy/+OCJBI6OMpvxoUuiLvE6F1lebhw= @@ -1431,6 +1432,8 @@ github.com/quasilyte/go-ruleguard v0.1.2-0.20200318202121-b00d7a75d3d8/go.mod h1 github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 h1:N/ElC8H3+5XpJzTSTfLsJV/mx9Q9g7kxmchpfZyxgzM= github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= +github.com/remeh/sizedwaitgroup v1.0.0 h1:VNGGFwNo/R5+MJBf6yrsr110p0m4/OX4S3DCy7Kyl5E= +github.com/remeh/sizedwaitgroup v1.0.0/go.mod h1:3j2R4OIe/SeS6YDhICBy22RWjJC5eNCJ1V+9+NVNYlo= github.com/remyoudompheng/bigfft v0.0.0-20170806203942-52369c62f446/go.mod h1:uYEyJGbgTkfkS4+E/PavXkNJcbFIpEtjt2B0KDQ5+9M= github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= github.com/rivo/uniseg v0.4.3 h1:utMvzDsuh3suAEnhH0RdHmoPbU648o6CvXxTx4SBMOw= @@ -2330,6 +2333,7 @@ gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA= gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc= +gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc= gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce h1:+JknDZhAj8YMt7GC73Ei8pv4MzjDUNPHgQWJdtMAaDU= gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce/go.mod h1:5AcXVHNjg+BDxry382+8OKon8SEWiKktQR07RKPsv1c= gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= diff --git a/go.work b/go.work index 13c88e6e9..a8f99ffa2 100644 --- a/go.work +++ b/go.work @@ -1,4 +1,4 @@ -go 1.19 +go 1.20 use ( . diff --git a/go.work.sum b/go.work.sum index d5ba6fb62..9d7af619d 100644 --- a/go.work.sum +++ b/go.work.sum @@ -128,6 +128,8 @@ cloud.google.com/go/workflows v1.9.0/go.mod h1:ZGkj1aFIOd9c8Gerkjjq7OW7I5+l6cSvT github.com/0chain/gorocksdb v0.0.0-20220406081817-640f6b0a3abb/go.mod h1:3i9d+2Osik7apjXERxpAlKxE7SilJKkHoW9Ihdjdfxs= github.com/0chain/gosdk v1.8.5 h1:vZGart1W3pijEF3mhGk3d4CBsrRo4iqLxORY7rRkIP4= github.com/0chain/gosdk v1.8.5/go.mod h1:JeaXA7gbncHoYbKixUnM58+7S2E9z7IKYY+fV0IQJ/o= +github.com/0chain/gosdk v1.10.1-0.20231114143641-51b9f0240b64 h1:DBDslN917WflFB3FFWBU9ISgym5tye0gBVw0N3m7Fmk= +github.com/0chain/gosdk v1.10.1-0.20231114143641-51b9f0240b64/go.mod h1:l/c+X5v6RMyzWUib6FFPdK5bTAzAJ7PojhEvELp3LGk= github.com/Azure/azure-sdk-for-go/sdk/azcore v0.21.1/go.mod h1:fBF9PQNqB8scdgpZ3ufzaLntG0AG7C1WjPMsiFOmfHM= github.com/Azure/azure-sdk-for-go/sdk/internal v0.8.3/go.mod h1:KLF4gFr6DcKFZwSuH8w8yEK6DpFl3LP5rhdvAb7Yz5I= github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v0.3.0/go.mod h1:tPaiy8S5bQ+S5sOiDlINkp7+Ef339+Nz5L5XO+cnOHo= @@ -263,7 +265,11 @@ github.com/klauspost/compress v1.15.6/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHU github.com/klauspost/cpuid v1.3.1 h1:5JNjFYYQrZeKRJ0734q51WCEEn2huer72Dc7K+R/b6s= github.com/klauspost/cpuid/v2 v2.0.6/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= github.com/klauspost/cpuid/v2 v2.0.13/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c= +github.com/klauspost/cpuid/v2 v2.1.1/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY= +github.com/klauspost/cpuid/v2 v2.2.3/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY= github.com/klauspost/reedsolomon v1.9.16/go.mod h1:eqPAcE7xar5CIzcdfwydOEdcmchAKAP/qs14y4GCBOk= +github.com/klauspost/reedsolomon v1.11.8 h1:s8RpUW5TK4hjr+djiOpbZJB4ksx+TdYbRH7vHQpwPOY= +github.com/klauspost/reedsolomon v1.11.8/go.mod h1:4bXRN+cVzMdml6ti7qLouuYi32KHJ5MGv0Qd8a47h6A= github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= @@ -276,6 +282,8 @@ github.com/matryer/is v1.4.1/go.mod h1:8I/i5uYgLzgsgEloJE1U6xx5HkBQpAZvepWuujKwM github.com/mattn/go-colorable v0.1.12 h1:jF+Du6AlPIjs2BiUiQlKOX0rt3SujHxPnksPKZbaA40= github.com/mattn/go-isatty v0.0.14 h1:yVuAays6BHfxijgZPzw+3Zlu5yQgKGP2/hcQbHb7S9Y= github.com/mattn/go-runewidth v0.0.13 h1:lTGmDsbAYt5DmK6OnoV7EuIF1wEIFAcxld6ypU4OSgU= +github.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM= +github.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8= github.com/mitchellh/pointerstructure v1.2.0/go.mod h1:BRAsLI5zgXmw97Lf6s25bs8ohIXc3tViBH44KcwB2g4= github.com/mmcloughlin/addchain v0.4.0/go.mod h1:A86O+tHqZLMNO4w6ZZ4FlVQEadcoqkyU72HC5wJ4RlU= github.com/naoina/go-stringutil v0.1.0/go.mod h1:XJ2SJL9jCtBh+P9q5btrd/Ylo8XwT/h1USek5+NqSA0= @@ -297,6 +305,8 @@ github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrb github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= github.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA= github.com/protolambda/bls12-381-util v0.0.0-20220416220906-d8552aa452c7/go.mod h1:IToEjHuttnUzwZI5KBSM/LOOW3qLbbrHOEfp3SbECGY= +github.com/remeh/sizedwaitgroup v1.0.0 h1:VNGGFwNo/R5+MJBf6yrsr110p0m4/OX4S3DCy7Kyl5E= +github.com/remeh/sizedwaitgroup v1.0.0/go.mod h1:3j2R4OIe/SeS6YDhICBy22RWjJC5eNCJ1V+9+NVNYlo= github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY= github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= @@ -322,6 +332,7 @@ github.com/valyala/fasttemplate v1.2.2/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+ github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673/go.mod h1:N3UwUGtsrSj3ccvlPHLoLsHnpR27oXr4ZE984MbSER8= github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= +github.com/zeebo/blake3 v0.2.3 h1:TFoLXsjeXqRNFxSbk35Dk4YtszE/MQQGK10BH4ptoTg= go.dedis.ch/kyber/v3 v3.0.13 h1:s5Lm8p2/CsTMueQHCN24gPpZ4couBBeKU7r2Yl6r32o= go.dedis.ch/kyber/v3 v3.0.13/go.mod h1:kXy7p3STAurkADD+/aZcsznZGKVHEqbtmdIzvPfrs1U= go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738 h1:VcrIfasaLFkyjk6KNlXQSzO+B0fZcnECiDrKJsfxka0= @@ -384,7 +395,6 @@ google.golang.org/grpc v1.51.0/go.mod h1:wgNDFcnuBGmxLKI/qn4T+m5BtEBYXJPvibbUPsA google.golang.org/protobuf v1.28.0 h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw= google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= gopkg.in/cheggaaa/pb.v1 v1.0.28/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw= -gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= k8s.io/klog v1.0.0 h1:Pt+yjF5aB1xDSVbau4VsWe+dQNzA0qv1LlXdC2dF6Q8= rsc.io/tmplfunc v0.0.3/go.mod h1:AG3sTPzElb1Io3Yg4voV9AGZJuleGAwaVRxL9M49PhA=