You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In certain cases, your S3 bucket may fit one (or both) of the following criteria:
116
+
Use --no-copy when you want to avoid creating a duplicate copy of the data in your cluster account. Using this flag can significantly speed up Datastore creation time by preventing the copy process.
117
117
118
-
(1) the bucket is continually updating with new data which you want included in a Grid Datastore
119
-
(2) the bucket is particularly large (leading to long Datastore creation times)
118
+
:::info
119
+
We recommend that you use this flag when creating a private S3 Datastore within a BYOC cluster.
120
120
121
-
In these cases, you can pass the `--no-copy` flag to the `grid datastore create` command. This flag will
122
-
prevent Grid from making a copy of the dataset, which significantly speeds up Datastore creation time.
121
+
Also consider using this option when the your dataset size is greater than `500 GB`.
123
122
124
-
:::info
123
+
If your Datastore was from a public S3 bucket, having a copy in the cluster can be advantageous if someone deletes or modifys the public bucket. `--no-copy` does not prevent you from deleting the file in S3 as thats managed by the user. If thats done the Datastore will show that the file still exists since the metadata is cached, On access S3 will indicate that the file does not exist anymore.
125
124
126
125
When using this flag, you cannot remove files from your bucket. If you'd like to add files, please create a new version of the Datastore after you've added files to your bucket.
Copy file name to clipboardExpand all lines: docs/getting-started/typical-workflow-cli-user.md
+6-1Lines changed: 6 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -325,7 +325,7 @@ cd grid-tutorials/getting-started
325
325
pip install -r requirements.txt
326
326
```
327
327
328
-
now run the following command to train a resnet18 on 2 GPUs
328
+
now run the following command to train a resnet18 on 1 GPU
329
329
330
330
```bash
331
331
python flash-image-classifier.py \
@@ -387,6 +387,11 @@ Make sure you are in the /grid-tutorials/getting-started directory for the tutor
387
387
388
388
Now kick off the run with grid run
389
389
390
+
:::note
391
+
To convert the below run into a multi-gpu Run please set the `--instance_type` parameter to a machine with multiple gpus
392
+
and update the `--gpus` argument to the desired number of GPUs. See [here](../platform/4_machines.md) for a table of machine types and their hardware specs.
f. You will be redirected to IAM -> Roles. Select your just created
155
+
156
+
g. Click on Add Permissions -> Create Inline Policy. Paste the output `permission policy` from the CLI output after replacing your bucket name and click on Review Policy
0 commit comments