Cloud OT datasets uploading speed

[miasik]

The documentations states:
You set your training configuration as usual. Define the remote server and options, when hitting "Start Training", it will upload the concepts and training configuration to the remote server.
I have a large dataset. Something like 400Gb in 500k files (images+captions). So, I have some questions related to it:

1. Does OT calculate datasets sizes when it creates PODs?
2. Can OT upload in parallel to save time?
3. How does OT reserve cloud space for samples, backups, saves?

[dxqbYD]

ad 2) it currently uses 4 parallel uploads
ad 1) and 3): you have to decide the manually, parameter "volume size" on the cloud tab, in GB

with this large a dataset you might want to create your pod on the runpod website though, so you can use a persistent storage volume and not loose your upload when you delete the pod.

you can still use the pre-installed OneTrainer template here: https://www.runpod.io/console/explore/1a33vbssq9
then enter the pod id as "cloud id" into the cloud tab