Supabase storage slow upload speed (internally / server-side / edge-function -> storage)

[0x426]

Hey there,

I've noticed, that the upload of a file (pdf file with about 70kb) from a supabase edge function to the supabase storage takes round about up to 700ms. On the client side it takes about 900 - 1000ms (custom database triggers, RLS, etc.).

The edge functions runs completely in the service_role context, so it bypasses any RLS, custom database triggers or further checks.

The code for the server-side upload looks simplified like following:

const supabaseClient: SupabaseClient = createClient(
    'https://my-supabase-project.co',
    Deno.env.get('SUPABASE_SERVICE_ROLE_KEY'), {
        auth: {
            autoRefreshToken: false,
            persistSession: false,
            detectSessionInUrl: false
        }
    }
);

---

// Prepare the file storage object
const fileStorageObject: IFile = {
    path:           filePath,
    data:           pdfDoc,
    name:           fileName,
    upsert:         true,
    metadata: {
        mimetype:   'application/pdf',
        size:       pdfDoc.byteLength
    }
};

---

// Process file upload
const {
    data,
    error
} = await supabaseClient.storage
    .from(bucketId)
    .upload(`${file.path}/${file.name}`, file.data, {
        cacheControl: '604800',
        contentType: file?.metadata?.mimetype,
        upsert: file?.upsert || false
    });

So absolutely nothing special here.

I'm logging everything what happens in the supabase edge functions, there I noticed the high amount of time it needs to process the upload, here a few examples:

2024-09-30T20:08:16.628Z | f840a368-3748-4feb-b1ed-28d51f327a3e | my-edge-function | 342ms | Successfully post processed file.
2024-09-30T20:08:17.386Z | f840a368-3748-4feb-b1ed-28d51f327a3e | my-edge-function | 1098ms | Successfully uploaded and finished processing of file.

-> 756ms upload time
---

2024-09-30T20:07:53.995Z | 64e6da8c-1040-4767-bd93-ec7dd70e74c8 | my-edge-function | 282ms | Successfully post processed file.
2024-09-30T20:07:54.731Z | 64e6da8c-1040-4767-bd93-ec7dd70e74c8 | my-edge-function | 1018ms | Successfully uploaded and finished processing of file.

-> 736ms upload time
---

2024-09-30T20:01:24.259Z | d421bd2c-4257-4ab7-b68d-ab55506607db | my-edge-function | 692ms | Successfully post processed file.
2024-09-30T20:01:27.125Z | d421bd2c-4257-4ab7-b68d-ab55506607db | my-edge-function | 3558ms | Successfully uploaded and finished processing of file.

-> 2866ms upload time
---

2024-09-30T20:00:47.028Z | df1a9250-c2cd-45e4-b693-2c51bb677a93 | my-edge-function | 272ms | Successfully post processed file.
2024-09-30T20:00:47.751Z | df1a9250-c2cd-45e4-b693-2c51bb677a93 | my-edge-function | 996ms | Successfully uploaded and finished processing of file.

-> 724ms upload time
---

2024-09-30T19:58:12.190Z | 6a5dd53b-ad45-446d-9feb-8f7dc4b960ad | my-edge-function | 748ms | Successfully post processed file.
2024-09-30T19:58:12.867Z | 6a5dd53b-ad45-446d-9feb-8f7dc4b960ad | my-edge-function | 1426ms | Successfully uploaded and finished processing of file.

-> 678ms upload time

From the storage logs, I can see some additional stuff like this (simplified & summarized):

"[Lifecycle]: ObjectCreated:Post my-supabase-project/example/path/file.pdf"
user_agent: Deno/1.45.2 (variant; SupabaseEdgeRuntime/1.58.9)
responseTime: 589.0064429994673
role: service_role
region: eu-central-1

I'm explicitly calling all my edge functions with the region header to prevent any overhead of load balancers for routing. As well as re-deployed every edge function with the latest CLI version.

Something got my attention: hostname: storage-api-lb-eu-central-1-ext.storage.supabase.com

It seems like the storage upload makes some external load balancer routing roundtrip, even if calling internally?

---

The avg cpu time over the last 7 days is at about 243ms, the avg memory usage at about 6.36MB.

I'd expect something about ~ 100ms for a backend side process and a file with just a few kb.

Has anyone else had the same experience or can tell me how I can improve this?

Thanks in advance.

[alextbogdanov]

Yes, I recently had to upload 90k photos to the supabase storage and each was around 100kb. My connection speed was 200mbps and the whole process took about 30+ hours which was a shock to me.

[didier]

Nothing here? It's still so so slow.