[Web] Trouble loading > 2GB ONNX model (those with separate blocks)

[sedghi]

Describe the issue

Cannot load bigger models that generates multiple blocks neither from the binary or uri

To reproduce

I have a repository, a simple Vite front-end application. You can take a look at it in here

basically in the App.tsx i have

const onnxSession = await ort.InferenceSession.create(bytes, {
        executionProviders: ["webgpu"],
        enableMemPattern: false,
        enableCpuMemArena: false,
        extra: {
          session: {
            disable_prepacking: "1",
            use_device_allocator_for_initializers: "1",
            use_ort_model_bytes_directly: "1",
            use_ort_model_bytes_for_initializers: "1",
          },
        },
        interOpNumThreads: 4,
        intraOpNumThreads: 2,
      })

You need to download the SAM models that I exported to ONNX:

• sam_l: https://ohif-assets.s3.us-east-2.amazonaws.com/SAM/sam_h.zip
• sam_h: https://ohif-assets.s3.us-east-2.amazonaws.com/SAM/sam_l.zip

(i just exported the onnx files using this repo)

The code works great with sam_l (both URI and binary). I guess it's because it generates two files. However, since sam_h is a large model, it generates numerous files, and I can't make it load through either URI or binary.

You can download my repository and place the sam_l and sam_h ONNX formats in the public folder, then run yarn start. Uncomment the following lines in src/App.tsx:

// load("http://localhost:5173/sam_h/vit_h_encoder.onnx")
load("http://localhost:5173/sam_h/vit_h_decoder.onnx")

You can see that the decoder still runs, but not the encoder, as it is probably referencing the smaller blocks.

Urgency

it is urgent i guess?

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.3

Execution Provider

'webgpu' (WebGPU)

[sedghi]

Bump?