[QNN] Multi-graph models fail on second iteration with DMA execution error 1100

[notaJiminLee]

🐛 Describe the bug

Summary

When running QNN-delegated multi-graph models with --iteration 2 or higher, the execution fails at the first graph (0x1) of the second iteration with Error 1100. Single-graph models work correctly with any number of iterations.

Steps to Reproduce

1. Export a multi-graph model (e.g., LLM) to PTE with QNN backend
2. Run with iteration > 1:

./qnn_executor_runner --model_path model.pte --iteration 2 --input_list_path input_list.txt

Reproduced with SmolLM2 (135M) and Llama 3.2 1B

Expected Behavior

All iterations should complete successfully, as they do with single-graph models.

Actual Behavior

• Single-graph model: All iterations succeed
• Multi-graph model:
  • Iteration 1: All graphs execute successfully (result 0)
  • Iteration 2: First graph (0x1) immediately fails (result 1100)

Error Logs

[INFO] [Qnn ExecuTorch]:  <I> operator() started. graph = 0x1
[ERROR] [Qnn ExecuTorch]:  <E> Dma execution failed on the skel side. result = 1100 transport error = 0
[ERROR] [Qnn ExecuTorch]:  <E> Graph forward failed in execution with err 1100
[ERROR] [Qnn ExecuTorch]: qnn_graph_execute failed. Error 1100
Aborted

What I've Tried (All Failed)

• use_multi_contexts=True during export - No effect
• --shared_buffer true at runtime - No effect
• --warm_up 1 --iteration 1 - Fails (warm_up counts as first execution)

Versions

Unable to run collect_env.py on Android device. Manual environment info:

Environment

• Device: Samsung Galaxy S24 (SM8650 / Snapdragon 8 Gen 3)
• Android Version: 14
• ExecuTorch Version: 0.7.0
• QNN SDK Version: 2.28.0.241029

cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin

[shewu-quic]

Hi @notaJiminLee,

To run LLM models, you can use qnn_llama_runner, or alternatively, you can run them using a script like the one below.

python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-1b_instruct --model_mode hybrid --prefill_ar_len 128 --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1

if you already have a pre-compiled .pte model, you can perform inference by providing the flag --pre_gen_pte and specifying the folder that contains the .pte model. Taking LLAMA3.2 as an example:

python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2 --model_mode hybrid --prefill_ar_len 32 --max_seq_len 128 --prompt "what is 1+1" --pre_gen_pte ${FOLDER_TO_PRE_GEN_PTE}

[notaJiminLee]

Hi @shewu-quic ,
Thanks for the clarification.

My use case is benchmarking inference latency only - I don't need actual text generation or token decoding. I just want to measure the execution time of the model's forward pass across multiple iterations.

For this purpose, I was using qnn_executor_runner with --iteration N flag. The model loads and the first iteration completes successfully, but it fails on the second iteration with DMA Error 1100.

Could you clarify:

1. Is qnn_executor_runner not intended for LLM models, even for simple latency benchmarking?
2. If so, what is the technical reason? (e.g., multi-graph context handling, etc.)
3. Does qnn_llama_runner handle multiple iterations correctly for the same model?

I confirmed that single-graph models (e.g., MixNet-S) work fine with --iteration 10 on qnn_executor_runner, so this seems specific to multi-graph models.

[shewu-quic]

Understood. You want to measure execution time regardless of the kv cache update. It should be fine to use qnn_executor_runner for this purpose. In my test, it is also work. Could you please try using the most recent qnn version, for example 2.37?

1. Is qnn_executor_runner not intended for LLM models, even for simple latency benchmarking?
2. If so, what is the technical reason? (e.g., multi-graph context handling, etc.)

qnn_executor_runner should support LLM models. Generally, LLMs will account for kv cache update time, and we have implemented some optimizations to improve token rate.
For multi-graph, you can set the method_index to choose between kv_forward or prefill forward in qnn_executor_runner when using a hybrid_mode model.

3. Does qnn_llama_runner handle multiple iterations correctly for the same model?

Yes, it should be correct for the same model with multiple iterations.