vllm + ipex support AMX now？

[guixinliu1995]

Describe the issue

I running the newest vllm and 2.6.0 ipex on a gnr platform for prefill and decode, may I ask dose this call AMX?
If dose, how to make sure that?

[azhuvath]

Profilers like Intel VTune can highlight it as part of vectorization.

It should use AMX provided you are following these steps.

git clone https://github.com/vllm-project/vllm.git
cd vllm
docker build -f docker/Dockerfile.cpu --tag vllm-cpu-env --target vllm-openai .

docker run --rm
--net=host
-v ~/.cache/huggingface:/root/.cache/huggingface
--privileged=true
--shm-size=16g
-e VLLM_CPU_KVCACHE_SPACE=40
-e VLLM_CPU_OMP_THREADS_BIND="0-61|64-125|128-189|192-253"
-e VLLM_CPU_SGL_KERNEL=1
vllm-cpu-env
--model=ond-ai/od-agent-1.4-Qwen3-8B-ckpt-2
--dtype=bfloat16
-tp=4
--enable-chunked-prefill
--enable-prefix-caching

You will have to adjust VLLM_CPU_OMP_THREADS_BIND based on your system and how NUMA is defined.