Skip to content
Back to pull request #6341

llama : save and restore kv cache for single seq id #264

Sign in to view logs

llama : save and restore kv cache for single seq id

llama : save and restore kv cache for single seq id #264

bench-server-baseline (phi-2, q4_0)

bench-server-baseline (phi-2, q4_0)
succeeded Apr 5, 2024 in 12m 50s