Skip to content

Commit 6121963

Browse files
committed
fix dummy request.
Signed-off-by: Fanrong Li <[email protected]>
1 parent b1dd9e6 commit 6121963

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

tensorrt_llm/_torch/pyexecutor/llm_request.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -468,7 +468,11 @@ def create_dummy_requests(
468468
req.paged_kv_block_ids = []
469469
if is_gen:
470470
req.state = LlmRequestState.GENERATION_IN_PROGRESS
471-
req.prompt_len = token_num - 1
471+
if token_num > 1:
472+
req.prompt_len = token_num - 1
473+
else:
474+
req.add_new_token(1, 0)
475+
req.prompt_len = 1
472476
req.py_draft_tokens = [1] * max_num_draft_tokens
473477
req.py_prompt_len = req.prompt_len
474478
requests.append(req)

0 commit comments

Comments
 (0)