Code

Use vLLM to load LLMs#230

Code

Open

kyriediculous wants to merge 2 commits intolivepeer:mainlivepeer/ai-worker:mainfrom joinhive-ai:llmjoinhive-ai/ai-worker:llmCopy head branch name to clipboard

+676-243

Commits

Commits on Oct 17, 2024

• llm: use vLLM

  kyriediculous

  committed
  b7e5606

Commits on Oct 18, 2024

• fixup: additional params

  kyriediculous

  committed
  6f9256d