Code

add support for multinode tp#751

Code

Merged

shihaobai merged 52 commits intomainModelTC/lightllm:mainfrom

multinodeModelTC/lightllm:multinodeCopy head branch name to clipboard

Mar 1, 2025

Commits

Commits on Feb 16, 2025

• supporting multinode

  jayfeather9

  committed
  65428a1

  Copy full SHA for 65428a1

• fix format

  jayfeather9

  committed
  a26d48d

  Copy full SHA for a26d48d

• add cuda() with device id

  jayfeather9

  committed
  64ccb4f

  Copy full SHA for 64ccb4f

Commits on Feb 18, 2025

• Merge branch 'main' into multinode

  

  root

  committed
  b3de424

  Copy full SHA for b3de424

Commits on Feb 20, 2025

• fix multinode abort

  

  shihaobai

  committed
  3fd6e48

  Copy full SHA for 3fd6e48

• support chunked prefill

  

  shihaobai

  committed
  c0b1146

  Copy full SHA for c0b1146

• Merge branch 'main' into multinode

  

  root

  committed
  855e037

  Copy full SHA for 855e037

Commits on Feb 21, 2025

• modify dist_utils & remove child_ips

  

  Feiyang Wu

  committed
  8ad585a

  Copy full SHA for 8ad585a

• Merge branch 'multinode' of https://github.com/ModelTC/lightllm into multinode

  

  Feiyang Wu

  committed
  e0844d3

  Copy full SHA for e0844d3

Commits on Feb 23, 2025

• fix chunked_prefill for multinode

  

  baishihao

  committed
  fa3f826

  Copy full SHA for fa3f826

• Merge branch 'multinode' of https://github.com/ModelTC/lightllm into multinode

  

  baishihao

  committed
  7e92ff6

  Copy full SHA for 7e92ff6

• merge main

  

  baishihao

  committed
  4a83a6a

  Copy full SHA for 4a83a6a

• fix health

  

  shihaobai

  committed
  461ec65

  Copy full SHA for 461ec65

Commits on Feb 24, 2025

• update port

  

  shihaobai

  committed
  b15a487

  Copy full SHA for b15a487

Commits on Feb 26, 2025

• 修改 rank 配置。

  hiworldwzj

  committed
  4e419df

  Copy full SHA for 4e419df

Commits on Feb 27, 2025

• refactor multinode

  

  shihaobai

  committed
  8237f19

  Copy full SHA for 8237f19

• Merge branch 'main' into multinode

  

  shihaobai

  committed
  b55edd7

  Copy full SHA for b55edd7

• fix get_dp_size

  

  shihaobai

  committed
  958b83d

  Copy full SHA for 958b83d

• remove tp_rank of get_available_gpu_memory

  

  shihaobai

  committed
  fffb99e

  Copy full SHA for fffb99e

• fix chunked prefill

  

  shihaobai

  committed
  ecd495c

  Copy full SHA for ecd495c

• fix dist_utils

  

  shihaobai

  committed
  82a756f

  Copy full SHA for 82a756f

• multinode utils

  

  shihaobai

  committed
  814f095

  Copy full SHA for 814f095

• update router mulitnode mananger

  

  shihaobai

  committed
  b33b3b4

  Copy full SHA for b33b3b4

• fix chunked prefill

  

  shihaobai

  committed
  296a579

  Copy full SHA for 296a579

• refomat

  

  shihaobai

  committed
  2afd14b

  Copy full SHA for 2afd14b

• fix

  hiworldwzj

  authored
  7061bfb

  Copy full SHA for 7061bfb

Commits on Feb 28, 2025

• refactor order

  

  shihaobai

  committed
  0bde847

  Copy full SHA for 0bde847

• fix

  

  shihaobai

  committed
  39b90bf

  Copy full SHA for 39b90bf

• update httpserver sync

  

  shihaobai

  committed
  429f9c3

  Copy full SHA for 429f9c3

• update

  

  shihaobai

  committed
  4377c20

  Copy full SHA for 4377c20

• remove cudagraph_step_length

  

  shihaobai

  committed
  4abb4a1

  Copy full SHA for 4abb4a1

• modify the default value of current_waiting_num

  

  shihaobai

  committed
  7646d6e

  Copy full SHA for 7646d6e

• update

  

  shihaobai

  committed
  8446544

  Copy full SHA for 8446544

• fix

  

  shihaobai

  committed
  877b98f

  Copy full SHA for 877b98f

• fix visualserver

  

  shihaobai

  committed
  8ff7ed6

  Copy full SHA for 8ff7ed6

• fix

  

  shihaobai

  committed
  249dea7

  Copy full SHA for 249dea7

• update mem_manager

  

  shihaobai

  committed
  64bdc11

  Copy full SHA for 64bdc11

• fix start rank params.

  hiworldwzj

  authored
  d68e0d7

  Copy full SHA for d68e0d7

• fix

  

  shihaobai

  committed
  2461069

  Copy full SHA for 2461069

Commits on Mar 1, 2025

• fix

  hiworldwzj

  authored
  b3dbecd

  Copy full SHA for b3dbecd

• fix

  hiworldwzj

  authored
  ea4dc98

  Copy full SHA for ea4dc98

• fix

  hiworldwzj

  authored
  1cffab4

  Copy full SHA for 1cffab4

• update docs

  

  shihaobai

  committed
  4f68164

  Copy full SHA for 4f68164

• fix

  hiworldwzj

  authored
  2b3f07a

  Copy full SHA for 2b3f07a

• fix

  hiworldwzj

  authored
  068663a

  Copy full SHA for 068663a

• fix

  hiworldwzj

  authored
  d14fcea

  Copy full SHA for d14fcea

• fix

  

  shihaobai

  committed
  46dbbb2

  Copy full SHA for 46dbbb2

• fix

  hiworldwzj

  authored
  72f4eb3

  Copy full SHA for 72f4eb3

• fix

  hiworldwzj

  authored
  a640f72

  Copy full SHA for a640f72

• fix

  hiworldwzj

  authored
  5cd5dbf

  Copy full SHA for 5cd5dbf

• fix

  hiworldwzj

  authored
  5d13dc8

  Copy full SHA for 5d13dc8

• reformat

  

  shihaobai

  committed
  ce4d3eb

  Copy full SHA for ce4d3eb