使用100%A100运行报错

[3mpt]

背景 使用100%A100 用vllm部署
INFO 03-03 21:25:17 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20250303-212517.pkl... INFO 03-03 21:25:17 model_runner_base.py:149] Completed writing input of failed execution to /tmp/err_execute_model_input_20250303-212517.pkl. CRITICAL 03-03 21:25:17 launcher.py:99] MQLLMEngine is already dead, terminating server process INFO: 127.0.0.1:44754 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [39513] [rank0]:[W303 21:25:18.275990903 ProcessGroupNCCL.cpp:1250] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator())
报错。
同样的步骤使用50% A100 正常运行