Enhancing Ascend 910A Training Efficiency in LlamaFactory with NPU

[zhou-wjjw]

What does this PR do?

The training efficiency of the Ascend 910A has been significantly enhanced by leveraging the full computational power of the NPU and the capabilities of torch_npu. This enhancement has led to a remarkable tenfold increase in training efficiency.

[hiyouga]

We should first check if the torch_npu package is available, such as

LLaMA-Factory/src/llmtuner/chat/vllm_engine.py

Lines 11 to 14 in d6ca785

	if is_vllm_available():
	from vllm import AsyncEngineArgs, AsyncLLMEngine, RequestOutput, SamplingParams
	from vllm.lora.request import LoRARequest
	from vllm.sequence import MultiModalData

[zhou-wjjw]

We should first check if the torch_npu package is available, such as

LLaMA-Factory/src/llmtuner/chat/vllm_engine.py

Lines 11 to 14 in d6ca785

	if is_vllm_available():
	from vllm import AsyncEngineArgs, AsyncLLMEngine, RequestOutput, SamplingParams
	from vllm.lora.request import LoRARequest
	from vllm.sequence import MultiModalData

Alright, so it looks like VLLM doesn't support Ascend. No worries, I'll just tweak the code a bit and see if I can get it working.

[zhou-wjjw]

We should first check if the torch_npu package is available, such as

LLaMA-Factory/src/llmtuner/chat/vllm_engine.py

Lines 11 to 14 in d6ca785

	if is_vllm_available():
	from vllm import AsyncEngineArgs, AsyncLLMEngine, RequestOutput, SamplingParams
	from vllm.lora.request import LoRARequest
	from vllm.sequence import MultiModalData

Yeah, I think updating the docs for now and letting the devs figure out how to handle it could be a good way to go. Just make it clear in the documentation that VLLM and Ascend aren't playing nicely together at the moment. That way, the developers can see the issue straight away and decide for themselves how they want to tackle it, based on what works best for their specific project.

[statelesshz]

Hmm... If you want to use LLaMA-Factory on Ascend910A, this is the modification method I would recommend.

[hiyouga]

It now works, LGTM