Skip to content

Latest commit

 

History

History
64 lines (61 loc) · 2.96 KB

NOTES.md

File metadata and controls

64 lines (61 loc) · 2.96 KB
Raw

Notes

chatglm2-6b - AttributeError: can't set attribute

Download chatglm2-6b from hugginface, convert to OpenVINO IR files and run with benchmark.py, the following error may occur：

AttributeError: can't set attribute

Reproduced with https://huggingface.co/THUDM/chatglm2-6b 7fabe56db91e085c9c027f56f1c654d137bdba40
As on https://huggingface.co/THUDM/chatglm2-6b/discussions/99
Solution: update tokenization_chatglm.py as following: 

          self.vocab_file = vocab_file
          self.tokenizer = SPTokenizer(vocab_file)
 +        kwargs.pop("eos_token", None)
 +        kwargs.pop("pad_token", None)
 +        kwargs.pop("unk_token", None)
          self.special_tokens = {
              "<bos>": self.tokenizer.bos_id,
              "<eos>": self.tokenizer.eos_id,

The solution works for chatglm3-6b as well.

Qwen-7B-Chat-Int4 - Torch not compiled with CUDA enabled

Convert Qwen-7B-Chat-Int4 to OpenVINO IR files run with convert.py, the following error may occur：

raise AssertionError("Torch not compiled with CUDA enabled")

Reproduced with https://huggingface.co/Qwen/Qwen-7B-Chat-Int4 8750247cc50f2a7bb84bef322f7707159b700723
Solution: update modeling_qwen.py as following: 

-SUPPORT_CUDA = torch.cuda.is_available()
+SUPPORT_CUDA = False
 SUPPORT_BF16 = SUPPORT_CUDA and torch.cuda.is_bf16_supported()

Baichuan2-7B-Chat - AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'

Convert Baichuan2-7B-Chat to OpenVINO IR files run with convert.py, the following error may occur：

AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'

Reproduced with https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat 84603cde5ebffb6084e476cfaeceaf0b8b91fe54
Reference to https://huggingface.co/baichuan-inc/Baichuan2-7B-Base/discussions/2
Solution: update tokenization_baichuan.py as following: 

         eos_token = AddedToken(eos_token, lstrip=False, rstrip=False) if isinstance(eos_token, str) else eos_token
         unk_token = AddedToken(unk_token, lstrip=False, rstrip=False) if isinstance(unk_token, str) else unk_token
         pad_token = AddedToken(pad_token, lstrip=False, rstrip=False) if isinstance(pad_token, str) else pad_token
+        self.vocab_file = vocab_file
+        self.add_bos_token = add_bos_token
+        self.add_eos_token = add_eos_token
+        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
+        self.sp_model.Load(vocab_file)
         super().__init__(
             bos_token=bos_token,
             eos_token=eos_token,
             clean_up_tokenization_spaces=clean_up_tokenization_spaces,
             **kwargs,
         )
-        self.vocab_file = vocab_file
-        self.add_bos_token = add_bos_token
-        self.add_eos_token = add_eos_token
-        self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
-        self.sp_model.Load(vocab_file)