load_pretrained_model() missing 1 required positional argument: 'model_name' when evaluating VILA-7b

[huayicong23]

Description
running the following command results in a missing 1 required positional argument error，the original code in vila.py is :
self._tokenizer, self._model, self._image_processor, self._max_length = load_pretrained_model(pretrained, self.model_name, device_map=self.device_map, attn_implementation=attn_implementation)
if I modify it like llava_onevision.py:
self._tokenizer, self._model, self._image_processor, self._max_length = load_pretrained_model(pretrained, None, self.model_name, device_map=self.device_map, attn_implementation=attn_implementation)
This will result in following error:

Example Command

Error message

environments

[pufanyi]

Hello! I'm not sure but based on my experience, maybe you can check if you have installed deepspeed and flash-attn. It seems that VILA requires these 2 packages, but they need to install separately. If these two packages are not installed, this strange error message will be triggered.

[pufanyi]

Just run

python -m pip install flash-attn --no-build-isolation
python -m pip install deepspeed

[pufanyi]

And as a reminder, when you run VILA, you need to install llava from VILA repo instead of LLaVA repo, which is:

git clone [email protected]:NVlabs/VILA.git
cd VILA
python -m pip install -e .