model download issue

[Kkkassini]

model = AutoModel.from_pretrained("mPLUG/DocOwl1.5-Omni")

throws KeyError: 'mplug_docowl'

[HAWLYQ]

Hi, @Kkkassini , please run inference code with our model as the instruction in DocOwl1.5

For example

model_path='./mPLUG/DocOwl1.5-Omni'
docowl=DocOwlInfer(ckpt_path=model_path, anchors='grid_9', add_global_img=True)
print('load model from ', model_path)

image='./DocDownstream-1.0/imgs/DUE_Benchmark/DocVQA/pngs/rnbx0223_193.png'
query='What is the Compound Annual Growth Rate (CAGR) for total assets?'
answer=docowl.inference(image, query)
print(answer)```

[Kkkassini]

Hi, @Kkkassini , please run inference code with our model as the instruction in DocOwl1.5

For example

model_path='./mPLUG/DocOwl1.5-Omni'
docowl=DocOwlInfer(ckpt_path=model_path, anchors='grid_9', add_global_img=True)
print('load model from ', model_path)

image='./DocDownstream-1.0/imgs/DUE_Benchmark/DocVQA/pngs/rnbx0223_193.png'
query='What is the Compound Annual Growth Rate (CAGR) for total assets?'
answer=docowl.inference(image, query)
print(answer)```

Thanks, but I still get

Repo id must be in the form 'repo_name' or 'namespace/repo_name': './mPLUG/DocOwl1.5-stage1'. Use repo_type argument if needed.

Should I download it before?

[HAWLYQ]

Hi, @Kkkassini , please run inference code with our model as the instruction in DocOwl1.5
For example

model_path='./mPLUG/DocOwl1.5-Omni'
docowl=DocOwlInfer(ckpt_path=model_path, anchors='grid_9', add_global_img=True)
print('load model from ', model_path)

image='./DocDownstream-1.0/imgs/DUE_Benchmark/DocVQA/pngs/rnbx0223_193.png'
query='What is the Compound Annual Growth Rate (CAGR) for total assets?'
answer=docowl.inference(image, query)
print(answer)```

Thanks, but I still get

Repo id must be in the form 'repo_name' or 'namespace/repo_name': './mPLUG/DocOwl1.5-stage1'. Use repo_type argument if needed.

Should I download it before?

Yes, you should download the model before~

[AR-javis]

@HAWLYQ are you planning to add support for mplug_docowl architecture in transformers library in the future? HF throws architecture not supported error.

[HAWLYQ]

@HAWLYQ are you planning to add support for mplug_docowl architecture in transformers library in the future? HF throws architecture not supported error.

Hi, @AR-javis we temporarily don't plan to support mplug_docowl in the transformers library~ we may consider it in the next version update. Thanks for your advice~