Evaluation of custom models and datasets.

[juxingyiwan]

VLMEVALKIT is a pretty convenient evaluation tool for MLLMs. I hope that the esteemed authors can create a framework for VLMEVALKIT that supports the evaluation of custom models and custom datasets. This framework can define a unified MLLM input-output interface and the conversion format for datasets.

[kennymckormick]

Hi, @juxingyiwan ,
we are going to first implement for custom multiple-choice datasets, and will create a tutorial for it. Stay Tuned!