Skip to content
@Q-Future

Visual Evaluation with Foundation Models

We are working towards a future that one foundation model can be a multi-purpose expert for low-level visual perception and visual evaluation.

👁️‍🗨️ Low-level Visual Perception in the Foundation Model Era

🔖Aiming at next-era cornerstone research

Low-level Visual Perception | Multi-Modality Large Language Models | Visual Quality Assessment

📖Main Projects

  • Co-Instruct: Homepage, Repo, Demo. Open-ended visual quality comparer (up to 4 images), low-level visual assistant, an improved version of ②Q-Instruct [CVPR 2024].

  • Q-Align [ICML 2024]: Homepage, Repo, Demo. A unified visual scorer for images and videos, via text-instructed alignment on multi-modality foundation models; can efficiently fine-tune to more datasets with stable good performance. State-of-the-art on IQA, VQA, and IAA.

  • Q-Instruct [CVPR 2024]: Homepage, Repo, 200K Dataset, Technical Report A large-scale instruction tuning dataset to improve low-level perceptual abilities of foundation models.

  • Q-Bench+ [ICLR2024, Spotlight]: Homepage, Repo, Data-Single, Data-Pair, Preprint The first low-level benchmark for foundation models on low-level vision.

🖋️Extension Projects

  • Q-Boost: Homepage A discussion on boosting the IQA performance for non-specially-IQA-aligned MLLMs.

  • [Pending]Chinese-Q-Bench/质衡: Homepage, Repo The first attempt to test multi-lingual abilities on low-level vision.

Maintained by Teo Wu@Singapore and Zicheng Zhang@Shanghai.

Pinned Loading

  1. Q-Align Q-Align Public

    ③[ICML2024] [IQA, IAA, VQA] All-in-one Foundation Model for visual scoring. Can efficiently fine-tune to downstream datasets.

    Python 277 18

  2. Q-Bench Q-Bench Public

    ①[ICLR2024 Spotlight] (GPT-4V/Gemini-Pro/Qwen-VL-Plus+16 OS MLLMs) A benchmark for multi-modality LLMs (MLLMs) on low-level vision and visual quality assessment.

    Jupyter Notebook 248 12

  3. Q-Instruct Q-Instruct Public

    ②[CVPR 2024] Low-level visual instruction tuning, with a 200K dataset and a model zoo for fine-tuned checkpoints.

    Python 198 8

  4. A-Bench A-Bench Public

    [LMM + AIGC] What do we expect from LMMs as AIGI evaluators and how do they perform?

    148 3

  5. Q-Bench-Video Q-Bench-Video Public

    A benchmark for video quality understanding of LMMs

    Python 105

  6. Co-Instruct Co-Instruct Public

    ④[ECCV 2024 Oral, Comparison among Multiple Images!] A study on open-ended multi-image quality comparison: a dataset, a model and a benchmark.

    65 4

Repositories

Showing 10 of 14 repositories
  • LMM-PCQA Public

    Official repo for `LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM', ACM MM2024 Oral

    Q-Future/LMM-PCQA’s past year of commit activity
    Python 11 0 0 0 Updated Nov 2, 2024
  • Q-Ground Public

    Official codes for "Q-Ground: Image Quality Grounding with Large Multi-modality Models", ACM MM2024 (Oral)

    Q-Future/Q-Ground’s past year of commit activity
    26 0 2 0 Updated Oct 25, 2024
  • R-Bench Public
    Q-Future/R-Bench’s past year of commit activity
    Jupyter Notebook 8 0 0 0 Updated Oct 13, 2024
  • Q-Bench-Video Public

    A benchmark for video quality understanding of LMMs

    Q-Future/Q-Bench-Video’s past year of commit activity
    Python 105 0 0 0 Updated Oct 2, 2024
  • Co-Instruct Public

    ④[ECCV 2024 Oral, Comparison among Multiple Images!] A study on open-ended multi-image quality comparison: a dataset, a model and a benchmark.

    Q-Future/Co-Instruct’s past year of commit activity
    65 4 3 0 Updated Sep 29, 2024
  • A-Bench Public

    [LMM + AIGC] What do we expect from LMMs as AIGI evaluators and how do they perform?

    Q-Future/A-Bench’s past year of commit activity
    148 3 0 0 Updated Sep 27, 2024
  • Compare2Score Public

    [Neurips 24 Spotlight] Training in Pairs + Inference on Single Image with Anchors

    Q-Future/Compare2Score’s past year of commit activity
    Python 18 MIT 1 1 0 Updated Aug 24, 2024
  • Q-Align Public

    ③[ICML2024] [IQA, IAA, VQA] All-in-one Foundation Model for visual scoring. Can efficiently fine-tune to downstream datasets.

    Q-Future/Q-Align’s past year of commit activity
    Python 277 18 12 0 Updated Aug 12, 2024
  • Q-Instruct Public

    ②[CVPR 2024] Low-level visual instruction tuning, with a 200K dataset and a model zoo for fine-tuned checkpoints.

    Q-Future/Q-Instruct’s past year of commit activity
    Python 198 8 11 0 Updated Aug 12, 2024
  • Q-Bench Public

    ①[ICLR2024 Spotlight] (GPT-4V/Gemini-Pro/Qwen-VL-Plus+16 OS MLLMs) A benchmark for multi-modality LLMs (MLLMs) on low-level vision and visual quality assessment.

    Q-Future/Q-Bench’s past year of commit activity
    Jupyter Notebook 248 12 1 0 Updated Aug 12, 2024