Hotfix inception_score (0.1.14)

🚨 Hotfix Released: inception_score Calculation Bug Fixed

A bug affecting inception_score in version 0.1.14 has been fixed.
Other versions are unaffected.

We apologize for the inconvenience and appreciate your understanding.
For questions or support, feel free to reach out.

IQA-PyTorch v0.1.14 Release Notes

📢 Major Changes

🚩🚨 New Metrics

• Add topiq_nr-swin-face metric for face quality assessment 3868bb1
• Add compare2score and deepdc metrics 5c42b5f
• Add fid_dinov2 cdedf6d
• Add mmd_rbf_kernel support for FID 73f8c0e
• Add qualiclip, qualiclip+ and its variances trained on different datasets 40dd5b8
• Add sfid 7f0a07f

🛠️ Enhancements

• Add cfiqa dataset and train topiq_nr-face with it 4d382cb
• Add face iqa benchmark results 24caee0
• Update test script ec4acc8
• Merge dataset info into default_dataset_configs.yml 167a79b

🐛 Bug Fixes

• Fix cpu bug in brisque 24abb8d
• Fix range check with eps=1e-6 61cf2fd
• Fix order of distorted and ref images f3ac5db
• Remove inaccurate image validity check ada32dd

📚 Development Updates

• Add api interface load_dataset 53f75ea

👥 Contributors

Thank you to our new contributors for their valuable input:

• add compare2score and deepdc by @h4nwei in #215
• add mmd_rbf_kernel support for FID by @Luciennnnnnn in #232
• clean dataset processing by @LorenzoAgnolucci in #238
• add QualiCLIP by @LorenzoAgnolucci in #242
• Make it possible to use pyiqa as a dependency on MacOS by @ternarybits in #252
• Fix order of distorted and reference images by @Alexkkir in #258
• Remove inaccurate image validity check by @bbergeron0 in #261

📝 For complete details, see the Full Changelog

IQA-PyTorch v0.1.13 Release Notes

📢 Major Changes

📦 Model Weights Migration

• All weights moved to Hugging Face: Now available at chaofengc/IQA-PyTorch-Weights
  • ⚡ Enables faster and more stable downloads
  • 🤝 Allows community contributions for model weights
  • 🌏 For users in mainland China: Use the Hugging Face mirror with:

    export HF_ENDPOINT=https://hf-mirror.com

⚡ Performance Improvements

• Enhanced Metric Efficiency: All metrics now process 1080×800 images with:
  • ⏱️ Sub-second inference time on NVIDIA V100 GPU
  • 💾 Memory efficient: Most metrics require <6GB GPU memory. Exceptions: qalign and qalign_8bit
  • 📊 Detailed results available in efficiency benchmark

📚 Development Updates

• Dataset Organization: Split information consolidated into meta CSV files
  • 🔍 Available at chaofengc/IQA-PyTorch-Datasets-metainfo

🐛 Bug Fixes

• 🔧 Fixed weight loading functionality (5bc4df8)
• 🔄 Removed imgaug for NumPy 2.x compatibility (336831)
• 🛠️ Resolved LIQE num_patch issue (5a3b3e3) - Thanks @avermilov

✨ New Features

📊 New Metrics

• Add perceptual color difference metric msswd proposed in MS-SWD (ECCV2024). (3835ad9)
• Added lpips+ and lpips-vgg+ (b57dede)
• Implemented piqe metric (5b5afb3)
• Added MATLAB-compatible niqe_matlab and brisque_matlab (065c2b7)
• Introduced qalign_8bit and qalign_4bit (e225d8f)

🛠️ Enhanced Functionality

• Added approximate score ranges for metrics (cbba398)
• Implemented optional input validation (0aa6f71)

⚡ Performance & Usability Improvements

🚀 Speed Optimizations

• 📥 Faster loading through simplified imports and lazy loading with class mapper cache
• ⚡ 5x GPU performance boost for nrqm and pi metrics
• 🔋 Enhanced efficiency for ilniqe and piqe through matrix operations

💻 Command Line Interface

• 🔧 Improved CLI syntax:

  pyiqa [metric_name(s)] -t [image_path or dir] -r [image_path or dir] --device cuda --verbose

📖 Documentation

• 📚 Comprehensive documentation updates and improvements

👥 Contributors

Thank you to our new contributors for their valuable input:

• 🌟 @jaycecd: Score range improvements (#180)
• 🌟 @Luciennnnnnn: Cache strategy enhancement (#192)
• 🌟 @RistoAle97: Typo fixes (#197)

📝 For complete details, see the Full Changelog

pyiqa v.0.1.12

❗Important Change: Remove set_random_seed in test process.

In this version, set_random_seed has been removed due to its potential negative impact on the training process. For metrics that were previously tested on multiple crops, we have adopted a uniform cropping approach to ensure consistent results. This method aims to cover the entire image as much as possible, providing a fairer evaluation.
Please note that this change may lead to slight differences in the following metrics. Below is a comparison of results on sample images:

Metrics	I03.bmp	I04.bmp	I06.bmp	I08.bmp	I19.bmp	average
ahiq	-0.1059	0.5855	0.7066	0.5176	0.2566	0.3921
v0.1.12	-0.0996	0.5952	0.7331	0.5710	0.2196	0.4039
hyperiqa	0.1572	0.7397	0.7056	0.7204	0.2898	0.5226
v0.1.12	0.1692	0.7270	0.6972	0.7130	0.3105	0.5234
maniqa	0.2518	0.6176	0.4449	0.5197	0.2309	0.430
v0.1.12	0.2320	0.6033	0.4573	0.5201	0.2670	0.415
tres	27.86	85.95	87.19	95.53	80.02	75.31
v0.1.12	25.67	85.64	85.31	96.49	77.49	74.12

📢 Fix Bugs

1. Fix bug in inference of liqe. 0508684

✨ New features

1. Improve codes to allow training of lpips, dists, pieapp 036f2cc
2. Add distributed training. 18f123e

🛠️ Improvements

1. Update demo codes. c8ae6a7
2. Add alert to make README more readable. bee2c14

🍻 New Contributors

• Add ARNIQA by @LorenzoAgnolucci at #158
• Update requirements @shakedel by at #164

Many thanks to the valuable contributions 🤗 !

Full Changelog: v0.1.11...v0.1.12

pyiqa v0.1.11

📢 Fix Bugs

1. 🐛 fix topiq_nr-face multiple inference error
2. 📝 update distributed train

✨ New features

You can now use pyiqa in terminal like this:

# list all available metrics
pyiqa -ls

# test with default settings
pyiqa [metric_name(s)] --target [image_path or dir] --ref [image_path or dir]

1. ✨ add wadiqam pretrained models
2. ✨ add qalign
3. 🔨 add scandir_images func
4. 🚩 add inception_score
5. 🚩 add console entry point with pyiqa command

🛠️ Improvements

1. add star-history
2. 🔧 set seed for every forward in test mode

🍻 New Contributors

• Add UNIQUE by @zwx8981 at #146

Many thanks to the valuable contributions 🤗 !

Full Changelog: v0.1.10...v0.1.11

pyiqa v0.1.10

📢 Fix Bugs

1. fix vertical flip in data transforms 872e6ca
2. fix bug of fid calculation for different sizes 2da824d
3. fix maniqa hyperparameter error 33a2829
4. fix device handling 6b1547f

✨ New features

1. Add huggingface link to download datasets 031fafd
2. Add liqe liqe_mix by @zwx8981
3. Add nima-koniq and nima-spaq bd385c7

🛠️ Improvements

1. Improve training pipeline 43fb392
2. Add more error messages in fid c36d31c
3. Improve error message for FR and demo codes 921bd75

🍻 New Contributors

• Improve SSIM numerical stability by transforming operations into float64 by @Luciennnnnnn in #125
• Add LIQE by @zwx8981 in #127
  Many thanks to their valuable contributions 🤗 !

Full Changelog: v0.1.8...v0.1.10

pyiqa v0.1.8

📢 Fix Bugs

1. Fix lower_better for FID be5d8c7
2. Change mad option test_y_channel to be false by default cf84ffb
3. Set read rgb to true in inference model to force read RGB image ad93cd4

✨ New features

1. Add topiq_fr, topiq_nr, topiq_nr-face, topiq_iaa etc. introduced by our paper TOPIQ
2. Add st-lpips by @abhijay9 in #93. Many thanks ❤️ !
3. Add laion_aes introduced by LAION-Aesthetics_Predictor
4. Add datasets PIQ2023, GFIQA f46faae

🛠️ Improvements

1. Add documents: https://iqa-pytorch.readthedocs.io
2. Update to torchvision>=0.13, torch>=1.12
3. Improve dataset api 6fbae36
4. Meta information files for training can be download automatically now.
5. Update evaluation protocol and results 2770a7e
6. Update results of maniqa 7ee5ea6
7. Add assertion for brisque to force gray input 4e2c707

New Contributors

• @abhijay9 made their first contribution in #93

Full Changelog: v0.1.7...v0.1.8

pyiqa v0.1.7

📢 Important Changes & Bug Fix

1. Fix verbose option in FID 49b2297
2. Fix grad backpropagation for as_loss=True e027618
3. Fix niqe with gray scale input 212ecef

✨ New features

1. Add metric uranker b142d1c
2. Add metric maniqa-koniq, maniqa-kadid fe95923
3. Add metric clipscore for image-caption matching ecb3e5e
4. Add metric entropy to calculate gray scale image entropy like matlab 5f6d4fb
5. Add pytest cases for results calibration, datasets loading and gradient backward. bc5e135

🛠️ Improvements

1. Recursively find images in folder for FID calculation d7ade54
2. Add metric output doc in model cards

Hotfix of NRQM & PI

🚨 Hotfix: NRQM & PI Calculation Bug Resolved

We have identified and resolved a critical bug in the NRQM calculation within our toolbox. The issue stemmed from the SSIM (Structural Similarity Index) function, where only the structure similarity score was being utilized. We apologize for any inconvenience this may have caused.

As a result of this bug, the PI, which is determined by the formula PI = 0.5 * (10 - NRQM + NIQE), was also affected.

With this hotfix, we have:

• Updated the NRQM calculation to correctly incorporate all relevant components of the SSIM function.
• Adjusted the PI calculation to reflect the corrected NRQM values.

We strongly recommend using the latest release to benefit from these crucial fixes.

If you encounter any issues or have further questions, please don't hesitate to reach out to our support team. Thank you for your understanding and continued support.

IQA-PyTorch v0.1.6

⚠️ Important Changes & Bug Fix

1. Fix OOM on GPU for NRQM fcb7f6e
2. Fix device problem for gradient calculation 08f8850
3. Fix bug for small image test of NIQE 8d7462d
4. Fix default dataset config 1baa70e
5. Fix clip installation error 53d176f
6. feat: add psnry for y colorspace; ssimc for RGB ssim

New features

1. Add CNNIQA, TreS
2. Pass forward argument to inference model
3. Add loss reduction when using metric as loss f03d7f1

Improvements

1. Update benchmark results
2. Update clipiqa+