iceclear / clip-iqa Goto Github PK

View Code? Open in Web Editor NEW

296.0 4.0 18.0 30.99 MB

[AAAI 2023] Exploring CLIP for Assessing the Look and Feel of Images

License: Other

Python 99.88% Shell 0.12%

iqa clip

clip-iqa's Introduction

Exploring CLIP for Assessing the Look and Feel of Images (AAAI 2023)

Paper

Jianyi Wang, Kelvin C.K. Chan, Chen Change Loy

S-Lab, Nanyang Technological University

TODO

Colab demo
~~MMEditing update~~
~~Code release~~

Dependencies and Installation

The same as MMEditing, support the latest version 0.16.1.

# Create a conda environment and activate it
conda create -n clipiqa python=3.8 -y
conda activate clipiqa
# Install PyTorch following official instructions, e.g.
conda install pytorch=1.10 torchvision cudatoolkit=11.3 -c pytorch
# Install pre-built MMCV using MIM.
pip3 install openmim
mim install mmcv-full==1.5.0
# Install CLIP-IQA from the source code.
git clone [email protected]:IceClear/CLIP-IQA.git
cd CLIP-IQA
pip install -r requirements.txt
pip install -e .

Running Examples

Test CLIP-IQA on KonIQ-10k

python demo/clipiqa_koniq_demo.py

Test CLIP-IQA on Live-iWT

python demo/clipiqa_liveiwt_demo.py

Train CLIP-IQA+ on KonIQ-10k

# Support dist training as MMEditing
python tools/train.py configs/clipiqa/clipiqa_coop_koniq.py

Test CLIP-IQA+ on KonIQ-10k (Checkpoint)

python demo/clipiqa_koniq_demo.py --config configs/clipiqa/clipiqa_coop_koniq.py --checkpoint ./iter_80000.pth

[Note] You may change prompts for different datasets, please refer to config files for details.

[Note] For testing on a single image, please refer to here for details.

Other Implementations

Demo

✨ Versatile Quality Assessment

✨ Demo for IQA on SPAQ

✨ Demo for Abstract Perception on AVA

For more evaluation, please refer to our paper for details.

Citation

If our work is useful for your research, please consider citing:

@inproceedings{wang2022exploring,
    author = {Wang, Jianyi and Chan, Kelvin CK and Loy, Chen Change},
    title = {Exploring CLIP for Assessing the Look and Feel of Images},
    booktitle = {AAAI},
    year = {2023}
}

License

This project is licensed under NTU S-Lab License 1.0. Redistribution and use should follow this license.

Acknowledgement

This project is based on MMEditing and CLIP. Thanks for their awesome works.

Contact

If you have any question, please feel free to reach me out at [email protected].

clip-iqa's People

Contributors

Stargazers

Watchers

Forkers

teowu yzhq97 yingchaojiefeng redredredc yhtbyr tarah7579 xianchaozhang yoshida21 cyanymore zemooo7 sunpro108 david-hown haitian2du trishamazu harry2345 kp-forks aaririri yaos666

clip-iqa's Issues

Different image inferences with same result

Hi,

Im doing different tests with demo/clipiqa_single_image_demo.py and the attribute_list = ['Quality', 'Brightness', 'Sharpness', 'Noisiness', 'Colorfulness', 'Contrast'].

First, I’ve seen that fitting a good size to the input image is essential because in some cases the result is NaN. Is there a fixed size that should be adjusted in the inbound image?

In the tests I’m doing, regardless of the resize, I see that it doesn’t matter the input image the result is always the same, you know what might be?

Example:

MSI@DESKTOP-FEG9P7H MINGW64 /e/2. Projects/Image Quality/CLIP-IQA (v2-3.8)
$ python demo/clipiqa_single_image_demo.py --config configs/clipiqa/clipiqa_attribute_test.py --file_path dataset/good_1.jpg 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
['Quality', 'Brightness', 'Sharpness', 'Noisiness', 'Colorfulness', 'Contrast', 'Quality']
[0.9892578  0.98876953 0.99853516 0.06512451 0.74316406 0.66796875]
(clipiqa) 
MSI@DESKTOP-FEG9P7H MINGW64 /e/2. Projects/Image Quality/CLIP-IQA (v2-3.8)
$ python demo/clipiqa_single_image_demo.py --config configs/clipiqa/clipiqa_attribute_test.py --file_path dataset/bad_3.jpeg 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
['Quality', 'Brightness', 'Sharpness', 'Noisiness', 'Colorfulness', 'Contrast', 'Quality']
[0.9892578  0.98876953 0.99853516 0.06512451 0.74316406 0.66796875]
(clipiqa) 
MSI@DESKTOP-FEG9P7H MINGW64 /e/2. Projects/Image Quality/CLIP-IQA (v2-3.8)
$ python demo/clipiqa_single_image_demo.py --config configs/clipiqa/clipiqa_attribute_test.py --file_path dataset/blur_2.tif
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
['Quality', 'Brightness', 'Sharpness', 'Noisiness', 'Colorfulness', 'Contrast', 'Quality']
[0.9892578  0.98876953 0.99853516 0.06512451 0.74316406 0.66796875]
(clipiqa)

About dataset setting.

Hi!
I am just wondering about the CLIPIQA+ experiment mentioned in the paper. In Table 1, it says that the model was trained on Koniq and tested on Koniq, LIVE-itW, and SPAQ. So, I was curious about the specifics of the training process. Did it involve training on the official Koniq-train dataset, selecting the best parameters on Koniq-valid, and testing on Koniq-test, LIVE-itW, and SPAQ?

stack expects each tensor to be equal size

我用我自己的数据集训练，为啥会出现这个错误

Would it be possible to convert this to onnx?

Non public data availability

Hello, can this code be tested and trained on non-public datasets, such as one's own dataset? How to set up and process data?

L1DIS metric support problem

Hi IceClear,
I just try to train CLIP-IQA+ on KonIQ-10k as your instruct:

python tools/train.py configs/clipiqa/clipiqa_coop_koniq.py

But encounter the following error, it seems only PSNR/SSIM supported.
Did this mean I have to change the config option L1DIS to PSNR/SSIM?

[                                                  ] 0/2015, elapsed: 0s, ETA:Traceback (most recent call last):
  File "tools/train.py", line 146, in <module>
    main()
  File "tools/train.py", line 135, in main
    train_model(
  File "d:\code\clip-iqa\mmedit\apis\train.py", line 106, in train_model
    _non_dist_train(
  File "d:\code\clip-iqa\mmedit\apis\train.py", line 362, in _non_dist_train
    runner.run(data_loaders, cfg.workflow, cfg.total_iters)
  File "C:\Users\A\.conda\envs\clipiqa\lib\site-packages\mmcv\runner\iter_based_runner.py", line 134, in run
    iter_runner(iter_loaders[i], **kwargs)
  File "C:\Users\A\.conda\envs\clipiqa\lib\site-packages\mmcv\runner\iter_based_runner.py", line 67, in train
    self.call_hook('after_train_iter')
  File "C:\Users\A\.conda\envs\clipiqa\lib\site-packages\mmcv\runner\base_runner.py", line 309, in call_hook
    getattr(hook, fn_name)(self)
  File "d:\code\clip-iqa\mmedit\core\evaluation\eval_hooks.py", line 42, in after_train_iter
    results = single_gpu_test(
  File "d:\code\clip-iqa\mmedit\apis\test.py", line 43, in single_gpu_test
    result = model(
  File "C:\Users\A\.conda\envs\clipiqa\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\A\.conda\envs\clipiqa\lib\site-packages\mmcv\parallel\data_parallel.py", line 50, in forward
    return super().forward(*inputs, **kwargs)
  File "C:\Users\A\.conda\envs\clipiqa\lib\site-packages\torch\nn\parallel\data_parallel.py", line 166, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "C:\Users\A\.conda\envs\clipiqa\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\A\.conda\envs\clipiqa\lib\site-packages\mmcv\runner\fp16_utils.py", line 110, in new_func
    return old_func(*args, **kwargs)
  File "d:\code\clip-iqa\mmedit\models\restorers\basic_restorer.py", line 79, in forward
    return self.forward_test(lq, gt, **kwargs)
  File "d:\code\clip-iqa\mmedit\models\restorers\clipiqa.py", line 190, in forward_test
    results = dict(eval_result=self.evaluate(output, gt))
  File "d:\code\clip-iqa\mmedit\models\restorers\clipiqa.py", line 156, in evaluate
    value = self.allowed_metrics[metric](output_img, gt_img)
KeyError: 'L1DIS'

Train

hello ! thanks your great work, i want to know how to train my own datasets ？and how to make the datasets?

Image transformation methods

Hi there, thank you for publishing such a great work. May I ask what is the resolution for CLIP-IQA? The resolution I found in coopclipiqa.py file is 224x224, but the resize transformation in config files are all commented out. So does this mean that there are some code for image resizing somewhere else or the image resolution is kept the same? The resolution are also not included in the paper as well.

Model loading

When we try to run clipiqa_single_image_demo.py and load pre-trained model, there will be the following problems:

unexpected key in source state_dict: generator.clipmodel_0.logit_scale, generator.clipmodel_0.prompt_learner.ctx,.......
missing keys in source state_dict: generator.clip_model.positional_embedding, generator.clip_model.text_projection,......

How should I solve it?

如何在cpu上运行demo

Current iteration 300000 exceeds cumulative_periods

Traceback (most recent call last):
File "tools/train.py", line 146, in
main()
File "tools/train.py", line 135, in main
train_model(
File "/home/www/application/models/CLIP-IQA/mmedit/apis/train.py", line 106, in train_model
_non_dist_train(
File "/home/www/application/models/CLIP-IQA/mmedit/apis/train.py", line 362, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_iters)
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/runner/iter_based_runner.py", line 134, in run
iter_runner(iter_loaders[i], **kwargs)
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/runner/iter_based_runner.py", line 60, in train
self.call_hook('before_train_iter')
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 309, in call_hook
getattr(hook, fn_name)(self)
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/runner/hooks/lr_updater.py", line 140, in before_train_iter
self.regular_lr = self.get_regular_lr(runner)
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/runner/hooks/lr_updater.py", line 75, in get_regular_lr
_lr_group = [
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/runner/hooks/lr_updater.py", line 76, in
self.get_lr(runner, _base_lr)
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/runner/hooks/lr_updater.py", line 387, in get_lr
idx = get_position_from_periods(progress, self.cumulative_periods)
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/runner/hooks/lr_updater.py", line 415, in get_position_from_periods
raise ValueError(f'Current iteration {iteration} exceeds '

A question encountered in clipiqa_single_image_demo.py

After running this code：output, attributes = restoration_inference(model, os.path.join(args.file_path), return_attributes=True)

output and attributes are the same. Why return the same two data?

Code running result：

attributes>>>>>>>>>>>>>>>>>>
tensor([[0.9644, 0.3345, 0.9995, 0.0357, 0.2766, 0.3577]])
output>>>>>>>>>>>>>>>>>>
[[0.96435547 0.33447266 0.9995117 0.03567505 0.27661133 0.35766602]]

Code without OpenMMLab integration?

@IceClear Any chance to get the code without using OpenMMLab? Would make it easier for experimenting. Thanks for the awesome work by the way.

Result nan

Hello,

I have been doing tests with the clipiqa_single_image_demo.py and the "test degradation attributes" and sometimes the result is nan.
It is true that for some images it returns scores, but usually not.

python demo/clipiqa_single_image_demo.py --config configs/clipiqa/clipiqa_attribute_test.py --checkpoint checkpoint/iter_80000.pth --file_path dataset/img.jpg

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
['Quality', 'Brightness', 'Sharpness', 'Noisiness', 'Colorfulness', 'Contrast', 'Quality']
[0.9892578  0.98876953 0.99853516 0.06488037 0.7416992  0.66796875]

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
['Quality', 'Brightness', 'Sharpness', 'Noisiness', 'Colorfulness', 'Contrast', 'Quality']
[nan nan nan nan nan nan]

Do you know why this can be and how to fix it?

how to infer image directly

how to infer image directly not by load image form path ，it means that I just input a image array to the main function not input a image path？

can you provide koniq10k_distributions_sets.csv and koniq_resize.txt

thanks for this excellent work! can you provide the csv file and the txt file for training?
thanks in advance.

interpolation

Thank you for your work
In the paper you mentioned interpolation scheme -- do you have code to interpolate embedding?
And whats best way to customize CLIP? why you choose mmedit?

evaluate() got an unexpected keyword argument 'gpu_collect' while training clipiqa_coop_koniq

python tools/train.py configs/clipiqa/clipiqa_coop_koniq.py

2023-10-16 08:47:54,004 - mmedit - INFO - workflow: [('train', 1)], max: 500 iters
2023-10-16 08:47:54,004 - mmedit - INFO - Checkpoints will be saved to C:\Users\pdo-labuser\CLIP-IQA\work_dirs\clipiqa_coop_koniq by HardDiskBackend.
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 2015/2015, 17.2 task/s, elapsed: 117s, ETA: 0sTraceback (most recent call last):
File "tools/train.py", line 146, in
main()
File "tools/train.py", line 135, in main
train_model(
File "c:\users\pdo-labuser\clip-iqa\mmedit\apis\train.py", line 106, in train_model
_non_dist_train(
File "c:\users\pdo-labuser\clip-iqa\mmedit\apis\train.py", line 362, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_iters)
File "C:\Users\pdo-labuser\anaconda3\envs\clipiqa\lib\site-packages\mmcv\runner\iter_based_runner.py", line 134, in run
iter_runner(iter_loaders[i], **kwargs)
File "C:\Users\pdo-labuser\anaconda3\envs\clipiqa\lib\site-packages\mmcv\runner\iter_based_runner.py", line 67, in train
self.call_hook('after_train_iter')
File "C:\Users\pdo-labuser\anaconda3\envs\clipiqa\lib\site-packages\mmcv\runner\base_runner.py", line 309, in call_hook
getattr(hook, fn_name)(self)
File "c:\users\pdo-labuser\clip-iqa\mmedit\core\evaluation\eval_hooks.py", line 48, in after_train_iter
self.evaluate(runner, results)
File "c:\users\pdo-labuser\clip-iqa\mmedit\core\evaluation\eval_hooks.py", line 57, in evaluate
eval_res = self.dataloader.dataset.evaluate(
TypeError: evaluate() got an unexpected keyword argument 'gpu_collect'

Please help with resolution

inference in onnx

Hello,
do you have plans to provide the ONNX model weights and inference code at a later time?
Thanks.

怎么在cpu上运行啊

clipiqa+

我按照github上的iqa+的模型训练为什么srcc和plcc指标一半都达不到

如何确定模型训练好了

在该模型的训练过程中，采用
2024-07-24 10:32:17,498 - mmedit - INFO - Iter(val) [262000]
2024-07-24 10:32:23,660 - mmedit - INFO - Iter [262100/300000] lr_generator: 7.783e-05, eta: 1:30:52, time: 0.964, data_time: 0.926, memory: 808, loss_pix: 0.0056, loss: 0.0056
2024-07-24 10:32:29,773 - mmedit - INFO - Iter [262200/300000] lr_generator: 7.743e-05, eta: 1:30:37, time: 0.061, data_time: 0.023, memory: 808, loss_pix: 0.0058, loss: 0.0058
2024-07-24 10:32:35,682 - mmedit - INFO - Iter [262300/300000] lr_generator: 7.702e-05, eta: 1:30:21, time: 0.059, data_time: 0.022, memory: 808, loss_pix: 0.0064, loss: 0.0064
2024-07-24 10:32:42,025 - mmedit - INFO - Iter [262400/300000] lr_generator: 7.662e-05, eta: 1:30:06, time: 0.063, data_time: 0.024, memory: 808, loss_pix: 0.0049, loss: 0.0049
2024-07-24 10:32:47,888 - mmedit - INFO - Iter [262500/300000] lr_generator: 7.622e-05, eta: 1:29:50, time: 0.059, data_time: 0.023, memory: 808, loss_pix: 0.0065, loss: 0.0065
2024-07-24 10:32:53,844 - mmedit - INFO - Iter [262600/300000] lr_generator: 7.582e-05, eta: 1:29:35, time: 0.060, data_time: 0.022, memory: 808, loss_pix: 0.0077, loss: 0.0077
2024-07-24 10:32:59,860 - mmedit - INFO - Iter [262700/300000] lr_generator: 7.542e-05, eta: 1:29:19, time: 0.060, data_time: 0.022, memory: 808, loss_pix: 0.0071, loss: 0.0071
2024-07-24 10:33:05,857 - mmedit - INFO - Iter [262800/300000] lr_generator: 7.502e-05, eta: 1:29:03, time: 0.060, data_time: 0.024, memory: 808, loss_pix: 0.0062, loss: 0.0062
2024-07-24 10:33:12,004 - mmedit - INFO - Iter [262900/300000] lr_generator: 7.463e-05, eta: 1:28:48, time: 0.061, data_time: 0.023, memory: 808, loss_pix: 0.0052, loss: 0.0052
2024-07-24 10:34:51,582 - mmedit - INFO - Exp name: clipiqa_coop_koniq
2024-07-24 10:34:51,582 - mmedit - INFO - Iter(val) [263000] L1DIS: 0.0579
之前的模型保存均为loss最小或者val准确率最大，目前该模型的训练指标是啥

RecursionError: maximum recursion depth exceeded

运行过程中突然
ile "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in collate
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 77, in
return [collate(samples, samples_per_gpu) for samples in transposed]
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/site-packages/mmcv/parallel/collate.py", line 23, in collate
if not isinstance(batch, Sequence):
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/abc.py", line 98, in instancecheck
return _abc_instancecheck(cls, instance)
File "/home/www/application/miniconda3/envs/clipiqa/lib/python3.8/abc.py", line 102, in subclasscheck
return _abc_subclasscheck(cls, subclass)
RecursionError: maximum recursion depth exceeded

Pretrained model iter80000.pth not loading

Hi @IceClear , I am trying to run the demo for a single image, but the pretrained model provided is taking forever to load. Infact, doesn't load at all.

This is the line with the issue:

model = init_model(
        args.config, args.checkpoint, device=torch.device('cuda', args.device))

This is the output I get:

Initializing a generic context
Initial context: "X X X X X X X X X X X X X X X X"
Number of context words (tokens): 16
2023-02-20 03:16:18.505897: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-20 03:16:18.939280: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.

After which it just doesn't complete loading.
Im using config file: ./configs/clipiqa/clipiqa_coop_koniq.py
and checkpoint: ./iter_80000.pth as provided and mentioned.

Please advice.

Only return single attribute value for inferencing single image demo

Hi @IceClear,

Thank you for open sourcing this impressive work! I have a question regarding the clipiqa_single_image_demo.py script. When I use this script to do inference on a single image, I notice that only the first value of the attribute list is returned. The output looks something like this: [0.74316406] (using the example rock image).

The print info would be something like:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[0.74316406]

Could you please provide some guidance on how to obtain all the attribute values, such as 'Brightness', 'Sharpness', 'Noisiness', 'Colorfulness', and 'Contrast', which may differ from each other?

Thank you very much for your help!

init model error

when initing the model the error
*** urllib.error.URLError: <urlopen error CLIPIQAFixed: <urlopen error [Errno 104] Connection reset by peer>>

How do I train my own data set？

HI~ Thank you for your work！
I want to train a model that recognizes cars and boats, is the train config and CSV file labeled correctly?

problem: CLIPIQA is not in the model registry

Hi!
I try to run clipiqa_single_image_demo.py, but it tells me "CLIPIQA is not in the model registry".
Could you please tell me how to fix this problem? Thanks!

Arise RuntimeError: CUDA out of memory for larget image

Hi IceClear,
Thank you for your impressive work! I encounter OOM issue when handle large image with clipiqa_single_image_demo.py, such as 8M/12M pictures. I want to know is there any memery limited for the CLIP-IQA model. Or whether my env runtime had configuration issue?

Here's the detail output logs:

clipiqa) PS D:\code\CLIP-IQA> python .\demo\clipiqa_single_image_demo.py --file_path=../dataset/IMG_20230316_105935577.jpg
Traceback (most recent call last):
File ".\demo\clipiqa_single_image_demo.py", line 62, in
main()
File ".\demo\clipiqa_single_image_demo.py", line 38, in main
output, attributes = restoration_inference(model, os.path.join(args.file_path), return_attributes=True)
File "d:\code\clip-iqa\mmedit\apis\restoration_inference.py", line 79, in restoration_inference
result = model(test_mode=True, **data)
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\mmcv\runner\fp16_utils.py", line 110, in new_func
return old_func(*args, **kwargs)
File "d:\code\clip-iqa\mmedit\models\restorers\basic_restorer.py", line 79, in forward
return self.forward_test(lq, gt, **kwargs)
File "d:\code\clip-iqa\mmedit\models\restorers\clipiqa.py", line 182, in forward_test
output, attribute_prob = self.generator(lq)
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "d:\code\clip-iqa\mmedit\models\backbones\sr_backbones\coopclipiqa.py", line 314, in forward
logits_per_image, logits_per_text = self.clip_model(image, self.tokenized_prompts[i].to(image.device), self.pos_embedding)
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "d:\code\clip-iqa\mmedit\models\components\clip\model.py", line 373, in forward
image_features = self.encode_image(image, pos_embedding)
File "d:\code\clip-iqa\mmedit\models\components\clip\model.py", line 355, in encode_image
return self.visual(image.type(self.dtype), pos_embedding=pos_embedding)
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "d:\code\clip-iqa\mmedit\models\components\clip\model.py", line 158, in forward
x = self.attnpool(x, return_token, pos_embedding)
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "d:\code\clip-iqa\mmedit\models\components\clip\model.py", line 72, in forward
x, _ = F.multi_head_attention_forward(
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\torch\nn\functional.py", line 5101, in multi_head_attention_forward
attn_output, attn_output_weights = _scaled_dot_product_attention(q, k, v, attn_mask, dropout_p)
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\torch\nn\functional.py", line 4847, in _scaled_dot_product_attention
attn = softmax(attn, dim=-1)
File "C:\Users\A.conda\envs\clipiqa\lib\site-packages\torch\nn\functional.py", line 1680, in softmax
ret = input.softmax(dim)
RuntimeError: CUDA out of memory. Tried to allocate 8.86 GiB (GPU 0; 12.00 GiB total capacity; 9.59 GiB already allocated; 0 bytes free; 9.70 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Could u please give me some advice?

Run on CPU?

Is there a straightforward way to run the model on CPU?

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/A9dCpjHPfE or add me on WeChat (ID: van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

	OpenMMLab 1.0 branch	OpenMMLab 2.0 branch
MMEngine		0.x
MMCV	1.x	2.x
MMDetection	0.x 、1.x、2.x	3.x
MMAction2	0.x	1.x
MMClassification	0.x	1.x
MMSegmentation	0.x	1.x
MMDetection3D	0.x	1.x
MMEditing	0.x	1.x
MMPose	0.x	1.x
MMDeploy	0.x	1.x
MMTracking	0.x	1.x
MMOCR	0.x	1.x
MMRazor	0.x	1.x
MMSelfSup	0.x	1.x
MMRotate	0.x	1.x
MMYOLO		0.x

Attention: please create a new virtual environment for OpenMMLab 2.0.