Giter Club home page Giter Club logo

vztu / rapique Goto Github PK

View Code? Open in Web Editor NEW
46.0 1.0 6.0 1.27 MB

[IEEE OJSP'2021] "RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content", Zhengzhong Tu, Xiangxu Yu, Yilin Wang, Neil Birkbeck, Balu Adsumilli, Alan C. Bovik

Home Page: https://ieeexplore.ieee.org/document/9463703/

License: MIT License

MATLAB 50.56% Python 45.48% Shell 3.96%
video-quality-assessment video-quality image-quality-assessment user-generated-content deep-learning ffmpeg video

rapique's Introduction

PWC PWC PWC

RAPIQUE

An official implementation of Rapid and Accurate Video Quality Evaluator (RAPIQUE) proposed in [IEEE OJSP2021] RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content. Arxiv. IEEExplore(Open Access!) and [PCS2021] Efficient User-Generated Video Quality Prediction. IEEExplore. Note that the temporal features can be used as standalone features in company with spatial models to boost performance on motion-intensive models. Check out the temporal-only modules in [ICIP21] A Temporal Statistics Model For UGC Video Quality Prediction. IEEExplore

Check out our BVQA resource list and performance benchmark/leaderboard results in https://github.com/vztu/BVQA_Benchmark.

For more evaluation codes, please check out VIDEVAL

Requirements

  • MATLAB >= 2019
    • Deep learning toolbox (ResNet-50)
  • Python3
  • Sklearn
  • FFmpeg
  • Git LFS

Performances

SRCC / PLCC

Methods KoNViD-1k LIVE-VQC YouTube-UGC All-Combined
TLVQM 0.7101 / 0.7037 0.7988 / 0.8025 0.6693 / 0.6590 0.7271 / 0.7342
VIDEVAL 0.7832 / 0.7803 0.7522 / 0.7514 0.7787 / 0.7733 0.7960 / 0.7939
MDVSFA 0.7812 / 0.7856 0.7382 / 0.7728 - / - - / -
RAPIQUE 0.8031 / 0.8175 0.7548 / 0.7863 0.7591 / 0.7684 0.8070 / 0.8229

Scatter plots and fitted logistic curves on these datasets:

KonVid-1k LIVE-VQC YouTube-UGC All-Combined

Speed

The unit is average secs/video.

Methods 540p 720p 1080p 4k@60
Video-BLIINDS 341.1 839.1 1989.9 16129.2
VIDEVAL 61.9 146.5 354.5 1716.3
TLVQM 34.5 78.9 183.8 969.3
RAPIQUE 13.5 17.3 18.3 112

Performance vs. Speed

Demos

Feature Extraction Only

demo_compute_RAPIQUE_feats.m

You need to specify the parameters

Evaluation of BVQA Model

We proposed several evaluation methods for BIQA/BVQA models. Please check out [ICASSP21] Regression or classification? New methods to evaluate no-reference picture and video quality models IEEExplore for details.

  • For regression evaluation:
$ python evaluate_bvqa_features_regression.py
  • For binary classification evaluation:
$ python evaluate_bvqa_features_binary_classification.py
  • For ordinal classification evaluation:
$ python evaluate_bvqa_features_ordinal_classification.py

Citation

If you use this code for your research, please cite our papers.

@article{tu2021rapique,
  title={RAPIQUE: Rapid and accurate video quality prediction of user generated content},
  author={Tu, Zhengzhong and Yu, Xiangxu and Wang, Yilin and Birkbeck, Neil and Adsumilli, Balu and Bovik, Alan C},
  journal={IEEE Open Journal of Signal Processing},
  volume={2},
  pages={425--440},
  year={2021},
  publisher={IEEE}
}
@article{tu2021ugc,
  title={UGC-VQA: Benchmarking blind video quality assessment for user generated content},
  author={Tu, Zhengzhong and Wang, Yilin and Birkbeck, Neil and Adsumilli, Balu and Bovik, Alan C},
  journal={IEEE Transactions on Image Processing},
  year={2021},
  publisher={IEEE}
}
@inproceedings{tu2021efficient,
  title={Efficient User-Generated Video Quality Prediction},
  author={Tu, Zhengzhong and Chen, Chia-Ju and Wang, Yilin and Birkbeck, Neil and Adsumilli, Balu and Bovik, Alan C},
  booktitle={2021 Picture Coding Symposium (PCS)},
  pages={1--5},
  year={2021},
  organization={IEEE}
}
@inproceedings{tu2021temporal,
  title={A Temporal Statistics Model For UGC Video Quality Prediction},
  author={Tu, Zhengzhong and Chen, Chia-Ju and Wang, Yilin and Birkbeck, Neil and Adsumilli, Balu and Bovik, Alan C},
  booktitle={2021 IEEE International Conference on Image Processing (ICIP)},
  pages={1454--1458},
  year={2021},
  organization={IEEE}
}
@inproceedings{tu2021regression,
  title={Regression or classification? New methods to evaluate no-reference picture and video quality models},
  author={Tu, Zhengzhong and Chen, Chia-Ju and Chen, Li-Heng and Wang, Yilin and Birkbeck, Neil and Adsumilli, Balu and Bovik, Alan C},
  booktitle={ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={2085--2089},
  year={2021},
  organization={IEEE}
}

rapique's People

Contributors

vztu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

rapique's Issues

MSU Video Quality Metrics Benchmark Invitation

Hello! We kindly invite you to participate in our video quality metrics benchmark. You can submit RAPIQUE (or any other your metrics) to the benchmark, following the submission steps, described here. The dataset distortions refer to compression artifacts on professional and user-generated content. The full dataset is used to measure methods overall performance, so we do not share it to avoid overfitting. Nevertheless, we provided the open part of it (around 1,000 videos) within our paper "Video compression dataset and benchmark of learning-based video-quality metrics", accepted to NeurIPS 2022.

The .mat file is broken

@vztu Thanks for your sharing.When I want to load the feature file provided, I meet the error:
using MatlabR2019b:
错误使用 load
无法读取 MAT 文件 \RAPIQUE-main\feat_files\YOUTUBE_UGC_RAPIQUE_feats.mat。不是二进制 MAT 文件。请尝试执行 load
-ASCII 以便以文本形式读取。

using Python:
ValueError: Unknown mat file type, version 50, 51
Could you help me?Thank you very much!!!

The speed of the RAPIQUE?

@vztu Thanks for your sharing. I have read the paper《RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content》,and "TABLE IV" shows the RAPIQUE's runtime is 17.3. Could you explain the 17.3 for me in details ? I don't find the length of the 1080p video ? 17.3 specifies whether 17.3fps or 17.3 seconds for the whole 1080p video ? And because I find that the RAPIQUE is not end to end, does “17.3” include "compute features time "?Thank you very much!!!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.