- [Multi-Reference-Based Cross-Scale Feature Fusion for Compressed Video Super Resolution ]
PyTorch implementation of Multi-Reference-Based Cross-Scale Feature Fusion for Compressed Video Super Resolution
Feel free to contact: [email protected].
- UBUNTU 20.04/18.04
- CUDA 10.1
- PYTORCH 1.6
- packages: TQDM, LMDB, PYYAML, OPENCV-PYTHON, SCIKIT-IMAGE
Suppose that you have installed CUDA 10.1, then:
git clone --depth=1 https://github.com/lowlow-chen/RCVSR
conda create -n stdf python=3.7 -y
conda activate rcvsr
python -m pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
python -m pip install tqdm lmdb pyyaml opencv-python scikit-imageBuild DCNv2.
cd ops/dcn/
bash build.sh(Optional) Check if DCNv2 works.
python simple_check.pyThe DCNv2 source files here is different from the open-sourced version due to incompatibility. [issue]
Download the raw dataset
> MFQEv2 dataset includes 108 lossless YUV sequences for training, and 18 test sequences recommended by ITU-T.
Compress sequences
We now compress both training and test sequences by HM16.5 at LDP mode, QP=27-42. we will get:
mfqe/
├── train_raw_yuv/
│── train_qp_yuv/
│ ├── QP32/
│── train_qp_down/
│ ├── QP27/
├── test_raw_yuv/
│── test_qp_yuv/
│ ├── QP32/
│── test_qp_down/
│ ├── QP27/
└── README.mdEdit YML
We now edit option_R3_mfqev2_2G.yml.
Suppose the folder mfqe/ is placed at /home/***/mfqe/, then you should assign /home/***/mfqe/ to dataset -> train -> root in YAML.
R3: one of the network structures provided in the paper;mfqev2: MFQEv2 dataset will be adopted;2G: 2 GPUs will be used for the below training. Similarly, you can also editoption_R3_mfqev2_1G.ymlif needed.
Generate LMDB
We now generate LMDB to speed up IO during training.
python create_lmdb_mfqev2.py --opt_path option_R3_mfqev2_2G.ymlNow you will get all needed data:
mfqe/
├── train_raw_yuv/
│── train_qp_yuv/
│ ├── QP32/
│── train_qp_down/
│ ├── QP27/
├── test_raw_yuv/
│── test_qp_yuv/
│ ├── QP32/
│── test_qp_down/
│ ├── QP27/
└── README.md
├── LDP_train_gt_qp27.lmdb/
└── LDP_train_lq_qp27.lmdb/Finally, the mfqe dataset root will be sym-linked to the folder ./data/ automatically.
So that we and programmes can access mfqe dataset at
./data/directly.
# 2 GPUs
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 --master_port=12354 train.py --opt_path option_R3_mfqev2_2G.yml
# 1 GPU
#CUDA_VISIBLE_DEVICES=0 python train.py --opt_path option_R3_mfqev2_1G.ymlPretrained models can be found here: update later...
Test MFQEv2 dataset after training
CUDA_VISIBLE_DEVICES=1 python test.py --opt_path option_R3_mfqev2_1G.ymlTest MFQEv2 dataset without training
If you did not run create_lmdb for training, you should first sym-link MFQEv2 dataset to ./data/.
mkdir data/
ln -s /your/path/to/mfqe/ data/MFQEv2Test your own video
First download the pre-trained model, and then run:
CUDA_VISIBLE_DEVICES=0 python test_one_video.pySee test_one_video.py for more details.
You can use, redistribute, and adapt the material for non-commercial purposes, as long as you give appropriate credit by citing the following paper and indicating any changes that you've made.
@article{RCVSR,
title={Multi-Reference-Based Cross-Scale Feature Fusion for Compressed Video Super Resolution},
author={Chen, Lu and Ye, Mao and Ji, Luping and Li, Shuai and Guo, Hongwei},
journal={IEEE Transactions on Broadcasting},
year={2024},
publisher={IEEE}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent