Awesome Image Harmonization

A curated list of resources including papers, datasets, and relevant links pertaining to image harmonization.

Contributing

Contributions are welcome. If you wish to contribute, feel free to send a pull request. If you have suggestions for new sections to be included, please raise an issue and discuss before sending a pull request.

Online Demo
Leaderboard
Color Transfer
Papers
Datasets
Other Resources
Related Topics

Online Demo

Try this online demo for image harmonization and have fun!

Leaderboard

The leaderboard of SOTA image harmonization methods can be found here.

Color Transfer

We summarize different color transfer strategies which could be used for image harmonization task here.

Papers

Supervised deep learning methods

Wenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, Liqing Zhang: "High-Resolution Image Harmonization via Collaborative Dual Transformations." CVPR (2022) [arXiv] [dataset] (high-resolution)
Zhongyun Bao, Chengjiang Long, Gang Fu, Daquan Liu, Yuanzhen Li, Jiaming Wu, Chunxia Xiao: "Deep Image-based Illumination Harmonization." CVPR (2022) [arXiv] (rendered images)
Yucheng Hang, Bin Xia, Wenming Yang, Qingmin Liao: "SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonization." CVPR (2022) [arXiv] [code]
Ziyue Zhu, Zhao Zhang, Zheng Lin, Ruiqi Wu, Chunle Guo: "Image Harmonization by Matching Regional References." arXiv preprint arXiv:2204.04715 (2022) [arXiv]
Jingtang Liang, Xiaodong Cun, and Chi-Man Pun: "Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization." arXiv preprint arXiv:2109.05750 (2021) [arXiv] [code] (high-resolution)
Wenyan Cong, Junyan Cao, Li Niu, Jianfu Zhang, Xuesong Gao, Zhiwei Tang, Liqing Zhang: "Deep Image Harmonization by Bridging the Reality Gap." arXiv preprint arXiv:2109.06671 (2021) [arXiv] [dataset] (rendered images)
Zhongyun Hu, Ntumba Elie Nsampi, Xue Wang, Qing Wang: "NeSF: Neural Shading Field for Image Harmonization." arXiv preprint arXiv:2112.01314 (2021) [arXiv] (rendered images)
Zonghui Guo, Dongsheng Guo, Haiyong Zheng, Zhaorui Gu, Bing Zheng, Junyu Dong: "Image Harmonization with Transformer." ICCV (2021) [pdf] [supp] [code]
Yifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, Zhangyang Wang: "SSH: A Self-Supervised Framework for Image Harmonization." ICCV (2021) [pdf] [supp] [arXiv] [code]
Jun Ling, Han Xue, Li Song, Rong Xie, Xiao Gu: "Region-Aware Adaptive Instance Normalization for Image Harmonization." CVPR (2021) [pdf] [supp] [arXiv] [code]
Zonghui Guo, Haiyong Zheng, Yufeng Jiang, Zhaorui Gu, Bing Zheng: "Intrinsic Image Harmonization." CVPR (2021) [pdf] [supp] [code]
Wenyan Cong, Li Niu, Jianfu Zhang, Jing Liang, Liqing Zhang: "BargainNet: Background-Guided Domain Translation for Image Harmonization." ICME (2021) [arXiv] [code]
Konstantin Sofiiuk, Polina Popenova, Anton Konushin: "Foreground-aware Semantic Representations for Image Harmonization." WACV (2021) [pdf] [supp] [arXiv] [code]
Guoqing Hao, Satoshi Iizuka, Kazuhiro Fukui: "Image Harmonization with Attention-based Deep Feature Modulation." BMVC (2020) [pdf] [supp] [code]
Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, Liqing Zhang: "DoveNet: Deep Image Harmonization via Domain Verification." CVPR (2020) [pdf] [supp] [arXiv] [code].
Xiaodong Cun, Chi-Man Pun: "Improving the Harmony of the Composite Image by Spatial-Separated Attention Module." IEEE Trans. Image Process. 29: 4759-4771 (2020) [pdf] [arXiv] [code]
Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Xin Lu, Ming-Hsuan Yang: "Deep Image Harmonization." CVPR (2017) [pdf] [supp] [arXiv] [code]

Unsupervised deep learning methods

Anand Bhattad, David A. Forsyth: "Cut-and-Paste Neural Rendering." arXiv preprint arXiv: 2010.05907 (2020) [arXiv] [supp]
Fangneng Zhan, Shijian Lu, Changgong Zhang, Feiying Ma, Xuansong Xie:"Adversarial Image Composition with Auxiliary Illumination." ACCV (2020) [pdf] [arXiv]
Bor-Chun Chen, Andrew Kae: "Toward Realistic Image Compositing With Adversarial Learning." CVPR (2019) [pdf]

Traditional methods

Shuangbing Song, Fan Zhong, Xueying Qin, Changhe Tu: "Illumination Harmonization with Gray Mean Scale." Advances in Computer Graphics. CGI (2020) [pdf]
Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, Alexei A. Efros: "Learning a Discriminative Model for the Perception of Realism in Composite Images." ICCV (2015) [pdf] [arXiv] [code]
Su Xue, Aseem Agarwala, Julie Dorsey, Holly E. Rushmeier: "Understanding and improving the realism of image composites." ACM Trans. Graph. 31(4): 84:1-84:10 (2012) [pdf]
Kalyan Sunkavalli, Micah K. Johnson, Wojciech Matusik, Hanspeter Pfister: "Multi-scale image harmonization." ACM Trans. Graph. 29, 4 (2010) [pdf]
Jue Wang, Maneesh Agrawala, Michael F. Cohen. 2007: "Soft scissors: an interactive tool for realtime high quality matting." ACM Trans. Graph. 26, 3 (2007) [pdf]
Jean-François Lalonde, Alexei A. Efros: "Using Color Compatibility for Assessing Image Realism." ICCV (2007) [pdf] [code]
Daniel Cohen-Or, Olga Sorkine, Ran Gal, Tommer Leyvand, Ying-Qing Xu: "Color harmonization." ACM Trans. Graph. 25, 3 (2006) [pdf]
Jiaya Jia, Jian Sun, Chi-Keung Tang, Heung-Yeung Shum: "Drag-and-drop pasting." ACM Trans. Graph. 25, 3 (2006) [pdf]
Patrick Pérez, Michel Gangnet, Andrew Blake: "Poisson image editing." ACM Trans. Graph. 22, 3 (2003) [pdf]

Datasets

iHarmony4: It contains four subdatasets: HCOCO, HAdobe5k, HFlickr, Hday2night, with a total of 73,146 pairs of unharmonized images and harmonized images. [pdf] [link]
GMSDataset: It contains 183 images with image resolution of 1940*1440. It consists of 16 different objects and for each object, one source image and 11 target images in different background scenes and illumination conditions are captured. [pdf] [link] (access code: ekn2)
HVIDIT: A dataset built upon VIDIT (Virtual Image Dataset for Illumination Transfer) dataset for image harmonization. It contains 3007 images of 276 scenes for training and 329 images of 24 scenes for testing. [pdf] [link]
RHHarmony: A rendered human harmonization dataset, which contains 15,000 ground-truth rendered images and has the potential to generate 135,000 composite rendered images. [pdf] [link]
RealHM: A Real-world HarMonization dataset, which contains 216 real composite images with manually harmonized outputs. [pdf] [link]
ccHarmony: An image harmonization dataset constructed based on the images captured with color checker (cc). It contains 10 synthetic composite images for each of 426 foregrounds from 350 real images, leading to 4260 pairs of synthetic composite images and ground-truth real images. [pdf] [link]

tsing-coco / awesome-image-harmonization Goto Github PK