Awesome Image Composition

A curated list of resources including papers, datasets, and relevant links pertaining to image composition (object insertion). The goal of image composition is inserting one foreground into a background image to get a realistic composite image, by addressing the inconsistencies (appearance, geometry, and semantic inconsistency) between foreground and background. Generally speaking, image composition could be used to combine the visual elements from different images.

Welcome to follow WeChat public account "Newly AIGCer" or Zhihu Column "Newly CVer" to get the latest information about image composition!

Contributing

Contributions are welcome. If you wish to contribute, feel free to send a pull request. If you have suggestions for new sections to be included, please raise an issue and discuss before sending a pull request.

Table of Contents

• Online Demo
• Survey
• Toolbox
• Papers
• Datasets
• Evaluation

Online Demo

Try this online demo for image composition and have fun!

Survey

• Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, Liqing Zhang: "Making Images Real Again: A Comprehensive Survey on Deep Image Composition." arXiv preprint arXiv:2106.14490 (2021). [arXiv] [slides]

Toolbox

We integrate 10+ image composition related functions into libcom (the library of image composition), including image blending, standard/painterly image harmonization, shadow generation, object placement, generative composition, quality evaluation, etc. The ultimate goal of this library is solving all the problems related to image composition with simple import libcom.

Papers

1. Image Blending

  Awesome-Image-Blending

2. Image Harmonization

  Awesome-Image-Harmonization

3. Object Shadow Generation

  Awesome-Object-Shadow-Generation

4. Object Reflection Generation

  Awesome-Object-Reflection-Generation

5. Object Placement

  Awesome-Object-Placement

6. Perspective Transformation

• Junhong Gou, Bo Zhang, Li Niu, Jianfu Zhang, Jianlou Si, Chen Qian, Liqing Zhang: "Virtual Accessory Try-On via Keypoint Hallucination." arXiv preprint arXiv:2310.17131 (2023) [arXiv]
• Bo Zhang, Yue Liu, Kaixin Lu, Li Niu, Liqing Zhang: "Spatial Transformation for Image Composition via Correspondence Learning." arXiv preprint arXiv:2207.02398 (2022) [arXiv]
• Fangneng Zhan, Hongyuan Zhu, Shijian Lu: "Spatial Fusion GAN for Image Synthesis." CVPR (2019) [pdf]
• Chen-Hsuan Lin, Ersin Yumer, Oliver Wang, Eli Shechtman, Simon Lucey: "ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing." CVPR (2018) [pdf] [code]

7. Occlusion

• Lingxiao Li, Kaixiong Gong, Weihong Li, Xili Dai, Tao Chen, Xiaojun Yuan, Xiangyu Yue: "BIFRÖST: 3D-Aware Image compositing with Language Instructions." NeurIPS (2024) [arXiv] [code]
• Jonghyun Lee, Hansam Cho, Youngjoon Yoo, Seoung Bum Kim, Yonghyun Jeong: "Compose and Conquer: Diffusion-Based 3D Depth Aware Composable Image Synthesis." ICLR (2024) [pdf] [code]
• Zan Li, Wencheng Wang, Fei Hou: "Image Composition with Depth Registration." IJCAI (2023) [paper]
• Fangneng Zhan, Jiaxing Huang, Shijian Lu: "Hierarchy Composition GAN for High-fidelity Image Synthesis." Transactions on cybernetics (2021) [arXiv]
• Samaneh Azadi, Deepak Pathak, Sayna Ebrahimi, Trevor Darrell: "Compositional GAN: Learning Image-Conditional Binary Composition." IJCV (2020) [arXiv] [code]

8. Resolution/Sharpness/Noise Discrepancy

• Jizhizi Li, Jing Zhang, Stephen J.Maybank, Dacheng Tao: "Bridging Composite and Real: Towards End-to-End Deep Image Matting." IJCV (2021) [pdf] [code]

9. Foreground Object Search

  Awesome-Foreground-Object-Search

10. Generative Image Composition

  Awesome-Generative-Image-Composition

Datasets

• Datasets for image harmonization [link]
• Datasets for object shadow generation [link]
• Datasets for object placement [link]
• Datasets for foreground object search [link]
• Datasets for perspective transformation [link]
• Datasets for generative image composition [link]

Evaluation

• Composite-Image-Evaluation