Edit and Generate Anything In Any Image and Video.
Motivation of this repo: Segment Anything offers powerful perception capabilities and interfaces with points and boxes. We believe that the interface ability has the potential to greatly drive the development of generation and editing applications. We will be posting some application developments from our lab here that are compatible with both Segment Anything and Generation, thanks.
- Customization (LoRA or Dreambooth)
- Video editing with point interaction
- Hugging Face Demo
- ...
Please follow the segment anything to creating the environment and download the model checkpoint (vit_l, vit_l or vit_b)
Hugging Face Demo is now available, but please note that it may run slowly due to being currently executed on a low-end CPU.
The ultimate goal of this task is to achieve control over the generation of complex scenes, such as dense crowds and department stores.
See the here. Using points to edit and control. Enjoy it!
Replace the person in the video:
Customize the clothes of the person in the video:
More results:
| Input Video | Output Video | ||
 |
 |
 |
 |
| "Input Video" | "A soldier is dancing" | "Donald Trump is dancing" | "Iron Man is dancing" |
| Input Video | Output Video | ||
|
 |
 |
 |
| "Input Video" | "A man is dancing in skirt and wearing black stockings" | "A man is dancing in jeans" | "A man is dancing in shorts" |
Image to paragraph is available, please refer to Image2Paragraph, and enjoy!
Image to 3D using SAM is available, please refer to Anything3D, and enjoy!
face_1000.mp4
- Segment Anything
- Caption-Anything (hugging face demo)
- Stable Diffusion
If you find this project helpful for your research, please consider citing the following BibTeX entry.
@article{kirillov2023segany,
title={Segment Anything},
author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan-Yen and Doll{\'a}r, Piotr and Girshick, Ross},
journal={arXiv:2304.02643},
year={2023}
}