20230426-154651.mp4_20230426_154749.mp4
The demo video has a nice progress bar when generating the new image, which requires to launch the
advanced.py
. See Run for more details.
This example implements the StableDiffusionControlNet
plugin (along with a Canny
plugin to generate the annotations) with carefree-drawboard
π¨, here's how you can use it (you can also see the demo video above):
- Drag/Upload an image to the drawboard π¨.
- Click the
Canny
plugin at thert
corner of the selectingImageNode
(it is the bottom-most one), the annotation should be generated in no time. - Click the
StableDiffusionControlNet
plugin at thert
corner of the selected annotationImageNode
. - Fill in some parameters, submit, wait, and see the result.
- You may notice that the
ControlNet
plugin poped up no matter whatImageNode
you are selecting. This is as expected becauseControlNet
can indeed accept any image as input. π- See Details for more details!
This example has quite a few prerequisites, see the original blog for more details.
cfdraw run
We use app
as the default entry name. If your script is named other than app.py
(e.g. {name}.py
), then run:
cfdraw run --module {name}
For example, we provide an advanced.py
, which utilized send_progress
to show some nice progress bars on carefree-drawboard
π¨. To launch it, run:
cfdraw run --module advanced
- We utilized
cache_resource
to avoid re-initializing models every hot-rerun.- This is useful when we are focusing on the plugin styles/logic.
- At production stage, we can call the initialization function at the very beginning to pre-load the models.
- We used
follow=True
andnodeConstraint=NodeConstraints.MULTI_NODE
, so the plugin will and only will be displayed when the multipleNode
s are selected.
See Plugin Positioning for more details.
-
We specified
useModal=True
for the plugin, so it will always popup as a modal. -
We specified lots of
definitions
, in order to align with the parameters exposed by thediffusers
library.
See
IFieldDefinition
for more details.
- We used a special internal method:
filter
in the first two lines of theprocess
method:
path_data = self.filter(data.nodeDataList, SingleNodeType.PATH)[0]
image_data = self.filter(data.nodeDataList, SingleNodeType.IMAGE)[0]
That's because in Stable Diffusion Inpainting, we need to get two images: the original image and the mask image. In this example, we utilize the Sketch
plugin (which will generate a PathNode
) in carefree-drawboard
π¨ to draw the mask, and use a normal ImageNode
to represent the original image.
So, the nodeDataList
should be of length 2, and we can use the filter
method to extract the PathNode
and the ImageNode
respectively.
After the extraction, we can use the common internal method load_image
to load PIL.Image
from src
.
- We specified
upload_root
to./
(incfconfig.py
), so the images/projects will be saved to the cwd (current working directory).
Default
upload_root
is~/.cache/carefree-draw
.