Add CLIP to ivy models #34
Conversation
…n the stateful API
…to feat/clip-model
ivy_models/clip/clip.py
Outdated
| return new_mapping | ||
|
|
||
|
|
||
| def load_clip(name: str, pretrained=True): |
There was a problem hiding this comment.
this should be named clip_<version> to match previous conventions
| return image.convert("RGB") | ||
|
|
||
|
|
||
| def _transform(n_px): |
There was a problem hiding this comment.
there's an equivalent in ivy_models_tests/image_helpers.py
can we also refactor some of the helpers here to that file or folder? thanks!
ps: the image helpers were initially used during tests only, but if it is deemed better in ivy_models, please feel free to refactor this in another PR
There was a problem hiding this comment.
The CLIP image transforms depend on the variant of the model that we load, so I thought it was better to handle it there for convenience. Also, as you mentioned, I think it'd be better to have some helpers to load the appropriate image preprocessing pipeline in ivy_models so that users can easily load them directly. Currently, users will have to write the appropriate image processing pipeline for each model that they load and that won't be cool for the UX.
There was a problem hiding this comment.
we have the transform pipeline there, this function looks quite similar to that imo, the resize and centercrop args are configurable as well. do you think it can be improved with the rgb flag? Thanks!
ref: https://github.com/unifyai/models/blob/c87584bf7be32c8b2ed58e3978452645e9ff36a0/ivy_models_tests/helpers/image_helpers.py#L30
|
Hey! Thanks for the changes! One quick point, I noticed the |
No description provided.