Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for List[Image] #1763

Closed
wants to merge 1 commit into from
Closed

Conversation

ryanh-ai
Copy link

@ryanh-ai ryanh-ai commented Nov 6, 2024

When providing List[dspy.Image] for input field, the ChatAdapter was formatting the list of images as a string and overflowing the input context for LM's from multiple providers.

Changes were done very quickly to unblock active work, but this is tested locally across Bedrock and OpenAI providers.

When providing `List[dspy.Image]` for input field, the `ChatAdapter` was formatting the list of images as a string and overflowing the input context for LM's from multiple providers.

Changes were done very quickly to unblock active work, but this is tested locally across Bedrock and OpenAI providers.
@ryanh-ai
Copy link
Author

ryanh-ai commented Nov 6, 2024

@isaacbmiller FYI, per Discord thread on this issue

@okhat
Copy link
Collaborator

okhat commented Nov 6, 2024

Thanks a ton @ryanh-ai ! I guess in general do we need more complex collections of images? Will people also want to do dict[str, list[Image]] and all kinds of crazy things?

If we only wanna support list[Image], then this PR is good if @isaacbmiller approves it.

@okhat okhat requested a review from isaacbmiller November 6, 2024 14:49
@isaacbmiller
Copy link
Collaborator

I think that people will at a minimum want to do things like dict[str, Image] which I am not sure even works currently.

I think this PR shouldn't be merged and I can work on a scalable solution for adapters for all different types including Image

@okhat okhat marked this pull request as draft November 6, 2024 21:41
@ryanh-ai
Copy link
Author

ryanh-ai commented Nov 7, 2024

Agree above on the likely need for dictionary support.

I also found an edge case when length of image list is 1. I will update the patch when I adjust that, but not invest too much more in this until you determine best path forward. I'm happy to test as you do!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants