Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add: Resize the image size to meet OpenAI requirements #458

Merged
merged 7 commits into from
Feb 19, 2024

Conversation

mingming-ma
Copy link
Collaborator

@mingming-ma mingming-ma commented Feb 15, 2024

Explanation

This updates make sure users' input image's size is within OpenAI GPT-4V limit.

The OpenAI requirements for images can be found at Is there a limit to the size of the image I can upload?

Yes, we restrict image uploads to 20MB per image.

And Calculating costs

detail: high images are first scaled to fit within a 2048 x 2048 square, maintaining their aspect ratio.

I found this tool browser-image-compression focus on the image compression in web browser is great to handle it, supports both file size and px constraints.

I set the values in imageCompressionOptions as:

      const imageCompressionOptions = {
        maxSizeMB: 20,
        maxWidthOrHeight: 2048,
        useWebWorker: true,
      };

The useWebWorker is optional, means the tool will use multi-thread web worker for the compression, fallback to run in main-thread (default: true). Although it is default true, I added it to align with the document's Usage

How to test

Use some large size image in the prompt, I got a 30MB image from here

Before

image

After

image

Fixes #395
Fixes #465

@mingming-ma mingming-ma self-assigned this Feb 15, 2024
@mingming-ma mingming-ma added this to the Release 1.3 milestone Feb 15, 2024
@Amnish04 Amnish04 requested review from Amnish04 and humphd February 15, 2024 20:51
Copy link
Collaborator

@Amnish04 Amnish04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested this and it works well! Just a few suggestions I commented.

src/components/OptionsButton.tsx Outdated Show resolved Hide resolved
src/components/PromptForm/DesktopPromptForm.tsx Outdated Show resolved Hide resolved
@Amnish04
Copy link
Collaborator

I was also wondering if we should set a lower maximum size of image upload for better performance, or would that be a negligible difference?

@mingming-ma
Copy link
Collaborator Author

@Amnish04 Yeah, that's a good way to try. if we set lower value of maxSizeMB, it should improve the response since the payload now have smaller image, and also help reduce the costs.

Maybe we add options in follow up PRs for the user to select uploaded image with different compression rate or wanted size. User can adjust it accordingly by the result.

Copy link

cloudflare-workers-and-pages bot commented Feb 15, 2024

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 0ef37ba
Status: ✅  Deploy successful!
Preview URL: https://e5c9b2e2.console-overthinker-dev.pages.dev
Branch Preview URL: https://mingming-resizeimage-395.console-overthinker-dev.pages.dev

View logs

@mingming-ma mingming-ma requested a review from Amnish04 February 15, 2024 23:57
@Amnish04
Copy link
Collaborator

@Amnish04 Yeah, that's a good way to try. if we set lower value of maxSizeMB, it should improve the response since the payload now have smaller image, and also help reduce the costs.

Maybe we add options in follow up PRs for the user to select uploaded image with different compression rate or wanted size. User can adjust it accordingly by the result.

Sounds good, could you please file a follow up for that.

src/components/OptionsButton.tsx Outdated Show resolved Hide resolved
src/components/OptionsButton.tsx Outdated Show resolved Hide resolved
src/components/OptionsButton.tsx Outdated Show resolved Hide resolved
src/components/PromptForm/DesktopPromptForm.tsx Outdated Show resolved Hide resolved
src/lib/utils.ts Outdated Show resolved Hide resolved
@mingming-ma mingming-ma requested a review from humphd February 16, 2024 19:44
@mingming-ma mingming-ma modified the milestones: Release 1.3, Release 1.4 Feb 16, 2024
@mingming-ma
Copy link
Collaborator Author

@humphd During refactoring the code, I realized can also solve the dynamic import issue 😄 Let me know if other updates needed. Thanks a lot for the feedback!

Copy link
Collaborator

@humphd humphd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better! One small fix.

src/lib/utils.ts Outdated

// Make sure image's size is within 20MB and 2048x2048 resolution
// https://platform.openai.com/docs/guides/vision/is-there-a-limit-to-the-size-of-the-image-i-can-upload
export const imageCompressionOptions = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this inline below into your compressImageToBase64 function, since we don't use this externally now.

@mingming-ma mingming-ma requested a review from humphd February 17, 2024 15:47
@mingming-ma mingming-ma merged commit b87536c into main Feb 19, 2024
4 checks passed
@mingming-ma mingming-ma deleted the mingming/resizeimage#395 branch April 10, 2024 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants