Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Huggingface retry generate_image with delay #2745

Conversation

primaryobjects
Copy link
Contributor

@primaryobjects primaryobjects commented Apr 21, 2023

Background

The command generate_image with huggingface may return an error response due to the service queuing the request. This results in the image failing to generate. A delay may be invoked and the request retried, in order to return a successful image.

cannot identify image file <_io.BytesIO object at 0x7f6eac20ef40>
b'{"error":"Model CompVis/stable-diffusion-v1-4 is currently loading","estimated_time":20.0}'

Changes

  • Check for error in response from huggingface and retry with delay up to 10 times.
  • Delay in seconds is obtained from huggingface response.

Documentation

  • No additional documentation required for users.

Test Plan

  • Tested locally by running with generate_image command.
  • Added unit tests for successful and failure cases. See Report.

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes

@k-boikov k-boikov added the enhancement New feature or request label Apr 21, 2023
ntindle
ntindle previously approved these changes Apr 22, 2023
@ntindle
Copy link
Member

ntindle commented Apr 23, 2023

Do me a solid and update this so we can merge

@codecov
Copy link

codecov bot commented Apr 23, 2023

Codecov Report

Patch coverage: 88.88% and project coverage change: -7.32 ⚠️

Comparison is base (06e81b7) 49.70% compared to head (6501f38) 42.38%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2745      +/-   ##
==========================================
- Coverage   49.70%   42.38%   -7.32%     
==========================================
  Files          64       64              
  Lines        3026     3048      +22     
  Branches      507      510       +3     
==========================================
- Hits         1504     1292     -212     
- Misses       1402     1689     +287     
+ Partials      120       67      -53     
Impacted Files Coverage Δ
autogpt/commands/image_gen.py 58.66% <88.88%> (+32.25%) ⬆️

... and 14 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@github-actions github-actions bot added size/l and removed size/m labels Apr 23, 2023
@primaryobjects
Copy link
Contributor Author

primaryobjects commented Apr 23, 2023

Code coverage added for all cases here.

python -m pytest .\tests\test_image_gen.py::TestImageGenFailure
== test session starts ==
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)                                 [100%] 
plugins: anyio-3.6.2, asyncio-0.21.0, benchmark-4.0.0, cov-4.0.0, integration-0.2.3, mock-3.10.0, vcr-1.0.2                                                                                               ============= 
asyncio: mode=strict
collected 3 items

tests\test_image_gen.py ... [100%]

== 3 passed in 0.63s ====
python -m pytest .\tests\test_image_gen.py::TestImageGen -k huggingface
== test session starts ==
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
plugins: anyio-3.6.2, asyncio-0.21.0, benchmark-4.0.0, cov-4.0.0, integration-0.2.3, mock-3.10.0, vcr-1.0.2
asyncio: mode=strict
collected 3 items / 2 deselected / 1 selected

tests\test_image_gen.py . [100%]

== 1 passed, 2 deselected in 59.96s ==

@primaryobjects primaryobjects requested a review from ntindle April 24, 2023 02:20
@github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Apr 24, 2023
@github-actions
Copy link
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

@ntindle
Copy link
Member

ntindle commented May 12, 2023

Fixed the tests partially by removing them 😅

If you can add a test that's appropriate

@ntindle ntindle added this to the v0.3.1 Release milestone May 12, 2023
@vercel vercel bot temporarily deployed to Preview May 13, 2023 19:29 Inactive
@github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label May 13, 2023
@github-actions
Copy link
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

@vercel
Copy link

vercel bot commented May 14, 2023

Deployment failed with the following error:

Resource is limited - try again in 1 hour (more than 100, code: "api-deployments-free-per-day").

@github-actions github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label May 14, 2023
@github-actions
Copy link
Contributor

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

@vercel
Copy link

vercel bot commented May 14, 2023

Deployment failed with the following error:

Resource is limited - try again in 56 minutes (more than 100, code: "api-deployments-free-per-day").

@lc0rp lc0rp assigned lc0rp and unassigned ntindle May 14, 2023
@p-i- p-i- modified the milestones: v0.3.1 Release, v0.3.2-release May 14, 2023
@vercel
Copy link

vercel bot commented May 14, 2023

Deployment failed with the following error:

Resource is limited - try again in 34 minutes (more than 100, code: "api-deployments-free-per-day").

@ntindle ntindle merged commit f424fac into Significant-Gravitas:master May 16, 2023
ppetermann pushed a commit to ppetermann/Auto-GPT that referenced this pull request May 22, 2023
Co-authored-by: Media <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>
Co-authored-by: k-boikov <[email protected]>
Co-authored-by: merwanehamadi <[email protected]>
Co-authored-by; lc0rp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request size/l
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

8 participants