Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up CI #4930

Merged
merged 19 commits into from
Jul 10, 2023
Merged

Speed up CI #4930

merged 19 commits into from
Jul 10, 2023

Conversation

Pwuts
Copy link
Member

@Pwuts Pwuts commented Jul 9, 2023

Background

CI takes 5 - 6 minutes to run (on average). This is awful to work with.

With this PR, average CI time goes down to 1.5 - 2 minutes

Warning
Before this is merged, the cassettes must be formatted: https://github.com/Significant-Gravitas/Auto-GPT-test-cassettes/blob/master/format_cassettes.py

Changes

  • Faster VCR lookup: match cassettes on a hash of the request body instead of the entire body
  • Better VCR cassette longevity: more strict formatting and more extensive stripping of headers in stored cassettes
  • Added debug logging to execute_python_file
  • Improved logging in @retry_api
  • Do not retry insufficient_quota errors in @retry_api
  • Deleted dead file tests/vcr/openai_filter.py

Documentation

I have tried adding comments :)

Test Plan

CI

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes.
  • I have run the following commands against my code to ensure it passes our linters:
    black .
    isort .
    mypy
    autoflake --remove-all-unused-imports --recursive --ignore-init-module-imports --ignore-pass-after-docstring autogpt tests --in-place

@netlify
Copy link

netlify bot commented Jul 9, 2023

Deploy Preview for auto-gpt-docs canceled.

Name Link
🔨 Latest commit 474c1af
🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/64ac21c8c386400008e3965d

@github-actions github-actions bot added the size/m label Jul 9, 2023
@Pwuts Pwuts force-pushed the ci/debug-slowness branch from 4a3c169 to fe10280 Compare July 9, 2023 20:54
@codecov
Copy link

codecov bot commented Jul 9, 2023

Codecov Report

Patch coverage: 92.85% and project coverage change: +0.03 🎉

Comparison is base (2b56996) 50.88% compared to head (64bc2d9) 50.92%.

❗ Current head 64bc2d9 differs from pull request most recent head 474c1af. Consider uploading reports for the commit 474c1af to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4930      +/-   ##
==========================================
+ Coverage   50.88%   50.92%   +0.03%     
==========================================
  Files         119      119              
  Lines        4889     4895       +6     
  Branches      646      646              
==========================================
+ Hits         2488     2493       +5     
- Misses       2218     2219       +1     
  Partials      183      183              
Impacted Files Coverage Δ
autogpt/commands/execute_code.py 71.96% <75.00%> (-0.16%) ⬇️
autogpt/llm/providers/openai.py 89.07% <100.00%> (+0.28%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@Pwuts Pwuts force-pushed the ci/debug-slowness branch from 321c729 to fcbe479 Compare July 9, 2023 21:33
@Pwuts Pwuts force-pushed the ci/debug-slowness branch from fcbe479 to f6b3388 Compare July 9, 2023 21:38
@Auto-GPT-Bot
Copy link
Contributor

You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged.

@Pwuts Pwuts force-pushed the ci/debug-slowness branch from 0c0bbad to c0b9380 Compare July 10, 2023 00:24
@github-actions github-actions bot added size/l and removed size/m labels Jul 10, 2023
@Pwuts Pwuts force-pushed the ci/debug-slowness branch 2 times, most recently from fd98cb1 to 4dfe620 Compare July 10, 2023 02:25
@Pwuts Pwuts force-pushed the ci/debug-slowness branch from 4dfe620 to faa4654 Compare July 10, 2023 02:30
@Pwuts Pwuts force-pushed the ci/debug-slowness branch from f57b888 to 1c8bdb5 Compare July 10, 2023 04:17
@Pwuts Pwuts force-pushed the ci/debug-slowness branch from 292850b to 2eca4f9 Compare July 10, 2023 05:13
@Pwuts Pwuts added this to the v0.4.5 Release milestone Jul 10, 2023
@Pwuts Pwuts marked this pull request as ready for review July 10, 2023 05:42
@Pwuts Pwuts requested a review from a team as a code owner July 10, 2023 05:42
waynehamadi
waynehamadi previously approved these changes Jul 10, 2023
collijk
collijk previously approved these changes Jul 10, 2023
@Pwuts Pwuts dismissed stale reviews from collijk and waynehamadi via 474c1af July 10, 2023 15:20
@Pwuts Pwuts requested a review from collijk July 10, 2023 15:20
@Pwuts Pwuts self-assigned this Jul 10, 2023
@Pwuts Pwuts merged commit 5e39dd1 into master Jul 10, 2023
@Pwuts Pwuts deleted the ci/debug-slowness branch July 10, 2023 15:26
abdulrahman305 added a commit to abdulrahman305/Auto-GPT that referenced this pull request Jul 10, 2023
Alaaelmziat pushed a commit to Alaaelmziat/Auto-GPT that referenced this pull request Jul 12, 2023
* Match requests in cassette by hash

* Strip requests more extensively for VCR

* Sort JSON keys on cassette save

* Strip max_tokens from cassettes

* Improve logging in retry decorator

* Raise when quota exceeded

* Clean up @retry_api

* Fix @retry_api

* Remove dead tests/vcr/openai_filter.py

* Add debug logging to execute_python_file

* Make Docker CI pass
dayofthedave pushed a commit to dayofthedave/Auto-GPT that referenced this pull request Jul 17, 2023
* Match requests in cassette by hash

* Strip requests more extensively for VCR

* Sort JSON keys on cassette save

* Strip max_tokens from cassettes

* Improve logging in retry decorator

* Raise when quota exceeded

* Clean up @retry_api

* Fix @retry_api

* Remove dead tests/vcr/openai_filter.py

* Add debug logging to execute_python_file

* Make Docker CI pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants