Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache Python Packages in the CI pipeline #4488

Merged
merged 12 commits into from
Jun 3, 2023
Merged

Cache Python Packages in the CI pipeline #4488

merged 12 commits into from
Jun 3, 2023

Conversation

waynehamadi
Copy link
Contributor

@waynehamadi waynehamadi commented May 31, 2023 •
edited
Loading

Background

Changes

Now python packages are cached in the ci pipeline.

Documentation

Test Plan

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes.
  • I have run the following commands against my code to ensure it passes our linters: 
    black .
isort .
mypy
autoflake --remove-all-unused-imports --recursive --ignore-init-module-imports autogpt tests --in-place

@vercel
Copy link

vercel bot commented May 31, 2023

Deployment failed with the following error:

Resource is limited - try again in 9 minutes (more than 100, code: "api-deployments-free-per-day").

@Auto-GPT-Bot
Copy link
Contributor

You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged.

@waynehamadi waynehamadi force-pushed the cache-python-packages-in-ci branch from fbfdbc4 to df04485 Compare May 31, 2023 01:09
@vercel
Copy link

vercel bot commented May 31, 2023

Deployment failed with the following error:

Resource is limited - try again in 4 minutes (more than 100, code: "api-deployments-free-per-day").

@codecov
Copy link

codecov bot commented May 31, 2023 •
edited
Loading

Codecov Report

Patch and project coverage have no change.

Comparison is base (63b79a8) 69.65% compared to head (253130d) 69.65%.

Additional details and impacted files 
@@           Coverage Diff           @@
##           master    #4488   +/-   ##
=======================================
  Coverage   69.65%   69.65%           
=======================================
  Files          72       72           
  Lines        3523     3523           
  Branches      562      562           
=======================================
  Hits         2454     2454           
  Misses        881      881           
  Partials      188      188

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@waynehamadi waynehamadi mentioned this pull request May 31, 2023
Closed
1 task
Pwuts
Pwuts requested changes May 31, 2023
View reviewed changes
Copy link
Member

@Pwuts Pwuts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would save a considerable amount of time to run the CI, nice! That cache needs to be cleaned regularly though, similar to the docker-ci and docker-cache-clean workflows.

@waynehamadi waynehamadi force-pushed the cache-python-packages-in-ci branch from df04485 to 55f0bf7 Compare May 31, 2023 14:28
@vercel
Copy link

vercel bot commented May 31, 2023 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
docs ⬜️ Ignored (Inspect) Jun 3, 2023 1:33pm

@waynehamadi waynehamadi force-pushed the cache-python-packages-in-ci branch from 55f0bf7 to 3512cbd Compare May 31, 2023 15:26
@waynehamadi waynehamadi force-pushed the cache-python-packages-in-ci branch from 3512cbd to 22e35fa Compare May 31, 2023 17:36
@waynehamadi
Copy link
Contributor Author

waynehamadi commented May 31, 2023 •
edited
Loading

@Pwuts

  • now the key for the cache has the date. This way it's computed every day.
  • removed useless exit 0
  • change git diff command to spot change in current_score to make it more precise
  • skipped a test that unfortunately we cannot maintain right now.

@BaseInfinity
Copy link
Contributor

BaseInfinity commented May 31, 2023 •
edited
Loading

This looks good to me! Quick question, what's the idea with including the date?

@waynehamadi
Copy link
Contributor Author

@BaseInfinity have the latest packages every day. otherwise, if our requirements.txt doesn't change for 2 weeks, our packages go stale.

Pwuts
Pwuts previously requested changes Jun 1, 2023
View reviewed changes
.github/workflows/ci.yml Outdated Show resolved Hide resolved
.github/workflows/ci.yml Outdated Show resolved Hide resolved
@BaseInfinity
Copy link
Contributor

@BaseInfinity have the latest packages every day. otherwise, if our requirements.txt doesn't change for 2 weeks, our packages go stale.

Ahh gotcha I wasn't aware we are pulling in the latest versions on a daily basis that makes sense

One potential gotcha I've run into by not locking down specific package versions is getting breaking changes unexpectedly, the flipside is we never updated packages lol so there's that lol

Any ways cool stuff

@Auto-GPT-Bot
Copy link
Contributor

You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged.

@Auto-GPT-Bot
Copy link
Contributor

You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged.

@richbeales richbeales dismissed Pwuts’s stale review June 3, 2023 14:48

stale/resolved

@richbeales richbeales merged commit 79ba85a into Significant-Gravitas:master Jun 3, 2023
kuwork added a commit to kuwork/Auto-GPT that referenced this pull request Jun 12, 2023
@kuwork
Merge branch 'cn-dev' into cn
7dbedd3 
* cn-dev: (64 commits)
  针对国内的情况，翻译成中文
  Update challenge scores
  Update version numbers for v0.4.0 release
  Add `replace_in_file` command (Significant-Gravitas#4565)
  Update bulletin with highlights for v0.4.0 release (Significant-Gravitas#4576)
  Skip flaky challenges (Significant-Gravitas#4573)
  Fix `test_web_selenium` (Significant-Gravitas#4554)
  Clean up CI git logic
  remove information retrieval challenge b from beaten challenges
  Fix CI git authentication and cassettes
  debug
  Fix CI git diff
  Fix CI git authorization
  Update submodule reference
  Update current score
  Cache Python Packages in the CI pipeline (Significant-Gravitas#4488)
  Fix pushing cassettes in CI
  Remove news about config (Significant-Gravitas#4553)
  Fix CI for internal PRs with CI changes (Significant-Gravitas#4552)
  Update BULLETIN.md
  ...

# Conflicts:
#	BULLETIN.md
#	CONTRIBUTING.md
#	autogpt/agent/agent.py
#	autogpt/app.py
#	autogpt/llm/llm_utils.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richbeales richbeales richbeales approved these changes

@Pwuts Pwuts Pwuts left review comments

Assignees
No one assigned
Labels
behaviour change size/m
Projects
AutoGPT development kanban
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@waynehamadi @Auto-GPT-Bot @BaseInfinity @richbeales @Pwuts