Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix errors in Mandatory Tasks of Benchmarks #4893

Conversation

yuta0x89
Copy link
Contributor

@yuta0x89 yuta0x89 commented Jul 5, 2023

Background

This PR is intended to fix the errors at the following URL.
https://github.com/Significant-Gravitas/Auto-GPT/actions/runs/5461866933

Changes

As can be seen in the URL below, an error occurs during the __init__ process of the api_requestor in the openai library, within original_init in tests/vcr/__init__.py.
https://github.com/Significant-Gravitas/Auto-GPT/actions/runs/5461866933/jobs/9940430084#step:7:202

This seems to be because config defined as a pytest.fixture in the tests/conftest.py is not being called during the execution of the pytest, causing os.environ["OPENAI_API_KEY"] to become None. So, modifying files, as shown below, will resolve this issue.

@challenge()
def test_write_file(
+   config: Config,  # or agent: Agent
    patched_api_requestor: None,
    monkeypatch: pytest.MonkeyPatch,
    level_to_run: int,
    challenge_name: str,
    workspace: Workspace,
    patched_make_workspace: pytest.fixture,
) -> None:

Test Plan

I performed the following pytests.

$ PROXY="http://localhost:8080" pytest -n auto --record-mode=all tests/challenges/basic_abilities/test_write_file.py
$ PROXY="http://localhost:8080" pytest -n auto --record-mode=all tests/challenges/memory/test_memory_challenge_a.py

Due to this commit, the error changed from

FAILED tests/challenges/basic_abilities/test_write_file.py::test_write_file - openai.error.AuthenticationError: No API key provided. You can set your API key in code using 'openai.api_key = <API-KEY>', or you can set the environment variable OPENAI_API_KEY=<API-KEY>). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path = <PATH>'. You can generate API keys in the OpenAI web interface. See https://onboard.openai.com for details, or email [email protected] if you have any questions.

to

FAILED tests/challenges/basic_abilities/test_write_file.py::test_write_file - openai.error.APIConnectionError: Error communicating with OpenAI: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa3acc83850>: Failed to establish a new connection: [Errno 111] Connection refused'))

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes.
  • I have run the following commands against my code to ensure it passes our linters:
    black .
    isort .
    mypy
    autoflake --remove-all-unused-imports --recursive --ignore-init-module-imports --ignore-pass-after-docstring autogpt tests --in-place

@netlify
Copy link

netlify bot commented Jul 5, 2023

Deploy Preview for auto-gpt-docs canceled.

Name Link
🔨 Latest commit d22143e
🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/64a5fc59e393460008d327dc

@github-actions github-actions bot added the size/s label Jul 5, 2023
@codecov
Copy link

codecov bot commented Jul 5, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.15 ⚠️

Comparison is base (9cf3501) 50.05% compared to head (f13d2e2) 49.90%.

❗ Current head f13d2e2 differs from pull request most recent head d22143e. Consider uploading reports for the commit d22143e to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4893      +/-   ##
==========================================
- Coverage   50.05%   49.90%   -0.15%     
==========================================
  Files         116      116              
  Lines        4799     4813      +14     
  Branches      643      649       +6     
==========================================
  Hits         2402     2402              
- Misses       2218     2226       +8     
- Partials      179      185       +6     

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@waynehamadi waynehamadi merged commit bfb45f2 into Significant-Gravitas:master Jul 5, 2023
@lc0rp lc0rp added this to the v0.4.4 Release milestone Jul 6, 2023
dayofthedave pushed a commit to dayofthedave/Auto-GPT that referenced this pull request Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants