Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verify model compatibility if OPENAI_FUNCTIONS is set #5075

Merged
merged 3 commits into from
Jul 31, 2023

Conversation

Pwuts
Copy link
Member

@Pwuts Pwuts commented Jul 29, 2023

Background

Not all models support the OpenAI Functions API. It would be nice if we can assume that we can use functions if config.openai_functions is set to True.

Changes

Add a validator to the Config which checks if smart_llm supports the Functions API in case openai_functions is enabled.

If these two settings have incompatible values, it throws an error:

$ ./run.sh --debug
All packages are installed.
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/reinier/code/agpt/Auto-GPT/autogpt/__main__.py", line 5, in <module>
    autogpt.app.cli.main()
  File "/home/reinier/.local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/reinier/.local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/reinier/.local/lib/python3.10/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "/home/reinier/.local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/reinier/.local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/reinier/.local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/reinier/code/agpt/Auto-GPT/autogpt/app/cli.py", line 121, in main
    run_auto_gpt(
  File "/home/reinier/code/agpt/Auto-GPT/autogpt/app/main.py", line 61, in run_auto_gpt
    config = ConfigBuilder.build_config_from_env(workdir=working_directory)
  File "/home/reinier/code/agpt/Auto-GPT/autogpt/config/config.py", line 336, in build_config_from_env
    config = cls.build_agent_configuration(config_dict_without_none_values)
  File "/home/reinier/code/agpt/Auto-GPT/autogpt/core/configuration/schema.py", line 52, in build_agent_configuration
    return cls.default_settings.__class__.parse_obj(final_configuration)
  File "pydantic/main.py", line 526, in pydantic.main.BaseModel.parse_obj
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Config
openai_functions
  Model gpt-4-0314 does not support OpenAI Functions. Please disable OPENAI_FUNCTIONS or choose a suitable model. (type=assertion_error)
Press any key to continue...

I also had to make a few changes to mitigate circular import errors.

Test Plan

CI

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes.
  • I have run the following commands against my code to ensure it passes our linters:
    black .
    isort .
    mypy
    autoflake --remove-all-unused-imports --recursive --ignore-init-module-imports --ignore-pass-after-docstring autogpt tests --in-place

@netlify
Copy link

netlify bot commented Jul 29, 2023

Deploy Preview for auto-gpt-docs failed.

Name Link
🔨 Latest commit 1e20c1a
🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/64c7d5fd66313a0008758f56

@codecov
Copy link

codecov bot commented Jul 29, 2023

Codecov Report

Patch coverage: 53.33% and project coverage change: -0.78% ⚠️

Comparison is base (b7f1df3) 51.94% compared to head (bff06d2) 51.17%.

❗ Current head bff06d2 differs from pull request most recent head 1e20c1a. Consider uploading reports for the commit 1e20c1a to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5075      +/-   ##
==========================================
- Coverage   51.94%   51.17%   -0.78%     
==========================================
  Files         116      116              
  Lines        4976     4948      -28     
  Branches      668      667       -1     
==========================================
- Hits         2585     2532      -53     
- Misses       2196     2218      +22     
- Partials      195      198       +3     
Files Changed Coverage Δ
autogpt/llm/providers/openai.py 89.07% <ø> (ø)
autogpt/command_decorator.py 90.90% <50.00%> (-9.10%) ⬇️
autogpt/config/config.py 79.54% <50.00%> (-1.05%) ⬇️
autogpt/models/command.py 68.00% <50.00%> (-5.92%) ⬇️
autogpt/llm/base.py 90.29% <100.00%> (+0.09%) ⬆️

... and 11 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Pwuts Pwuts added setup Issues with getting Auto-GPT setup on local machines conflicts Automatically applied to PRs with merge conflicts labels Jul 29, 2023
@Pwuts Pwuts added this to the v0.4.7 Release milestone Jul 29, 2023
@github-actions github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label Jul 29, 2023
@github-actions
Copy link
Contributor

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

@lc0rp lc0rp merged commit 3651d22 into master Jul 31, 2023
@lc0rp lc0rp deleted the config/verify-openai-function-support branch July 31, 2023 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
setup Issues with getting Auto-GPT setup on local machines size/m
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants