Rename evaluate_code to analyze_code
#1371
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Handles Docker errors separately, and prints a potentially helpful message for users.
|
@willcallender There are conflicts now |
nponeccop
previously approved these changes
Apr 16, 2023
github-actions
bot
added
the
conflicts
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
Sorry, but where can I see the conflicts? Normally I'd use the resolve conflicts button but it's greyed out for me. |
github-actions
bot
removed
the
conflicts
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
|
I've tested the changes and they still work as expected. |
github-actions
bot
added
the
conflicts
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
github-actions
bot
removed
the
conflicts
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
Pwuts
approved these changes
Apr 19, 2023
sindlinger
pushed a commit
to Orgsindlinger/Auto-GPT-WebUI
that referenced
this pull request
Sep 25, 2024
ChatGPT is less confused by this phrasing From my own observations and others (ie Significant-Gravitas#101 and Significant-Gravitas#286) ChatGPT seems to think that `evaluate_code` will actually run code, rather than just provide feedback. Since changing the phrasing to `analyze_code` I haven't seen the AI make this mistake. --------- Co-authored-by: Reinier van der Leer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
As seen in #101 and #286, GPT-3.5 misinterprets "evaluate" to mean "execute" rather than "analyze". As such, I changed the name of the function to "analyze" to make the purpose of the function more clear to the AI.
Changes
Renamed the
evaluate_codecommand toanalyze_codeeverywhere it appears, including the function names and the text as it's given to the AI.Documentation
I tested with several simple prompts, namely hello world programs in various languages, and I haven't seen this error since making the change.
Test Plan
I used the same basic AI personality for these tests, specifically I used Dev-GPT, an AI designed to autonomously develop, run, and test code. I gave it a single goal which was "Write and run a simple hello world program in [language]." I tested Python and Rust, it didn't always succeed but it never got confused by
evaluate_code.PR Quality Checklist