Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update prompt.py to clarify how to call commands #4027

Merged
merged 12 commits into from
May 15, 2023
Merged

Update prompt.py to clarify how to call commands #4027

merged 12 commits into from
May 15, 2023

Conversation

richbeales
Copy link
Contributor

@richbeales richbeales commented May 8, 2023 •
edited
Loading

Background

As discussed on the call recently, we tell the LLM to exclusively use the commands in double quotes, but the commands aren't listed in double quotes, and are underscore-separated and not space-separated.

Changes

Documentation

Test Plan

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes

@vercel
Copy link

vercel bot commented May 8, 2023 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
docs ⬜️ Ignored (Inspect) Visit Preview May 15, 2023 0:41am

@github-actions github-actions bot added the size/s label May 8, 2023
@rihp
Copy link
Contributor

rihp commented May 8, 2023

lgtm!

@k-boikov
Copy link
Contributor

k-boikov commented May 8, 2023

#RichTheLightning
LGTM

@Boostrix
Copy link
Contributor

Boostrix commented May 9, 2023 •
edited
Loading

nice, I've seen code reviews here where people pointed out discrepancies in that area.

the command string should probably be internally validated at the command mgr level using a regex to show a warning, especially since commands are likely to be added by external means, like plugins.

See also:

ntindle
ntindle previously requested changes May 10, 2023
View reviewed changes
autogpt/prompts/prompt.py Show resolved Hide resolved
@vercel vercel bot temporarily deployed to Preview May 12, 2023 18:04 Inactive
@k-boikov k-boikov self-assigned this May 13, 2023
@github-actions github-actions bot added size/xl and removed size/s labels May 13, 2023
@github-actions
Copy link
Contributor

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

1 similar comment
@github-actions
Copy link
Contributor

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

@vercel vercel bot temporarily deployed to Preview May 13, 2023 22:00 Inactive
@ntindle ntindle self-requested a review May 13, 2023 22:30
@github-actions
Copy link
Contributor

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

@vercel vercel bot temporarily deployed to Preview May 13, 2023 22:36 Inactive
@github-actions
Copy link
Contributor

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

@codecov
Copy link

codecov bot commented May 13, 2023 •
edited
Loading

Codecov Report

Patch and project coverage have no change.

Comparison is base (1a2f768) 60.72% compared to head (79b674a) 60.72%.

Additional details and impacted files 
@@           Coverage Diff           @@
##           master    #4027   +/-   ##
=======================================
  Coverage   60.72%   60.72%           
=======================================
  Files          73       73           
  Lines        3320     3320           
  Branches      475      475           
=======================================
  Hits         2016     2016           
  Misses       1164     1164           
  Partials      140      140
Impacted Files Coverage Δ
autogpt/prompts/prompt.py 54.00% <ø> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

k-boikov
k-boikov previously approved these changes May 13, 2023
View reviewed changes
@github-actions
Copy link
Contributor

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

@github-actions
Copy link
Contributor

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

@k-boikov k-boikov added this to the v0.3.2-release milestone May 14, 2023
@richbeales richbeales dismissed ntindle’s stale review May 14, 2023 18:06

fixed by Kris

@github-actions
Copy link
Contributor

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

@richbeales richbeales requested a review from k-boikov May 14, 2023 18:06
@github-actions
Copy link
Contributor

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

@Boostrix
Copy link
Contributor

FYI: I've seen fewer command hallucinations by adapting the prompt generator to encourage the LLM to respond within the bounds of the list of commands by also adding the length of the list to the prompt itself (aka "pick one between 1-21").

@k-boikov k-boikov merged commit 16b7e7a into master May 15, 2023
@k-boikov k-boikov deleted the richbeales-patch-3 branch May 15, 2023 12:55
DGdev91 added a commit to DGdev91/Auto-GPT that referenced this pull request May 16, 2023
@DGdev91
Update default prompt to make it the same as Significant-Gravitas#4027
fe7ff57
@DGdev91 DGdev91 mentioned this pull request May 16, 2023
Open
DGdev91 added a commit to DGdev91/Auto-GPT that referenced this pull request May 17, 2023
@DGdev91
Update default prompt to make it the same as Significant-Gravitas#4027
c4500d2
ppetermann pushed a commit to ppetermann/Auto-GPT that referenced this pull request May 22, 2023
@richbeales @k-boikov
@ntindle @ppetermann
Update prompt.py to clarify how to call commands (Significant-Gravita…
83a0dc9 
…s#4027)

* Update prompt.py to clarify how to call commands

* new cassettes

* try again with a new cassette

* one last try with the cassette

* black dot

---------

Co-authored-by: k-boikov <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@k-boikov k-boikov k-boikov approved these changes

@ntindle ntindle Awaiting requested review from ntindle

Assignees

@k-boikov k-boikov

Labels
prompt issue size/xl
Projects
AutoGPT development kanban
Archived in project
Milestone
v0.4.0 Release
Development

Successfully merging this pull request may close these issues.
5 participants
@richbeales @rihp @k-boikov @Boostrix @ntindle