Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the accuracy of the extract_dict_from_response method's JSON extraction. #5458

Conversation

HawkClaws
Copy link
Contributor

@HawkClaws HawkClaws commented Oct 2, 2023

Background

JSON parse error in extract_dict_from_response method

Changes 🏗️

extract_dict_from_response method in `autogpts\autogpt\autogpt\json_utils\utilities.p

PR Quality Scorecard ✨

  • Have you used the PR description template?   +2 pts
  • Is your pull request atomic, focusing on a single change?   +5 pts
  • Have you linked the GitHub issue(s) that this PR addresses?   +5 pts
  • Have you documented your changes clearly and comprehensively?   +5 pts
  • Have you changed or added a feature?   -4 pts
    • Have you added/updated corresponding documentation?   +4 pts
    • Have you added/updated corresponding integration tests?   +5 pts
  • Have you changed the behavior of AutoGPT?   -5 pts
    • Have you also run agbenchmark to verify that these changes do not regress performance?   +10 pts

Improve the accuracy of the extract_dict_from_response method's JSON extraction.

Improve the accuracy of the extract_dict_from_response method's JSON extraction.
Currently, the following response causes JSON extraction/parse errors.
This branch will resolve these issues.

Test Data

We have confirmed that JSON can be retrieved from the following text
The previous extract_dict_from_response gave an error.

Example 1:

raw JSON


{
"thoughts": {
"text": "I need to ask the user if they can provide information on a basic cola recipe.",
"reasoning": "Since the attempt to gather information via web search was unsuccessful, I will seek the user's cooperation.",
"plan": "I will ask the user, and if I receive a positive response, I will provide a basic cola recipe.",
"criticism": "Since the information gathering attempt was unsuccessful, we should consider ways to improve efficiency.",
"speak": "Can you provide information on a basic cola recipe?"
},
"command": {
"name": "ask_user",
"args": {
"question": "Can you provide information on a basic cola recipe?"
}
}
}


Example 2:

If the code block


{
  "thoughts": {
    "text": "I need to ask the user if they can provide information on a basic cola recipe.",
    "reasoning": "Since the attempt to gather information via web search was unsuccessful, I will seek the user's cooperation.",
    "plan": "I will ask the user, and if I receive a positive response, I will provide a basic cola recipe.",
    "criticism": "Since the information gathering attempt was unsuccessful, we should consider ways to improve efficiency.",
    "speak": "Can you provide information on a basic cola recipe?"
  },
  "command": {
    "name": "ask_user",
    "args": {
      "question": "Can you provide information on a basic cola recipe?"
    }
  }
}

Example 3:

If the code block contains the name json, then


{
  "thoughts": {
    "text": "I need to ask the user if they can provide information on a basic cola recipe.",
    "reasoning": "Since the attempt to gather information via web search was unsuccessful, I will seek the user's cooperation.",
    "plan": "I will ask the user, and if I receive a positive response, I will provide a basic cola recipe.",
    "criticism": "Since the information gathering attempt was unsuccessful, we should consider ways to improve efficiency.",
    "speak": "Can you provide information on a basic cola recipe?"
  },
  "command": {
    "name": "ask_user",
    "args": {
      "question": "Can you provide information on a basic cola recipe?"
    }
  }
}

Example 4:

JSON contained in a string


Based on the goal, I will determine the following command and provide information about the progress.

Goal:
To provide the highest quality cola recipe, including detailed instructions on ingredients, seasonings, carbonation, and blending methods.

Command Decision:
Next, I conducted a web search to provide a basic cola recipe, but I couldn't find any information. Therefore, I need to ask the user if they can provide information on a basic cola recipe.

JSON:

{
  "thoughts": {
    "text": "I need to ask the user if they can provide information on a basic cola recipe.",
    "reasoning": "Since the attempt to gather information via web search was unsuccessful, I will seek the user's cooperation.",
    "plan": "I will ask the user, and if I receive a positive response, I will provide a basic cola recipe.",
    "criticism": "Since the information gathering attempt was unsuccessful, we should consider ways to improve efficiency.",
    "speak": "Can you provide information on a basic cola recipe?"
  },
  "command": {
    "name": "ask_user",
    "args": {
      "question": "Can you provide information on a basic cola recipe?"
    }
  }
}

I'm awaiting the user's response.


@HawkClaws HawkClaws requested a review from Pwuts as a code owner October 2, 2023 06:15
@github-actions github-actions bot added the size/m label Oct 2, 2023
@netlify
Copy link

netlify bot commented Oct 2, 2023

Deploy Preview for auto-gpt-docs ready!

Name Link
🔨 Latest commit 4fbf241
🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/65562491d2bd490008decd1b
😎 Deploy Preview https://deploy-preview-5458--auto-gpt-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

ntindle
ntindle previously approved these changes Nov 5, 2023
@github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Nov 16, 2023
Copy link
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

@github-actions github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label Nov 16, 2023
Copy link
Contributor

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

@Pwuts
Copy link
Member

Pwuts commented Nov 16, 2023

LGTM!

@Pwuts Pwuts merged commit 6664eec into Significant-Gravitas:master Nov 16, 2023
10 of 15 checks passed
@Pwuts Pwuts added the invalid_json Groups issues and PRs related to invalid json error or similar label Nov 16, 2023
Pwuts added a commit that referenced this pull request Nov 30, 2023
- Fixed linting issues in `json_utils/utilities.py` and `test_json_utils.py`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid_json Groups issues and PRs related to invalid json error or similar size/m
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants