Command google_search never invoked anymore

[jaykayenn]

⚠️ Search for existing issues first ⚠️

• I have searched the existing issues, and there is no existing issue for my problem

Which Operating System are you using?

Linux

Which version of Auto-GPT are you using?

Latest Release

GPT-3 or GPT-4?

GPT-3.5

Steps to reproduce 🕹

Start a goal that's primarily searching the web

Current behavior 😯

Instead of searching Google or any other search engine, it will always 'browse website' instead. Even when the declared plan is to use search engines. Browsing the homepage of search engines and archive sites obviously yield no results.

Expected behavior 🤔

Use 'google_search' command like it used to before the latest release.

Your prompt 📝

# Search and summarize recent articles on the topic of XXX

Your Logs 📒

No response

[jaykayenn]

Google API and custom search ID already set in .env. No errors or warnings thrown.

[k-boikov]

Try prompting explicitly to use the google_search command.

[z-x-x136]

尝试明确提示使用 google_search 命令。

What do you mean? Can you give me an example

Thank

[jaykayenn]

Try prompting explicitly to use the google_search command.

I have, and it seems to understand. Yet, the generated command is 'browse_website' instead of 'google_search'. It just keeps scraping google com homepage.

[gsibert]

I believe command changed to just google but it returns disabled for me.

[yangkyeongmo]

I'm having the same issue. AutoGPT agent is ran by this command:
docker compose run --build --rm auto-gpt
Where docker-compose.yaml looks like this:

version: "3.9"
services:
  auto-gpt:
    image: significantgravitas/auto-gpt
    depends_on:
      - redis
    env_file:
      - .env
    environment:
      MEMORY_BACKEND: ${MEMORY_BACKEND:-redis}
      REDIS_HOST: ${REDIS_HOST:-redis}
    profiles: ["exclude-from-up"]
    volumes:
      - ./auto_gpt_workspace:/app/autogpt/auto_gpt_workspace
      - ./data:/app/data
      ## allow auto-gpt to write logs to disk
      - ./logs:/app/logs
      ## uncomment following lines if you have / want to make use of these files
      #- ./azure.yaml:/app/azure.yaml
      - ./ai_settings.yaml:/app/ai_settings.yaml
  redis:
    image: "redis/redis-stack-server:latest"

which is same with document.

From this environment, I found DEBUG message says it's allowed with commands without "google_search" nor "google" in it:
Commands:\n1. analyze_code: Analyze Code, args: \"code\": \"<full_code_string>\"\n2. read_audio_from_file: Convert Audio to text, args: \"filename\": \"<filename>\"\n3. execute_python_file: Execute Python File, args: \"filename\": \"<filename>\"\n4. execute_shell: Execute Shell Command, non-interactive commands only, args: \"command_line\": \"<command_line>\"\n5. execute_shell_popen: Execute Shell Command, non-interactive commands only, args: \"command_line\": \"<command_line>\"\n6. append_to_file: Append to file, args: \"filename\": \"<filename>\", \"text\": \"<text>\"\n7. delete_file: Delete file, args: \"filename\": \"<filename>\"\n8. list_files: List Files in Directory, args: \"directory\": \"<directory>\"\n9. read_file: Read file, args: \"filename\": \"<filename>\"\n10. write_to_file: Write to file, args: \"filename\": \"<filename>\", \"text\": \"<text>\"\n11. clone_repository: Clone Repository, args: \"url\": \"<repository_url>\", \"clone_path\": \"<clone_path>\"\n12. generate_image: Generate Image, args: \"prompt\": \"<prompt>\"\n13. improve_code: Get Improved Code, args: \"suggestions\": \"<list_of_suggestions>\", \"code\": \"<full_code_string>\"\n14. send_tweet: Send Tweet, args: \"tweet_text\": \"<tweet_text>\"\n15. browse_website: Browse Website, args: \"url\": \"<url>\", \"question\": \"<what_you_want_to_find_on_website>\"\n16. write_tests: Write Tests, args: \"code\": \"<full_code_string>\", \"focus\": \"<list_of_focus_areas>\"\n17. delete_agent: Delete GPT Agent, args: \"key\": \"<key>\"\n18. get_hyperlinks: Get text summary, args: \"url\": \"<url>\"\n19. get_text_summary: Get text summary, args: \"url\": \"<url>\", \"question\": \"<question>\"\n20. list_agents: List GPT Agents, args: () -> str\n21. message_agent: Message GPT Agent, args: \"key\": \"<key>\", \"message\": \"<message>\"\n22. start_agent: Start GPT Agent, args: \"name\": \"<name>\", \"task\": \"<short_task_desc>\", \"prompt\": \"<prompt>\"\n23. Task Complete (Shutdown): \"task_complete\", args: \"reason\": \"<reason>\"\n\n
Sorry that it's hard to read. But this commands list does not contain "google" nor original "google_search" in it. I get original "google_search" is changed to "google" but for now it does not seem to be included in default docker image.

The workaround was to run with 0.2.2 version image.

[jaykayenn]

That's what I thought. Google_search.py is still in the commands folder, but doesn't get picked up into commandRegistry.

[anonhostpi]

@yangkyeongmo added whitespace and removed escape sequences for readability

Commands:
  1. analyze_code: Analyze Code, args: "code": "<full_code_string>"
  2. read_audio_from_file: Convert Audio to text, args: "filename": "<filename>"
  3. execute_python_file: Execute Python File, args: "filename": "<filename>"
  4. execute_shell: Execute Shell Command, non-interactive commands only, args: "command_line": "<command_line>"
  5. execute_shell_popen: Execute Shell Command, non-interactive commands only, args: "command_line": "<command_line>"
  6. append_to_file: Append to file, args: "filename": "<filename>", "text": "<text>"
  7. delete_file: Delete file, args: "filename": "<filename>"
  8. list_files: List Files in Directory, args: "directory": "<directory>"
  9. read_file: Read file, args: "filename": "<filename>"
  10. write_to_file: Write to file, args: "filename": "<filename>", "text": "<text>"
  11. clone_repository: Clone Repository, args: "url": "<repository_url>", "clone_path": "<clone_path>"
  12. generate_image: Generate Image, args: "prompt": "<prompt>"
  13. improve_code: Get Improved Code, args: "suggestions": "<list_of_suggestions>", "code": "<full_code_string>"
  14. send_tweet: Send Tweet, args: "tweet_text": "<tweet_text>"
  15. browse_website: Browse Website, args: "url": "<url>", "question": "<what_you_want_to_find_on_website>"
  16. write_tests: Write Tests, args: "code": "<full_code_string>", "focus": "<list_of_focus_areas>"
  17. delete_agent: Delete GPT Agent, args: "key": "<key>"
  18. get_hyperlinks: Get text summary, args: "url": "<url>"
  19. get_text_summary: Get text summary, args: "url": "<url>", "question": "<question>"
  20. list_agents: List GPT Agents, args: () -> str
  21. message_agent: Message GPT Agent, args: "key": "<key>", "message": "<message>"
  22. start_agent: Start GPT Agent, args: "name": "<name>", "task": "<short_task_desc>", "prompt": "<prompt>"
  23. Task Complete (Shutdown): "task_complete", args: "reason": "<reason>"

[webbfly]

Any new development in this area? Or some workaround?

[yangkyeongmo]

Any new development in this area? Or some workaround?

@webbfly Have you tried using previous(0.2.2) version? It worked for me.

[ZylerKade]

I've noticed this issue as well. Worked for me in v0.2.2 but not since switching to stable v.0.3.0.

I have both Google API Key and Search ID set in my .env but it just uses the browse_website function and not Google.

Using GPT 4 API.

I also checked my Google API Key results to see if any traffic was using it, and I see no traffic on my API since making the switch.

[benqqqq]

Hi, I think the root cause is that there are two commands with the same "name" google (the first argument) in the following code snippets:

(Google Official search)
https://github.com/Significant-Gravitas/Auto-GPT/blob/3bf5934b20b906f977776e2d41033027ecb5df9b/autogpt/commands/google_search.py#L40-L47

(DuckDuckGo search)
https://github.com/Significant-Gravitas/Auto-GPT/blob/3bf5934b20b906f977776e2d41033027ecb5df9b/autogpt/commands/google_search.py#L14-L15

This causes the command registry to overwrite each other, as seen in the code snippet
https://github.com/Significant-Gravitas/Auto-GPT/blob/dcd6aa912b992d7c024ea1b668d3249b201b0c2c/autogpt/commands/command.py#L61-L62

There are two workarounds available:

1. Comment out the GOOGLE_API_KEY in the .env file, which will make it use DuckDuckGo for searching instead.
   #GOOGLE_API_KEY=xxxx

2. Use different names in the above two code snippets for these two commands.

[jaykayenn]

@benqqqq
Confirmed. Renaming the DDG command enables googe_search again.
But then it runs into a byte object JSON error.
TypeError: Object of type 'bytes' is not JSON serializable

EDIT: Oops. Renaming the commands solves the conflict, but 'google_search' is coded to run DDG... which is currently broken. There should be an IF..ELSE switch to select Google if the API key is defined. Otherwise, there's nowhere for the logic to invoke 'google_official_search' whether you have an API key or not.

[moltra]

is this related to #4120 (comment)

[Boostrix]

There should be an IF..ELSE switch to select Google if the API key is defined. Otherwise, there's nowhere for the logic to invoke 'google_official_search' whether you have an API key or not.

the most flexible option is making the underlying backend configurable as part of the argument - and documenting all available options via the description string of the command, that way the LLM itself can figure out ways to progress, without you having to do a thing:

• browse_website should support "modes" for different back-ends (curl, selenium etc) #4044

Basic approach explained here: #4128 (comment)

[jaykayenn]

There should be an IF..ELSE switch to select Google if the API key is defined. Otherwise, there's nowhere for the logic to invoke 'google_official_search' whether you have an API key or not.

the most flexible option is making the underlying backend configurable as part of the argument - and documenting all available options via the description string of the command, that way the LLM itself can figure out ways to progress, without you having to do a thing:

• browse_website should support "modes" for different back-ends (curl, selenium etc) #4044

Basic approach explained here: #4128 (comment)

Ah yes, agreed. I keep forgetting we have an LLM to work with, haha.

So once the naming conflict is resolved, the AI is actually doing as it's told.

• 'google_search' command is coded to use DDG (broken, needs package update)
• The command description for 'google_official_search' doesn't give the AI any reason to invoke it over 'google_search'

Am I reading it right?

[Boostrix]

the basic idea is to stop hard-coding stuff and instead move these things into arguments (which could use some sane defaults) - if there's a problem, the LLM can replace the default.

Anyway, it does make sense to mention sane alternatives in the description string - or it will try random stuff, same applies for things like the "model" value or URL endpoints.

I've been able to reduce a good amount of warnings and errors that way, simply forwarding errors and warnings to the LLM and allowing hard-coded stuff to be specified via the params of the action/command string.

that being said, I do sometimes watch it trying to come up with non-existing command strings still - I suppose it would make sense to add some sort of "immutable" annotation to the string to make it obvious to the LLM that it should not tinker with some of these things

Note that this approach works for a number of env/CFG settings, too - obviously, you won't want to expose your API key and have the LLM tinker with that (kinda pointless), but things like rate limit etc are all game.

As long as you provide sufficient "meta information" to the LLM via the description string, it will automatically figure out stuff - here it's simply picking the right tool for the job to calculate an expression:

#4143

[jonasdebeukelaer]

@jaykayenn
Dunno if you've already fixed it, but I had to change this line to fix

   [result.encode("utf-8", "ignore") for result in results]

to

   [result.encode("utf-8", "ignore").decode("utf-8") for result in results]

in the safe_google_results function. Was missing the re-decoding

[GurGaller]

Is there a workaround for people running autogpt on Docker?
I can't get it to use Google Search at all

[mecizeldec]

Same issue with Docker.

[ZylerKade]

Using the combination of suggestions from @benqqqq and @jonasdebeukelaer I was able to get mine running smoothly again with proper Google searches via the Google API (running in VS Code + Docker Container):

1. Renamed Duck Duck Go query name, changed Line 14 from:

https://github.com/Significant-Gravitas/Auto-GPT/blob/3bf5934b20b906f977776e2d41033027ecb5df9b/autogpt/commands/google_search.py#L14

to

@command("ddg", "Duck Duck Go", '"query": "<query>"', not CFG.google_api_key)

2. Added line fix, changed Line 113 from:

https://github.com/Significant-Gravitas/Auto-GPT/blob/3bf5934b20b906f977776e2d41033027ecb5df9b/autogpt/commands/google_search.py#L113

to

[result.encode("utf-8", "ignore").decode("utf-8") for result in results]

Now everything running smoothly for me again.

Note: I also updated my DuckDuckGo Search per discussion comments in #4120 - Not sure if that has anything to do with it as I didn't test in between the DDG update and making the other changes. But worth noting.

Update DuckDuckGo Search:

pip install --upgrade duckduckgo-search

[k-boikov]

Fixed with #3606 and #4226