Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Used a regex expression for simple URL validation and added tests #3763

Conversation

OmriGM
Copy link
Contributor

@OmriGM OmriGM commented May 3, 2023

Background

  • Used a regex expression for simple URL validation
  • Added additional tests to url validators

Changes

The expression before:

url.startswith("http://") and not url.startswith("https://"):

The expression after:

re.match(r"^https?://", url):

Also, added unit test to tests/unit/test_url_validation.py module, specifically to validate_url() which is the main function in the module

Documentation

Test Plan

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes

@vercel
Copy link

vercel bot commented May 3, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 15, 2023 10:54pm

@github-actions github-actions bot added the size/l label May 3, 2023
@codecov
Copy link

codecov bot commented May 3, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.01 🎉

Comparison is base (bf33f4a) 62.66% compared to head (469d6ef) 62.67%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3763      +/-   ##
==========================================
+ Coverage   62.66%   62.67%   +0.01%     
==========================================
  Files          74       74              
  Lines        3426     3427       +1     
  Branches      504      504              
==========================================
+ Hits         2147     2148       +1     
  Misses       1123     1123              
  Partials      156      156              
Impacted Files Coverage Δ
autogpt/url_utils/validators.py 93.10% <100.00%> (+0.24%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@Boostrix
Copy link
Contributor

Boostrix commented May 3, 2023

this sort of regex-based validation routine would also be useful in other places, like file name/path validation (for shell command execution)
A number of folks keep reporting that Auto-GPT is trying use absolute paths and even the ~ (home dir) inside the workspace.

@vercel vercel bot temporarily deployed to Preview May 4, 2023 06:32 Inactive
@OmriGM
Copy link
Contributor Author

OmriGM commented May 4, 2023

@ntindle BTW, there is a failing test (tests/integration/test_execute_code.py: test_execute_python_file()) that is dependent on the docker daemon running on the local machine, was that on purpose?
Here is the error message of the failing test (before I started the docker daemon on my local computer)

./tests/integration/test_execute_code.py::test_execute_python_file Failed: [undefined]assert 'Error: Error...directory\'")' == 'Hello udaxihhexd!\n'
  - Hello udaxihhexd!
  + Error: Error while fetching server API version: 500 Server Error for http+docker://localhost/version: Internal Server Error ("b'dial unix docker.raw.sock: connect: no such file or directory'")
python_test_file = '/private/var/folders/58/jb5nf1y51b1btsjnnsn9jsdc0000gn/T/pytest-of-omrigrossman/pytest-13/test_execute_python_file0/home/users/monty/auto_gpt_workspace/tmph3oy_5_j.py'
random_string = 'udaxihhexd'

    def test_execute_python_file(python_test_file: str, random_string: str):
        result = sut.execute_python_file(python_test_file)
>       assert result == f"Hello {random_string}!\n"
E       assert 'Error: Error...directory\'")' == 'Hello udaxihhexd!\n'
E         - Hello udaxihhexd!
E         + Error: Error while fetching server API version: 500 Server Error for http+docker://localhost/version: Internal Server Error ("b'dial unix docker.raw.sock: connect: no such file or directory'")

@p-i-
Copy link
Contributor

p-i- commented May 5, 2023

This is a mass message from the AutoGPT core team.
Our apologies for the ongoing delay in processing PRs.
This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to:
https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

@vercel vercel bot temporarily deployed to Preview May 5, 2023 07:08 Inactive
@anonhostpi
Copy link

I think it maybe smarter to use a library for validation. The repository may find URL/URI/Path validation useful in other areas.

@k-boikov k-boikov added this to the v0.3.2-release milestone May 14, 2023
@k-boikov k-boikov self-assigned this May 15, 2023
@vercel vercel bot temporarily deployed to Preview May 15, 2023 22:54 Inactive
@ntindle
Copy link
Member

ntindle commented May 16, 2023

Are we sure Regex is the way to go here? Aren’t there entire libraries for validation?

@Boostrix
Copy link
Contributor

This would at least seem like a start, maybe the OP would like to adopt the feature and extend it over time?

@lc0rp
Copy link
Contributor

lc0rp commented May 19, 2023

@OmriGM would you consider something like this?

urllib:

from urllib.parse import urlparse
parsed_url = urlparse("https://www.google.com")
assert bool(parsed_url.scheme and parsed_url.netloc)

Or the validators package pip install validators

...
import validators
assert validators.url("http://google.com")
...

We'd like to get this into 0.3.2.

Please also update the branch.

@vercel
Copy link

vercel bot commented May 19, 2023

Deployment failed with the following error:

Resource is limited - try again in 16 minutes (more than 100, code: "api-deployments-free-per-day").

@richbeales richbeales merged commit a08fc85 into Significant-Gravitas:master May 19, 2023
ppetermann pushed a commit to ppetermann/Auto-GPT that referenced this pull request May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

8 participants