internal resource request bug

[REal0day]

Background

The web_requests.py module attempts to identify if a document is internal or not. There are a few ways to bypass these. A common attack for access internal resources is SSRF. More information can be found here: https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Server%20Side%20Request%20Forgery/README.md

Changes

I added a few extra items to the list for checking to see if the program is access internal resources. The startswith() function only checks the first part of the string, which could disallow legitimate domains too like http://localhost.something.com. The biggest change was to add '127.0.0.1' as a localhost.

Documentation

I didn't add any comments, as I can imagine this might just add clutter to the overall program itself, with everyone commenting little changes like this.

Test Plan

I wrote a list of internal links and attempted to access them and found that they were not caught.

PR Quality Checklist

• My pull request is atomic and focuses on a single change.
• I have thoroughly tested my changes with multiple different prompts.
• I have considered potential risks and mitigations for my changes.
• I have documented my changes clearly and comprehensively.
• I have not snuck in any "extra" small tweaks changes