Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update website_discovery.py #3921

Merged
merged 4 commits into from
Dec 4, 2024
Merged

Update website_discovery.py #3921

merged 4 commits into from
Dec 4, 2024

Conversation

underdarknl
Copy link
Contributor

fixes #3913

Changes

This makes sure we don't create websites on services that only begin with http such as http_proxy. As those urls cannot be handled by Boefjes later on in the chain.

Issue link

Closes #3913

Demo

Please add some proof in the form of screenshots or screen recordings to show (off) new functionality, if there are interesting new features for end-users.

QA notes

See the ISsue for an IP that returns a http_proxy service when nmapped, it should no longer also create a website on that service

Code Checklist

  • All the commits in this PR are properly PGP-signed and verified.
  • This PR only contains functionality relevant to the issue.
  • I have written unit tests for the changes or fixes I made.
  • I have checked the documentation and made changes where necessary.
  • I have performed a self-review of my code and refactored it to the best of my abilities.
  • Tickets have been created for newly discovered issues.
  • For any non-trivial functionality, I have added integration and/or end-to-end tests.
  • I have informed others of any required .env changes files if required and changed the .env-dist accordingly.
  • I have included comments in the code to elaborate on what is not self-evident from the code itself, including references to issues and discussions online, or implicit behavior of an interface.

Checklist for code reviewers:

Copy-paste the checklist from the docs/source/templates folder into your comment.


Checklist for QA:

Copy-paste the checklist from the docs/source/templates folder into your comment.

@underdarknl underdarknl requested a review from a team as a code owner December 2, 2024 10:25
Copy link
Contributor

@dekkers dekkers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change looks good, but I think we should also look into why we get "http-proxy" as service name because if I remember correctly from seeing this earlier the service name should be just "http". If we don't fix that we risk not scanning websites that should be scanned.

@underdarknl
Copy link
Contributor Author

for some reason, the remote host that triggered this, no longer returns http-proxy at the moment. However, it might just be that :8080 and :8443 are named http-proxy in the nmap services file?

Copy link

sonarcloud bot commented Dec 4, 2024

@dekkers dekkers merged commit 86bb96f into main Dec 4, 2024
33 checks passed
@dekkers dekkers deleted the fix/websites-http-services branch December 4, 2024 17:25
jpbruinsslot added a commit that referenced this pull request Dec 11, 2024
* main: (21 commits)
  Bump django from 5.0.9 to 5.0.10 in /rocky (#3940)
  Do not let enabling plugins affect the global plugin cache (#3944)
  Fix typing in more places and configure mypy to follow imports (#3932)
  Updates CWE archive to 4.16 (#3943)
  Report flaws (#3880)
  Translations update from Hosted Weblate (#3939)
  Fix report recipe API (#3942)
  Boefje runonce functionality in scheduler (#3906)
  fix: 🔨 do not store CDN findings (#3931)
  Dont check for Locations on local Ip's. (#3894)
  add unpkg.com to disallowed hostnames in CSP (#3927)
  Update website_discovery.py (#3921)
  Add export http boefje (#3901)
  Bump python-multipart from 0.0.9 to 0.0.18 in /bytes (#3925)
  Fix layout issues on scheduled reports page (#3930)
  Create scheduled report with zero objects selectable (#3907)
  Improve the KATalogus `/plugins` endpoint performance (#3892)
  Add bgp.jsonl and bgp-meta.json to .gitignore (#3928)
  Update pre-commit and all hooks (#3923)
  add support for detecting Lame dns delegations on ip ranges (#3899)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

security.txt downloader gets activated on http-proxy scheme style urls
3 participants