Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix jsonb 'contained by' query #3643

Merged
merged 12 commits into from
Oct 14, 2024
Merged

Fix jsonb 'contained by' query #3643

merged 12 commits into from
Oct 14, 2024

Conversation

jpbruinsslot
Copy link
Contributor

@jpbruinsslot jpbruinsslot commented Oct 8, 2024

Changes

Queries using lists as filter values when using it in combination with a 'contained_by' operator (<@ ) were incorrectly interpreted as text, and as such would create an incorrect SQL query.

Issue link

Closes #3627
References #3299 (comment)

QA notes

Api functionality from scheduler to filter on the /tasks and /queues endpoint using jsonb operator are mainly addressed here. Since we don't yet use this, this pr should not intervene with current functionality. So mainly filtering of tasks in rocky should be main area where something might happen. Checking that out would probably be a good thing.

I'm trying to find some good qa examples, to test the functionality in this pr. But having issues with using curl and how characters are getting escaped and interpreted by fastapi. So stay tuned for that.


Code Checklist

  • All the commits in this PR are properly PGP-signed and verified.
  • This PR only contains functionality relevant to the issue.
  • I have written unit tests for the changes or fixes I made.
  • I have checked the documentation and made changes where necessary.
  • I have performed a self-review of my code and refactored it to the best of my abilities.
  • Tickets have been created for newly discovered issues.
  • For any non-trivial functionality, I have added integration and/or end-to-end tests.
  • I have informed others of any required .env changes files if required and changed the .env-dist accordingly.
  • I have included comments in the code to elaborate on what is not self-evident from the code itself, including references to issues and discussions online, or implicit behavior of an interface.

Checklist for code reviewers:

Copy-paste the checklist from the docs/source/templates folder into your comment.


Checklist for QA:

Copy-paste the checklist from the docs/source/templates folder into your comment.

@jpbruinsslot jpbruinsslot added the mula Issues related to the scheduler label Oct 8, 2024
@jpbruinsslot jpbruinsslot self-assigned this Oct 8, 2024
@jpbruinsslot jpbruinsslot marked this pull request as ready for review October 9, 2024 12:47
@jpbruinsslot jpbruinsslot requested a review from a team as a code owner October 9, 2024 12:47
@jpbruinsslot jpbruinsslot marked this pull request as draft October 9, 2024 13:02
@jpbruinsslot jpbruinsslot marked this pull request as ready for review October 9, 2024 14:54
Copy link
Contributor

@ammar92 ammar92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work. Tiny remark, looks good otherwise

mula/scheduler/storage/filters/casting.py Show resolved Hide resolved
@jpbruinsslot
Copy link
Contributor Author

Nice work. Tiny remark, looks good otherwise

Thanks for reviewing 💪

@stephanie0x00
Copy link
Contributor

Checklist for QA:

  • I have checked out this branch, and successfully ran a fresh make reset.
  • I confirmed that there are no unintended functional regressions in this branch:
    • I have managed to pass the onboarding flow
    • Objects and Findings are created properly
    • Tasks are created and completed properly
  • I confirmed that the PR's advertised feature or hotfix works as intended.
  • I checked the logs for errors and/or warnings and made issues where necessary

What works:

Seems to work. Scheduled a bunch of tasks and then filtered on each of the task filters to see if it would work. Also checked the logs for the scheduler, rocky, octopoes_api, octopoes_api_worker and bytes to check for any error messages that might be side effects. No obvious bugs found.

What doesn't work:

n/a

Bug or feature?:

n/a

@underdarknl underdarknl merged commit 40c7bbe into main Oct 14, 2024
14 checks passed
@underdarknl underdarknl deleted the fix/mula/jsonb-filtering branch October 14, 2024 13:25
Copy link

sonarcloud bot commented Oct 14, 2024

jpbruinsslot added a commit that referenced this pull request Oct 24, 2024
* main: (25 commits)
  Add sterr to output list (#3649)
  feat: ✨ add Shodan InternetDB boefje (#2615)
  Add search endpoint for schedules for scheduler (#3695)
  Fix/report naming (#3666)
  Add delete schedule functionality for schedules in the scheduler (#3678)
  Fix javascript and component template in prod environments (#3672)
  Fix first order dangling affirmation delete (#3682)
  Do not run dh_strip_nondeterminism in Debian packaging (#3674)
  Github action should trigger if workflow definition changes (#3680)
  Updated packages (#3694)
  Silence KATFindingType not found error in JobHandler (#3686)
  Add docs for xtdb analyze bits. (#3688)
  Fix empty vulnerability reports (#3662)
  Fix jsonb 'contained by' query (#3643)
  Add plugins to findings report (#3657)
  Translations update from Hosted Weblate (#3673)
  Update upload_raw.py (#3645)
  Add rocky worker service to debian packages (#3619)
  Fix/yielded objects (#3669)
  make some things look better (#3661)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mula Issues related to the scheduler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix jsonb filtering in scheduler
6 participants