-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: adds values filter for map column #190
Conversation
Codecov Report
@@ Coverage Diff @@
## main #190 +/- ##
============================================
+ Coverage 78.83% 78.90% +0.06%
- Complexity 902 906 +4
============================================
Files 80 80
Lines 3661 3673 +12
Branches 420 421 +1
============================================
+ Hits 2886 2898 +12
Misses 599 599
Partials 176 176
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
This comment has been minimized.
This comment has been minimized.
Optional<String> valueFilter = | ||
getValuesFilterForMapColumn( | ||
filter.getLhs(), rhs, filter.getOperator(), paramsBuilder, executionContext); | ||
valueFilter.stream().forEach(str -> builder.append(str)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
valueFilter.stream().forEach(str -> builder.append(str)); | |
valueFilter.ifPresent(builder::append); |
since there is no collection within Optional
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
String valCol = convertExpressionToMapValueColumn(lhs); | ||
return Optional.of( | ||
String.format( | ||
"%s %s %s AND ", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: we can take AND
outside this function, so that it can be reused elsewhere as a terminal filter condition (i.e. when there is no other filter after it).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, ok.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
Description
This pull request introduces an additional value filter for map columns, aiming to enhance performance by utilizing the underlying inverted index on the
__keys
and__values
columns of the Pinot implementation.This optimisation is specifically applied to queries like
mapValue(tags__keys, 'flags', tags__values) = '0'
.This approach is the same as the one followed in the deprecated
CONTAINS_KEYVALUE
.Ref:
query-service/query-service-impl/src/main/java/org/hypertrace/core/query/service/pinot/QueryRequestToPinotSQLConverter.java
Line 181 in b89ebf4
Testing
Modified the existing test case.
Checklist: