Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance unnest support in decoupled mode #17550

Draft
wants to merge 100 commits into
base: master
Choose a base branch
from
Draft

Enhance unnest support in decoupled mode #17550

wants to merge 100 commits into from

Conversation

kgyrtkirk
Copy link
Member

  • adds DruidRelFieldTrimmer
  • Unnest now uses unnestFieldType instead of blindly taking the rowType
  • removed the possibly problematic column reuse in UnnestCleanupRule (the trimmer should take care of that)
  • lots of iq file changes because earlier no proper trimming was done below LogicalCorrelate and Unnest -s

@kgyrtkirk
working?
7021b3f
@kgyrtkirk
x
54b057e
@kgyrtkirk
clenaup
00837c5
@kgyrtkirk
some updates
314d3f7
@kgyrtkirk
try-unnestfiled-instead-rowtype
2ceefdc 
(cherry picked from commit c4eb2cf1b82c7a332d57701e430ac582a986593f)
@kgyrtkirk
more wood
4e0a37a
@kgyrtkirk
more wood
e7061ad
@kgyrtkirk
run trimmer
b45b3ba
@kgyrtkirk
ok-but alon trim should have affected native query - right?
d38215f
@kgyrtkirk
make some assert function and fail with that
322ec81
@kgyrtkirk
make helper method
e8f06bf
@kgyrtkirk
one way
04ab28f
@kgyrtkirk
rename: d1/dbl1
c49fc27
@kgyrtkirk
Revert "rename: d1/dbl1"
0d03cc5 
This reverts commit c49fc27.
@kgyrtkirk
Reapply "rename: d1/dbl1"
d4b4c94 
This reverts commit 0d03cc5.
@kgyrtkirk
dbl1/etc
7777e48
@kgyrtkirk
dbl1/etc
0d07cf1
@kgyrtkirk
fix some
bd72b95
@kgyrtkirk
fix some more
88e57f9
@kgyrtkirk
fix windowtests
62664e3
@kgyrtkirk
fix some more
0874843
@kgyrtkirk
fix nondefault
ced72b3
@kgyrtkirk
update iq files
fc31c8a
@kgyrtkirk
Merge remote-tracking branch 'apache/master' into rename-d1-dbl1
a1b9f52
@kgyrtkirk
fixups after merge
247ee17
@kgyrtkirk
Merge branch 'rename-d1-dbl1' into unnest-relfieldtrimmer-unnestfield…
1379c9a 
…type
@kgyrtkirk
update test
8dfd57f
@kgyrtkirk
undo trials; re-introduce expr if needed (will be trimmed in most cases)
0d09c18
@kgyrtkirk
updates/etc
96f572e
@kgyrtkirk
aa
7d9693b
github-advanced-security[bot]
github-advanced-security bot found potential problems Dec 7, 2024
View reviewed changes
sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidRelFieldTrimmer.java
return makeIdentityMapping(input);
}

protected TrimResult dummyProject(int fieldCount, RelNode input,

Check notice

Code scanning / CodeQL

Missing Override annotation

This method overrides [RelFieldTrimmer.dummyProject](1); it is advisable to add an Override annotation.
sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidRelFieldTrimmer.java Outdated
return result(input, mapping);
}

public TrimResult trimFields(

Check notice

Code scanning / CodeQL

Deprecated method or constructor invocation Note test

Invoking
CalciteTestBase.makeColumnExpression
Loading
should be avoided because it has been deprecated.
@github-actions github-actions bot added Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 Area - Querying
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@kgyrtkirk