Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: df patched upgrade to 2024-03-05, requiring new DF fixes #9901

Closed

Conversation

wiedld
Copy link
Contributor

@wiedld wiedld commented Apr 1, 2024

Opened on wrong base. Closing.

alamb and others added 10 commits March 28, 2024 12:29
… dictionaries (apache#9679)

* Add test for multiple count distincts on a dictionary

* Fix accumulator merge bug

* Fix cleanup code
…r common subexpr elimination optimization (apache#9685)

* test(9678): reproducer of short-circuiting causing expr elimination to error

* fix(9678): populate visited stack for short-circuited expressions, during the common-expr elimination optimization

* test(9678): reproducer for optimizer error (in common_subexpr_eliminate), as seen in other test case

* chore: extract id_array into abstraction, to make it more clear the relationship between the two visitors

* refactor: tweak the fix and make code more explicit (JumpMark, node_to_identifier)

* fix: get the series_number and curr_id with the correct self.current_idx, before the various incr/decr

* chore: remove unneeded conditional check (already done earlier), and add code comments

* Refine documentation in common_subexpr_eliminate.rs

* chore: cleanup -- fix 1 doc comment and consolidate common-expr-elimination test with other expr test

---------

Co-authored-by: Andrew Lamb <[email protected]>
… not always stay in sync with the updated TreeNode traversal
…, while keeping the (stack-popped) symbol used for alias.
…expr_set, while keeping the (stack-popped) symbol used for alias."

This reverts commit 049bf09.
…ing does not always stay in sync with the updated TreeNode traversal"

This reverts commit d59a8de.
…re-find the correct expression during re-write. (apache#9871)

* test(9870): reproducer of error with jumping traversal patterns in common-expr-elimination traversals

* refactor: remove the IdArray ordered idx, since the idx ordering does not always stay in sync with the updated TreeNode traversal

* refactor: use the only reproducible key (expr_identifer) for expr_set, while keeping the (stack-popped) symbol used for alias.

* refactor: encapsulate most of the logic within ExprSet, and delineate the expr_identifier from the alias symbol

* test(9870): demonstrate that the sqllogictests are now passing
* fix: Remove supported coalesce types

* Use comparison_coercion

* Fix test

* Fix

* Add comment

* More

* fix
)

* fix CI clippy

* fix scalar size test

* fix tests

* fix tests
@github-actions github-actions bot added logical-expr Logical plan and expressions physical-expr Physical Expressions optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) substrait labels Apr 1, 2024
@wiedld
Copy link
Contributor Author

wiedld commented Apr 1, 2024

Opened on wrong base. Closing.

@wiedld wiedld closed this Apr 1, 2024
@wiedld wiedld deleted the iox-10349/df-upgrade-with-patches branch June 21, 2024 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate logical-expr Logical plan and expressions optimizer Optimizer rules physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt) substrait
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants