Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore: Move aggregate statistics optimizer test from core to optimizer crate #12783

Merged
merged 7 commits into from
Oct 9, 2024

Conversation

jayzhan211
Copy link
Contributor

Which issue does this PR close?

Closes #.

Rationale for this change

Part of #11502
When we move the file, there is dependency issue that we keep it in core crate before. Now we move it to the ideal place.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added optimizer Optimizer rules core Core DataFusion crate labels Oct 7, 2024
Signed-off-by: jayzhan211 <[email protected]>
@jayzhan211 jayzhan211 changed the title [chore] Move aggregate statistics optimizer test from core to optimizer crate Chore: Move aggregate statistics optimizer test from core to optimizer crate Oct 7, 2024
@jayzhan211 jayzhan211 marked this pull request as draft October 7, 2024 09:20
Signed-off-by: jayzhan211 <[email protected]>
Signed-off-by: jayzhan211 <[email protected]>
@jayzhan211 jayzhan211 marked this pull request as ready for review October 8, 2024 07:05
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @jayzhan211

datafusion-physical-expr = { workspace = true }
datafusion-physical-plan = { workspace = true }
itertools = { workspace = true }

[dev-dependencies]
datafusion-functions-aggregate = { workspace = true }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to avoid this dependency? (I think we could remove it as a follow on PR, but I didn't quite understand why it was needed in the first place)

Looks like it is just count_udaf (which we could perhaps mock up) 🤔

Copy link
Contributor Author

@jayzhan211 jayzhan211 Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can't use function that hasn't implement accumulator here, for physical optimizer, I think we should use the real function

Error if I switch to stub

test aggregate_statistics::tests::test_count_partial_with_nulls_indirect_child ... FAILED
test aggregate_statistics::tests::test_count_partial_direct_child ... FAILED
test aggregate_statistics::tests::test_count_partial_with_nulls_direct_child ... FAILED
test aggregate_statistics::tests::test_count_partial_indirect_child ... FAILED
test aggregate_statistics::tests::test_count_inexact_stat ... FAILED
test aggregate_statistics::tests::test_count_with_nulls_inexact_stat ... FAILED

successes:

successes:

failures:

---- aggregate_statistics::tests::test_count_partial_with_nulls_indirect_child stdout ----
Error: NotImplemented("no impl for stub")

---- aggregate_statistics::tests::test_count_partial_direct_child stdout ----
Error: NotImplemented("no impl for stub")

---- aggregate_statistics::tests::test_count_partial_with_nulls_direct_child stdout ----
Error: NotImplemented("no impl for stub")

---- aggregate_statistics::tests::test_count_partial_indirect_child stdout ----
Error: NotImplemented("no impl for stub")

---- aggregate_statistics::tests::test_count_inexact_stat stdout ----
Error: NotImplemented("no impl for stub")

---- aggregate_statistics::tests::test_count_with_nulls_inexact_stat stdout ----
Error: NotImplemented("no impl for stub")

@jayzhan211 jayzhan211 merged commit d83b3b2 into apache:main Oct 9, 2024
26 checks passed
@jayzhan211 jayzhan211 deleted the aggregate-statistic-test branch October 9, 2024 03:38
@jayzhan211
Copy link
Contributor Author

Thanks @alamb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate optimizer Optimizer rules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants