planner: enhance stats derive suitable for memo derive and traditional logical tree derive #58252

AilinKid · 2024-12-13T08:23:58Z

What problem does this PR solve?

Issue Number: ref #51664

Problem Summary:

What changed and how does it work?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

tiprow · 2024-12-13T08:24:16Z

Hi @AilinKid. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

codecov · 2024-12-13T08:47:00Z

Codecov Report

Attention: Patch coverage is 93.90244% with 5 lines in your changes missing coverage. Please review.

Project coverage is 73.3954%. Comparing base (1fa2955) to head (17a0400).
Report is 39 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #58252        +/-   ##
================================================
+ Coverage   73.1670%   73.3954%   +0.2284%     
================================================
  Files          1680       1681         +1     
  Lines        463042     470382      +7340     
================================================
+ Hits         338794     345239      +6445     
- Misses       103444     104290       +846     
- Partials      20804      20853        +49

Flag	Coverage Δ
integration	`43.2490% <93.5897%> (?)`
unit	`72.3902% <89.0243%> (+0.0896%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.6910% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`44.5557% <ø> (-1.4641%)`	⬇️

pkg/planner/core/casetest/cascades/memo_test.go

pkg/planner/core/collect_column_stats_usage.go

pkg/planner/core/rule_collect_plan_stats.go

pkg/planner/core/operator/logicalop/logical_cte.go

AilinKid · 2024-12-18T09:17:53Z

/test unit-test

tiprow · 2024-12-18T09:18:16Z

@AilinKid: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

AilinKid · 2024-12-18T10:53:04Z

/retest-required

tiprow · 2024-12-18T10:53:26Z

@AilinKid: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest-required

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

AilinKid · 2024-12-18T10:59:24Z

/ok-to-test

AilinKid · 2024-12-18T11:10:55Z

/retest-required

Signed-off-by: arenatlx <[email protected]>

AilinKid · 2024-12-23T04:28:22Z

/ok-to-test

qw4990 · 2024-12-23T05:04:47Z

pkg/planner/core/operator/logicalop/logical_aggregation.go

-	if len(colGroups) == 0 {
-		return nil
-	}
+func (*LogicalAggregation) getGroupNDVs(childProfile *property.StatsInfo, gbyCols []*expression.Column) []property.GroupNDV {


Can we even remove these getGroupNDVs later on?

ti-chi-bot · 2024-12-23T06:09:19Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hawkingrei, qw4990

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/planner/OWNERS~~ [hawkingrei,qw4990]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-12-23T06:09:23Z

[LGTM Timeline notifier]

Timeline:

2024-12-20 10:44:31.804694606 +0000 UTC m=+1212861.893497149: ☑️ agreed by hawkingrei.
2024-12-23 06:09:22.260230104 +0000 UTC m=+1455552.349032646: ☑️ agreed by qw4990.

Signed-off-by: arenatlx <[email protected]>

tiprow · 2024-12-24T07:09:58Z

@AilinKid: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
fast_test_tiprow	`17a0400`	link	true	`/test fast_test_tiprow`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/invalid-title sig/planner SIG: Planner size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 13, 2024

AilinKid changed the title ~~Enhance memo stats derive~~ planner: enhance stats derive suitable for memo derive and traditional logical tree derive Dec 13, 2024

ti-chi-bot bot removed the do-not-merge/invalid-title label Dec 13, 2024

hawkingrei self-requested a review December 17, 2024 09:24

hawkingrei reviewed Dec 17, 2024

View reviewed changes

pkg/planner/core/casetest/cascades/memo_test.go Show resolved Hide resolved

hawkingrei reviewed Dec 17, 2024

View reviewed changes

pkg/planner/core/collect_column_stats_usage.go Outdated Show resolved Hide resolved

hawkingrei reviewed Dec 17, 2024

View reviewed changes

pkg/planner/core/rule_collect_plan_stats.go Outdated Show resolved Hide resolved

hawkingrei reviewed Dec 18, 2024

View reviewed changes

pkg/planner/core/operator/logicalop/logical_cte.go Outdated Show resolved Hide resolved

ti-chi-bot bot added the ok-to-test Indicates a PR is ready to be tested. label Dec 18, 2024

AilinKid added 6 commits December 19, 2024 16:19

.

bde8291

Signed-off-by: arenatlx <[email protected]>

.

5532aed

Signed-off-by: arenatlx <[email protected]>

.

6b98340

Signed-off-by: arenatlx <[email protected]>

.

995c539

Signed-off-by: arenatlx <[email protected]>

.

ffcfc9a

Signed-off-by: arenatlx <[email protected]>

.

8991950

Signed-off-by: arenatlx <[email protected]>

AilinKid force-pushed the enhance-memo-stats-derive branch from a5aa1ed to 8991950 Compare December 19, 2024 08:21

AilinKid added 2 commits December 19, 2024 17:04

.

9a3c935

Signed-off-by: arenatlx <[email protected]>

.

d26d061

Signed-off-by: arenatlx <[email protected]>

hawkingrei approved these changes Dec 20, 2024

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 20, 2024

AilinKid added 3 commits December 23, 2024 12:20

.

ecf7735

Signed-off-by: arenatlx <[email protected]>

.

cad2c73

Signed-off-by: arenatlx <[email protected]>

.

db2390e

Signed-off-by: arenatlx <[email protected]>

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 23, 2024

qw4990 reviewed Dec 23, 2024

View reviewed changes

qw4990 approved these changes Dec 23, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 23, 2024

AilinKid added 2 commits December 24, 2024 15:06

.

3625251

Signed-off-by: arenatlx <[email protected]>

.

17a0400

Signed-off-by: arenatlx <[email protected]>

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 24, 2024

ti-chi-bot bot merged commit 8ecdb54 into pingcap:master Dec 24, 2024
23 of 24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner: enhance stats derive suitable for memo derive and traditional logical tree derive #58252

planner: enhance stats derive suitable for memo derive and traditional logical tree derive #58252

AilinKid commented Dec 13, 2024

tiprow bot commented Dec 13, 2024

codecov bot commented Dec 13, 2024 •

edited

Loading

AilinKid commented Dec 18, 2024

tiprow bot commented Dec 18, 2024

AilinKid commented Dec 18, 2024

tiprow bot commented Dec 18, 2024

AilinKid commented Dec 18, 2024

AilinKid commented Dec 18, 2024

AilinKid commented Dec 23, 2024

qw4990 Dec 23, 2024

ti-chi-bot bot commented Dec 23, 2024

ti-chi-bot bot commented Dec 23, 2024

tiprow bot commented Dec 24, 2024

planner: enhance stats derive suitable for memo derive and traditional logical tree derive #58252

planner: enhance stats derive suitable for memo derive and traditional logical tree derive #58252

Conversation

AilinKid commented Dec 13, 2024

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

tiprow bot commented Dec 13, 2024

codecov bot commented Dec 13, 2024 • edited Loading

Codecov Report

AilinKid commented Dec 18, 2024

tiprow bot commented Dec 18, 2024

AilinKid commented Dec 18, 2024

tiprow bot commented Dec 18, 2024

AilinKid commented Dec 18, 2024

AilinKid commented Dec 18, 2024

AilinKid commented Dec 23, 2024

qw4990 Dec 23, 2024

Choose a reason for hiding this comment

ti-chi-bot bot commented Dec 23, 2024

ti-chi-bot bot commented Dec 23, 2024

[LGTM Timeline notifier]

tiprow bot commented Dec 24, 2024

codecov bot commented Dec 13, 2024 •

edited

Loading