fix(optimizer): fix lookup join and agg distribution derive

[chenzl25]

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

PLEASE DO NOT LEAVE THIS EMPTY !!!

Please explain IN DETAIL what the changes are in this PR and why they are needed:

• Lack of considering output_indices for BatchLookupJoin.
• Lack of considering UpstreamHashShard for BatchHashAgg, BatchSortAgg, StreamHashAgg.
• This bug found by enabling batch lookup join by default. feat(batch): enable batch lookup join by default #6818

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

If your pull request contains user-facing changes, please specify the types of the changes, and create a release note. Otherwise, please feel free to remove this section.

Types of user-facing changes

Please keep the types that apply to your changes, and remove those that do not apply.

• Installation and deployment
• Connector (sources & sinks)
• SQL commands, functions, and operators
• RisingWave cluster configuration changes
• Other (please specify in the release note below)

Release note

Please create a release note for your changes. In the release note, focus on the impact on users, and mention the environment or conditions where the impact may occur.

Refer to a related PR or issue link (optional)

[codecov]

Codecov Report

Merging #6833 (c751355) into main (4a7dc72) will decrease coverage by 0.00%.
The diff coverage is 91.66%.

@@            Coverage Diff             @@
##             main    #6833      +/-   ##
==========================================
- Coverage   73.22%   73.22%   -0.01%     
==========================================
  Files        1026     1026              
  Lines      164162   164168       +6     
==========================================
+ Hits       120207   120210       +3     
- Misses      43955    43958       +3     

Flag	Coverage Δ
rust	73.22% <91.66%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ntend/src/optimizer/plan_node/batch_lookup_join.rs	57.06% <88.88%> (+0.88%)	⬆️
...frontend/src/optimizer/plan_node/batch_hash_agg.rs	66.07% <100.00%> (ø)
...frontend/src/optimizer/plan_node/batch_sort_agg.rs	75.60% <100.00%> (ø)
...rontend/src/optimizer/plan_node/stream_hash_agg.rs	92.53% <100.00%> (-1.50%)	⬇️
src/common/src/types/ordered_float.rs	32.03% <0.00%> (-0.20%)	⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[BowenXiao1999]

LGTM

[BowenXiao1999]

Before this pr, we can not derive the group_key here?

[chenzl25]

Before this pr, we will miss a exchange here.

[BugenZhao]

Will #6642 be caused by this? 🤔

[BugenZhao]

I guess the batch_plan must be distributed?

[chenzl25]

Currently, our query execution mode by default is local, so we need to set it to distributed.

[BugenZhao]

Seems we always generate distributed plan. For local plan, we use the local_batch_plan field.

risingwave/src/frontend/planner_test/src/lib.rs

Lines 468 to 479 in 672c049

	let batch_plan = match logical_plan.gen_batch_distributed_plan() {
	Ok(batch_plan) => batch_plan,
	Err(err) => {
	ret.batch_error = Some(err.to_string());
	break 'batch;
	}
	};
	// Only generate batch_plan if it is specified in test case
	if self.batch_plan.is_some() {
	ret.batch_plan = Some(explain_plan(&batch_plan));
	}

[chenzl25]

You are right. I can remove it in this #6818 PR together.

set rw_batch_enable_lookup_join = true;
set query_mode = distributed;