feat(streaming): handle multiple edges with no-op fragment

[BugenZhao]

Signed-off-by: Bugen Zhao [email protected]I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

As explained in #9319. When the fragment graph builder on the frontend finds it going to add a second edge between the fragments, it'll manually introduce a fragment with a single NoOp operator ("unknown" below) and insert it between them. The exchange between the upstream and NoOp will be NoShuffle, while the one between NoOp and the downstream will be the original shuffle type. This is the solution 1.2 proposed in Multiple Edges in the Stream Graph.

I've also tried to do this in optimizer, but eventually found the approach in this PR most intuitive and accurate: it only inserts NoOp if necessary and no false positives exist. BTW, since we've found that NoShuffle exchange is much more friendly to scaling than we thought after #8694, I guess we may have no fear about that. :)

Resolve #9319.

Checklist For Contributors

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• I have demonstrated that backward compatibility is not broken by breaking changes and created issues to track deprecated features to be removed in the future. (Please refer to the issue)
• All checks passed in ./risedev check (or alias, ./risedev c)

Checklist For Reviewers

• I have requested macro/micro-benchmarks as this PR can affect performance substantially, and the results are shown.

Documentation

• My PR DOES NOT contain user-facing changes.

Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

• Installation and deployment
• Connector (sources & sinks)
• SQL commands, functions, and operators
• RisingWave cluster configuration changes
• Other (please specify in the release note below)

Release note

[chenzl25]

LGTM! What an interesting case.

[codecov]

Codecov Report

Merging #9320 (e8165f8) into main (0c381b2) will decrease coverage by 0.01%.
The diff coverage is 74.07%.

@@            Coverage Diff             @@
##             main    #9320      +/-   ##
==========================================
- Coverage   70.79%   70.79%   -0.01%     
==========================================
  Files        1225     1227       +2     
  Lines      203343   203448     +105     
==========================================
+ Hits       143955   144026      +71     
- Misses      59388    59422      +34     

Flag	Coverage Δ
rust	70.79% <74.07%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/stream/src/executor/mod.rs	50.95% <ø> (ø)
src/stream/src/executor/no_op.rs	0.00% <0.00%> (ø)
src/stream/src/from_proto/mod.rs	0.00% <ø> (ø)
src/stream/src/from_proto/no_op.rs	0.00% <0.00%> (ø)
...tend/src/stream_fragmenter/graph/fragment_graph.rs	97.40% <100.00%> (+0.57%)	⬆️
src/frontend/src/stream_fragmenter/mod.rs	79.83% <100.00%> (+7.26%)	⬆️

... and 5 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more