-
Notifications
You must be signed in to change notification settings - Fork 130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add loop regions to the frontend's capabilities #1475
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not yet, need to discuss. Please answer questions in the meantime
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make minor changes and feel free to merge.
@@ -303,7 +308,7 @@ def replace(self, name: str, new_name: str): | |||
:param name: Name to find. | |||
:param new_name: Name to replace. | |||
""" | |||
raise NotImplementedError() | |||
pass | |||
|
|||
@abc.abstractmethod |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They may have empty contents bur are still @abc.abstractmethod
s. What was the intention?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Initially the idea was to never instantiate a ControlFlowBlock - only subclasses of it, each of which being a graph (States and Control flow regions). However, with the change to use break/return/continue blocks, we now have blocks being used which should NOT contain further elements - so are no longer a graph. That's why these methods now all of a sudden need to not throw NotImplementedErrors. Or rather, they could be left as throwing those errors, but then at each point where we iterate over nodes of a graph we would need to make sure to not try and iterate into a control flow block, only into states and regions etc.
So that's the background - and now why I left the abstractmethod decorator: It should still be the case that you never instantiate a BlockGraphView directly and always explicitly subclass it. In 90% of use cases, that would mean implementing these methods here, since you would usually be making a graph type object.
- Improve CFG incompatibility warning message - Adhere to LLVM loop terminology (tail -> latch) - Minor type safety improvements
This is a continuation of #1617 (superseded and closed by this PR), with a lot of the work being done by @luca-patrignani. # Conditional Blocks This PR implements Conditional Blocks, which are a native way of semantically expressing conditional branching in an SDFG. This replaces the traditional "state machine only" way of expressing conditional branching, with two main goals: 1. **Simplify SDFG analysis and optimization by clearly exposing conditional branching.** Previously, detecting and treating conditional branches required expensive analysis of the control flow graph structure, which had to be performed repeatedly and was error prone. By contrast, Conditional Blocks can be generated by a frontend using semantic information from the source language, entirely circumventing this step. 2. **Address code generation issues.** Code generation relies on a series of control flow detections to generate appropriate code that is not full of `goto` statements for each state transition. However, just as in the above issue, this process is error prone and often leads to invalid code being generated for complex control flow constructs (e.g., conditionals inside of loops with conditional break, continue, return, etc.). By exposing _all_ regular control flow (i.e., loops and conditional branching) with native SDFG constructs, this step can be skipped in code generation. ### Anatomy of Conditional Blocks `ConditionalBlock`s are a type of `ControlFlowBlock` which contains a series of **branches**. Each branch is represented by a full `ControlFlowRegion` and has a condition in the form of a `CodeBlock` attached to it. When a `ConditionalBlock` is executed, the conditions are checked in the insertion order of the branches, and if a matching condition was found, that branch (and only that branch) is executed. When the executed branch finishes executing, the `ConditionalBlock`'s successor is next. If no condition matches, no branch is executed. The condition for a single branch at a time may be `None`, which represents a wildcard or `else` case that is executed if no conditions match. ### Code Generation Changes Code generation (when using this feature) is drastically simplified with respect to control flow: no more control flow detection is performed. Instead, regular control flow constructs are only generated from the new native SDFG constructs ([`LoopRegion`s](#1475) and `ConditionalBlock`s), and any other state transition is either only used for sequential ordering (unconditional transitions to a single, direct successor), or leads to a `goto`. This makes code generation significantly less error prone and simpler to work with. ### Compatibility This feature is implemented minimally invasive and with full backwards compatibility for now. Just as with [`LoopRegion`s](#1475), this feature is only used if an explicit `use_experimental_cfg_blocks` flag is set to `True` in compatible frontends (currently only Python frontend, Fortran frontend integration is coming soon). If an SDFG makes use of these experimental blocks, some passes and transformations will no longer be applied automatically in pipelines. Transformations that handle these blocks correctly can be explicitly marked with `@transformation.experimental_cfg_block_compatible` to apply them on SDFGs with experimental blocks. ### Inlining Conditional blocks can be inlined through a utility function to traditional SDFG state machines. This is automatically done by compatible frontends if the experimental CFG blocks feature is turned off. ### Visualization Components The visualization components are being worked on separately in spcl/dace-webclient#173. This PR does not depend on the visualization components to be merged. --------- Co-authored-by: Luca Patrignani <[email protected]> Co-authored-by: luca-patrignani <[email protected]>
This PR lets the Python and Fortran frontends (optionally) generate
LoopRegion
s for DaCe programs. This forms the third core element of the plan to make loops first class citizens of SDFGs.This PR is fully backwards compatible.
LoopRegion
s are always generated from new Python DaCe programs, and the legacy way of constructing a while / for loop is gone to remove complexity. To provide backwards compatibility, theseLoopRegion
s are by default immediately inlined into a traditional single level state machine loop as soon as program parsing is completed, before simplification and / or validation. However, an optional boolean parameteruse_experimental_cfg_blocks
can be set to True when declaring a DaCe program in Python to enable their use, which skips this inlining step.Example use:
The Fortran frontend similarly only utilizes
LoopRegions
if an additional parameteruse_experimenatl_cfg_blocks
is passed to the parser together with the program.Many passes and transformations (including in simplify) do not yet have the capability of handling the new, hierarchical SDFGs. To not break the pipeline and to provide backwards compatibility, a new decorator
@single_level_sdfg_only
has been added, which can be (and has been) placed over any pass or transformation that is not compatible with the new style SDFGs. Passes annotated with this decorator are skipped in all pipelines where they occur and instead generate warnings that they were skipped due to compatibility issues.For more information on
LoopRegion
s please refer to the PR that introduced them.Important Note about disabled tests:
Certain Python frontend loop tests have been disabled. Specifically, this concerns tests where either the loop structure (using continue/break) or other conditional statements cause the generation of control flow that looks irregular before the simplification pass is applied. The reason being that the frontend generates branches with one branch condition being set to constant
False
when generating continue / break / return, or while/for-else clauses. These branches are trivially removed during simplification, but not running simplification (as part of our CI does) leads to irregular control flow which is handled poorly by our codegen-time control flow detection. This error has so far gone unnoticed in these tests because of sheer luck, but is now exposed through a ever so slightly different state machine being generated by control flow region and loop inlining.The goal is for a subsequent PR to completely adapt codegen to make use of the control flow region constructs, thereby fixing this issue and re-enabling the tests. For more information about the issue, see #635 and #1586.
Linked to: https://github.com/orgs/spcl/projects/10/views/4?pane=issue&itemId=42047238 and https://github.com/orgs/spcl/projects/10/views/4?pane=issue&itemId=42151188