Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed cancellation propagation when task group host is in a shielded scope #648

Merged
merged 8 commits into from
Dec 14, 2023

Conversation

agronholm
Copy link
Owner

This makes the following changes:

  1. Cancel scopes now track their child scopes
  2. When entering a cancel scope, it moves its host task from the parent scope's task set to its own, and returns it when exiting
  3. Cancel scopes propagate cancellation directly to any child scope that isn't shielded or explicitly cancelled (as explicitly cancelled scopes would have their own cancel callbacks)
  4. Task groups now track their tasks separately from their cancel scopes

Fixes #642.

Copy link
Collaborator

@gschaffner gschaffner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me! the logic you describe makes sense and the implementation looks to match it. also, the implementation appears to clean up after the new reference cycles.

src/anyio/_backends/_asyncio.py Outdated Show resolved Hide resolved
Comment on lines 496 to 511
def _deliver_cancellation_to_parent(self) -> None:
"""Start cancellation effort in the farthest directly cancelled parent scope"""
def _restart_cancellation_in_parent(self) -> None:
"""Start cancellation effort in the closest directly cancelled parent scope"""
scope = self._parent_scope
scope_to_cancel: CancelScope | None = None
while scope is not None:
if scope._cancel_called and scope._cancel_handle is None:
scope_to_cancel = scope
if scope._cancel_called:
if scope._cancel_handle is None:
scope._deliver_cancellation(scope)

break

# No point in looking beyond any shielded scope
if scope._shield:
break

scope = scope._parent_scope

if scope_to_cancel is not None:
scope_to_cancel._deliver_cancellation()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would the following simplification be correct?:

def _restart_cancellation_in_parent(self) -> None:
        """Restart cancellation effort in the parent scope if it was directly cancelled"""
        scope = self._parent_scope
        if scope is not None and scope._cancel_called and scope._cancel_handle is None:
            scope._deliver_cancellation(scope)

it passes all tests. i think that _deliver_cancellation_to_parent previously needed to walk up beyond self._parent_scope because self._parent_scope.__exit__ was not guaranteed to call self._parent_scope._deliver_cancellation_to_parent. but with the call in __exit__ unconditional now, i think walking up beyond self._parent_scope is not needed.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it looks good. I've pushed this change now.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops—i've just realized that this simplification is actually incorrect. suggested changes: a7284b7...bfef03b

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's embarrassing that I didn't realize that myself. Reverted.

@agronholm
Copy link
Owner Author

All ready for merging then?

Copy link
Collaborator

@gschaffner gschaffner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are a couple docstrings/comments that got out of sync, but otherwise looks good to merge!

src/anyio/_backends/_asyncio.py Outdated Show resolved Hide resolved
src/anyio/_backends/_asyncio.py Show resolved Hide resolved
@gschaffner
Copy link
Collaborator

gschaffner commented Dec 14, 2023

might be good to add 0214c34, as that bug almost slipped by.

@agronholm
Copy link
Owner Author

might be good to add 0214c34, as that bug almost slipped by.

Done.

Copy link
Collaborator

@gschaffner gschaffner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great! apologies for the review being a bit hectic :)

@agronholm
Copy link
Owner Author

No worries! Would you mind approving the PR if it looks good?

@agronholm
Copy link
Owner Author

Hm, that didn't do the trick. It doesn't look like you're on the collaborator list. Would you like to be?

@gschaffner
Copy link
Collaborator

sure!

@gschaffner
Copy link
Collaborator

invite accepted; should be merge-able now.

@agronholm agronholm merged commit 44ca5ea into master Dec 14, 2023
16 checks passed
@agronholm agronholm deleted the fix-642 branch December 14, 2023 09:10
@agronholm
Copy link
Owner Author

Thanks for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Opening a CancelScope with shield=True also shields sibling tasks on asyncio backend
2 participants