refactor(streaming): retrieve epoch from task local storage #9488
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Bugen Zhao [email protected]I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Epoch changes periodically in streaming executors, while providing a correct value is crucial to several components like
LruCache
,StateTable
,PROCTIME()
. For the simplicity of the interface design, we do not pass the value everywhere. Instead, we make these components to hold other copies of the value and let the executor callupdate_epoch
when it needs to be updated.This could be really error-prone if we accidentally forget to call that! For example, in #9459 we find that a lot of executors forget to update the epoch for the LRU cache, so the eviction will be problematic implicitly.
In this PR we attempt to provide the epoch in the task-local storage. Components can directly call
curr_epoch
to get the current epoch without any context, as long as it's in the scope of a running executor. We've also refactored thePROCTIME
(#9088) to retrieve the epoch from this value.This could be further used by
LruCache
andStateTable
in the future.NOTE: Per offline discussions, we agree that we should be really cautious and think twice before introducing such "global variables". Abusing it will hurt the maintainability of our system.
Checklist For Contributors
./risedev check
(or alias,./risedev c
)Checklist For Reviewers
Documentation
Click here for Documentation
Types of user-facing changes
Please keep the types that apply to your changes, and remove the others.
Release note