Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

op-batcher: Move decision about data availability type to channel submission time #12002

Merged
merged 31 commits into from
Sep 25, 2024

Conversation

geoknee
Copy link
Contributor

@geoknee geoknee commented Sep 19, 2024

Closes #11609

For now, I decided to not make any changes to the decision logic itself (only to when it is triggered and what happens when a change is necessary).

Changing the trigger for the decision opens the opportunity to make a more accurate decision about which DA type to choose, since some transaction data is now in scope. However, this would not address the larger assumption in the current logic, which is about how many blobs are typically included in a tx. This can be lower than the target number on chains with low throughput as channels must be closed before they time out. To address this assumption, we could store the previous number of blobs per tx to make a better estimate than the current one (which assumes we always include the target number). This is saved for future work.

Tests: the modified implementation passes existing tests, including an end to end test for switching DA type.

@geoknee geoknee requested review from a team as code owners September 19, 2024 13:15
@geoknee geoknee mentioned this pull request Sep 19, 2024
@sebastianst sebastianst removed the request for review from bitwiseguy September 19, 2024 16:46
Copy link
Member

@sebastianst sebastianst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should definitely add a few unit tests of TxData() and the requeue function (currently called Rebuild).

op-batcher/batcher/channel_config_provider.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved

This comment was marked as off-topic.

@zhiqiangxu
Copy link
Contributor

If the Data Availability type is determined at submission time, how are MaxFrameSize and MaxFramesPerTx decided? I believe there's a kind of interdependency here.

This comment was marked as off-topic.

@geoknee
Copy link
Contributor Author

geoknee commented Sep 23, 2024

If the Data Availability type is determined at submission time, how are MaxFrameSize and MaxFramesPerTx decided? I believe there's a kind of interdependency here.

In dynamic DA mode, the batcher maintains two configs (one for each DA type blobs and calldata). One of those configs is the default (i.e. initially blobs, but generally whatever was used when the previous channel was submitted). The channel can then be built optimistically with the default config. When it comes to submit the channel, if the data availability type needs to change, channels which haven't been submitted will be discarded and rebuilt with the appropriate config (including params like MaxFrameSize. Does that make sense to you?

@zhiqiangxu
Copy link
Contributor

If the Data Availability type is determined at submission time, how are MaxFrameSize and MaxFramesPerTx decided? I believe there's a kind of interdependency here.

In dynamic DA mode, the batcher maintains two configs (one for each DA type blobs and calldata). One of those configs is the default (i.e. initially blobs, but generally whatever was used when the previous channel was submitted). The channel can then be built optimistically with the default config. When it comes to submit the channel, if the data availability type needs to change, channels which haven't been submitted will be discarded and rebuilt with the appropriate config (including params like MaxFrameSize. Does that make sense to you?

Yeah it makes sense, thank you!

@sebastianst
Copy link
Member

In dynamic DA mode, the batcher maintains two configs (one for each DA type blobs and calldata). One of those configs is the default (i.e. initially blobs, but generally whatever was used when the previous channel was submitted). The channel can then be built optimistically with the default config. When it comes to submit the channel, if the data availability type needs to change, channels which haven't been submitted will be discarded and rebuilt with the appropriate config (including params like MaxFrameSize. Does that make sense to you?

Worth adding that we're effectively still choosing the config at the same point in time when we create the channel. That's because right after a channel is being submitted, a new channel is created, and then we choose the config that was just selected 2 seconds ago.

This comment was marked as off-topic.

@geoknee geoknee force-pushed the gk/batcher-da-simplified branch from 9a46625 to cbf5751 Compare September 24, 2024 09:01
do not return nextTxData from channel which was discarded by requeue
Copy link
Member

@sebastianst sebastianst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding the additional tests!

op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Show resolved Hide resolved
op-batcher/batcher/channel_manager.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager_test.go Outdated Show resolved Hide resolved
op-batcher/batcher/channel_manager_test.go Show resolved Hide resolved
Copy link
Contributor

semgrep-app bot commented Sep 24, 2024

Semgrep found 1 sol-style-return-arg-fmt finding:

  • packages/contracts-bedrock/src/libraries/Blueprint.sol

Named return arguments to functions must be appended with an underscore (_)

Ignore this finding from sol-style-return-arg-fmt.

@geoknee geoknee force-pushed the gk/batcher-da-simplified branch from 5da2bce to 510464d Compare September 24, 2024 20:21
Copy link
Contributor

semgrep-app bot commented Sep 25, 2024

Semgrep found 2 sol-style-input-arg-fmt findings:

Inputs to functions must be prepended with an underscore (_)

Ignore this finding from sol-style-input-arg-fmt.

@geoknee geoknee added this pull request to the merge queue Sep 25, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Sep 25, 2024
@geoknee geoknee added this pull request to the merge queue Sep 25, 2024
Merged via the queue into develop with commit 106993f Sep 25, 2024
63 checks passed
@geoknee geoknee deleted the gk/batcher-da-simplified branch September 25, 2024 13:40
samlaf pushed a commit to samlaf/optimism that referenced this pull request Nov 10, 2024
…mission time (ethereum-optimism#12002)

* tidy up godoc

* move data availability config decision to channel submission time

instead of channel creation time

Also, cache the ChannelConfig whenever we switch DA type so it is used by default for new channels

* fix test

* formatting changes

* respond to PR comments

* add unit test for Requeue method

* reduce number of txs in test block

* improve test (more blocks in queue)

* hoist pending tx management up

* wip

* tidy up test

* wip

* fix

* refactor to do requeue before calling nextTxData

* introduce ErrInsufficientData

do not return nextTxData from channel which was discarded by requeue

* run test until nonzero data is returned by TxData

* break up and improve error logic

* fix test to anticipate ErrInsufficientData

* after requeuing, call nextTxData again

* remove unecessary checks

* move err declaration to top of file

* add some comments and whitespace

* hoist lock back up to TxData

* rename variable to blocksToRequeue

* remove panic

* add comment

* use deterministic rng and nonecompressor in test

* test: increase block size to fill channel more quickly

* remove ErrInsufficientData

replace with io.EOF as before

* tidy up

* typo
@geoknee geoknee added the A-op-batcher Area: op-batcher label Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-op-batcher Area: op-batcher
Projects
None yet
Development

Successfully merging this pull request may close these issues.

op-batcher: Assess da-type choice when submitting a channel
3 participants