codec/types: avoid unnecessary allocations for NewAnyWithCustomTypeURL on error #8605
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…L on error Avoids a bleed out attack in which a node can be made to allocate memory slowly or very fast in small strides, by sending bad data to code that invokes NewAnyWithCustomTypeURL, in which we unconditionally returned a new Any object. On a 64-bit machine, this would waste 96 bytes per invocation even on error. Added a test to ensure zero allocations with a fixed error returned. Also added a benchmark which shows reduction in wasted allocations and wasted CPU time: ```shell $ benchstat before.txt after.txt name old time/op new time/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 142ns ± 6% 55ns ±12% -61.65% (p=0.000 n=9+10) name old alloc/op new alloc/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 96.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) ``` Fixes #8537
odeke-em
requested review from
alessio,
robert-zaremba,
clevinson and
tac0turtle
|
I kindly request that we backport this change too. |
|
@Mergifyio backport release/v0.41.x |
|
Command
|
on merge the backport should be handled unless conflicts arise. |
|
Gotcha, thanks Marko!
…On Wed, Feb 17, 2021 at 1:54 AM Marko ***@***.***> wrote:
I kindly request that we backport this change too.
on merge the backport should be handled unless conflicts arise.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#8605 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABFL3VZCNTKOUQ65YNXFTY3S7OG4XANCNFSM4XYA6RLQ>
.
|
Codecov Report
@@ Coverage Diff @@
## master #8605 +/- ##
=======================================
Coverage 61.47% 61.47%
=======================================
Files 658 658
Lines 37583 37585 +2
=======================================
+ Hits 23104 23106 +2
Misses 12067 12067
Partials 2412 2412
|
alessio
approved these changes
Feb 17, 2021
alessio
deleted the
codec-types-avoid-wasting-Any-allocations-on-error
branch
|
Command
|
mergify bot
pushed a commit
that referenced
this pull request
Feb 17, 2021
…L on error (#8605) Avoids a bleed out attack in which a node can be made to allocate memory slowly or very fast in small strides, by sending bad data to code that invokes NewAnyWithCustomTypeURL, in which we unconditionally returned a new Any object. On a 64-bit machine, this would waste 96 bytes per invocation even on error. Added a test to ensure zero allocations with a fixed error returned. Also added a benchmark which shows reduction in wasted allocations and wasted CPU time: ```shell $ benchstat before.txt after.txt name old time/op new time/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 142ns ± 6% 55ns ±12% -61.65% (p=0.000 n=9+10) name old alloc/op new alloc/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 96.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) ``` Fixes #8537 (cherry picked from commit 56fc3fc)
alessio
pushed a commit
that referenced
this pull request
Feb 17, 2021
…L on error (#8605) (#8606) Avoids a bleed out attack in which a node can be made to allocate memory slowly or very fast in small strides, by sending bad data to code that invokes NewAnyWithCustomTypeURL, in which we unconditionally returned a new Any object. On a 64-bit machine, this would waste 96 bytes per invocation even on error. Added a test to ensure zero allocations with a fixed error returned. Also added a benchmark which shows reduction in wasted allocations and wasted CPU time: ```shell $ benchstat before.txt after.txt name old time/op new time/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 142ns ± 6% 55ns ±12% -61.65% (p=0.000 n=9+10) name old alloc/op new alloc/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 96.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta NewAnyWithCustomTypeURLWithErrorReturned-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) ``` Fixes #8537 (cherry picked from commit 56fc3fc) Co-authored-by: Emmanuel T Odeke <[email protected]>
|
Good job. PS: I thought we backport only fixes and significant changes? This is a micro optimization of an unhappy path |
|
Great question Robert! I asked for a backport as it isn’t just an
optimization but supposedly if someone could trivially send unmarshallable
data in a fast loop they could cause memory to be allocated unnecessarily,
which is why I called it a slow bleed — better for security to seal off
such seemingly simple paths but that can cause unnecessary allocations.
…On Wed, Feb 17, 2021 at 5:32 AM Robert Zaremba ***@***.***> wrote:
Good job.
PS: I thought we backport only fixes and significant changes? This is a
micro optimization of an *unhappy* path
PS2: All optimizations are always good 👏
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#8605 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABFL3VZ2AJNWINUGZHSNO2TS7PAOVANCNFSM4XYA6RLQ>
.
|
|
Stable release updates are for improvements and bugfixes that don't break user code. And no, not all optimizations are always welcome. Only low risk optimizations are welcome, e.g. this one ;-) |
How that can happen? Is there a way a user could exploit it? I think any TX will go out of gas and other allocations would be way more significant before something here will happen. |
|
In relation to the motivations, it came up on my radar as a path that could
be taken when ensuring that code related to ADR 20 was working alright.
In regards to users exploiting it, am not too sure to craft up a direct
answer but the subtlety is that if there is any adversary, assume that
given the BFT properties of Tendermint, that they’ll be willing to play the
long game, and any path that can be exploited will be exploited. Also it is
a low risk and correctness change.
…On Wed, Feb 17, 2021 at 5:39 AM Robert Zaremba ***@***.***> wrote:
if someone could trivially send unmarshallable data in a fast loop
How that can happen? Is there a way a user could exploit it? I think any
TX will go out of gas and other allocations would be way more significant
before something here will happen.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#8605 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABFL3V3QGJ3TUQ3DIZ3H6W3S7PBILANCNFSM4XYA6RLQ>
.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Avoids a bleed out attack in which a node can be made to allocate
memory slowly or very fast in small strides, by sending bad data
to code that invokes NewAnyWithCustomTypeURL, in which we
unconditionally returned a new Any object. On a 64-bit machine,
this would waste 96 bytes per invocation even on error.
Added a test to ensure zero allocations with a fixed error returned.
Also added a benchmark which shows reduction in wasted allocations and
wasted CPU time:
Fixes #8537
Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.
docs/) or specification (x/<module>/spec/)godoccomments.Unreleasedsection inCHANGELOG.mdFiles changedin the Github PR explorerCodecov Reportin the comment section below once CI passes