[Flight] Add Serialization of Typed Arrays / ArrayBuffer / DataView #26954
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
CLA Signed
React Core Team
sebmarkbage
force-pushed
the
flightbinary
branch
2 times, most recently
from
dfdc0e0 to
729759d
Compare
This lets us have different environments operate on different types, such as passing the raw typed array when possible.
sebmarkbage
force-pushed
the
flightbinary
branch
from
729759d to
287cf4a
Compare
gnoff
approved these changes
Jun 29, 2023
timneutkens
pushed a commit
to vercel/next.js
that referenced
this pull request
Jul 4, 2023
This PR updates the vendored react dependencies using `pnpm sync-react` ### React upstream changes - facebook/react#27028 - facebook/react#27027 - facebook/react#27019 - facebook/react#26954 - facebook/react#26987 - facebook/react#26985 - facebook/react#26933 - facebook/react#26625 - facebook/react#27011 - facebook/react#27008 - facebook/react#26997 - facebook/react#26989 - facebook/react#26955 - facebook/react#26963 - facebook/react#26983 - facebook/react#26914 - facebook/react#26951 - facebook/react#26977 - facebook/react#26958 - facebook/react#26940 - facebook/react#26939 - facebook/react#26887 - facebook/react#26947 - facebook/react#26945 - facebook/react#26942 - facebook/react#26938 - facebook/react#26844 - facebook/react#25510 - facebook/react#26932 - facebook/react#26896 - facebook/react#26913 - facebook/react#26888 - facebook/react#26827 - facebook/react#26889 - facebook/react#26877 - facebook/react#26873 - facebook/react#26880 - facebook/react#26842 - facebook/react#26858 - facebook/react#26754 - facebook/react#26753 - facebook/react#26881 ### Related Closes #49409 (by facebook/react#26977) fix NEXT-1189 Co-authored-by: Shu Ding <[email protected]>
timneutkens
pushed a commit
to vercel/next.js
that referenced
this pull request
Jul 6, 2023
Fixes #49409 ### React upstream changes - facebook/react#27045 - facebook/react#27051 - facebook/react#27032 - facebook/react#27031 - facebook/react#27029 - facebook/react#27028 - facebook/react#27027 - facebook/react#27019 - facebook/react#26954 - facebook/react#26987 - facebook/react#26985 - facebook/react#26933 - facebook/react#26625 - facebook/react#27011 - facebook/react#27008 - facebook/react#26997 - facebook/react#26989 - facebook/react#26955 - facebook/react#26963 - facebook/react#26983 - facebook/react#26914 - facebook/react#26951 - facebook/react#26977 - facebook/react#26958 - facebook/react#26940 - facebook/react#26939 - facebook/react#26887 - facebook/react#26947 - facebook/react#26945 - facebook/react#26942 - facebook/react#26938 - facebook/react#26844 - facebook/react#25510 - facebook/react#26932 - facebook/react#26896 - facebook/react#26913 - facebook/react#26888 - facebook/react#26827 - facebook/react#26889 - facebook/react#26877 - facebook/react#26873 - facebook/react#26880 - facebook/react#26842 - facebook/react#26858 - facebook/react#26754 - facebook/react#26753 - facebook/react#26881 --------- Co-authored-by: Jiachi Liu <[email protected]>
shuding
added a commit
to vercel/next.js
that referenced
this pull request
Jul 8, 2023
This PR updates the vendored react dependencies using `pnpm sync-react` ### React upstream changes - facebook/react#27028 - facebook/react#27027 - facebook/react#27019 - facebook/react#26954 - facebook/react#26987 - facebook/react#26985 - facebook/react#26933 - facebook/react#26625 - facebook/react#27011 - facebook/react#27008 - facebook/react#26997 - facebook/react#26989 - facebook/react#26955 - facebook/react#26963 - facebook/react#26983 - facebook/react#26914 - facebook/react#26951 - facebook/react#26977 - facebook/react#26958 - facebook/react#26940 - facebook/react#26939 - facebook/react#26887 - facebook/react#26947 - facebook/react#26945 - facebook/react#26942 - facebook/react#26938 - facebook/react#26844 - facebook/react#25510 - facebook/react#26932 - facebook/react#26896 - facebook/react#26913 - facebook/react#26888 - facebook/react#26827 - facebook/react#26889 - facebook/react#26877 - facebook/react#26873 - facebook/react#26880 - facebook/react#26842 - facebook/react#26858 - facebook/react#26754 - facebook/react#26753 - facebook/react#26881 ### Related Closes #49409 (by facebook/react#26977) fix NEXT-1189 Co-authored-by: Shu Ding <[email protected]>
EdisonVan
pushed a commit
to EdisonVan/react
that referenced
this pull request
Apr 15, 2024
…acebook#26954) This uses the same mechanism as [large strings](facebook#26932) to encode chunks of length based binary data in the RSC payload behind a flag. I introduce a new BinaryChunk type that's specific to each stream and ways to convert into it. That's because we sometimes need all chunks to be Uint8Array for the output, even if the source is another array buffer view, and sometimes we need to clone it before transferring. Each type of typed array is its own row tag. This lets us ensure that the instance is directly in the right format in the cached entry instead of creating a wrapper at each reference. Ideally this is also how Map/Set should work but those are lazy which complicates that approach a bit. We assume both server and client use little-endian for now. If we want to support other modes, we'd convert it to/from little-endian so that the transfer protocol is always little-endian. That way the common clients can be the fastest possible. So far this only implements Server to Client. Still need to implement Client to Server for parity. NOTE: This is the first time we make RSC effectively a binary format. This is not compatible with existing SSR techniques which serialize the stream as unicode in the HTML. To be compatible, those implementations would have to use base64 or something like that. Which is what we'll do when we move this technique to be built-in to Fizz.
bigfootjon
pushed a commit
that referenced
this pull request
Apr 18, 2024
…26954) This uses the same mechanism as [large strings](#26932) to encode chunks of length based binary data in the RSC payload behind a flag. I introduce a new BinaryChunk type that's specific to each stream and ways to convert into it. That's because we sometimes need all chunks to be Uint8Array for the output, even if the source is another array buffer view, and sometimes we need to clone it before transferring. Each type of typed array is its own row tag. This lets us ensure that the instance is directly in the right format in the cached entry instead of creating a wrapper at each reference. Ideally this is also how Map/Set should work but those are lazy which complicates that approach a bit. We assume both server and client use little-endian for now. If we want to support other modes, we'd convert it to/from little-endian so that the transfer protocol is always little-endian. That way the common clients can be the fastest possible. So far this only implements Server to Client. Still need to implement Client to Server for parity. NOTE: This is the first time we make RSC effectively a binary format. This is not compatible with existing SSR techniques which serialize the stream as unicode in the HTML. To be compatible, those implementations would have to use base64 or something like that. Which is what we'll do when we move this technique to be built-in to Fizz. DiffTrain build for commit d9c3331.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This uses the same mechanism as large strings to encode chunks of length based binary data in the RSC payload behind a flag.
I introduce a new BinaryChunk type that's specific to each stream and ways to convert into it. That's because we sometimes need all chunks to be Uint8Array for the output, even if the source is another array buffer view, and sometimes we need to clone it before transferring.
Each type of typed array is its own row tag. This lets us ensure that the instance is directly in the right format in the cached entry instead of creating a wrapper at each reference. Ideally this is also how Map/Set should work but those are lazy which complicates that approach a bit.
We assume both server and client use little-endian for now. If we want to support other modes, we'd convert it to/from little-endian so that the transfer protocol is always little-endian. That way the common clients can be the fastest possible.
So far this only implements Server to Client. Still need to implement Client to Server for parity.
NOTE: This is the first time we make RSC effectively a binary format. This is not compatible with existing SSR techniques which serialize the stream as unicode in the HTML. To be compatible, those implementations would have to use base64 or something like that. Which is what we'll do when we move this technique to be built-in to Fizz.