Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(spanner): implement generation and propagation of "x-goog-spanner-request-id" Header #11048

Merged
merged 14 commits into from
Dec 18, 2024

Conversation

odeke-em
Copy link
Contributor

@odeke-em odeke-em commented Oct 29, 2024

In tandem this specification

https://orijtech.notion.site/x-goog-spanner-request-id-always-on-gRPC-header-to-aid-in-quick-debugging-of-errors-14aba6bc91348091a58fca7a505c9827

This change adds sending over the "x-goog-spanner-request-id" header
for every unary and streaming call, in the form:

<version>.<processId>.<clientId>.<channelId>.<requestCountForClient>.<rpcCountForRequest>

where:

  • version is the version of the specification
  • processId is a randomly generated uint64 singleton for the lifetime of a process
  • clientId is the monotonically increasing id/number of gRPC Spanner clients created
  • requestCountForClient is the monotonically increasing number of requests made by the client
  • channelId currently at 1 is the Id of the client for Go
  • rpcCountForRequest is the number of RPCs/retries within a specific request

This header is to be sent on both unary and streaming calls and it'll
help debug latencies for customers. On an error, customers can assert against
.Error and retrieve the associated .RequestID and log it, or even better
it'll be printed out whenever errors are logged.

Importantly making randIdForProcess to be a uint6 which is 64bits and not
a UUID4 which is 128bits which surely massively reduces the possibility of collisions
to ensure that high QPS applications can function and accept bursts of traffic
without worry, as the prior design used uint32 aka 32 bits for
which just 50,000 new processes being created could get the probability
of collisions to 25%, with this new change a company would have to
create 82 million QPS every second for 1,000 years for a 1% collision
with 2.6e18 for which the collision would be 1%.
Using 64-bits still provides really good protection whereby for a 1% chance of collision,
we would need 810 million objects, so we have good protection.
However, Google Cloud Spanner's backend has to store every one of the always on
headers for a desired retention period hence 64-bits is a great balance between collision
protection vs storage.

Fixes #11073

@product-auto-label product-auto-label bot added the api: spanner Issues related to the Spanner API. label Oct 29, 2024
@odeke-em odeke-em force-pushed the spanner-request-id-header branch 5 times, most recently from 1938e60 to 463cd50 Compare November 1, 2024 13:39
@odeke-em odeke-em marked this pull request as ready for review November 1, 2024 13:53
@odeke-em odeke-em requested review from a team as code owners November 1, 2024 13:53
@odeke-em odeke-em changed the title spanner: prototype and lay down foundation for x-spanner-request-id header spanner: implement generation and propagation of "x-spanner-request-id" Header Nov 1, 2024
@odeke-em odeke-em force-pushed the spanner-request-id-header branch from 7785a60 to 921c239 Compare November 1, 2024 14:08
@odeke-em
Copy link
Contributor Author

odeke-em commented Nov 3, 2024

Kindly cc-ing @olavloite @tharoldD @willpoint

@odeke-em odeke-em force-pushed the spanner-request-id-header branch from 3d50ba7 to 3c79e77 Compare November 5, 2024 01:55
@odeke-em odeke-em changed the title spanner: implement generation and propagation of "x-spanner-request-id" Header feat(spanner): implement generation and propagation of "x-spanner-request-id" Header Nov 5, 2024
@odeke-em odeke-em force-pushed the spanner-request-id-header branch 2 times, most recently from 066caaa to ef110da Compare November 5, 2024 01:59
@odeke-em
Copy link
Contributor Author

odeke-em commented Nov 5, 2024

Kindly cc-ing you @harshachinta.

@odeke-em odeke-em changed the title feat(spanner): implement generation and propagation of "x-spanner-request-id" Header feat(spanner): implement generation and propagation of "x-goog-spanner-request-id" Header Nov 5, 2024
@odeke-em odeke-em force-pushed the spanner-request-id-header branch from e9edb17 to 18cd8d4 Compare November 5, 2024 04:22
@rahul2393
Copy link
Contributor

@odeke-em can you please fix the vet and tests issues here?

@rahul2393 rahul2393 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 8, 2024
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 8, 2024
spanner/batch.go Outdated Show resolved Hide resolved
spanner/client.go Outdated Show resolved Hide resolved
spanner/grpc_client.go Outdated Show resolved Hide resolved
spanner/grpc_client.go Outdated Show resolved Hide resolved
spanner/sessionclient.go Outdated Show resolved Hide resolved
spanner/transaction.go Outdated Show resolved Hide resolved
@odeke-em odeke-em force-pushed the spanner-request-id-header branch 6 times, most recently from 9a9f77c to a35c97c Compare November 20, 2024 03:36
spanner/errors.go Outdated Show resolved Hide resolved
spanner/grpc_client.go Outdated Show resolved Hide resolved
spanner/grpc_client.go Outdated Show resolved Hide resolved
spanner/grpc_client.go Outdated Show resolved Hide resolved
spanner/read.go Outdated Show resolved Hide resolved
spanner/request_id_header.go Outdated Show resolved Hide resolved
spanner/request_id_header.go Outdated Show resolved Hide resolved
spanner/request_id_header.go Outdated Show resolved Hide resolved
spanner/request_id_header.go Show resolved Hide resolved
spanner/sessionclient.go Show resolved Hide resolved
@olavloite olavloite added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 17, 2024
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 17, 2024
@odeke-em odeke-em force-pushed the spanner-request-id-header branch from cf9dfea to 367ecbd Compare December 17, 2024 15:19
@olavloite olavloite enabled auto-merge (squash) December 17, 2024 15:29
@olavloite olavloite added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 18, 2024
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 18, 2024
…est-id" Header

In tandem with the specification:

    https://orijtech.notion.site/x-goog-spanner-request-id-always-on-gRPC-header-to-aid-in-quick-debugging-of-errors-14aba6bc91348091a58fca7a505c9827

this change adds sending over the "x-goog-spanner-request-id" header
for every unary and streaming call, in the form:

	<version>.<processId>.<clientId>.<channelId>.<requestCountForClient>.<rpcCountForRequest>

where:
* version is the version of the specification
* processId is a randomly generated uint64 singleton for the lifetime of a process
* clientId is the monotonically increasing id/number of gRPC Spanner clients created
* requestCountForClient is the monotonically increasing number of requests made by the client
* channelId currently at 1 is the Id of the client for Go
* rpcCountForRequest is the number of RPCs/retries within a specific request

This header is to be sent on both unary and streaming calls and it'll
help debug latencies for customers. On an error, customers can assert against
.Error and retrieve the associated .RequestID and log it, or even better
it'll be printed out whenever errors are logged.

Importantly making randIdForProcess to be a uint6 which is 64bits and not
a UUID4 which is 128bits which surely massively reduces the possibility of collisions
to ensure that high QPS applications can function and accept bursts of traffic
without worry, as the prior design used uint32 aka 32 bits for
which just 50,000 new processes being created could get the probability
of collisions to 25%, with this new change a company would have to
create 82 million QPS every second for 1,000 years for a 1% collision
with 2.6e18 for which the collision would be 1%.
Using 64-bits still provides really good protection whereby for a 1% chance of collision,
we would need 810 million objects, so we have good protection.
However, Google Cloud Spanner's backend has to store every one of the always on
headers for a desired retention period hence 64-bits is a great balance between collision
protection vs storage.

Fixes googleapis#11073
…nnelID is derived from sessionClient.connPool
We have to re-insert the request-id even after gax.Invoke->grpc
internals clear it. Added test to validate retries.
This change accounts for logic graciously raised by Knut
along with his test contribution.
auto-merge was automatically disabled December 18, 2024 07:07

Head branch was pushed to by a user without write access

@odeke-em odeke-em force-pushed the spanner-request-id-header branch from 367ecbd to 4cf4e11 Compare December 18, 2024 07:07
@olavloite olavloite added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 18, 2024
@olavloite olavloite enabled auto-merge (squash) December 18, 2024 07:10
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 18, 2024
@olavloite olavloite merged commit 10960c1 into googleapis:main Dec 18, 2024
8 checks passed
@odeke-em odeke-em deleted the spanner-request-id-header branch December 18, 2024 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the Spanner API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

spanner: propagate x-goog-spanner-request-id header on every call and increment it appropriately per retry
4 participants