-
Notifications
You must be signed in to change notification settings - Fork 123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add support for BatchWriteAtLeastOnce #2520
Conversation
google-cloud-spanner/src/main/java/com/google/cloud/spanner/DatabaseClient.java
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/DatabaseClient.java
Outdated
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/DatabaseClient.java
Outdated
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/DatabaseClient.java
Outdated
Show resolved
Hide resolved
* | ||
* @return ServerStream\<BatchWriteResponse> | ||
*/ | ||
ServerStream<BatchWriteResponse> batchWriteAtLeastOnce(Iterable<MutationGroup> mutationGroups) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When should I create multiple MutationGroups instead of just one large mutation group? What's the tradeoff between a large number of small MutationGroups vs a few large MutationGroups? Is there any limits on the number of mutations that I may have in a MutationGroup?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When should I create multiple MutationGroups instead of just one large mutation group?
Dependent mutations (to be committed together) should be part of a single MutationGroup.
For example, the following should be part of same MutationGroup:
- Inserting rows into both parent and child tables. If they're part of different MutationGroups, it will result in a failure when a commit on the child table’s row is attempted first.
- Inserting rows into tables T1, T2; T2 has a foreign key pointing to T1. If they're part of different MutationGroups, it will result in a failure when a commit on T2 is attempted first.
If there's a need to create just one large MutationGroup, it's better to use the existing Commit RPC for it.
What's the tradeoff between a large number of small MutationGroups vs a few large MutationGroups?
Depends on the use case. The bottomline is we should only put mutations meant to be committed together in a group, otherwise, should prefer a different group.
Is there any limits on the number of mutations that I may have in a MutationGroup?
Considering that in the base case, each mutation group could be committed into its own transaction, which has a limit of 40k mutations, the same limit would apply to the MutationGroup also.
google-cloud-spanner/src/main/java/com/google/cloud/spanner/MutationGroup.java
Outdated
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/MutationGroup.java
Outdated
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/MutationGroup.java
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/SessionImpl.java
Outdated
Show resolved
Hide resolved
google-cloud-spanner/src/test/java/com/google/cloud/spanner/it/ITWriteTest.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, with a couple of nits on the documentation. This is a non-trivial feature, so making sure that we make the documentation as clear as possible will help our users make the most out of this.
google-cloud-spanner/src/main/java/com/google/cloud/spanner/DatabaseClient.java
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/DatabaseClient.java
Outdated
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/DatabaseClientImpl.java
Show resolved
Hide resolved
google-cloud-spanner/src/main/java/com/google/cloud/spanner/MutationGroup.java
Show resolved
Hide resolved
google-cloud-spanner/src/test/java/com/google/cloud/spanner/it/ITWriteTest.java
Show resolved
Hide resolved
* feat: add support for BatchWriteAtleastOnce * test: add batchwrite() support to MockSpannerServiceImpl * test: add commit timestamp to proto * test: add commit timestamp to proto * test: add commit timestamp to proto * consume the stream in tests * refactor tests * refactor tests * test if mutations are correctly applied * null check * skip for emulator * add method documentation * add method documentation * add method documentation * remove autogenerated code * remove autogenerated tests * batchWriteAtleastOnce -> batchWriteAtLeastOnce * batchWriteAtleastOnceWithOptions -> batchWriteAtLeastOnceWithOptions * changes based on updated batch write API * add copyright and doc * address review comments * address review comments * add more documentation --------- Co-authored-by: Arpan Mishra <[email protected]>
No description provided.