feat: add support for Proto Columns #2779

harshachinta · 2024-01-10T13:31:19Z

[DO NOT MERGE]

This PR has the changes from

* feat: Importing Proto Changes Commit will be reverted, once PROTO changes are available publicly. * feat: Proto Message Implementation * feat: Adding support for enum * feat: Code refactoring Adding default implementation for newly added methods ByteArray compatability changes for Proto Messages * docs: Adding Java docs for all the newly added methods. * test: Sample Proto & Generated classes for unit test * feat: Adding bytes/proto & int64/enum compatability Adding Additional check for ChecksumResultSet * test: Adding unit tests * test: Adding unit tests for ValueBinder.java * feat: refactoring to add support for getValue & other minor changes * feat: Minor refactoring 1. Adding docs and formatting the code. 2. Adding additional methods for enum and message which accepts descriptors. * feat: Adding bytes/message & int64/enum compatability in Value * refactor: Minor refactoring * feat: Adding Proto Array Implementation * test: Implementing unit tests for array of protos and enums * refactor: adding clirr ignores * feat: Adding support for enum as Primary Key * feat: Code Review Changes, minor refactoring and adding docs * feat: Addressing review comments -Modified Docs/Comments -Minor Refactoring * refactor: Using Column instead of column to avoid test failures * feat: Minor refactoring -code review comments -adding function docs

#2211) * samples: Adding samples for updating & querying Proto messages & enums * style: linting * style: linting * docs: Adding function and class doc

* test: Adding Integration tests for Proto Messages & Enums * test: Adding additional test for Parameterized Queries, Primary Keys & Invalid Wire type errors. * style: Formatting * style: Formatting * test: Updating instance and db name * test: Adding inter compatability check while writing data

…ement-alpha

Co-authored-by: Pavol Juhos <[email protected]>

* feat: add code changes and tests for Proto columns DDL support * feat: add auto generated code * feat: code changes and tests for Proto columns DDL support * feat: add descriptors file * feat: code refactoring * feat: Integration tests and code refactoring * feat: code refactoring * feat: unit tests and clirr differences * feat: lint changes * feat: code refactor * feat: code refactoring * feat: code refactoring * feat: code refactoring * feat: add java docs to new methods * feat: lint formatting * feat: lint formatting changes * feat: lint formatting * feat: lint formatting * feat: test exception cases * feat: code refactoring * feat: add java docs and refactoring * feat: add java docs * feat: java docs refactor * feat: remove overload method setProtoDescriptors that accepts file path as input to avoid unexpected issues * feat: remove updateDdl method overload to update proto descriptor

…to validate main branch update

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

generated-files-bot · 2024-01-10T13:31:28Z

Warning: This pull request is touching the following templated files:

google-cloud-spanner/src/main/java/com/google/cloud/spanner/admin/database/v1/DatabaseAdminClient.java
google-cloud-spanner/src/test/java/com/google/cloud/spanner/admin/database/v1/DatabaseAdminClientTest.java

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

…oogleapis/java-spanner into proto-column-enhancement-alpha

olavloite

Looks generally good to me, but with some requests for optimizations. Feel free to defer those to a later PR if this is urgent to get merged.

google-cloud-spanner/src/main/java/com/google/cloud/spanner/AbstractResultSet.java

google-cloud-spanner/src/main/java/com/google/cloud/spanner/Type.java

olavloite · 2024-01-25T09:28:53Z

google-cloud-spanner/src/main/java/com/google/cloud/spanner/Value.java

+    v.forEach(
+        (message) -> {
+          if (message != null) {
+            serializedArray.add(ByteArray.copyFrom(message.toByteArray()));


nit: It would have been great if we could have done this conversion without copying the byte array twice (message.toByteArray() creates a new byte[], and the copyFrom method then makes a copy of that again).

I had a quick look, and could not find anything, so feel free to ignore.

olavloite · 2024-01-25T09:29:23Z

google-cloud-spanner/src/main/java/com/google/cloud/spanner/Value.java

+          "Proto message may not be null. Use MyProtoClass.getDefaultInstance() as a parameter value.");
+      checkNotNull();
+      try {
+        return (T) m.toBuilder().mergeFrom(value.getByteArray().toByteArray()).build();


Here also: Any possibility that we can do this conversion without copying to a byte array multiple times?

@olavloite I have applied the optimization suggested above to this case too. Please let me know if this is correct.

value.getByteArray().toByteArray()

is changed to

Base64.getDecoder() .wrap( CharSource.wrap(value.getBase64String()) .asByteSource(StandardCharsets.UTF_8) .openStream())

olavloite · 2024-01-25T09:30:06Z

google-cloud-spanner/src/main/java/com/google/cloud/spanner/Value.java

+          "Proto message may not be null. Use MyProtoClass.getDefaultInstance() as a parameter value.");
+      checkNotNull();
+      try {
+        return (T) m.toBuilder().mergeFrom(value.toByteArray()).build();


Here also: Can we do a more direct conversion?

google-cloud-spanner/src/main/java/com/google/cloud/spanner/Value.java

...loud-spanner/src/test/java/com/google/cloud/spanner/connection/RandomResultSetGenerator.java

jitpack.yml

gauravpurohit06 and others added 11 commits December 28, 2022 16:20

samples: Adding samples for updating & querying Proto messages & enums (

1fb6e94

#2211) * samples: Adding samples for updating & querying Proto messages & enums * style: linting * style: linting * docs: Adding function and class doc

Merge remote-tracking branch 'upstream/main' into proto-column-enhanc…

dd358c2

…ement-alpha

Configured jitpack.yml to use OpenJDK 11 (#2218)

53397e6

Co-authored-by: Pavol Juhos <[email protected]>

Merge branch 'main' into proto-column-enhancement-alpha

a70f259

teat: update pom file to run tests on cloud-devel region temporarily …

9513753

…to validate main branch update

🦉 Updates from OwlBot post-processor

6701f52

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

fix: revert host changes in pom.xml file

f63da18

Merge branch 'main' into proto-column-enhancement-alpha

5fbef25

harshachinta requested review from a team as code owners January 10, 2024 13:31

product-auto-label bot added size: xl Pull request size is extra large. api: spanner Issues related to the googleapis/java-spanner API. labels Jan 10, 2024

🦉 Updates from OwlBot post-processor

7f5bd98

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

harshachinta added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jan 10, 2024

harshachinta and others added 8 commits January 24, 2024 13:03

feat(spanner): revert autogenerated code

058b75c

🦉 Updates from OwlBot post-processor

e01d0c7

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Merge branch 'main' into proto-column-enhancement-alpha

fb482c4

feat(spanner): remove samples

45ad85e

🦉 Updates from OwlBot post-processor

ab35680

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

feat(spanner): remove clirr

d7fbc09

feat(spanner): skip emulator test

2de342f

feat(spanner): clirr

d18c28e

harshachinta requested a review from olavloite January 24, 2024 16:45

harshachinta and others added 3 commits January 24, 2024 17:36

feat(spanner): fix javadoc

3e46cd9

🦉 Updates from OwlBot post-processor

ee333a3

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

feat(spanner): fix javadoc

e105101

harshachinta and others added 5 commits January 24, 2024 17:53

feat(spanner): fix javadoc

7dfe81e

feat(spanner): fix javadoc

5491ca3

🦉 Updates from OwlBot post-processor

5e972fe

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Merge branch 'proto-column-enhancement-alpha' of https://github.com/g…

74c1e86

…oogleapis/java-spanner into proto-column-enhancement-alpha

feat(spanner): fix emulator skip

be99f94

olavloite approved these changes Jan 25, 2024

View reviewed changes

olavloite and others added 4 commits January 25, 2024 13:32

build: ignore all changes in v1 package

b8915b1

feat(spanner): add optimizations to deserialize proto messages

2f75cb0

feat(spanner): remove TODO

63c720d

feat(spanner): remove TODO

42fafc0

harshachinta removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jan 26, 2024

harshachinta merged commit 30d37dd into main Jan 26, 2024
27 checks passed

harshachinta deleted the proto-column-enhancement-alpha branch January 26, 2024 13:33

release-please bot mentioned this pull request Jan 26, 2024

chore(main): release 6.57.0 #2782

Merged

lqiu96 mentioned this pull request Feb 2, 2024

Spring-Spanner-Data: Support for the latest Spanner v6.57.0 features GoogleCloudPlatform/spring-cloud-gcp#2574

Open

harshachinta restored the proto-column-enhancement-alpha branch February 5, 2024 09:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for Proto Columns #2779

feat: add support for Proto Columns #2779

harshachinta commented Jan 10, 2024 •

edited

Loading

generated-files-bot bot commented Jan 10, 2024 •

edited

Loading

olavloite left a comment

olavloite Jan 25, 2024

olavloite Jan 25, 2024

harshachinta Jan 26, 2024

olavloite Jan 25, 2024

feat: add support for Proto Columns #2779

feat: add support for Proto Columns #2779

Conversation

harshachinta commented Jan 10, 2024 • edited Loading

generated-files-bot bot commented Jan 10, 2024 • edited Loading

olavloite left a comment

Choose a reason for hiding this comment

olavloite Jan 25, 2024

Choose a reason for hiding this comment

olavloite Jan 25, 2024

Choose a reason for hiding this comment

harshachinta Jan 26, 2024

Choose a reason for hiding this comment

olavloite Jan 25, 2024

Choose a reason for hiding this comment

harshachinta commented Jan 10, 2024 •

edited

Loading

generated-files-bot bot commented Jan 10, 2024 •

edited

Loading