feat: add backend query options #90

olavloite · 2020-03-03T17:39:40Z

Adds the ability to set QueryOptions when running Cloud Spanner queries. For now, only setting the query_optimizer_version is added.

QueryOptions can be configured through the following mechanisms:

Through the SPANNER_OPTIMIZER_VERSION environment variable.
At Spanner level using SpannerOptions.newBuilder().setDefaultQueryOptions(DatabaseId, QueryOptions).
At query level using Statement.newBuilder(String).withQueryOptions(QueryOptions).

If the options are configured through multiple mechanisms then:

Options set at an environment variable level will override options configured at the SpannerOptions level.
Options set at a query-level will override options set at either the SpannerOptions or environment variable level.

If no options are set, the optimizer version will default to:

The optimizer version the database is pinned to.
If the database is not pinned to a specific version, then the Cloud Spanner backend will use the "latest" version.

skuruppu

LGTM

A couple of requests:

I would like to verify that both ExecuteSql and ExecuteStreamingSql requests support query options. I think it does from what I understand of the implementation but thought I'd check.
If you could update the description of this PR to contain the information about the env variable and the precedence, that would be great. Example PR

skuruppu · 2020-03-09T00:36:54Z

google-cloud-spanner/src/main/java/com/google/cloud/spanner/AbstractReadContext.java

+    MultiUseReadOnlyTransaction(Builder builder) {
+      super(builder);
      checkArgument(
-          bound.getMode() != TimestampBound.Mode.MAX_STALENESS
-              && bound.getMode() != TimestampBound.Mode.MIN_READ_TIMESTAMP,
-          "Bounded staleness mode %s is not supported for multi-use read-only transactions."
-              + " Create a single-use read or read-only transaction instead.",
-          bound.getMode());
-      this.bound = bound;
-    }
-
-    MultiUseReadOnlyTransaction(
-        SessionImpl session,
-        ByteString transactionId,
-        Timestamp timestamp,
-        SpannerRpc rpc,
-        int defaultPrefetchChunks) {
-      super(session, rpc, defaultPrefetchChunks);
-      this.transactionId = transactionId;
-      this.timestamp = timestamp;
+          !(builder.bound != null && builder.transactionId != null)
+              && !(builder.bound == null && builder.transactionId == null),
+          "Either TimestampBound or TransactionId must be specified");
+      if (builder.bound != null) {
+        checkArgument(
+            builder.bound.getMode() != TimestampBound.Mode.MAX_STALENESS
+                && builder.bound.getMode() != TimestampBound.Mode.MIN_READ_TIMESTAMP,
+            "Bounded staleness mode %s is not supported for multi-use read-only transactions."
+                + " Create a single-use read or read-only transaction instead.",
+            builder.bound.getMode());
+        this.bound = builder.bound;
+      } else {
+        this.timestamp = builder.timestamp;
+        this.transactionId = builder.transactionId;
+      }


Since most of this refactoring and extra checks for arguments is not related to the query options work, would it be ok to move this to a separate PR?

I would prefer not to. The reason for introducing this builder pattern in this PR is that this PR requires an additional object to be passed in to the different transaction classes. This would add another parameter to the constructors of these classes, bringing the total number to 6 in this specific case. As a general rule of thumb, a method (or constructor) in Java should not take more than 4 arguments: https://rules.sonarsource.com/java/RSPEC-107

google-cloud-spanner/src/test/java/com/google/cloud/spanner/SpannerImplTest.java

google-cloud-spanner/src/test/java/com/google/cloud/spanner/SpannerOptionsTest.java

google-cloud-spanner/src/test/java/com/google/cloud/spanner/DatabaseClientImplTest.java

hengfengli · 2020-03-09T02:44:34Z

One thing I notice is that PartitionedDML also uses ExecuteSqlRequest, which needs to support QueryOptions as well.

java-spanner/google-cloud-spanner/src/main/java/com/google/cloud/spanner/PartitionedDMLTransaction.java

Lines 73 to 78 in d8b6438

    
           final ExecuteSqlRequest.Builder builder = 
        
               ExecuteSqlRequest.newBuilder() 
        
                   .setSql(statement.getSql()) 
        
                   .setQueryMode(QueryMode.NORMAL) 
        
                   .setSession(session.getName()) 
        
                   .setTransaction(TransactionSelector.newBuilder().setId(transactionId).build());

olavloite · 2020-03-10T09:33:20Z

LGTM

A couple of requests:

I would like to verify that both ExecuteSql and ExecuteStreamingSql requests support query options. I think it does from what I understand of the implementation but thought I'd check.

If you could update the description of this PR to contain the information about the env variable and the precedence, that would be great. Example PR

I've added update statements to ITQueryOptionsTest. All update statements are executed by the client library using the ExecuteSql RPC. (All queries use ExecuteStreamingSql). This verifies that QueryOptions work for both RPCs.
Done.

olavloite · 2020-03-10T10:01:29Z

One thing I notice is that PartitionedDML also uses ExecuteSqlRequest, which needs to support QueryOptions as well.

I've added a PartitionedDML statement to ITQueryOptionsTest to verify that QueryOptions are supported (or actually; currently ignored) for DML and PartitionedDML statements.

skuruppu · 2020-03-12T05:22:06Z

@olavloite you are welcome to merge this in as I've now merged in the generated code.

Adds support for setting QueryOptions that will be used by the backend to execute queries.

QueryOptions should be an option on a Statement instead of a parameter to the executeQuery method. By setting these options on a Statement, it is possible to use it with analyzeQuery as well.

This PR was generated using Autosynth. 🌈 <details><summary>Log from Synthtool</summary> ``` 2020-03-17 12:21:53,248 synthtool > Executing /tmpfs/src/git/autosynth/working_repo/synth.py. .github/ISSUE_TEMPLATE/bug_report.md .github/ISSUE_TEMPLATE/feature_request.md .github/ISSUE_TEMPLATE/support_request.md .github/PULL_REQUEST_TEMPLATE.md .github/release-please.yml .github/trusted-contribution.yml .kokoro/build.bat .kokoro/build.sh .kokoro/coerce_logs.sh .kokoro/common.cfg .kokoro/continuous/common.cfg .kokoro/continuous/dependencies.cfg .kokoro/continuous/integration.cfg .kokoro/continuous/java11.cfg .kokoro/continuous/java7.cfg .kokoro/continuous/java8-osx.cfg .kokoro/continuous/java8-win.cfg .kokoro/continuous/java8.cfg .kokoro/continuous/lint.cfg .kokoro/continuous/propose_release.cfg .kokoro/continuous/samples.cfg .kokoro/dependencies.sh .kokoro/linkage-monitor.sh .kokoro/nightly/common.cfg .kokoro/nightly/dependencies.cfg .kokoro/nightly/integration.cfg .kokoro/nightly/java11.cfg .kokoro/nightly/java7.cfg .kokoro/nightly/java8-osx.cfg .kokoro/nightly/java8-win.cfg .kokoro/nightly/java8.cfg .kokoro/nightly/lint.cfg .kokoro/nightly/samples.cfg .kokoro/presubmit/clirr.cfg .kokoro/presubmit/common.cfg .kokoro/presubmit/dependencies.cfg .kokoro/presubmit/integration.cfg .kokoro/presubmit/java11.cfg .kokoro/presubmit/java7.cfg .kokoro/presubmit/java8-osx.cfg .kokoro/presubmit/java8-win.cfg .kokoro/presubmit/java8.cfg .kokoro/presubmit/linkage-monitor.cfg .kokoro/presubmit/lint.cfg .kokoro/presubmit/samples.cfg .kokoro/release/bump_snapshot.cfg .kokoro/release/common.cfg .kokoro/release/common.sh .kokoro/release/drop.cfg .kokoro/release/drop.sh .kokoro/release/promote.cfg .kokoro/release/promote.sh .kokoro/release/publish_javadoc.cfg .kokoro/release/publish_javadoc.sh .kokoro/release/snapshot.cfg .kokoro/release/snapshot.sh .kokoro/release/stage.cfg .kokoro/release/stage.sh .kokoro/trampoline.sh CODE_OF_CONDUCT.md CONTRIBUTING.md LICENSE README.md codecov.yaml java.header license-checks.xml renovate.json samples/install-without-bom/pom.xml samples/pom.xml samples/snapshot/pom.xml samples/snippets/pom.xml 2020-03-17 12:21:53,455 synthtool > Wrote metadata to synth.metadata. ``` </details>

olavloite added the api: spanner Issues related to the googleapis/java-spanner API. label Mar 3, 2020

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Mar 3, 2020

olavloite requested a review from skuruppu March 3, 2020 17:40

skuruppu requested a review from hengfengli March 5, 2020 02:41

skuruppu approved these changes Mar 9, 2020

View reviewed changes

skuruppu force-pushed the autosynth branch 4 times, most recently from 9130062 to 59d2f8d Compare March 12, 2020 00:42

yoshi-automation force-pushed the autosynth branch from 59d2f8d to 2c7df30 Compare March 12, 2020 09:07

olavloite changed the base branch from autosynth to master March 12, 2020 12:38

olavloite added 4 commits March 12, 2020 13:39

feat: add backend query options

0b1a529

Adds support for setting QueryOptions that will be used by the backend to execute queries.

fix: set QueryOptions on Statement

160970b

QueryOptions should be an option on a Statement instead of a parameter to the executeQuery method. By setting these options on a Statement, it is possible to use it with analyzeQuery as well.

feat: add toBuilder() method to Statement

7e6b799

fix: code review comments

d27b3a7

olavloite force-pushed the query-options branch from 624f0e1 to d27b3a7 Compare March 12, 2020 12:53

fix: remove unused interface and class

096e3fb

olavloite merged commit e96e172 into master Mar 12, 2020

olavloite deleted the query-options branch March 12, 2020 15:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add backend query options #90

feat: add backend query options #90

olavloite commented Mar 3, 2020 •

edited

Loading

skuruppu left a comment

skuruppu Mar 9, 2020

olavloite Mar 10, 2020

hengfengli commented Mar 9, 2020

olavloite commented Mar 10, 2020

olavloite commented Mar 10, 2020

skuruppu commented Mar 12, 2020

feat: add backend query options #90

feat: add backend query options #90

Conversation

olavloite commented Mar 3, 2020 • edited Loading

skuruppu left a comment

Choose a reason for hiding this comment

skuruppu Mar 9, 2020

Choose a reason for hiding this comment

olavloite Mar 10, 2020

Choose a reason for hiding this comment

hengfengli commented Mar 9, 2020

olavloite commented Mar 10, 2020

olavloite commented Mar 10, 2020

skuruppu commented Mar 12, 2020

olavloite commented Mar 3, 2020 •

edited

Loading