upgrade clickhouse jdbc driver 0.3.2-patch10 #24424

mzitnik · 2022-11-30T08:25:09Z

Upgrade clickhouse jdbc driver to 0.3.2-patch10
@methodmissing

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

github-actions · 2022-11-30T09:05:00Z

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @apilloud for label java.
R: @Abacn for label build.
R: @johnjcasey for label io.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

codecov · 2022-11-30T09:05:33Z

Codecov Report

Merging #24424 (db0c348) into master (c4cad83) will decrease coverage by 0.18%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #24424      +/-   ##
==========================================
- Coverage   73.46%   73.28%   -0.19%     
==========================================
  Files         714      723       +9     
  Lines       96497    98227    +1730     
==========================================
+ Hits        70896    71981    +1085     
- Misses      24279    24924     +645     
  Partials     1322     1322

Flag	Coverage Δ
python	`82.66% <ø> (-0.52%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...s/python/apache_beam/io/gcp/bigquery_file_loads.py	`79.37% <0.00%> (-8.05%)`	⬇️
...l/job_management/v1/beam_expansion_api_pb2_grpc.py	`51.85% <0.00%> (-6.05%)`	⬇️
sdks/python/apache_beam/typehints/schemas.py	`84.55% <0.00%> (-4.40%)`	⬇️
...n/apache_beam/examples/complete/game/user_score.py	`59.70% <0.00%> (-3.09%)`	⬇️
sdks/python/apache_beam/utils/python_callable.py	`96.77% <0.00%> (-1.27%)`	⬇️
.../python/apache_beam/typehints/trivial_inference.py	`95.01% <0.00%> (-1.15%)`	⬇️
...on/apache_beam/runners/dataflow/dataflow_runner.py	`81.88% <0.00%> (-1.02%)`	⬇️
...dks/python/apache_beam/options/pipeline_options.py	`93.95% <0.00%> (-0.96%)`	⬇️
sdks/python/apache_beam/typehints/row_type.py	`99.11% <0.00%> (-0.89%)`	⬇️
sdks/python/apache_beam/transforms/external.py	`78.88% <0.00%> (-0.86%)`	⬇️
... and 39 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

johnjcasey · 2022-11-30T15:06:23Z

buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy

@@ -499,6 +499,7 @@ class BeamModulePlugin implements Plugin<Project> {
    def spark3_version = "3.1.2"
    def spotbugs_version = "4.0.6"
    def testcontainers_version = "1.16.3"
+    def testcontainers_version_clickhouse = "1.17.3"


What is the reason to not just update the testcontainers version globally?

@johnjcasey Yes i can, just was not sure that it is OK i will do that and push the change

@johnjcasey I have pushed the change testcontainers version

mshustov · 2022-12-05T09:46:30Z

sdks/java/io/clickhouse/build.gradle

@@ -49,7 +49,7 @@ idea {
  }
 }

-def clickhouse_jdbc_version = "0.2.6"
+def clickhouse_jdbc_version = "0.3.2-patch10"


Why not to the latest version (0.3.2-patch11)?

Cause some conflict with testcontainers_version need to have deeper look on that,but for now lets continue with 0.3.2-patch10

Sounds good

mshustov · 2022-12-05T09:47:51Z

sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/TableSchema.java

+      //      try {
+      String value = str;
+      /*
+      String value =


can be removed?

mshustov · 2022-12-05T09:50:56Z

sdks/java/io/clickhouse/src/test/java/org/apache/beam/sdk/io/clickhouse/BaseClickHouseTest.java

-        }
-      }
-    }
+        new ClickHouseContainer("clickhouse/clickhouse-server:22.5")


Why don't we test against the latest CH version?

Change to 22.9

mshustov · 2022-12-05T09:53:05Z

sdks/java/io/clickhouse/src/test/java/org/apache/beam/sdk/io/clickhouse/BaseClickHouseTest.java

  }

  @AfterClass
  public static void tearDown() {
    clickHouse.close();
-    zookeeper.close();


Shouldn't we close it?

mzitnik · 2022-12-06T06:44:04Z

@johnjcasey can you help with the PR

sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/ClickHouseWriter.java

sdks/java/io/clickhouse/src/test/java/org/apache/beam/sdk/io/clickhouse/AtomicInsertTest.java

sdks/java/io/clickhouse/src/test/java/org/apache/beam/sdk/io/clickhouse/TableSchemaTest.java

github-actions · 2022-12-19T12:13:59Z

Reminder, please take a look at this pr: @apilloud @Abacn @johnjcasey

github-actions · 2022-12-22T12:13:37Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kileys for label java.
R: @damccorm for label build.
R: @pabloem for label io.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

damccorm · 2022-12-22T14:15:42Z

stop reviewer notifications

damccorm · 2022-12-22T14:16:29Z

@johnjcasey has started this review and probably is the right person to complete it. I imagine responses may be slower since we are in the midst of the holiday season

github-actions · 2022-12-22T14:16:57Z

Stopping reviewer notifications for this pull request: requested by reviewer

johnjcasey · 2022-12-22T21:50:23Z

run java precommit

johnjcasey · 2022-12-22T21:50:42Z

run java_pulsar_io_direct precommit

mzitnik · 2023-01-02T10:25:07Z

@johnjcasey do we know on what Apache Beam version it will be merged?

johnjcasey · 2023-01-18T15:26:14Z

it will be 2.45

mshustov · 2023-01-18T17:53:18Z

@johnjcasey is there ETA for v2.45 release to communicate to our customers?

johnjcasey · 2023-01-25T15:55:30Z

The cut was last week, and we are closing out release blockers now. It could be as far away as 5 weeks from now, but more likely sooner

upgrade clickhouse jdbc driver 0.3.2-patch10

8246920

github-actions bot added build clickhouse io java labels Nov 30, 2022

github-actions bot added the Next Action: Reviewers label Nov 30, 2022

johnjcasey reviewed Nov 30, 2022

View reviewed changes

chnage testcontainers_version version to 1.17.3

3e2925e

mshustov reviewed Dec 5, 2022

View reviewed changes

mzitnik added 2 commits December 5, 2022 13:29

remove redundant code

f216f11

update tests to use clickhouse version to 22.9

b8b51c1

johnjcasey requested changes Dec 9, 2022

View reviewed changes

apply suggested changes

0b3699e

mzitnik requested a review from johnjcasey December 10, 2022 15:48

change back to 100K bulk size

db0c348

github-actions bot added the slow-review label Dec 19, 2022

github-actions bot removed the slow-review label Dec 22, 2022

johnjcasey approved these changes Dec 22, 2022

View reviewed changes

johnjcasey merged commit aba1752 into apache:master Jan 18, 2023

mzitnik mentioned this pull request Feb 16, 2023

[Bug]: BEAM SDKs Java IO ClickHouse dependency using ru.yandex instead of com.clickhouse issue #25511

Closed

15 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

upgrade clickhouse jdbc driver 0.3.2-patch10 #24424

upgrade clickhouse jdbc driver 0.3.2-patch10 #24424

mzitnik commented Nov 30, 2022

github-actions bot commented Nov 30, 2022

codecov bot commented Nov 30, 2022 •

edited

Loading

johnjcasey Nov 30, 2022

mzitnik Dec 5, 2022 •

edited

Loading

mzitnik Dec 5, 2022

mshustov Dec 5, 2022

mzitnik Dec 5, 2022 •

edited

Loading

mshustov Dec 5, 2022

mshustov Dec 5, 2022

mshustov Dec 5, 2022

mzitnik Dec 5, 2022

mshustov Dec 5, 2022

mzitnik Dec 5, 2022

mzitnik commented Dec 6, 2022

github-actions bot commented Dec 19, 2022

github-actions bot commented Dec 22, 2022

damccorm commented Dec 22, 2022

damccorm commented Dec 22, 2022

github-actions bot commented Dec 22, 2022

johnjcasey commented Dec 22, 2022

johnjcasey commented Dec 22, 2022

mzitnik commented Jan 2, 2023

johnjcasey commented Jan 18, 2023

mshustov commented Jan 18, 2023 •

edited

Loading

johnjcasey commented Jan 25, 2023

upgrade clickhouse jdbc driver 0.3.2-patch10 #24424

upgrade clickhouse jdbc driver 0.3.2-patch10 #24424

Conversation

mzitnik commented Nov 30, 2022

Upgrade clickhouse jdbc driver to 0.3.2-patch10 @methodmissing

GitHub Actions Tests Status (on master branch)

github-actions bot commented Nov 30, 2022

codecov bot commented Nov 30, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

mzitnik Dec 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mzitnik Dec 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mzitnik commented Dec 6, 2022

github-actions bot commented Dec 19, 2022

github-actions bot commented Dec 22, 2022

damccorm commented Dec 22, 2022

damccorm commented Dec 22, 2022

github-actions bot commented Dec 22, 2022

johnjcasey commented Dec 22, 2022

johnjcasey commented Dec 22, 2022

mzitnik commented Jan 2, 2023

johnjcasey commented Jan 18, 2023

mshustov commented Jan 18, 2023 • edited Loading

johnjcasey commented Jan 25, 2023

Upgrade clickhouse jdbc driver to 0.3.2-patch10
@methodmissing

codecov bot commented Nov 30, 2022 •

edited

Loading

mzitnik Dec 5, 2022 •

edited

Loading

mzitnik Dec 5, 2022 •

edited

Loading

mshustov commented Jan 18, 2023 •

edited

Loading