Skip to content

Commit

Permalink
Update Google dependencies. (#197)
Browse files Browse the repository at this point in the history
* Handle a null schema in the sink connector.

* Fix concurrent access on iteration of synchronized set.

* Reduce the ackIds iteration synchronization to the shortest period needed

* Clear list of ackIds as soon as acks are sent. This will make acks best effort, but also prevents us from forgetting to send some acks if a poll happens while there is an outstanding ack request

* Better error message when verifySubscription fails.

* When an exception happens on pull, just return an empty list to poll.

* Ensure partition number is non-negative in CPS source connector.

* Shade out the guava library so the connector can work with a newer version of Kafka that depends on a newer version of the guava library.

* Shade out the guava library so the connector can work with a newer version of Kafka that depends on a newer version of the guava library.

* Update versions of gRPC libraries used by Kafka Connector. This should hopefully fix issue #120.

* Remove bad oracle jdk7 environment from Travis runs.

* Minor formatting fixes.

* Add a new config property to the sink connector, metadata.publish. When this property is true, the following attributes are added to a message published to Cloud Pub/Sub via the sink connector:

kafka.topic: The Kafka topic on which the message was originally published
kafka.partition: The Kafka partition to which the message was originally published
kafka.offset: The offset of the message in the Kafka topic/partition
kafka.timestamp: The timestamp that Kafka associated with the message

* Calculate message size at the end rather than along the way.

* Remove the temporary variables for Kafka attributes

* Periodically recreate GRPC publishers and subscribers in order to avoid GOAWAY errors.

* Formatting/syntactic fixes

* Switch sink connector to client library.

* Remove commented-out code

* Fix java 7 compatibility

* Add parameters for timeouts for publishes to Cloud Pub/Sub.

* Fix handling of optional struct fields and error message for missing fields, which would show up as a message about unsupported nested types.

* Add test case for optional field where the value is present.

* Set maximum inbound message size on source connector to ensure it is possible to receive a message up to largest Pub/Sub message size (10MB).

* Added instructions to Kafka Connector about downloading from a release.

* Clear the list of futures whenever we flush. If we do not do this, then when an error occurs, all subsequent flush attempts will fail.

* Update Google dependencies to newer versions

* Back to older netty library; new one results in inability to connect.
  • Loading branch information
kamalaboulhosn authored Apr 2, 2019
1 parent 2e0614e commit 75be003
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions kafka-connector/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-all</artifactId>
<version>1.10.1</version>
<version>1.16.1</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
Expand All @@ -32,7 +32,7 @@
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-pubsub</artifactId>
<version>v1-rev381-1.23.0</version>
<version>v1-rev418-1.25.0</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
Expand All @@ -43,7 +43,7 @@
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-pubsub</artifactId>
<version>0.42.1-beta</version>
<version>1.55.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java -->
<dependency>
Expand Down Expand Up @@ -144,7 +144,7 @@
-->
<protocArtifact>com.google.protobuf:protoc:3.5.1:exe:${os.detected.classifier}</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:1.11.0:exe:${os.detected.classifier}</pluginArtifact>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:1.16.1:exe:${os.detected.classifier}</pluginArtifact>
<protoSourceRoot>${project.basedir}/src/main/proto</protoSourceRoot>
</configuration>
<executions>
Expand Down

0 comments on commit 75be003

Please sign in to comment.