-
Notifications
You must be signed in to change notification settings - Fork 146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recreation of GRPC publisher/subscribers #152
Merged
Merged
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
52f70e2
Handle a null schema in the sink connector.
kamalaboulhosn dcdc552
Merge branch 'master' of git://github.com/GoogleCloudPlatform/pubsub
kamalaboulhosn 763b35c
Fix concurrent access on iteration of synchronized set.
kamalaboulhosn 73fd048
Reduce the ackIds iteration synchronization to the shortest period ne…
kamalaboulhosn e43eba7
Clear list of ackIds as soon as acks are sent. This will make acks be…
kamalaboulhosn f752def
Merge branch 'master' into master
kamalaboulhosn dbfe894
Better error message when verifySubscription fails.
kamalaboulhosn 23f21e8
Merge branch 'master' of git://github.com/GoogleCloudPlatform/pubsub
kamalaboulhosn 641ec07
Merge branch 'master' into master
kamalaboulhosn e5cde4a
Merge branch 'master' of https://github.com/kamalaboulhosn/pubsub
kamalaboulhosn 73989b9
Merge branch 'master' of git://github.com/GoogleCloudPlatform/pubsub
kamalaboulhosn 8dd3c62
When an exception happens on pull, just return an empty list to poll.
kamalaboulhosn e7d7ddd
Ensure partition number is non-negative in CPS source connector.
kamalaboulhosn 5fb88ee
Merge branch 'master' into master
kamalaboulhosn 778226f
Shade out the guava library so the connector can work with a newer ve…
kamalaboulhosn bb34ebd
Shade out the guava library so the connector can work with a newer ve…
kamalaboulhosn e0f7c0a
Merge branch 'master' of https://github.com/kamalaboulhosn/pubsub
kamalaboulhosn 1ced89e
Merge remote-tracking branch 'upstream/master'
kamalaboulhosn 4fdbf1e
Update versions of gRPC libraries used by Kafka Connector. This shoul…
kamalaboulhosn 4a22105
Remove bad oracle jdk7 environment from Travis runs.
kamalaboulhosn 68be95b
Merge remote-tracking branch 'upstream/master'
kamalaboulhosn 9629623
Merge remote-tracking branch 'upstream/master'
kamalaboulhosn cdf663b
Minor formatting fixes.
kamalaboulhosn ad98f27
Add a new config property to the sink connector, metadata.publish. Wh…
kamalaboulhosn 98c5198
Merge remote-tracking branch 'upstream/master'
kamalaboulhosn a0196bd
Calculate message size at the end rather than along the way.
kamalaboulhosn 680d927
Remove the temporary variables for Kafka attributes
kamalaboulhosn 811aef8
Periodically recreate GRPC publishers and subscribers in order to avo…
kamalaboulhosn e58cf6a
Merge remote-tracking branch 'upstream/master'
kamalaboulhosn b4b024b
Formatting/syntactic fixes
kamalaboulhosn 514d9e6
Switch sink connector to client library.
kamalaboulhosn 36d4b9b
Remove commented-out code
kamalaboulhosn 58c022a
Fix java 7 compatibility
kamalaboulhosn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there no way to check on the channel itself, like by calling isTerminated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a possibility we might be able to do that in the future. A second change will have to be made to not throw exceptions in the CloudPubSubSinkTask when flushing messages. When an exception is thrown, everything shuts down. That means introducing retry and making some significant changes to the class. Once we have that, then we can throw an exception when we have an error, detect when termination occurs, and recreate the publisher.
There is some urgency for a fix to this problem for a customer and so my hope was to use this to solve the problem in the meantime.