Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-889] CheckpointDir option does not accept relative path and req… #1272

Closed
wants to merge 1 commit into from

Conversation

amitsela
Copy link
Member

@amitsela amitsela commented Nov 3, 2016

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

  • Make sure the PR title is formatted like:
    [BEAM-<Jira issue #>] Description of pull request
  • Make sure tests pass via mvn clean verify. (Even better, enable
    Travis-CI on your fork and ensure the whole test matrix passes).
  • Replace <Jira issue #> in the title with the actual Jira issue
    number, if there is one.
  • If this contribution is large, please file an Apache
    Individual Contributor License Agreement.

…uires protocol.

@amitsela
Copy link
Member Author

amitsela commented Nov 3, 2016

This seems much more friendly (and less code!).

Spark already does it's due-diligence on the provided checkpoint dir so no need to double verify.
Still warn if prefix does not much a reliable storage.

@amitsela
Copy link
Member Author

amitsela commented Nov 3, 2016

Jenkins failed on Dataflow IT test, unrelated.

@amitsela
Copy link
Member Author

amitsela commented Nov 3, 2016

R: @tgroh

@amitsela
Copy link
Member Author

amitsela commented Nov 4, 2016

R: @jbonofre if you're available ;-)

@jbonofre
Copy link
Member

jbonofre commented Nov 4, 2016

Sure ! Thanks !

@tgroh
Copy link
Member

tgroh commented Nov 4, 2016

The commit message doesn't seem to match the actual change, which is basically "handle relative paths in Spark CheckpointDir"

other than that, LGTM.

@amitsela
Copy link
Member Author

amitsela commented Nov 4, 2016

@tgroh it describes the issue instead of the solution.. mostly to match the ticket.
I'll change the commit message (leaving the ticket id) as it does make more sense.
Thanks!

@jbonofre
Copy link
Member

jbonofre commented Nov 4, 2016

LGTM

@asfgit asfgit closed this in 14e093a Nov 4, 2016
@amitsela amitsela deleted the BEAM-889 branch November 4, 2016 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants