Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel promotion tasks in bwc_pkg_promote_all do not fail correctly #127

Open
blag opened this issue Jan 9, 2019 · 4 comments
Open

Parallel promotion tasks in bwc_pkg_promote_all do not fail correctly #127

blag opened this issue Jan 9, 2019 · 4 comments

Comments

@blag
Copy link
Contributor

blag commented Jan 9, 2019

Creating an issue as per this comment.

Implementing promotion tasks in parallel does not work correctly - the process_completion task is never run, the workflow simply fails after all four tasks fail.

Here is the snippet:

  promote_all:
    next:
      - do:
          - promote_bwc_enterprise
          - promote_st2_auth_ldap
          - promote_st2flow
          - promote_bwc_ui
  promote_bwc_enterprise:
    action: st2ci.st2_pkg_promote_enterprise
    input:
      package: bwc-enterprise
      distro_version: <% ctx().pkg_distro_version %>
      release: <% ctx().release %>
      version: <% ctx().version %>
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_bwc_enterprise: <% ctx().version + '-' + result().output.revision %>
        do:
          - process_completion
      - when: <% failed() %>
        do:
          - process_completion
  promote_st2_auth_ldap:
    action: st2ci.st2_pkg_promote_enterprise
    input:
      package: st2-auth-ldap
      distro_version: <% ctx().pkg_distro_version %>
      release: <% ctx().release %>
      version: <% ctx().version %>
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_st2_auth_ldap: <% ctx().version + '-' + result().output.revision %>
        do:
          - process_completion
      - when: <% failed() %>
        do:
          - process_completion
  promote_st2flow:
    action: st2ci.st2_pkg_promote_enterprise
    input:
      package: st2flow
      distro_version: <% ctx().pkg_distro_version %>
      release: <% ctx().release %>
      version: <% ctx().version %>
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_st2flow: <% ctx().version + '-' + result().output.revision %>
        do:
          - process_completion
      - when: <% failed() %>
        do:
          - process_completion
  promote_bwc_ui:
    action: st2ci.st2_pkg_promote_enterprise
    input:
      package: bwc-ui
      distro_version: <% ctx().pkg_distro_version %>
      release: <% ctx().release %>
      version: <% ctx().version %>
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_bwc_ui: <% ctx().version + '-' + result().output.revision %>
        do:
          - process_completion
      - when: <% failed() %>
        do:
          - process_completion

  process_completion:
    action: core.noop
    join: all

    next:
      - when: <% succeeded() and     (ctx().promoted_bwc_enterprise and ctx().promoted_st2_auth_ldap and ctx().promoted_st2flow and ctx().promoted_bwc_ui) %>
        publish:
          - promoted: true
        do:
          - set_notify_success
      - when: <% succeeded() and not (ctx().promoted_bwc_enterprise and ctx().promoted_st2_auth_ldap and ctx().promoted_st2flow and ctx().promoted_bwc_ui) %>
        publish:
          - promoted: false
        do:
          - set_notify_failure
@m4dcoder
Copy link
Contributor

m4dcoder commented Jan 9, 2019

@blag The reason why the join don't work is because each task on success and on failure transition to process_completion. The workflow engine is basically waiting for all the transition to reach it which in this case will never because when: succeeded and when: failed are mutually exclusive. The original workflow transition to process_completion on task completion. The equivalent in orquesta is to not leave out when in the transition so it defaults to on complete. The alternative is to create a separate process_completion_on_failure for when: failed.

@m4dcoder m4dcoder mentioned this issue Jan 9, 2019
7 tasks
@blag
Copy link
Contributor Author

blag commented Jan 9, 2019

@m4dcoder I tried both of these parallel workflows and they both had the same behavior.

  promote_all:
    next:
      - do:
          - promote_bwc_enterprise
          - promote_st2_auth_ldap
          - promote_st2flow
          - promote_bwc_ui
  promote_bwc_enterprise:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_bwc_enterprise: <% ctx().version + '-' + result().output.revision %>
        do:
          - process_completion
      - when: <% failed() %>
        do:
          - process_completion_on_failure
  promote_st2_auth_ldap:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_st2_auth_ldap: <% ctx().version + '-' + result().output.revision %>
        do:
          - process_completion
      - when: <% failed() %>
        do:
          - process_completion_on_failure
  promote_st2flow:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_st2flow: <% ctx().version + '-' + result().output.revision %>
        do:
          - process_completion
      - when: <% failed() %>
        do:
          - process_completion_on_failure
  promote_bwc_ui:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_bwc_ui: <% ctx().version + '-' + result().output.revision %>
        do:
          - process_completion
      - when: <% failed() %>
        do:
          - process_completion_on_failure

  process_completion:
    action: core.noop
    join: all
    next:
      - do:
          - set_status_and_notify

  process_completion_on_failure:
    action: core.noop
    join: all
    next:
      - do:
          - set_status_and_notify

  set_status_and_notify:
    next:
      - when: <% succeeded() and     (ctx().promoted_bwc_enterprise and ctx().promoted_st2_auth_ldap and ctx().promoted_st2flow and ctx().promoted_bwc_ui) %>
        publish:
          - promoted: true
        do:
          - set_notify_success
      - when: <% succeeded() and not (ctx().promoted_bwc_enterprise and ctx().promoted_st2_auth_ldap and ctx().promoted_st2flow and ctx().promoted_bwc_ui) %>
        publish:
          - promoted: false
        do:
          - set_notify_failure
  promote_all:
    next:
      - do:
          - promote_bwc_enterprise
          - promote_st2_auth_ldap
          - promote_st2flow
          - promote_bwc_ui
  promote_bwc_enterprise:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_bwc_enterprise: <% ctx().version + '-' + result().output.revision %>
      - do:
          - process_completion
  promote_st2_auth_ldap:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_st2_auth_ldap: <% ctx().version + '-' + result().output.revision %>
      - do:
          - process_completion
  promote_st2flow:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_st2flow: <% ctx().version + '-' + result().output.revision %>
      - do:
          - process_completion
  promote_bwc_ui:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - promoted_bwc_ui: <% ctx().version + '-' + result().output.revision %>
      - do:
          - process_completion

  process_completion:
    action: core.noop
    join: all
    next:
      - when: <% succeeded() and     (ctx().promoted_bwc_enterprise and ctx().promoted_st2_auth_ldap and ctx().promoted_st2flow and ctx().promoted_bwc_ui) %>
        publish:
          - promoted: true
        do:
          - set_notify_success
      - when: <% succeeded() and not (ctx().promoted_bwc_enterprise and ctx().promoted_st2_auth_ldap and ctx().promoted_st2flow and ctx().promoted_bwc_ui) %>
        publish:
          - promoted: false
        do:
          - set_notify_failure

Both workflows failed immediately after all four parallel tasks finished - without ever running process_completion or any subsequent tasks.

Did I misunderstand what you meant?

@m4dcoder
Copy link
Contributor

Bug identified at StackStorm/orquesta#112

@blag
Copy link
Contributor Author

blag commented May 17, 2019

This can now proceed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants