-
Notifications
You must be signed in to change notification settings - Fork 628
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add compatibility tests for v5.0.x #2 #2396
Changes from 26 commits
b5c7185
329d8e0
b86a65a
536e37d
d5c758b
2759086
f123c50
1a4cc7d
c6508d9
b52332b
0bfbd52
0880657
a18213a
65f66e9
8774891
f6fba04
0ca1496
c385f3b
63f4b05
3bb23c8
69eeeb8
ee16871
1dc3c08
e01789d
285a161
95c29f7
6d4289f
cc81215
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
{ | ||
"chain-a": ["release-v5.0.x", "v4.1.0", "v3.3.0", "v2.4.0"], | ||
"chain-b": ["release-v5.0.x", "v4.1.0", "v3.3.0", "v2.4.0"], | ||
"entrypoint": ["TestClientTestSuite"], | ||
"test": [ | ||
"TestClientUpdateProposal_Succeeds" | ||
], | ||
"chain-binary": ["simd"], | ||
"chain-image": ["ghcr.io/cosmos/ibc-go-simd"] | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
{ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. these values get expanded so the workflow runs with every permutation of these values. |
||
"chain-a": ["release-v5.0.x", "v4.1.0"], | ||
"chain-b": ["release-v5.0.x", "v4.1.0"], | ||
"entrypoint": ["TestIncentivizedTransferTestSuite"], | ||
"test": [ | ||
"TestMsgPayPacketFee_AsyncSingleSender_Succeeds", | ||
"TestMsgPayPacketFee_InvalidReceiverAccount", | ||
"TestMultiMsg_MsgPayPacketFeeSingleSender", | ||
"TestMsgPayPacketFee_SingleSender_TimesOut", | ||
"TestPayPacketFeeAsync_SingleSender_NoCounterPartyAddress", | ||
"TestMsgPayPacketFee_AsyncMultipleSenders_Succeeds" | ||
], | ||
"chain-binary": ["simd"], | ||
"chain-image": ["ghcr.io/cosmos/ibc-go-simd"] | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
{ | ||
"chain-a": ["release-v5.0.x", "v4.1.0", "v3.3.0", "v2.4.0"], | ||
"chain-b": ["release-v5.0.x", "v4.1.0", "v3.3.0", "v2.4.0"], | ||
"entrypoint": ["TestTransferTestSuite"], | ||
"test": [ | ||
"TestMsgTransfer_Succeeds_Nonincentivized", | ||
"TestMsgTransfer_Fails_InvalidAddress", | ||
"TestMsgTransfer_Timeout_Nonincentivized", | ||
"TestSendEnabledParam", | ||
"TestReceiveEnabledParam" | ||
], | ||
"chain-binary": ["simd"], | ||
"chain-image": ["ghcr.io/cosmos/ibc-go-simd"] | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
on: | ||
workflow_call: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is a re-usable workflow that we can call for each test suite. |
||
inputs: | ||
docker-tag: | ||
description: 'Docker tag being used' | ||
required: true | ||
type: string | ||
test-suite: | ||
description: 'Test suite to run' | ||
required: true | ||
type: string | ||
|
||
|
||
jobs: | ||
load-test-matrix: | ||
outputs: | ||
test-matrix: ${{ steps.set-test-matrix.outputs.test-matrix }} | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- run: | | ||
# use jq -c to put the full json contents on a single line. This is required when using the json body | ||
# to create the matrix in the following job. | ||
test_matrix="$(cat .github/compatibility-test-matrices/${{ inputs.docker-tag }}/${{ inputs.test-suite }}.json | jq -c)" | ||
echo $test_matrix | ||
echo "::set-output name=test-matrix::$test_matrix" | ||
id: set-test-matrix | ||
e2e: | ||
runs-on: ubuntu-latest | ||
needs: load-test-matrix | ||
strategy: | ||
fail-fast: false | ||
matrix: ${{ fromJSON(needs.load-test-matrix.outputs.test-matrix) }} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Aha, so this magic uses There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes the previous job There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is indeed magic 😅 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the process of passing variables between steps is quite cumbersome unfortunately 😓 |
||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/setup-go@v3 | ||
with: | ||
go-version: 1.18 | ||
- name: Run e2e Test | ||
run: | | ||
cd e2e | ||
make e2e-test entrypoint=${{ matrix.entrypoint }} test=${{ matrix.test }} | ||
env: | ||
# each test has its own set of variables to specify which images are used. | ||
CHAIN_IMAGE: "${{ matrix.chain-image }}" | ||
CHAIN_A_TAG: "${{ matrix.chain-a }}" | ||
CHAIN_B_TAG: "${{ matrix.chain-b }}" | ||
CHAIN_BINARY: "${{ matrix.chain-binary }}" | ||
RLY_TAG: "v2.0.0" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,13 @@ | ||
name: Compatibility E2E | ||
on: | ||
workflow_dispatch: | ||
release-branch: | ||
description: 'Release branch to test' | ||
required: true | ||
type: choice | ||
options: | ||
- release/v5.0.x | ||
inputs: | ||
release-branch: | ||
description: 'Release branch to test' | ||
required: true | ||
type: choice | ||
options: | ||
- release/v5.0.x | ||
Comment on lines
+4
to
+10
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So do we need to add all release branches we want to test with here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes - this is just for a dropdown, if we prefer, we can have a text box and enter anything we like. I just think the dropdown is better UX There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree! |
||
|
||
env: | ||
REGISTRY: ghcr.io | ||
|
@@ -47,42 +48,29 @@ jobs: | |
docker build . -t "${REGISTRY}/${ORG}/${IMAGE_NAME}:${{ needs.determine-docker-tag.outputs.docker-tag }}" | ||
docker push "${REGISTRY}/${ORG}/${IMAGE_NAME}:${{ needs.determine-docker-tag.outputs.docker-tag }}" | ||
load-test-matrix: | ||
outputs: | ||
test-matrix: ${{ steps.set-test-matrix.outputs.test-matrix }} | ||
needs: determine-docker-tag | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- run: | | ||
# use jq -c to put the full json contents on a single line. This is required when using the json body | ||
# to create the matrix in the following job. | ||
test_matrix="$(cat .github/compatibility-test-matrices/${{ needs.determine-docker-tag.outputs.docker-tag }}.json | jq -c)" | ||
echo $test_matrix | ||
echo "::set-output name=test-matrix::$test_matrix" | ||
id: set-test-matrix | ||
transfer: | ||
needs: | ||
- build-release-image | ||
- determine-docker-tag | ||
uses: ./.github/workflows/e2e-compatibility-workflow-call.yaml | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is how workflows can be re-used in github actions. |
||
with: | ||
docker-tag: "${{ needs.determine-docker-tag.outputs.docker-tag }}" | ||
test-suite: "transfer" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. these strings map to sub directories, I'm open to alternatives if there are any suggestions! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure I understand fully here. Could you provide me with the verbose output? 😅 I initially thought you meant they correspond to the directory under There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will we also need to add blocks for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe we could rename the value to be something more explicit like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm easy. I think its fine to leave as is There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Whatever is fine by me! |
||
|
||
e2e-tests: | ||
runs-on: ubuntu-latest | ||
client: | ||
needs: | ||
- load-test-matrix | ||
- build-release-image | ||
strategy: | ||
fail-fast: false | ||
matrix: ${{ fromJSON(needs.load-test-matrix.outputs.test-matrix) }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/setup-go@v3 | ||
with: | ||
go-version: 1.18 | ||
- name: Run e2e Test | ||
run: | | ||
cd e2e | ||
make e2e-test entrypoint=${{ matrix.entrypoint }} test=${{ matrix.test }} | ||
env: | ||
# each test has its own set of variables to specify which images are used. | ||
CHAIN_IMAGE: "${{ matrix.chain-image }}" | ||
CHAIN_A_TAG: "${{ matrix.chain-a-tag }}" | ||
CHAIN_B_TAG: "${{ matrix.chain-b-tag }}" | ||
CHAIN_BINARY: "${{ matrix.chain-binary }}" | ||
RLY_TAG: "v2.0.0" | ||
- determine-docker-tag | ||
uses: ./.github/workflows/e2e-compatibility-workflow-call.yaml | ||
with: | ||
docker-tag: "${{ needs.determine-docker-tag.outputs.docker-tag }}" | ||
test-suite: "client" | ||
|
||
incentivized-transfer: | ||
needs: | ||
- build-release-image | ||
- determine-docker-tag | ||
uses: ./.github/workflows/e2e-compatibility-workflow-call.yaml | ||
with: | ||
docker-tag: "${{ needs.determine-docker-tag.outputs.docker-tag }}" | ||
test-suite: "incentivized-transfer" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so clean! ❤️