Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[E2E - Upgrade Testing] Add CI pipelines to test upgrades and chain integrity post-upgrade #1235

Closed
Tracked by #1181
niccoloraspa opened this issue Apr 12, 2022 · 2 comments
Closed
Tracked by #1181

[E2E - Upgrade Testing] Add CI pipelines to test upgrades and chain integrity post-upgrade #1235

niccoloraspa opened this issue Apr 12, 2022 · 2 comments
Assignees
@niccoloraspa @czarcas7ic
Labels
T:tests

Comments

@niccoloraspa
Copy link
Member

niccoloraspa commented Apr 12, 2022 •
edited by p0mvn
Loading

Background

Now that we have a e2e testing process in place to the test basic functionality of the chain, we can expand our test suite to test chain upgrades and make sure that basic properties are being preserved after the update

Tasks

Acceptance Criteria

  • Have an automated flow to test an upgrade and perform basic tests after the upgrade to ensure chain integrity.
@niccoloraspa niccoloraspa mentioned this issue Apr 12, 2022
Closed
3 tasks
@niccoloraspa niccoloraspa changed the title CI tests of upgrade succeeding, and basic properties being preserved post-upgrade Add CI pipelines to test upgrades and chain integrity post-upgrade Apr 12, 2022
@faddat
Copy link
Member

faddat commented Apr 18, 2022 •
edited
Loading

@nikever do you have a way around the github self-hosted runners lack of concurrency / extreme expense of the permium runners they sell?

basically, since their runners really do not match up to the performance of what most node operators use, I am not sure that this path can work, unless we are restricting the state that is used somehow (genesis export maybe?)

  • it would work. It would just work very slowly

@daniel-farina daniel-farina moved this to 🔍 Needs Review in Osmosis Chain Development Apr 21, 2022
This was referenced Apr 21, 2022
Merged
Merged
Closed
@czarcas7ic czarcas7ic mentioned this issue Apr 23, 2022
Closed
13 tasks
mergify bot pushed a commit that referenced this issue Apr 23, 2022
@p0mvn
refactor: implement Dockerized chain initialization in e2e tests (#1330)
c8725e9 
Closes: #XXX

## What is the purpose of the change

This is a follow-up PR to #1293 and builds upon its work. It is part of the e2e test chain upgrade epic #1235 

This PR introduces the ability to run chain initialization in a Docker container by running the following:
```
docker run -v < path >:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test
```
All chain data is placed at the given `< path >` that is mounted as a volume on the container. In addition, this PR introduces documentation about the current state of the e2e tests. 

## Brief change log

- [pull chain temp folder creation our of chain to e2e package](c175289)
- [create image and makefile steps to initialize chain state](de89119)
- [allow for running the upgrade initialization in Docker by providing a data dir](dabb683)
- [improve abstractions for chain initialization and add README](cf0d8a3)
- improve README

## Testing and Verifying

- ran e2e tests locally a few times
- `make build-e2e-chain-init`
- `make docker-build-e2e-chain-init`
- `docker run -v /home/roman/cosmos/osmosis/tmp:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test`

All steps worked as desired

## Documentation and Release Note

  - Does this pull request introduce a new feature or user-facing behavior changes? no
  - Is a relevant changelog entry added to the `Unreleased` section in `CHANGELOG.md`? no
  - How is the feature or change documented? improved `tests/e2e/README.md`

## Next Steps

The next step is to switch our chain initialization logic in `func (s *IntegrationTestSuite) configureChain(chainId string)` to use Dockertest and the newly introduced container. Then, mount the genesis and configs on the Osmosis containers, against which the e2e tests are executed
@p0mvn p0mvn mentioned this issue Apr 24, 2022
Merged
8 tasks
p0mvn added a commit that referenced this issue Apr 24, 2022
@p0mvn
refactor: implement Dockerized chain initialization in e2e tests (#1330)
c71f077 
Closes: #XXX

## What is the purpose of the change

This is a follow-up PR to #1293 and builds upon its work. It is part of the e2e test chain upgrade epic #1235 

This PR introduces the ability to run chain initialization in a Docker container by running the following:
```
docker run -v < path >:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test
```
All chain data is placed at the given `< path >` that is mounted as a volume on the container. In addition, this PR introduces documentation about the current state of the e2e tests. 

## Brief change log

- [pull chain temp folder creation our of chain to e2e package](c175289)
- [create image and makefile steps to initialize chain state](de89119)
- [allow for running the upgrade initialization in Docker by providing a data dir](dabb683)
- [improve abstractions for chain initialization and add README](cf0d8a3)
- improve README

## Testing and Verifying

- ran e2e tests locally a few times
- `make build-e2e-chain-init`
- `make docker-build-e2e-chain-init`
- `docker run -v /home/roman/cosmos/osmosis/tmp:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test`

All steps worked as desired

## Documentation and Release Note

  - Does this pull request introduce a new feature or user-facing behavior changes? no
  - Is a relevant changelog entry added to the `Unreleased` section in `CHANGELOG.md`? no
  - How is the feature or change documented? improved `tests/e2e/README.md`

## Next Steps

The next step is to switch our chain initialization logic in `func (s *IntegrationTestSuite) configureChain(chainId string)` to use Dockertest and the newly introduced container. Then, mount the genesis and configs on the Osmosis containers, against which the e2e tests are executed
@p0mvn p0mvn moved this from 🔍 Needs Review to 🏃 In Progress in Osmosis Chain Development Apr 28, 2022
czarcas7ic added a commit that referenced this issue Apr 29, 2022
@p0mvn @czarcas7ic
backport: e2e tests (#1193) (#1338)
f82a6fc 
* Setup e2e tests on a single chain; add balances query test (#1193)

* create e2e image and a makefile step to build

* progress

* e2e tests in ci

* use root distroless image and correct volume path

* remove chain b references

* implement query balances

* implement TestQueryBalances

* trigger worflow

* trigger

* test-e2e Makefile step

* fmt and sleep if service unavailable

* README

* restore branches

* add changelog entry

* exclude e2e from regular tests

* -E flag for grep exclusion

* grep

* go mod tidy --compat=1.17

* manually tidy go.mod

* second e2e chain with expanded test (#1206)

* e2e with IBC tx and test (#1216)

* second chain with tests

* add hermes, comment out balance query for now

* eventually

* Update Makefile

Co-authored-by: Roman <[email protected]>

* remove gas fees and unused functions

* readded check

* Nicco changes to own hermes image

* Remove unused hermes.Dockerfile

* Use a single Dockerfile for both debug and official image

* readd build hermes in makefile

* remove hermes

* Set correct golang image and use correct debug image tag

Co-authored-by: Adam Tucker <[email protected]>
Co-authored-by: Roman <[email protected]>
Co-authored-by: Niccolo Raspa <[email protected]>

* missing e2e ci job

* cleanup Makefile and ci workflows (#1203)

* cleanup makefile and ci workflows

* update changelog

* fix sim test

* fix makefile and rename docker repo

* fix make test-cover (#1219)

* refactor: begin modularizing e2e tests in preparation for upgrade (#1293)

* begin modularizing e2e test in preparation for upgrade

* rename common package to util

* move chain related constants from util to chain package

* fix genesis.go

* exctract initNodes into the genesis package

* remove genesis package, move all logic to chain

* continue cleaning up chain package and refactoring e2e

* store chains in a slice

* reuse common cdc from util package

* lexicographical reorder of functions in config.go of chain package

* clean up names

* refactor: implement Dockerized chain initialization in e2e tests (#1330)

Closes: #XXX

## What is the purpose of the change

This is a follow-up PR to #1293 and builds upon its work. It is part of the e2e test chain upgrade epic #1235 

This PR introduces the ability to run chain initialization in a Docker container by running the following:
```
docker run -v < path >:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test
```
All chain data is placed at the given `< path >` that is mounted as a volume on the container. In addition, this PR introduces documentation about the current state of the e2e tests. 

## Brief change log

- [pull chain temp folder creation our of chain to e2e package](c175289)
- [create image and makefile steps to initialize chain state](de89119)
- [allow for running the upgrade initialization in Docker by providing a data dir](dabb683)
- [improve abstractions for chain initialization and add README](cf0d8a3)
- improve README

## Testing and Verifying

- ran e2e tests locally a few times
- `make build-e2e-chain-init`
- `make docker-build-e2e-chain-init`
- `docker run -v /home/roman/cosmos/osmosis/tmp:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test`

All steps worked as desired

## Documentation and Release Note

  - Does this pull request introduce a new feature or user-facing behavior changes? no
  - Is a relevant changelog entry added to the `Unreleased` section in `CHANGELOG.md`? no
  - How is the feature or change documented? improved `tests/e2e/README.md`

## Next Steps

The next step is to switch our chain initialization logic in `func (s *IntegrationTestSuite) configureChain(chainId string)` to use Dockertest and the newly introduced container. Then, mount the genesis and configs on the Osmosis containers, against which the e2e tests are executed

* Setup e2e tests on a single chain; add balances query test (#1193)

* create e2e image and a makefile step to build

* progress

* e2e tests in ci

* use root distroless image and correct volume path

* remove chain b references

* implement query balances

* implement TestQueryBalances

* trigger worflow

* trigger

* test-e2e Makefile step

* fmt and sleep if service unavailable

* README

* restore branches

* add changelog entry

* exclude e2e from regular tests

* -E flag for grep exclusion

* grep

* go mod tidy --compat=1.17

* manually tidy go.mod

Co-authored-by: Adam Tucker <[email protected]>
Co-authored-by: Adam Tucker <[email protected]>
Co-authored-by: Niccolo Raspa <[email protected]>
@czarcas7ic czarcas7ic mentioned this issue May 2, 2022
Merged
@p0mvn p0mvn changed the title Add CI pipelines to test upgrades and chain integrity post-upgrade [Upgrade Testing] Add CI pipelines to test upgrades and chain integrity post-upgrade May 4, 2022
@p0mvn p0mvn changed the title [Upgrade Testing] Add CI pipelines to test upgrades and chain integrity post-upgrade [E2E - Upgrade Testing] Add CI pipelines to test upgrades and chain integrity post-upgrade May 4, 2022
@czarcas7ic czarcas7ic moved this from 🏃 In Progress to 🔍 Needs Review in Osmosis Chain Development May 4, 2022
@p0mvn p0mvn removed their assignment May 9, 2022
@mattverse mattverse mentioned this issue May 31, 2022
Closed
2 tasks
@czarcas7ic czarcas7ic moved this from Needs Review 🔍 to In Progress🏃 in Osmosis Chain Development Jun 23, 2022
@ValarDragon
Copy link
Member

Can we close this issue? cc @p0mvn

@p0mvn p0mvn closed this as completed Jul 15, 2022
Repository owner moved this from In Progress🏃 to Done ✅ in Osmosis Chain Development Jul 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@niccoloraspa niccoloraspa

@czarcas7ic czarcas7ic

Labels
T:tests
Projects
Osmosis Chain Development
Archived in project
Milestone
No milestone
Development

No branches or pull requests
5 participants
@niccoloraspa @ValarDragon @faddat @p0mvn @czarcas7ic