-
Notifications
You must be signed in to change notification settings - Fork 607
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[E2E] Setup e2e tests on a single chain; add balances query test #1193
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1193 +/- ##
==========================================
- Coverage 21.03% 21.03% -0.01%
==========================================
Files 196 196
Lines 25349 25351 +2
==========================================
Hits 5332 5332
- Misses 19054 19059 +5
+ Partials 963 960 -3
Continue to review full report at Codecov.
|
Are a lot of the files in the e2e folder copied from somewhere? Can we perhaps note its source either in a README.md or a doc.go in the package? |
It's copied most likely from my original work in either Gaia or Umee. I don't think there's a need to reference the original. Instead, I propose that we really flesh out the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀
@@ -37,7 +37,7 @@ jobs: | |||
- name: Display go version | |||
run: go version | |||
- name: Run all tests | |||
run: go test -mod=readonly -timeout 30m -coverprofile=coverage.txt -tags='norace' -covermode=atomic `go list ./... | grep -v simapp` | |||
run: go test -mod=readonly -timeout 30m -coverprofile=coverage.txt -tags='norace' -covermode=atomic `go list ./... | grep -E -v 'simapp|e2e'` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: we should instead encapsulate this in a make target.
See the changes I made here: https://github.com/cosmos/gaia/blob/main/Makefile#L167-L205
I don't think it's too much trouble to do that in this PR. WDYT?
go.mod
Outdated
@@ -29,6 +29,27 @@ require ( | |||
gopkg.in/yaml.v2 v2.4.0 | |||
) | |||
|
|||
require ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you run go mod tidy --compat=1.17
? Weird, these should be bundled with the existing indirect
require block below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, definitely looks like something went not-quite-right here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Had to manually move them to the same block and run go mod tidy
@@ -0,0 +1,13 @@ | |||
# End-to-end Tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't worry @ValarDragon I'll beef up this doc after merge.
Overall LGTM 🚀 I'm actually reverting to the previous |
So the context on why I originally named it |
That makes absolutely sense but in our setup we are just running a working copy of the Ideally we should try to use the same |
Sounds good to me! |
s.Require().NoError(cdc.UnmarshalJSON(appGenState[banktypes.ModuleName], &bankGenState)) | ||
|
||
bankGenState.DenomMetadata = append(bankGenState.DenomMetadata, banktypes.Metadata{ | ||
Description: "An example stable token", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh wow, I didn't know there was a description field available in the bank module's per-asset metadata
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't review scaffolding in depth. But the test looks great / its pretty exciting if it 'justworks.TM'.
Should we do the makefile change in this PR or a separate one?
a34561d
to
3c4336b
Compare
Making the makefile change here and then will merge |
On another look, I think the makefile should be addressed in a separate PR. Otherwise, the scope ends up being too large. I will merge this and address it in a separate PR |
* create e2e image and a makefile step to build * progress * e2e tests in ci * use root distroless image and correct volume path * remove chain b references * implement query balances * implement TestQueryBalances * trigger worflow * trigger * test-e2e Makefile step * fmt and sleep if service unavailable * README * restore branches * add changelog entry * exclude e2e from regular tests * -E flag for grep exclusion * grep * go mod tidy --compat=1.17 * manually tidy go.mod (cherry picked from commit 427e667) # Conflicts: # .github/workflows/test.yml # go.mod # go.sum
* create e2e image and a makefile step to build * progress * e2e tests in ci * use root distroless image and correct volume path * remove chain b references * implement query balances * implement TestQueryBalances * trigger worflow * trigger * test-e2e Makefile step * fmt and sleep if service unavailable * README * restore branches * add changelog entry * exclude e2e from regular tests * -E flag for grep exclusion * grep * go mod tidy --compat=1.17 * manually tidy go.mod
* create e2e image and a makefile step to build * progress * e2e tests in ci * use root distroless image and correct volume path * remove chain b references * implement query balances * implement TestQueryBalances * trigger worflow * trigger * test-e2e Makefile step * fmt and sleep if service unavailable * README * restore branches * add changelog entry * exclude e2e from regular tests * -E flag for grep exclusion * grep * go mod tidy --compat=1.17 * manually tidy go.mod
* create e2e image and a makefile step to build * progress * e2e tests in ci * use root distroless image and correct volume path * remove chain b references * implement query balances * implement TestQueryBalances * trigger worflow * trigger * test-e2e Makefile step * fmt and sleep if service unavailable * README * restore branches * add changelog entry * exclude e2e from regular tests * -E flag for grep exclusion * grep * go mod tidy --compat=1.17 * manually tidy go.mod
* Setup e2e tests on a single chain; add balances query test (#1193) * create e2e image and a makefile step to build * progress * e2e tests in ci * use root distroless image and correct volume path * remove chain b references * implement query balances * implement TestQueryBalances * trigger worflow * trigger * test-e2e Makefile step * fmt and sleep if service unavailable * README * restore branches * add changelog entry * exclude e2e from regular tests * -E flag for grep exclusion * grep * go mod tidy --compat=1.17 * manually tidy go.mod * second e2e chain with expanded test (#1206) * e2e with IBC tx and test (#1216) * second chain with tests * add hermes, comment out balance query for now * eventually * Update Makefile Co-authored-by: Roman <[email protected]> * remove gas fees and unused functions * readded check * Nicco changes to own hermes image * Remove unused hermes.Dockerfile * Use a single Dockerfile for both debug and official image * readd build hermes in makefile * remove hermes * Set correct golang image and use correct debug image tag Co-authored-by: Adam Tucker <[email protected]> Co-authored-by: Roman <[email protected]> Co-authored-by: Niccolo Raspa <[email protected]> * missing e2e ci job * cleanup Makefile and ci workflows (#1203) * cleanup makefile and ci workflows * update changelog * fix sim test * fix makefile and rename docker repo * fix make test-cover (#1219) * refactor: begin modularizing e2e tests in preparation for upgrade (#1293) * begin modularizing e2e test in preparation for upgrade * rename common package to util * move chain related constants from util to chain package * fix genesis.go * exctract initNodes into the genesis package * remove genesis package, move all logic to chain * continue cleaning up chain package and refactoring e2e * store chains in a slice * reuse common cdc from util package * lexicographical reorder of functions in config.go of chain package * clean up names * refactor: implement Dockerized chain initialization in e2e tests (#1330) Closes: #XXX ## What is the purpose of the change This is a follow-up PR to #1293 and builds upon its work. It is part of the e2e test chain upgrade epic #1235 This PR introduces the ability to run chain initialization in a Docker container by running the following: ``` docker run -v < path >:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test ``` All chain data is placed at the given `< path >` that is mounted as a volume on the container. In addition, this PR introduces documentation about the current state of the e2e tests. ## Brief change log - [pull chain temp folder creation our of chain to e2e package](c175289) - [create image and makefile steps to initialize chain state](de89119) - [allow for running the upgrade initialization in Docker by providing a data dir](dabb683) - [improve abstractions for chain initialization and add README](cf0d8a3) - improve README ## Testing and Verifying - ran e2e tests locally a few times - `make build-e2e-chain-init` - `make docker-build-e2e-chain-init` - `docker run -v /home/roman/cosmos/osmosis/tmp:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test` All steps worked as desired ## Documentation and Release Note - Does this pull request introduce a new feature or user-facing behavior changes? no - Is a relevant changelog entry added to the `Unreleased` section in `CHANGELOG.md`? no - How is the feature or change documented? improved `tests/e2e/README.md` ## Next Steps The next step is to switch our chain initialization logic in `func (s *IntegrationTestSuite) configureChain(chainId string)` to use Dockertest and the newly introduced container. Then, mount the genesis and configs on the Osmosis containers, against which the e2e tests are executed * Setup e2e tests on a single chain; add balances query test (#1193) * create e2e image and a makefile step to build * progress * e2e tests in ci * use root distroless image and correct volume path * remove chain b references * implement query balances * implement TestQueryBalances * trigger worflow * trigger * test-e2e Makefile step * fmt and sleep if service unavailable * README * restore branches * add changelog entry * exclude e2e from regular tests * -E flag for grep exclusion * grep * go mod tidy --compat=1.17 * manually tidy go.mod Co-authored-by: Adam Tucker <[email protected]> Co-authored-by: Adam Tucker <[email protected]> Co-authored-by: Niccolo Raspa <[email protected]>
Closes: #1124
Description
The package e2e defines an integration testing suite used for full end-to-end
testing functionality.
The file e2e_suite_test.go defines the testing suite and contains the core
bootstrapping logic that creates a testing environment via Docker containers.
A testing network is created dynamically with 2 test validators.
The file e2e_test.go contains the actual end-to-end integration tests that
utilize the testing suite.
Currently, there is a single test in
e2e_test.go
to query the balances of a validator.More tests are to be added in future PRs
Test Run
Proof of successful e2e test run: https://github.com/p0mvn/osmosis/runs/5830443179?check_suite_focus=true
Next Steps
The following can be split into separate tasks / PRs and tackled one by one:
For contributor use:
docs/
) or specification (x/<module>/spec/
)Unreleased
section inCHANGELOG.md
Files changed
in the Github PR explorer