Test Network Testing Framework #6489
Conversation
| if err := apiSrv.Start(config); err != nil { | ||
| errCh := make(chan error) | ||
|
|
||
| go func() { | ||
| if err := apiSrv.Start(config); err != nil { | ||
| errCh <- err | ||
| } | ||
| }() | ||
|
|
||
| select { | ||
| case err := <-errCh: | ||
| return err | ||
| case <-time.After(5 * time.Second): // assume server started successfully |
There was a problem hiding this comment.
I did not realize Tendermint HTTP client actually blocks -- this fixes that so we can properly cleanup below.
Codecov Report
@@ Coverage Diff @@
## master #6489 +/- ##
==========================================
+ Coverage 52.17% 56.68% +4.51%
==========================================
Files 199 478 +279
Lines 12155 28777 +16622
==========================================
+ Hits 6342 16313 +9971
- Misses 5433 11306 +5873
- Partials 380 1158 +778 |
| cdc.MustUnmarshalJSON(cfg.GenesisState[authtypes.ModuleName], &authGenState) | ||
|
|
||
| authGenState.Accounts = genAccounts | ||
| cfg.GenesisState[authtypes.ModuleName] = cdc.MustMarshalJSON(authGenState) |
|
Failures are due to my apparent misunderstanding of Tendermint's multi-process capabilities -- specifically the RPC. It really is only meant to run in a single process, so starting multiple RPC clients is not going to work :-/ The only stopgap I can think of is to only allow one use of a network at a time and within a network, only one single validator's RPC server/client is started (which is OK). This might be an ok tradeoff for now. |
|
Still getting race conditions in Tendermint :-/ I would imagine each node has its own unique address book (set of |
There was a problem hiding this comment.
Great work @alexanderbez ! Looks like a lot of heavy lifting.
I would like to have the configurability to use another app besides simapp for this functionality. I think that would basically mean having:
- an app constructor on
Config, and - test suites that have a constructor which takes a
Configobject
Co-authored-by: Aaron Craelius <[email protected]>
Co-authored-by: Aaron Craelius <[email protected]>
Co-authored-by: Aaron Craelius <[email protected]>
|
Thanks @aaronc.
Done.
I don't follow. Can you elaborate, please? |
|
Bump @aaronc @AdityaSripal on a re-review & comment followup. I'd like to get this merged so I can start addressing the CLI testing refactor. |
There was a problem hiding this comment.
@alexanderbez I added some TODOs to #6423 to track the TxGenerator migration and also to address reusability of IntegrationTestSuites. This looks like a great start.
Basically we could have a func TestIntegrationTestSuite(t *testing.T) {
suite.Run(t, new IntegrationTestSuite(testutil.DefaultConfig())
}I added a TODO to #6423 about doing this in a separate PR. |
|
IMHO that might be overkill. |
What's overkill? |
Why would that be overkill? Seems pretty simple and isn't one of the points to be able to reuse integration tests in gaia and other apps? |
|
|
|
I don’t think that’s true. Each integration test suite is scoped to a
package but when we are testing an app such as gaia or regen, the app
developer likely wants to verify that the app has correctly wired together
the various modules. We are not deploying simapp to a live network, we are
deploying other apps. For regen’s codebase most of the code is imported
from other repos but l still want to have tests of that code wired together
and the primary test suite l will want to run are these integration tests
for each of the modules we’re importing running against the regen, not
simapp, app constructor.
|
|
Mhhh, ok I'm still not really following what |
Yep, like I said it can be addressed later and I put a TODO in #6423. |
Description
Introduction of the
testutilpackage. This package allows the creation of an entirely in-process testing cluster with fully operational Tendermint nodes constructed withSimApp. Each node has an RPC & API exposed. In addition, the network exposes aLocalclient that can be used to directly interface with Tendermint's RPC. The test network is entirely configurable. In the future, we should explore how to pass dynamic app constructors so other applications/projects can utilize this.This is the foundation of #6423. With this framework, we'll be able to easily spin up and tear down networks for rapid integration testing and development. Once we refactor our CLI handling, we can entirely ditch all the current "fixtures".
I've included a "liveness" test as an example and replacement of the ad-hoc script we have. I've also included an integration test suite for x/bank API handlers.
/cc @aaronc @jackzampolin @anilcse @marbar3778
Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.
docs/) or specification (x/<module>/spec/)godoccomments.Unreleasedsection inCHANGELOG.mdFiles changedin the Github PR explorer