R4R: Simulate transactions for fee distribution, simulate inflation, and fix a multitude of bugs discovered in the process of doing so #2501

cwgoes · 2018-10-16T00:19:06Z

Closes #1870
Closes #2368
Closes #2485

Add transactions (and maybe invariants?) for the new fee distribution module.

Standard checklist:

Linked to github-issue with discussion and accepted design OR link to spec that describes this work.
Wrote tests
Updated relevant documentation (docs/)
Added entries in PENDING.md with issue #
rereviewed Files changed in the github PR explorer

For Admin Use:

Added appropriate labels to PR (ex. wip, ready-for-review, docs)
Reviewers Assigned
Squashed all commits, uses message "Merge pull request #XYZ: [title]" (coding standards)

rigelrozanski · 2018-10-16T01:14:53Z

Thanks for starting this Chris!

codecov · 2018-10-16T21:33:03Z

Codecov Report

Merging #2501 into develop will decrease coverage by 0.28%.
The diff coverage is 46.82%.

@@             Coverage Diff             @@
##           develop    #2501      +/-   ##
===========================================
- Coverage    60.32%   60.04%   -0.29%     
===========================================
  Files          150      150              
  Lines         8613     8684      +71     
===========================================
+ Hits          5196     5214      +18     
- Misses        3069     3116      +47     
- Partials       348      354       +6

cwgoes · 2018-10-16T22:44:43Z

I think we need to add some sanity checks:

For nonexistent delegation reward info on WithdrawDelegatorReward (maybe the delegator never delegated to the validator)
For nonexistent validator reward info on WithdrawValidatorRewardsAll (maybe the provided address isn't even a validator)
For nonexistent delegation reward info on WithdrawDelegatorRewardsAll (maybe the address has never delegated)

Also, when I comment out those three messages, supply conservation is broken:

Running quick Gaia simulation. This may take several minutes...
=== RUN   TestFullGaiaSimulation
Starting SimulateFromSeed with randomness created with seed 42
Starting the simulation from time Sun Jan 31 13:59:47 UTC 26140, unixtime 762734152787
Simulating... block 27/400, operation 200/203.  expected loose tokens to equal total steak held by accounts - pool.LooseTokens: 9738, sum of account tokens: 9737
Too many logs to display, instead writing to simulation_log_2018-10-17 00:43:37.txt
--- FAIL: TestFullGaiaSimulation (2.88s)
    util.go:111: 
FAIL
FAIL	github.com/cosmos/cosmos-sdk/cmd/gaia/app	2.899s
make: *** [Makefile:172: test_sim_gaia_fast] Error 1

I also fixed a bug in the simulation where power wasn't set correctly for last-commit, so we may not really have been simulating fee distribution at all before.

rigelrozanski · 2018-10-17T00:20:04Z

All good points -> with regards to fee distribution conservation of tokens - this very well could be due to the rounding down of tokens when you withdraw (and you are due a non-integer amount). - one easy way to deal with this in the short term is to just donate any trimmings of tokens to the community tax pool - again this will be extremely small amounts as we're going to be dealing with nano-atoms here.

With regards to sanity checks, yeah I guess some sanity checks are missing, but obviously we don't want to be panicking if the handler gets a bad message, just returning an error early on.

rigelrozanski · 2018-10-17T00:27:30Z

Beyond token conservation we could also maybe check accum invariants - however I don’t think we need to scope that in immediately

cwgoes · 2018-10-18T20:19:40Z

Hmm this passes now but I think we need to actually charge fees in simulation.

cwgoes · 2018-10-18T23:13:11Z

OK, I've found more problems - after disabling rounding the supply (which the simulation used to do) it fails even without any of the new fee distribution logic at all. I think we might be rounding somewhare in x/stake and not dealing with the remainder correctly.

rigelrozanski · 2018-10-19T01:01:50Z

after disabling rounding the supply (which the simulation used to do) it fails even without any of the new fee distribution logic at all. I think we might be rounding somewhare in x/stake and not dealing with the remainder correctly.

Can you highlight this change in with a commit?
@cwgoes

…/cosmos-sdk into cwgoes/check-supply-in-simulation

cwgoes · 2018-10-23T00:47:18Z

Interestingly, when I disable slashing I get an even faster simulation failure (same panic on individual > total accum).

sim fixes

… into cwgoes/check-supply-in-simulation

cwgoes · 2018-10-23T16:36:14Z

Cleaned up debugging prints, removed a few unnecessary changes, and updated PENDING.md to reflect what this PR changes.

This has passed 400-block single seed sim (locally and CI) & 100-block multi-seed sim (ran locally).

cwgoes · 2018-10-23T16:59:03Z

We should add accum invariants, but that can probably happen separately - #2573.

rigelrozanski

woohoo! Few minor comments throughout - the improper use of periods is driving me a bit mad

cmd/gaia/app/sim_test.go

x/stake/keeper/keeper.go

x/distribution/keeper/delegation.go

x/distribution/keeper/validator.go

x/distribution/types/delegator_info.go

types/decimal.go

x/distribution/types/validator_info.go

x/slashing/keeper.go

x/stake/keeper/keeper.go

jaekwon · 2018-10-24T00:42:20Z

x/stake/handler.go

@@ -150,6 +148,7 @@ func handleMsgEditValidator(ctx sdk.Context, msg types.MsgEditValidator, k keepe
 			return err.Result()
 		}
 		validator.Commission = commission
+		k.OnValidatorModified(ctx, msg.ValidatorAddr)


probably shouldn't be calling this before k.SetValidator(). Also hooks being called from handlers is weird.

I think this is the correct order, but we should change the name of the hook - #2538.

Update PENDING.md

1674625

cwgoes added C:x/staking T: Tests wip labels Oct 16, 2018

cwgoes requested review from ebuchman and rigelrozanski as code owners October 16, 2018 00:19

This PR now will do something else

2fff810

cwgoes changed the title ~~WIP: Assert supply conservation in simulation~~ WIP: Simulate transactions for fee distribution Oct 16, 2018

Merge branch 'develop' into cwgoes/check-supply-in-simulation

67258c4

cwgoes added 2 commits October 17, 2018 00:25

Add simulated distribution msgs

b700107

Correctly set power in vote info

1d4b595

cwgoes mentioned this pull request Oct 17, 2018

SDK 0.25 / Gaia-9000 release checklist #2220

Closed

23 tasks

Merge branch 'develop' into cwgoes/check-supply-in-simulation

74780c6

cwgoes mentioned this pull request Oct 18, 2018

WIP: Distribution fixes from simulation #2526

Merged

5 tasks

cwgoes added 4 commits October 18, 2018 21:08

Merge branch 'develop' into cwgoes/check-supply-in-simulation

f527163

Merge PR #2526: Distribution fixes from simulation

e099491

Merge branch 'develop' into cwgoes/check-supply-in-simulation

06c7e27

Update stake module simulation and fix linter errors

25ce0de

cwgoes changed the title ~~WIP: Simulate transactions for fee distribution~~ R4R: Simulate transactions for fee distribution (and associated fixes) Oct 18, 2018

cwgoes changed the title ~~R4R: Simulate transactions for fee distribution (and associated fixes)~~ WIP: Simulate transactions for fee distribution (and associated fixes) Oct 18, 2018

Simulation fixes work-in-progress

42ae267

Merge branch 'cwgoes/check-supply-in-simulation' of github.com:cosmos…

7f43860

…/cosmos-sdk into cwgoes/check-supply-in-simulation

jaekwon added 2 commits October 22, 2018 17:49

WIP intermediate...

ea565ba

WIP intermediate...

1630c0f

rigelrozanski force-pushed the cwgoes/check-supply-in-simulation branch from 4df3260 to 7f43860 Compare October 23, 2018 02:24

jaekwon and others added 12 commits October 22, 2018 19:48

WIP looking for bug...

fa3730a

simulation stop early with no proposers

5a559f7

re-enable simulation parameters

8999ee8

Merge pull request #2564 from cosmos/rigel/check-supply-in-simulation2

f82d6a5

sim fixes

Merge remote-tracking branch 'origin/jae/check-supply-in-simulation2'…

5790d01

… into cwgoes/check-supply-in-simulation

Remove debugging print statements

be66a49

Back to 50 initially bonded

c14fa12

Merge branch 'develop' into cwgoes/check-supply-in-simulation

33c8238

Update PENDING.md

f587b46

Back to on-operation=false, update PENDING.md

6bf7e02

Remove unnecessary change, clarify amt in sim_test.go

604de85

Cleanup, additional item in PENDING.md

edd56be

cwgoes added ready-for-review and removed wip labels Oct 23, 2018

rigelrozanski suggested changes Oct 23, 2018

View reviewed changes

cwgoes added 2 commits October 23, 2018 20:33

Address @rigelrozanski comments

306fb8f

Make linter happy

cd9373d

rigelrozanski approved these changes Oct 23, 2018

View reviewed changes

Store last total power as sdk.Int, not sdk.Dec

3d270ba

cwgoes merged commit 60d188d into develop Oct 23, 2018

cwgoes deleted the cwgoes/check-supply-in-simulation branch October 23, 2018 19:21

jaekwon reviewed Oct 24, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

R4R: Simulate transactions for fee distribution, simulate inflation, and fix a multitude of bugs discovered in the process of doing so #2501

R4R: Simulate transactions for fee distribution, simulate inflation, and fix a multitude of bugs discovered in the process of doing so #2501

cwgoes commented Oct 16, 2018 •

edited

Loading

rigelrozanski commented Oct 16, 2018

codecov bot commented Oct 16, 2018 •

edited

Loading

cwgoes commented Oct 16, 2018

rigelrozanski commented Oct 17, 2018 •

edited

Loading

rigelrozanski commented Oct 17, 2018 •

edited

Loading

cwgoes commented Oct 18, 2018 •

edited

Loading

cwgoes commented Oct 18, 2018

rigelrozanski commented Oct 19, 2018

cwgoes commented Oct 23, 2018 •

edited

Loading

cwgoes commented Oct 23, 2018

cwgoes commented Oct 23, 2018

rigelrozanski left a comment

jaekwon Oct 24, 2018

cwgoes Oct 24, 2018

R4R: Simulate transactions for fee distribution, simulate inflation, and fix a multitude of bugs discovered in the process of doing so #2501

R4R: Simulate transactions for fee distribution, simulate inflation, and fix a multitude of bugs discovered in the process of doing so #2501

Conversation

cwgoes commented Oct 16, 2018 • edited Loading

rigelrozanski commented Oct 16, 2018

codecov bot commented Oct 16, 2018 • edited Loading

Codecov Report

cwgoes commented Oct 16, 2018

rigelrozanski commented Oct 17, 2018 • edited Loading

rigelrozanski commented Oct 17, 2018 • edited Loading

cwgoes commented Oct 18, 2018 • edited Loading

cwgoes commented Oct 18, 2018

rigelrozanski commented Oct 19, 2018

cwgoes commented Oct 23, 2018 • edited Loading

cwgoes commented Oct 23, 2018

cwgoes commented Oct 23, 2018

rigelrozanski left a comment

Choose a reason for hiding this comment

jaekwon Oct 24, 2018

Choose a reason for hiding this comment

cwgoes Oct 24, 2018

Choose a reason for hiding this comment

cwgoes commented Oct 16, 2018 •

edited

Loading

codecov bot commented Oct 16, 2018 •

edited

Loading

rigelrozanski commented Oct 17, 2018 •

edited

Loading

rigelrozanski commented Oct 17, 2018 •

edited

Loading

cwgoes commented Oct 18, 2018 •

edited

Loading

cwgoes commented Oct 23, 2018 •

edited

Loading