Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add vesting account handling #232

Merged
merged 41 commits into from
Oct 20, 2021
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
97698d5
x/ditr handleMsg add go routines for refreching rewards and commision
huichiaotsou Oct 7, 2021
9a5283e
Merge branch 'v2/cosmos/stargate' into v2/aaron/update_rewards_commis…
huichiaotsou Oct 7, 2021
8e43f3f
merge
huichiaotsou Oct 8, 2021
91176e8
Merge branch 'v2/aaron/update_rewards_commission' of https://github.c…
huichiaotsou Oct 8, 2021
cece5af
move interval check to handle_block
huichiaotsou Oct 9, 2021
371b1a9
update comment
huichiaotsou Oct 9, 2021
4e6ff33
make shouldUpdateDelegatorRewardsAmounts a function
huichiaotsou Oct 12, 2021
ba35512
remove uneccessary update/refresh in handle msg
huichiaotsou Oct 12, 2021
8758b7f
fix no return value for shouldUpdateDelegatorRewardsAmounts
huichiaotsou Oct 12, 2021
30f23c5
add schema vesting_account
huichiaotsou Oct 13, 2021
c1a3588
schema
huichiaotsou Oct 15, 2021
6354fab
Merge branch 'v2/cosmos/stargate' into v2/aaron/vesting_account
huichiaotsou Oct 15, 2021
d600690
add structure in auth/handle_genesis.go
huichiaotsou Oct 15, 2021
e85ecf0
add types for JSON handling and for auth types/auth
huichiaotsou Oct 15, 2021
73719ad
add get genesis vesting accounts method in auth_accounts.go
huichiaotsou Oct 15, 2021
0616f38
add db.SaveVestingAccounts
huichiaotsou Oct 15, 2021
1f9f209
fix schema
huichiaotsou Oct 15, 2021
cc055ba
add schema(move create type COIN to auth.sql)
huichiaotsou Oct 19, 2021
e0e4f64
implement GetGenesisVestingAccounts and use in HandleGenesis
huichiaotsou Oct 19, 2021
6a53241
implement SaveVestingAccounts and the storing of 3 diff. vesting acco…
huichiaotsou Oct 19, 2021
91bdf17
add structs in types/auth.go: ContinuousVestingAccount/ DelayedVestin…
huichiaotsou Oct 19, 2021
563f8ea
modify sql comments
huichiaotsou Oct 19, 2021
7a5d81a
Merge remote-tracking branch 'remotes/origin/v2/aaron/vesting_account…
huichiaotsou Oct 19, 2021
31efda6
delete GetGenesisVestingAccounts from auth_accounts.go
huichiaotsou Oct 19, 2021
918ba27
modif comments GetGenesisVestingAccounts
huichiaotsou Oct 19, 2021
6fcac22
comment modif: Build vestingAccounts Array
huichiaotsou Oct 19, 2021
1b31894
linter issue: vestingAccountId -> vestingAccountID
huichiaotsou Oct 19, 2021
c006195
add unit test
huichiaotsou Oct 19, 2021
d655091
linter: Id -> ID
huichiaotsou Oct 19, 2021
71eb9f2
lint
MonikaCat Oct 19, 2021
600230c
remove fmt.Println
huichiaotsou Oct 20, 2021
39fc74d
fix comments, remove redundant store-to-db methods
huichiaotsou Oct 20, 2021
9be3448
rm unit test for save vesting account ftm
huichiaotsou Oct 20, 2021
234f23b
schema: length TEXT -> BIGINT
huichiaotsou Oct 20, 2021
4df7892
fix go.mod
huichiaotsou Oct 20, 2021
ace57db
rm custom row types from database/types/auth.go
huichiaotsou Oct 20, 2021
2dcdeaf
merge 2 cases(continuous and deleyed vesting account; move storeVesti…
huichiaotsou Oct 20, 2021
decd364
correctly merge 2 cases
huichiaotsou Oct 20, 2021
4d7e1ec
revert DBG to prev. version
huichiaotsou Oct 20, 2021
83c661a
Updated SaveVestingAccounts
MonikaCat Oct 20, 2021
2d39fd4
Merge branch 'v2/cosmos/stargate' into v2/aaron/vesting_account
mergify[bot] Oct 20, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions database/auth.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package database

import (
"encoding/json"
"fmt"

dbutils "github.com/forbole/bdjuno/v2/database/utils"
Expand Down Expand Up @@ -52,6 +53,41 @@ func (db *Db) saveAccounts(paramsNumber int, accounts []types.Account) error {
return nil
}

func (db *Db) SaveVestingAccounts(paramsNumber int, vestingAccount []types.VestingAccount) error {
if len(vestingAccount) == 0 {
return nil
}
stmt := `INSERT INTO vesting_account (address, original_vesting, end_time, start_time, vesting_periods) VALUES `
var params []interface{}

for i, account := range vestingAccount {
ai := i * paramsNumber
stmt += fmt.Sprintf("($%d,$%d,$%d,$%d,$%d),", ai+1, ai+2, ai+3, ai+4, ai+5)

originalVestingBz, err := json.Marshal(account.OriginalVesting)
if err != nil {
return err
}

VestingPeriodsBz, err := json.Marshal(account.VestingPeriods)
if err != nil {
return err
}

params = append(params, account.Address, string(originalVestingBz), account.EndTime, account.StartTime, string(VestingPeriodsBz))
}

stmt = stmt[:len(stmt)-1]
stmt += `ON CONFLICT (address) DO NOTHING`

_, err := db.Sql.Exec(stmt, params...)
huichiaotsou marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return err
}

return nil
}

// GetAccounts returns all the accounts that are currently stored inside the database.
func (db *Db) GetAccounts() ([]string, error) {
var rows []string
Expand Down
10 changes: 10 additions & 0 deletions database/schema/01-auth.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,14 @@
CREATE TABLE account
(
address TEXT NOT NULL PRIMARY KEY
);


CREATE TABLE vesting_account
(
address TEXT NOT NULL REFERENCES account (address) PRIMARY KEY,
original_vesting JSONB NOT NULL DEFAULT '{}',
MonikaCat marked this conversation as resolved.
Show resolved Hide resolved
end_time TIMESTAMP NOT NULL,
MonikaCat marked this conversation as resolved.
Show resolved Hide resolved
start_time TIMESTAMP NOT NULL,
vesting_periods JSONB NOT NULL DEFAULT '{}'
MonikaCat marked this conversation as resolved.
Show resolved Hide resolved
);
37 changes: 37 additions & 0 deletions modules/auth/auth_accounts.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package auth

import (
"encoding/json"
"strings"

"github.com/cosmos/cosmos-sdk/codec"
authttypes "github.com/cosmos/cosmos-sdk/x/auth/types"
Expand Down Expand Up @@ -32,6 +33,42 @@ func GetGenesisAccounts(appState map[string]json.RawMessage, cdc codec.Marshaler
return accounts, nil
}

// GetGenesisVestingAccounts parses the given appState and returns the desired genesis vesting account details
func GetGenesisVestingAccounts(appState map[string]json.RawMessage) ([]types.VestingAccount, error) {
log.Debug().Str("module", "auth (vesting)").Msg("parsing genesis")

var authState Accounts
if err := json.Unmarshal(appState[authttypes.ModuleName], &authState); err != nil {
return nil, err
}

// Store the accounts
accounts := []types.VestingAccount{}
for _, account := range authState.Accounts {
accountType := strings.Split(account.AccountType, ".")[3]

if accountType == "PeriodicVestingAccount" {
MonikaCat marked this conversation as resolved.
Show resolved Hide resolved
// Prepare vesting periods array for creating new Account instance
vestingPeriods := make([]types.VestingPeriod, len(account.VestingPeriods))
for index, period := range account.VestingPeriods {
vestingPeriods[index] = types.NewVestingPeriod(period.Length, period.Amount)
}

// Create new Account instance
vestingAccount := types.NewVestingAccount(
account.BaseVestingAccount.BaseAccount.Address,
account.BaseVestingAccount.OriginalVesting,
account.BaseVestingAccount.EndTime,
account.StartTime,
vestingPeriods,
)
accounts = append(accounts, vestingAccount)
}
}

return accounts, nil
}

// --------------------------------------------------------------------------------------------------------------------

// GetAccounts returns the account data for the given addresses
Expand Down
13 changes: 13 additions & 0 deletions modules/auth/handle_genesis.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
func (m *Module) HandleGenesis(_ *tmtypes.GenesisDoc, appState map[string]json.RawMessage) error {
log.Debug().Str("module", "auth").Msg("parsing genesis")
huichiaotsou marked this conversation as resolved.
Show resolved Hide resolved

// Handle Genesis account addresses
accounts, err := GetGenesisAccounts(appState, m.cdc)
if err != nil {
return fmt.Errorf("error while getting genesis accounts: %s", err)
Expand All @@ -23,5 +24,17 @@ func (m *Module) HandleGenesis(_ *tmtypes.GenesisDoc, appState map[string]json.R
return fmt.Errorf("error while storing genesis accounts: %s", err)
}

// Handle Genesis vesting account details
vestingAccounts, err := GetGenesisVestingAccounts(appState)
if err != nil {
return fmt.Errorf("error while getting genesis vesting accounts: %s", err)
}

paramsNumber := 5
err = m.db.SaveVestingAccounts(paramsNumber, vestingAccounts)
if err != nil {
return fmt.Errorf("error while storing genesis vesting accounts: %s", err)
}

return nil
}
31 changes: 31 additions & 0 deletions modules/auth/types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package auth

import sdk "github.com/cosmos/cosmos-sdk/types"

// --- Types are defined according to genesis file structure ---

type Accounts struct {
MonikaCat marked this conversation as resolved.
Show resolved Hide resolved
Accounts []AccountDetails `json:"accounts"`
}

type AccountDetails struct {
AccountType string `json:"@type"`
BaseVestingAccount BaseVestingAccount `json:"base_vesting_account"`
StartTime string `json:"start_time"`
VestingPeriods []Period `json:"vesting_periods"`
}

type BaseVestingAccount struct {
BaseAccount BaseAccount `json:"base_account"`
OriginalVesting []sdk.Coin `json:"original_vesting"`
EndTime string `json:"end_time"`
}

type BaseAccount struct {
Address string `json:"address"`
}

type Period struct {
Length string `json:"length"`
Amount []sdk.Coin `json:"amount"`
}
49 changes: 49 additions & 0 deletions types/auth.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package types

import sdk "github.com/cosmos/cosmos-sdk/types"

// Account represents a chain account
type Account struct {
Address string
Expand All @@ -11,3 +13,50 @@ func NewAccount(address string) Account {
Address: address,
}
}

// --------------- For Vesting Account ---------------

// Account represents a chain account
type VestingAccount struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, why are these re-declared since they are already available inside the SDK?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @RiccardoM , I added the types on the side because I couldn't really get the account type with Codec.. so I did with brute force to parse them into json. Do you know a way how I can parse/unpack the accounts?

Copy link
Contributor

@RiccardoM RiccardoM Oct 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@huichiaotsou In order to understand how to unpack accounts, it's worth to understand how they are handled from Cosmos and why they are packed in a strange way. The best way to do this is though a series of considerations:

  1. All the data returned from a gRPC call is serialized using Protobuf
  2. In order to represent a generic account, inside the Cosmos SDK there are multiple Go interfaces used. The most common are:
    1. AccountI, to represent a generic account
    2. VestingAccount to represent a generic vesting account
  3. Protobuf does not support Go interfaces by default. For this reason, the Cosmos team has decided to use the Any type to serialize them.

So, in order to read the details of a vesting account you need to do the following:

  1. Unpack that account from Any into an AccountI interface:
var accountI authtypes.AccountI
err := cdc.UnpackAny(account, &accountI)
if err != nil {
  return err
}
  1. Check if that accountI is also implementing the VestingAccountI interface:
vestingAccount, ok := accountI.(exported.VestingAccount)
if !ok {
  // The account is not a VestingAccount. Do as you wish (skip the for loop, return an error, ...)
}

From there, you can use vestingAccount however you wish since it will be of type VestingAccount.

Side note

Since they can be of various types (ContinousVestingAccount, DelayedVestingAccount and PeriodicVestingAccount), the best way to store VestingAccount instances is to simply serialize them as JSON and store them inside a table. However, this might make it really hard to get some useful insights that we might want to display on BigDipper (such as total vested/unvested tokens, next unvesting period, etc).

For this reason, I would like to ask @ryuash and @calvinkei how they are going to use the vesting data in order to decide how to store them accordingly.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default Big Dipper 2.0 will not be using this data in favor of Forbole X.
Inside the Forbole X wallet details page I would assume it would show xx tokens are locked until xx (maybe a countdown type of animation)

Maybe @calvinkei can clarify

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ryuash Wouldn't it be interesting for BigDipper to show a global amount of locked/unlocked tokens? Asking @kwunyeung also.

That might be interesting for users, no? To know when the next large unlock will happen or other interesting things

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the UI for showing vesting tokens on forbole x
Screenshot 2021-10-18 at 3 30 30 PM

So, it needs to support "total vesting tokens" and pagination. Probably filter & sort by date as well

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hahs, if we do some feature like that then we definitely cannot throw a JSONB in to psql.

I would need to get all possible vesting periods in the genesis file and query by xx time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@huichiaotsou In order to preserve forward-compatibility with new features that might come up I would suggest then adding a better handling for different vesting account types. We can do so by doing this:

// inside handler
db.StoreVestingAccount(vestingAccount)

// -------------------------

func (db *Database) StoreVestingAccount(vestingAccount exported.VestingAccount) error {
  if continousVestingAccount, ok := vestingAccount.(*vestingtypes.ContinousVestingAccount); ok {
    return db.storeContinousVestingAccount(continousVestingAccount)
  }

  if delayedVestingAccount, ok := vestingAccount.(*vestingtypes.DelayedVestingAccount); ok {
    return db.storeDelayedVestingAccount(continousVestingAccount)
  }

  if periodicVestingAccount, ok := vestingAccount.(*vestingtypes.PeriodicVestingAccount); ok {
    return db.storePeriodicVestingAccount(continousVestingAccount)
  }

  return fmt.Errorf("invalid vesting account type: %T", vestingAccount")

This way we can handling the storing of different types properly, with various periods etc

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RiccardoM I'm also interested in showing vesting amount on BD but I do agree with @ryuash that it could be in favour of Forbole X at some point if BD not showing them by default. I think we can have this feature showing on X first. I also wanna see if there will be any feedback or request on getting this feature on BD.

Agree not storing the vesting periods in JSONB as the structure is very clear that each record has an array of coins and a vested date. It's not any arbitrary json content where we may search in certain keys inside the json.

Address string
OriginalVesting sdk.Coins
EndTime string
StartTime string
VestingPeriods []VestingPeriod
}

// NewAccount builds a new VestingAccount instance
func NewVestingAccount(address string, originalVesting sdk.Coins, endTime string, startTime string, vestingPeriods []VestingPeriod) VestingAccount {
return VestingAccount{
Address: address,
OriginalVesting: originalVesting,
EndTime: endTime,
StartTime: startTime,
VestingPeriods: vestingPeriods,
}
}

func (u VestingAccount) Equal(v VestingAccount) bool {
for index, periodU := range u.VestingPeriods {
periodV := v.VestingPeriods[index]
if periodU.Amounts.String() != periodV.Amounts.String() {
return false
}
}
return u.Address == v.Address &&
u.OriginalVesting.String() == v.OriginalVesting.String() &&
u.EndTime == v.EndTime &&
u.StartTime == v.StartTime
}

type VestingPeriod struct {
Length string
Amounts sdk.Coins
}

func NewVestingPeriod(length string, amounts sdk.Coins) VestingPeriod {
return VestingPeriod{
Length: length,
Amounts: amounts,
}
}