Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protorev: Use hooks instead of message parsing to trigger backruns #5045

Merged
merged 76 commits into from
Jun 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
02ddd55
Add input and output
NotJeremyLiu Apr 18, 2023
75ea9cc
Update modules that use hook
NotJeremyLiu Apr 18, 2023
d556709
Merge branch 'main' into jl/add-in-and-out-coins-to-cl-swap-listener
NotJeremyLiu Apr 26, 2023
772af1b
Create new kvstore and methods for swpsToBackrun
NotJeremyLiu Apr 17, 2023
089d091
Connect protorev to cfmm and cl hooks
NotJeremyLiu Apr 17, 2023
1a6ed93
Implement CFMMHook logic in protorev
NotJeremyLiu Apr 17, 2023
dcc5549
Update comments, restrict join/exit pool logic to single denom in/out
NotJeremyLiu Apr 17, 2023
616f5dc
Remove old placeholder print
NotJeremyLiu Apr 17, 2023
ef2d317
Change init function to reset
NotJeremyLiu Apr 17, 2023
9e71597
add input and output coins in protorev cl hook
NotJeremyLiu Apr 27, 2023
99d8e9b
add necessary poolmanager expected keepers
NotJeremyLiu Apr 27, 2023
3212d18
add hook logic for cl pools and refactor
NotJeremyLiu Apr 27, 2023
9a73b34
implement new hook-based logic in the posthandler
NotJeremyLiu Apr 27, 2023
d89db40
Merge branch 'main' into jl/protorev-hook-based-logic
NotJeremyLiu May 2, 2023
a1b18ab
Merge branch 'main' into jl/protorev-hook-based-logic
NotJeremyLiu May 2, 2023
80a3db1
Merge branch 'main' into jl/protorev-hook-based-logic
NotJeremyLiu May 18, 2023
26c46a1
no-op unused hooks
NotJeremyLiu May 18, 2023
43b62ef
Remove ResetSwapToBackrun, only use DeleteSwapsToBackrun
NotJeremyLiu May 18, 2023
6c8ab71
Add swap tests for hooks
NotJeremyLiu May 18, 2023
fa03d8e
Add join/exit pool hook tests
NotJeremyLiu May 19, 2023
cc5f288
Add pool creation hook test
NotJeremyLiu May 19, 2023
8938ba5
Add pool creation tests, handle CL pools differently due to 0 liquidi…
NotJeremyLiu May 22, 2023
164615f
Update ExtractSwappedPools to no longer need tx
NotJeremyLiu May 22, 2023
d291e69
Update TestExtractSwappedPools for new hook-based logic
NotJeremyLiu May 22, 2023
087cc7c
Delete swaps without using user gas
NotJeremyLiu May 22, 2023
3e259bf
Update TestAnteHandle with new hook-based logic
NotJeremyLiu May 22, 2023
9d28bff
Update expected liquidity value for CL pool
NotJeremyLiu May 22, 2023
23c1396
Merge branch 'main' into jl/protorev-hook-based-logic
NotJeremyLiu May 22, 2023
840416e
Add changelog entry
NotJeremyLiu May 22, 2023
6c1944f
Remove unused messages in TestAnteHandle
NotJeremyLiu May 22, 2023
225f1b9
Remove no longer used functions
NotJeremyLiu May 22, 2023
cb4c85b
Check len on all hooks to avoid unintended behavior
NotJeremyLiu May 22, 2023
12b5c7d
lint
NotJeremyLiu May 22, 2023
65356d9
Move all mul logic into same fn in hook logic afterpoolCreated, add p…
NotJeremyLiu May 23, 2023
523c711
Separate getCoins logic into separate function
NotJeremyLiu May 23, 2023
031a52b
fix lint
NotJeremyLiu May 23, 2023
e2729f4
return sdk.Coins{} instead of nil
NotJeremyLiu May 23, 2023
65f5684
Merge branch 'main' into jl/protorev-hook-based-logic
NotJeremyLiu May 26, 2023
92933f1
remove unused import
NotJeremyLiu May 26, 2023
f88995c
Remove unnecessary code
NotJeremyLiu May 26, 2023
b1f8068
Merge branch 'main' into jl/protorev-hook-based-logic
NotJeremyLiu May 26, 2023
4c97712
clarify comment
NotJeremyLiu May 30, 2023
6c744ac
use NewCoins instead of Coins
NotJeremyLiu May 30, 2023
fe5f855
Merge branch 'main' into jl/protorev-hook-based-logic
NotJeremyLiu May 30, 2023
6faeab3
Revert "use NewCoins instead of Coins"
NotJeremyLiu May 30, 2023
6d7d411
add inline comment for why no return in err
NotJeremyLiu May 30, 2023
66c1e8c
use SetupTest instead of custom logic
NotJeremyLiu Jun 1, 2023
5363319
Restructure protorev hook updating to only process pools with coins
NotJeremyLiu Jun 1, 2023
e3a9998
Use denoms first, then coins in pool updating
NotJeremyLiu Jun 1, 2023
319f0fc
remove comment
NotJeremyLiu Jun 1, 2023
91f16f0
use poolmanager to get liquidity to avoid issues with CL
NotJeremyLiu Jun 2, 2023
c24c3b9
Add CL first, then balancer (and vice-versa) pool creation tests
NotJeremyLiu Jun 5, 2023
169006a
Merge branch 'main' into jl/protorev-hook-based-logic
NotJeremyLiu Jun 5, 2023
93e1edd
v16 version bump
NotJeremyLiu Jun 5, 2023
817fb28
implement ok checking pattern for map check
NotJeremyLiu Jun 5, 2023
5bda10f
fix typo
NotJeremyLiu Jun 5, 2023
7a69efa
Use expectPass bool in test
NotJeremyLiu Jun 5, 2023
2873238
reuse k.storeSwap in join/exit logic
NotJeremyLiu Jun 5, 2023
5393e78
Add direct unit test for StoreSwap helper method
NotJeremyLiu Jun 5, 2023
3aa26ea
Add direct unit test for GetComparablePoolLiquidity helper method
NotJeremyLiu Jun 5, 2023
7fc81f8
Add direct unit test for StoreJoinExitPoolSwaps helper method
NotJeremyLiu Jun 5, 2023
da7c839
Remove old function
NotJeremyLiu Jun 5, 2023
924a749
Add direct unit test for CompareAndStorePool helper method
NotJeremyLiu Jun 5, 2023
7fcdb55
Actually test CompareAndStorePool
NotJeremyLiu Jun 5, 2023
a0d2a3b
export AfterPoolCreatedWithCoins
NotJeremyLiu Jun 5, 2023
e5d17c6
Clarify why the test calls DeleteAllPoolsForBaseDenom
NotJeremyLiu Jun 5, 2023
22d528e
Move panic catching into GetComparablePoolLiquidity
NotJeremyLiu Jun 6, 2023
ed66bce
Add int overflow panic catching test
NotJeremyLiu Jun 6, 2023
61e7076
Revert "Add int overflow panic catching test"
NotJeremyLiu Jun 6, 2023
995b115
Add int overflow panic catch test - try 2
NotJeremyLiu Jun 6, 2023
026fe77
Add custom error message
NotJeremyLiu Jun 6, 2023
c18ec19
Add overflow catching logical branch tests for CompareAndStorePool
NotJeremyLiu Jun 6, 2023
c2db6d7
Add non-gamm pool error test for StoreJoinExitPoolSwaps()
NotJeremyLiu Jun 6, 2023
460da4a
Use s.SetupTest() and give testAcc pool shares
NotJeremyLiu Jun 6, 2023
61d3474
Add clarifying comment about gas meter
NotJeremyLiu Jun 6, 2023
8966b6a
Add comment explaining the base denoms check -> CompareAndStorePool l…
NotJeremyLiu Jun 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* [#4830](https://github.com/osmosis-labs/osmosis/pull/4830) Add gas cost when we AddToGaugeRewards, linearly increase with coins to add
* [#4886](https://github.com/osmosis-labs/osmosis/pull/4886) Implement MsgSplitRouteSwapExactAmountIn and MsgSplitRouteSwapExactAmountOut that supports route splitting.
* [#5000](https://github.com/osmosis-labs/osmosis/pull/5000) osmomath.Power panics for base < 1 to temporarily restrict broken logic for such base.
* [#5045] (https://github.com/osmosis-labs/osmosis/pull/5045) Implement hook-based backrunning logic for ProtoRev
* [#5281](https://github.com/osmosis-labs/osmosis/pull/5281) Add option to designate Reward Recipient to Lock and Incentives.
* [#4827] (https://github.com/osmosis-labs/osmosis/pull/4827) Protorev: Change highest liquidity pool updating from weekly to daily and change dev fee payout from weekly to after every trade.

Expand Down
2 changes: 2 additions & 0 deletions app/keepers/keepers.go
Original file line number Diff line number Diff line change
Expand Up @@ -685,13 +685,15 @@ func (appKeepers *AppKeepers) SetupHooks() {
// insert gamm hooks receivers here
appKeepers.PoolIncentivesKeeper.Hooks(),
appKeepers.TwapKeeper.GammHooks(),
appKeepers.ProtoRevKeeper.Hooks(),
),
)

appKeepers.ConcentratedLiquidityKeeper.SetListeners(
concentratedliquiditytypes.NewConcentratedLiquidityListeners(
appKeepers.TwapKeeper.ConcentratedLiquidityListener(),
appKeepers.PoolIncentivesKeeper.Hooks(),
appKeepers.ProtoRevKeeper.Hooks(),
),
)

Expand Down
2 changes: 1 addition & 1 deletion x/protorev/keeper/epoch_hook_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ func (s *KeeperTestSuite) TestUpdateHighestLiquidityPools() {
},
expectedBaseDenomPools: map[string]map[string]keeper.LiquidityPoolStruct{
"epochTwo": {
"uosmo": {Liquidity: sdk.NewInt(2000000), PoolId: 49},
"uosmo": {Liquidity: sdk.Int(sdk.NewUintFromString("999999999000000001000000000000000000")), PoolId: 49},
stackman27 marked this conversation as resolved.
Show resolved Hide resolved
},
},
},
Expand Down
222 changes: 222 additions & 0 deletions x/protorev/keeper/hooks.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
package keeper

import (
"errors"

sdk "github.com/cosmos/cosmos-sdk/types"

gammtypes "github.com/osmosis-labs/osmosis/v16/x/gamm/types"
"github.com/osmosis-labs/osmosis/v16/x/protorev/types"
)

type Hooks struct {
k Keeper
}

var (
_ gammtypes.GammHooks = Hooks{}
)

// Create new ProtoRev hooks.
func (k Keeper) Hooks() Hooks { return Hooks{k} }

// ----------------------------------------------------------------------------
// GAMM HOOKS
// ----------------------------------------------------------------------------

// AfterCFMMPoolCreated hook checks and potentially stores the pool via the highest liquidity method.
func (h Hooks) AfterCFMMPoolCreated(ctx sdk.Context, sender sdk.AccAddress, poolId uint64) {
NotJeremyLiu marked this conversation as resolved.
Show resolved Hide resolved
h.k.AfterPoolCreatedWithCoins(ctx, poolId)
}

// AfterJoinPool stores swaps to be checked by protorev given the coins entered into the pool.
func (h Hooks) AfterJoinPool(ctx sdk.Context, sender sdk.AccAddress, poolId uint64, enterCoins sdk.Coins, shareOutAmount sdk.Int) {
// Checked to avoid future unintended behavior based on how the hook is called
if len(enterCoins) != 1 {
return
}

h.k.StoreJoinExitPoolSwaps(ctx, sender, poolId, enterCoins[0].Denom, true)
}

// AfterExitPool stores swaps to be checked by protorev given the coins exited from the pool.
func (h Hooks) AfterExitPool(ctx sdk.Context, sender sdk.AccAddress, poolId uint64, shareInAmount sdk.Int, exitCoins sdk.Coins) {
// Added due to ExitSwapShareAmountIn both calling
// ExitPoolHook with all denoms of the pool and then also
// Swapping which triggers the after swap hook.
// So this filters out the exit pool hook call with all denoms
if len(exitCoins) != 1 {
return
}

h.k.StoreJoinExitPoolSwaps(ctx, sender, poolId, exitCoins[0].Denom, false)
}

// AfterCFMMSwap stores swaps to be checked by protorev given the coins swapped in the pool.
func (h Hooks) AfterCFMMSwap(ctx sdk.Context, sender sdk.AccAddress, poolId uint64, input sdk.Coins, output sdk.Coins) {
// Checked to avoid future unintended behavior based on how the hook is called
if len(input) != 1 || len(output) != 1 {
return
}

h.k.StoreSwap(ctx, poolId, input[0].Denom, output[0].Denom)
}

// ----------------------------------------------------------------------------
// CONCENTRATED LIQUIDITY HOOKS
// ----------------------------------------------------------------------------

// AfterConcentratedPoolCreated is a noop.
func (h Hooks) AfterConcentratedPoolCreated(ctx sdk.Context, sender sdk.AccAddress, poolId uint64) {
}

// AfterInitialPoolPositionCreated checks and potentially stores the pool via the highest liquidity method.
func (h Hooks) AfterInitialPoolPositionCreated(ctx sdk.Context, sender sdk.AccAddress, poolId uint64) {
h.k.AfterPoolCreatedWithCoins(ctx, poolId)
}

// AfterLastPoolPositionRemoved is a noop.
func (h Hooks) AfterLastPoolPositionRemoved(ctx sdk.Context, sender sdk.AccAddress, poolId uint64) {
}

// AfterConcentratedPoolSwap stores swaps to be checked by protorev given the coins swapped in the pool.
func (h Hooks) AfterConcentratedPoolSwap(ctx sdk.Context, sender sdk.AccAddress, poolId uint64, input sdk.Coins, output sdk.Coins) {
// Checked to avoid future unintended behavior based on how the hook is called
if len(input) != 1 || len(output) != 1 {
return
}

h.k.StoreSwap(ctx, poolId, input[0].Denom, output[0].Denom)
}

// ----------------------------------------------------------------------------
// HELPER METHODS
// ----------------------------------------------------------------------------

// StoreSwap stores a swap to be checked by protorev when attempting backruns.
func (k Keeper) StoreSwap(ctx sdk.Context, poolId uint64, tokenIn, tokenOut string) {
swapToBackrun := types.Trade{
Pool: poolId,
TokenIn: tokenIn,
TokenOut: tokenOut,
}

if err := k.AddSwapsToSwapsToBackrun(ctx, []types.Trade{swapToBackrun}); err != nil {
ctx.Logger().Error("Protorev error adding swap to backrun from storeSwap", err) // Does not return since logging is last thing in the function
}
}

// GetComparablePoolLiquidity gets the comparable liquidity of a pool by multiplying the amounts of the pool coins.
func (k Keeper) GetComparablePoolLiquidity(ctx sdk.Context, poolId uint64) (comparableLiquidity sdk.Int, err error) {
coins, err := k.poolmanagerKeeper.GetTotalPoolLiquidity(ctx, poolId)
if err != nil {
return sdk.Int{}, err
}

// Recover from overflow panic
defer func() {
if r := recover(); r != nil {
comparableLiquidity = sdk.Int{}
err = errors.New("Int overflow in GetComparablePoolLiquidity")
}
}()

comparableLiquidity = coins[0].Amount.Mul(coins[1].Amount)

return comparableLiquidity, nil
}

// StoreJoinExitPoolSwaps stores the swaps associated with GAMM join/exit pool messages in the store, depending on if it is a join or exit.
func (k Keeper) StoreJoinExitPoolSwaps(ctx sdk.Context, sender sdk.AccAddress, poolId uint64, denom string, isJoin bool) {
pool, err := k.gammKeeper.GetPoolAndPoke(ctx, poolId)
if err != nil {
return
}

// Get all the pool coins and iterate to get the denoms that make up the swap
coins := pool.GetTotalPoolLiquidity(ctx)

// Create swaps to backrun being the join coin swapped against all other pool coins
for _, coin := range coins {
if coin.Denom == denom {
continue
}
// Join messages swap in the denom, exit messages swap out the denom
if isJoin {
k.StoreSwap(ctx, poolId, denom, coin.Denom)
} else {
k.StoreSwap(ctx, poolId, coin.Denom, denom)
}
}
}

// AfterPoolCreatedWithCoins checks if the new pool should be stored as the highest liquidity pool
// for any of the base denoms, and stores it if so.
func (k Keeper) AfterPoolCreatedWithCoins(ctx sdk.Context, poolId uint64) {
baseDenoms, err := k.GetAllBaseDenoms(ctx)
if err != nil {
ctx.Logger().Error("Protorev error getting base denoms in AfterCFMMPoolCreated hook", err)
return
}

baseDenomMap := make(map[string]bool)
for _, baseDenom := range baseDenoms {
baseDenomMap[baseDenom.Denom] = true
}

pool, err := k.poolmanagerKeeper.GetPool(ctx, poolId)
if err != nil {
ctx.Logger().Error("Protorev error getting pool in AfterCFMMPoolCreated hook", err)
return
}

denoms, err := k.poolmanagerKeeper.RouteGetPoolDenoms(ctx, poolId)
if err != nil {
ctx.Logger().Error("Protorev error getting pool liquidity in afterPoolCreated", err)
return
}

// Pool must be active and the number of denoms must be 2
if pool.IsActive(ctx) && len(denoms) == 2 {
// Check if either of the denoms are base denoms (denoms in which we store highest liquidity
// pools for to create backrun routes). If so, we call CompareAndStorePool which will check
// if a pool already exists for the base denom pair, and if not, stores the new pool.
// If a pool does already exist for the base denom pair, it will compare the liquidity
// of the new pool with the stored pool, and store the new pool if it has more liquidity.
if _, ok := baseDenomMap[denoms[0]]; ok {
k.CompareAndStorePool(ctx, poolId, denoms[0], denoms[1])
}
if _, ok := baseDenomMap[denoms[1]]; ok {
k.CompareAndStorePool(ctx, poolId, denoms[1], denoms[0])
}
p0mvn marked this conversation as resolved.
Show resolved Hide resolved
}
}

// CompareAndStorePool compares the liquidity of the new pool with the liquidity of the stored pool, and stores the new pool if it has more liquidity.
func (k Keeper) CompareAndStorePool(ctx sdk.Context, poolId uint64, baseDenom, otherDenom string) {
storedPoolId, err := k.GetPoolForDenomPair(ctx, baseDenom, otherDenom)
if err != nil {
// Error means no pool exists for this pair, so we set it
k.SetPoolForDenomPair(ctx, baseDenom, otherDenom, poolId)
return
}
p0mvn marked this conversation as resolved.
Show resolved Hide resolved

// Get comparable liquidity for the new pool
newPoolLiquidity, err := k.GetComparablePoolLiquidity(ctx, poolId)
if err != nil {
ctx.Logger().Error("Protorev error getting newPoolLiquidity in compareAndStorePool", err)
return
p0mvn marked this conversation as resolved.
Show resolved Hide resolved
}

// Get comparable liquidity for the stored pool
storedPoolLiquidity, err := k.GetComparablePoolLiquidity(ctx, storedPoolId)
if err != nil {
ctx.Logger().Error("Protorev error getting storedPoolLiquidity in compareAndStorePool", err)
return
}
p0mvn marked this conversation as resolved.
Show resolved Hide resolved

// If the new pool has more liquidity, we set it
if newPoolLiquidity.GT(storedPoolLiquidity) {
k.SetPoolForDenomPair(ctx, baseDenom, otherDenom, poolId)
}
}
Loading