Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Off-chain router V20 release line #6785

Closed
wants to merge 156 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
156 commits
Select commit Hold shift + click to select a range
6fe6870
V20 SQS POC
p0mvn Oct 30, 2023
6932c71
logs for route debugging
p0mvn Oct 30, 2023
52b7b79
CLI for full range liquidity and return bucker index
p0mvn Oct 30, 2023
38931cb
fix localosmosis position creation
p0mvn Oct 31, 2023
3cc8840
fix GetTickLiquidityForFullRange query
p0mvn Oct 31, 2023
6ecdeb9
add rating logic for ordering initial pools
p0mvn Oct 31, 2023
df277bc
update localosmosis with stableswap pool
p0mvn Nov 1, 2023
5d0236d
add support for stableswap pools
p0mvn Nov 1, 2023
58632bf
document CLI for transmuter testing
p0mvn Nov 1, 2023
925a449
fix bug with ingesting cw pools and add info ingester logs
p0mvn Nov 1, 2023
4cf2c4b
add transmuter support
p0mvn Nov 1, 2023
5c0c1c4
initial taker fee implementation
p0mvn Nov 5, 2023
520aa15
utilize asset list from chain registry for getting token precision fo…
p0mvn Nov 5, 2023
c35548d
lower routes config
p0mvn Nov 6, 2023
57b9cdf
makefile changes
p0mvn Nov 6, 2023
a241634
tweak config params
p0mvn Nov 6, 2023
9861e75
various fixes and logs from debugging mainnet
p0mvn Nov 7, 2023
b98e355
pipe logs to file set via env
p0mvn Nov 7, 2023
bbfe049
use custom logger on ingester
p0mvn Nov 7, 2023
71732a2
move sqs logger up to app
p0mvn Nov 7, 2023
9d77d75
log level via env var
p0mvn Nov 7, 2023
57364be
propagate tvl error string instead of boolean flag
p0mvn Nov 7, 2023
4b299cb
fix bug with retrieving and parsing exponents
p0mvn Nov 7, 2023
0930fce
logger configs
p0mvn Nov 7, 2023
57a7957
debug route panic
p0mvn Nov 7, 2023
ed4200a
create GetPoolDenoms method on PoolI
p0mvn Nov 8, 2023
c8f721c
fix bug in denom parsing
p0mvn Nov 8, 2023
602ea09
format results per FE requirements
p0mvn Nov 9, 2023
bf91853
move quote into its own file
p0mvn Nov 9, 2023
2aa8652
implement effective swap fee
p0mvn Nov 9, 2023
c896087
implement effective spread factor
p0mvn Nov 9, 2023
f6d85cd
effective fee
p0mvn Nov 9, 2023
4bb14ed
fix issues around TVL sorting, add debugging helpers for logging
p0mvn Nov 10, 2023
6f9e0b1
refactor to extract routable pool implementations into their own package
p0mvn Nov 10, 2023
cf376ce
lint
p0mvn Nov 11, 2023
39afc35
fix all lints
p0mvn Nov 11, 2023
8d6b187
minor fixes
p0mvn Nov 13, 2023
8a400e3
begin implementing barebones logic for caching
p0mvn Nov 16, 2023
1d60568
custom serializer
p0mvn Nov 16, 2023
b6ad16d
updated
gzukel Nov 16, 2023
85cd200
updated
gzukel Nov 16, 2023
47097da
updated
gzukel Nov 16, 2023
5ee6d46
updated
gzukel Nov 16, 2023
89b75d9
updated
gzukel Nov 16, 2023
ed2f901
Revert "custom serializer"
p0mvn Nov 16, 2023
85f3edd
updated
gzukel Nov 16, 2023
4ef2724
updated, testing watch tower after re-deploy
gzukel Nov 17, 2023
62e8d63
add infrastructure for getting pools from store
p0mvn Nov 17, 2023
703bee5
mainnet Go test pools infrastructure. Inverse candidate routes. Broke…
p0mvn Nov 17, 2023
c40fd0a
Merge branch 'roman/sqs-poc-v20' of github.com:osmosis-labs/osmosis i…
p0mvn Nov 17, 2023
9331f42
basic tests from mainnet set up
p0mvn Nov 19, 2023
9b6383f
move router methods
p0mvn Nov 19, 2023
678301f
updates
p0mvn Nov 21, 2023
def9af9
remove dfs routes
p0mvn Nov 21, 2023
a592f7d
rename bfs to candidate routes
p0mvn Nov 21, 2023
59998d8
V20 SQS POC
p0mvn Oct 30, 2023
32f576e
logs for route debugging
p0mvn Oct 30, 2023
ab267be
CLI for full range liquidity and return bucker index
p0mvn Oct 30, 2023
dbca54b
fix localosmosis position creation
p0mvn Oct 31, 2023
f93754a
fix GetTickLiquidityForFullRange query
p0mvn Oct 31, 2023
8d951ff
add rating logic for ordering initial pools
p0mvn Oct 31, 2023
398018d
update localosmosis with stableswap pool
p0mvn Nov 1, 2023
0319571
add support for stableswap pools
p0mvn Nov 1, 2023
0e8a267
document CLI for transmuter testing
p0mvn Nov 1, 2023
01df30c
fix bug with ingesting cw pools and add info ingester logs
p0mvn Nov 1, 2023
e57a8d8
add transmuter support
p0mvn Nov 1, 2023
05d91c4
initial taker fee implementation
p0mvn Nov 5, 2023
51ed5d5
utilize asset list from chain registry for getting token precision fo…
p0mvn Nov 5, 2023
dbee820
lower routes config
p0mvn Nov 6, 2023
bb77738
makefile changes
p0mvn Nov 6, 2023
b5879a2
tweak config params
p0mvn Nov 6, 2023
09c816a
various fixes and logs from debugging mainnet
p0mvn Nov 7, 2023
7eccf87
pipe logs to file set via env
p0mvn Nov 7, 2023
c6033f8
use custom logger on ingester
p0mvn Nov 7, 2023
026ca9e
move sqs logger up to app
p0mvn Nov 7, 2023
2e54728
log level via env var
p0mvn Nov 7, 2023
ef1965e
propagate tvl error string instead of boolean flag
p0mvn Nov 7, 2023
a443780
fix bug with retrieving and parsing exponents
p0mvn Nov 7, 2023
50d0917
logger configs
p0mvn Nov 7, 2023
bd3b505
debug route panic
p0mvn Nov 7, 2023
db12f19
create GetPoolDenoms method on PoolI
p0mvn Nov 8, 2023
83bda04
fix bug in denom parsing
p0mvn Nov 8, 2023
9108455
format results per FE requirements
p0mvn Nov 9, 2023
ff0e042
move quote into its own file
p0mvn Nov 9, 2023
ff35bd4
implement effective swap fee
p0mvn Nov 9, 2023
b0d5b20
implement effective spread factor
p0mvn Nov 9, 2023
00cedd0
effective fee
p0mvn Nov 9, 2023
8f9c592
fix issues around TVL sorting, add debugging helpers for logging
p0mvn Nov 10, 2023
2870d4d
refactor to extract routable pool implementations into their own package
p0mvn Nov 10, 2023
6d36540
lint
p0mvn Nov 11, 2023
5edb1bb
fix all lints
p0mvn Nov 11, 2023
9a4a7d3
minor fixes
p0mvn Nov 13, 2023
51cb692
begin implementing barebones logic for caching
p0mvn Nov 16, 2023
a6fbb9a
custom serializer
p0mvn Nov 16, 2023
2da2f79
updated
gzukel Nov 16, 2023
66b735a
updated
gzukel Nov 16, 2023
7197f41
updated
gzukel Nov 16, 2023
5317aff
updated
gzukel Nov 16, 2023
eb9485f
updated
gzukel Nov 16, 2023
00fea0c
Revert "custom serializer"
p0mvn Nov 16, 2023
6895295
add infrastructure for getting pools from store
p0mvn Nov 17, 2023
9b7042b
mainnet Go test pools infrastructure. Inverse candidate routes. Broke…
p0mvn Nov 17, 2023
dfc16a8
updated
gzukel Nov 16, 2023
8b710c3
updated, testing watch tower after re-deploy
gzukel Nov 17, 2023
3e415bc
basic tests from mainnet set up
p0mvn Nov 19, 2023
208f52a
move router methods
p0mvn Nov 19, 2023
fb5307d
updates
p0mvn Nov 21, 2023
8475bfc
remove dfs routes
p0mvn Nov 21, 2023
53cc66e
rename bfs to candidate routes
p0mvn Nov 21, 2023
c553b6d
add route filtering and test USD / UMEE pair
p0mvn Nov 21, 2023
5e6fe94
Merge branch 'roman/sqs-poc-v20' of github.com:osmosis-labs/osmosis i…
p0mvn Nov 21, 2023
4461286
fix bug in split routes
p0mvn Nov 22, 2023
d2be244
initial locustfile configuration
p0mvn Nov 22, 2023
f2b1619
add router to load testing
p0mvn Nov 22, 2023
4ae4a7a
stage in locust
p0mvn Nov 22, 2023
bf9a889
stage in locust
p0mvn Nov 22, 2023
ea721d5
refactor routable pools to remove superfluos interfaces
p0mvn Nov 23, 2023
0dcbfd6
remove Router interface from APIs
p0mvn Nov 23, 2023
ea13771
write all pools to the same index
p0mvn Nov 23, 2023
c73aa86
delete Reverse
p0mvn Nov 23, 2023
5f50a4f
implement profiler and add a makefile command
p0mvn Nov 23, 2023
47248ba
Roman/sqs tickmodel refactor (#6920)
p0mvn Nov 26, 2023
b384c7f
update to v16.x (#6926)
czarcas7ic Nov 25, 2023
9433549
fix caching
p0mvn Nov 26, 2023
d37ab22
avoid denom sorting for routes
p0mvn Nov 26, 2023
4ae804c
add basic development environment docs
p0mvn Nov 27, 2023
1fead6d
makefile for mainnet state updates; minor bug fixes, update mainnet s…
p0mvn Nov 27, 2023
9dae009
Merge branch 'roman/sqs-poc-v20' of github.com:osmosis-labs/osmosis i…
p0mvn Nov 27, 2023
e635782
updated readme with space to trigger docker image
gzukel Nov 27, 2023
989fee5
update test pool state
p0mvn Nov 29, 2023
509eb5d
fix mainnet state makefile
p0mvn Nov 29, 2023
d47e1bb
remove CL validation in pool wrapper
p0mvn Nov 29, 2023
d03d0c4
ATOM/USDT candidate routes test
p0mvn Nov 29, 2023
cc1c077
reimplement splits with dynamic programming
p0mvn Nov 29, 2023
785f169
remove old split logic
p0mvn Nov 29, 2023
b061a7b
Merge branch 'roman/sqs-poc-v20' of github.com:osmosis-labs/osmosis i…
p0mvn Nov 29, 2023
5826cb2
Add healthcheck endpoint to sqs (#6935)
mattverse Nov 29, 2023
be7f7d1
Merge branch 'roman/sqs-poc-v20' of github.com:osmosis-labs/osmosis i…
p0mvn Nov 29, 2023
1fd2952
implement metrics endpoint
p0mvn Nov 29, 2023
6a1b078
Merge branch 'v20.x' into roman/sqs-poc-v20
p0mvn Nov 30, 2023
4887905
lint
p0mvn Nov 30, 2023
f82e283
lint more
p0mvn Nov 30, 2023
661e20d
lint more
p0mvn Nov 30, 2023
4ff5800
more lint
p0mvn Nov 30, 2023
f2941f5
lint
p0mvn Nov 30, 2023
e50f7cd
fix test
p0mvn Nov 30, 2023
8fb35a0
fix test
p0mvn Nov 30, 2023
9c4413d
fix test
p0mvn Nov 30, 2023
d7ccd64
Merge branch 'v20.x' into roman/sqs-poc-v20
p0mvn Nov 30, 2023
1404e24
Merge branch 'v20.x' into roman/sqs-poc-v20
p0mvn Nov 30, 2023
3c2d3fb
Merge branch 'v20.x' into roman/sqs-poc-v20
p0mvn Dec 1, 2023
a99eabd
refactor(SQS): app wiring and config v20.x (#6965)
p0mvn Dec 1, 2023
b5045ee
Merge branch 'v20.x' into roman/sqs-poc-v20
p0mvn Dec 1, 2023
7c5abcf
remove unused mock
p0mvn Dec 1, 2023
b9b7621
fix healthcheck
p0mvn Dec 2, 2023
f19fcf8
Merge branch 'v20.x' into roman/sqs-poc-v20
p0mvn Dec 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions .github/workflows/build-push-sqs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# This workflow pushes new docker images to osmolabs/sqs-dev on these events:
# 2. Every new commit to the roman/sqs-poc-v20 branch,
# tag with: sqs-v[0-9]+.[0-9]+.[0-9],
# branch push with sqs-v[0-9]+.x
# `osmolabs/sqs-dev:{SHORT_SHA}-$(date +%s)` is pushed.
# All the images above have support for linux/amd64 (not linux/arm64).
# All the images are based on an alpine image for easy debugging.

name: Build and Push SQS Images

on:
workflow_dispatch:
push:
tags:
- sqs-v[0-9]+.[0-9]+.[0-9]
branches:
- roman/sqs-poc-v20
- sqs-v[0-9]+.x

env:
RUNNER_BASE_IMAGE_ALPINE: alpine:3.17
SQS_DEV_IMAGE_REPOSITORY: osmolabs/sqs-dev
SQS_PROD_IMAGE_REPOSITORY: osmolabs/sqs

jobs:
docker:
runs-on: self-hosted
steps:
- name: Check out repo
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Find go version
run: |
GO_VERSION=$(cat go.mod | grep -E 'go [0-9].[0-9]+' | cut -d ' ' -f 2)
echo "GO_VERSION=$GO_VERSION" >> $GITHUB_ENV

- name: Create Docker Image Tag for release candidate on Tag Push
if: startsWith(github.ref, 'refs/tags/sqs-v')
run: |
GITHUB_TAG=${{ github.ref_name }}
echo "DOCKER_IMAGE_TAG=${GITHUB_TAG#v}" >> $GITHUB_ENV
echo "OSMOSIS_VERSION=${{ github.ref_name }}" >> $GITHUB_ENV

- name: Create Docker Image Tag for sqs-vx.x.x branch
if: startsWith(github.ref, 'sqs-v')
run: |
GITHUB_TAG=${{ github.ref_name }}
echo "DOCKER_IMAGE_TAG=${SHORT_SHA}-$(date +%s)" >> $GITHUB_ENV
echo "OSMOSIS_VERSION=${{ github.ref_name }}" >> $GITHUB_ENV

- name: Create Docker Image Tag for roman/sqs-poc-v20 branch
if: github.ref == 'refs/heads/roman/sqs-poc-v20'
run: |
SHORT_SHA=$(echo ${GITHUB_SHA} | cut -c1-8)
echo "DOCKER_IMAGE_TAG=${SHORT_SHA}-$(date +%s)" >> $GITHUB_ENV
echo "OSMOSIS_VERSION=$SHORT_SHA" >> $GITHUB_ENV

- name: Build and Push Docker Images Dev
uses: docker/build-push-action@v5
with:
file: Dockerfile
context: .
push: true
platforms: linux/amd64
build-args: |
GO_VERSION=${{ env.GO_VERSION }}
RUNNER_IMAGE=${{ env.RUNNER_BASE_IMAGE_ALPINE }}
GIT_VERSION=${{ env.OSMOSIS_VERSION }}
GIT_COMMIT=${{ github.sha }}
tags: |
${{ env.SQS_DEV_IMAGE_REPOSITORY }}:${{ env.DOCKER_IMAGE_TAG }}
${{ env.SQS_DEV_IMAGE_REPOSITORY }}:sqs-latest

- name: Build and Push Docker Images Prod
uses: docker/build-push-action@v5
with:
file: Dockerfile
context: .
push: true
platforms: linux/amd64
build-args: |
GO_VERSION=${{ env.GO_VERSION }}
RUNNER_IMAGE=${{ env.RUNNER_BASE_IMAGE_ALPINE }}
GIT_VERSION=${{ env.OSMOSIS_VERSION }}
GIT_COMMIT=${{ github.sha }}
tags: |
${{ env.SQS_PROD_IMAGE_REPOSITORY }}:${{ env.DOCKER_IMAGE_TAG }}
${{ env.SQS_PROD_IMAGE_REPOSITORY }}:sqs-latest
4 changes: 2 additions & 2 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@
"-test.timeout",
"30m",
"-test.run",
"TestKeeperTestSuite/TestYourName",
"TestKeeperTestSuite/TestGetTickLiquidityForFullRange_CreatePosition",
"-test.v"
],
},
Expand Down Expand Up @@ -299,7 +299,7 @@
"type": "go",
"request": "launch",
"mode": "test",
"program": "${workspaceFolder}/ingest/sqs/router",
"program": "${workspaceFolder}/ingest/sqs/router/usecase/route",
"args": [
"-test.timeout",
"30m",
Expand Down
27 changes: 27 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -654,6 +654,33 @@ sqs-update-mainnet-state:
mv taker_fees.json ingest/sqs/router/usecase/routertesting/parsing/taker_fees.json


###############################################################################
### SQS ###
###############################################################################

redis-start:
docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 -v ./redis-cache/:/data redis/redis-stack:7.2.0-v3

redis-stop:
docker container rm -f redis-stack

sqs-start:
./scripts/debug_builder.sh
build/osmosisd start

sqs-load-test-ui:
docker compose -f ingest/sqs/locust/docker-compose.yml up --scale worker=4

sqs-profile:
go tool pprof -http=:8080 http://localhost:9092/debug/pprof/profile?seconds=15

# Updates go tests with the latest mainnet state
# Make sure that the node is running locally
sqs-update-mainnet-state:
curl -X POST "http:/localhost:9092/store-state"
mv pools.json ingest/sqs/router/usecase/routertesting/parsing/pools.json
mv taker_fees.json ingest/sqs/router/usecase/routertesting/parsing/taker_fees.json

###############################################################################
### Release ###
###############################################################################
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,3 +83,4 @@ to use the SDK fork, and how to make / test updates to SDK branches.

LocalOsmosis is a containerized local Osmosis testnet used for trying out new features locally.
LocalOsmosis documentation can be found [here](https://github.com/osmosis-labs/osmosis/tree/main/tests/localosmosis)

6 changes: 3 additions & 3 deletions app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,13 +70,14 @@ import (
v9 "github.com/osmosis-labs/osmosis/v20/app/upgrades/v9"
_ "github.com/osmosis-labs/osmosis/v20/client/docs/statik"
"github.com/osmosis-labs/osmosis/v20/ingest"

"github.com/osmosis-labs/osmosis/v20/ingest/sqs"

"github.com/osmosis-labs/osmosis/v20/ingest/sqs/pools/common"
)

const appName = "OsmosisApp"
const (
appName = "OsmosisApp"
)

var (
// DefaultNodeHome default home directories for the application daemon
Expand Down Expand Up @@ -250,7 +251,6 @@ func NewOsmosisApp(

// Initialize the ingest manager for propagating data to external sinks.
app.IngestManager = ingest.NewIngestManager()

sqsConfig := sqs.NewConfigFromOptions(appOpts)

// Initialize the SQS ingester if it is enabled.
Expand Down
6 changes: 4 additions & 2 deletions app/keepers/keepers.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ import (
packetforward "github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v4/router"
packetforwardkeeper "github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v4/router/keeper"
packetforwardtypes "github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v4/router/types"

"github.com/osmosis-labs/osmosis/v20/ingest"

// IBC Transfer: Defines the "transfer" IBC port
Expand Down Expand Up @@ -151,8 +152,6 @@ type AppKeepers struct {
ConcentratedLiquidityKeeper *concentratedliquidity.Keeper
CosmwasmPoolKeeper *cosmwasmpool.Keeper

IngestManager ingest.IngestManager

// IBC modules
// transfer module
RawIcs20TransferAppModule transfer.AppModule
Expand All @@ -162,6 +161,9 @@ type AppKeepers struct {
HooksICS4Wrapper ibchooks.ICS4Middleware
PacketForwardKeeper *packetforwardkeeper.Keeper

// Ingest
IngestManager ingest.IngestManager

// keys to access the substores
keys map[string]*sdk.KVStoreKey
tkeys map[string]*sdk.TransientStoreKey
Expand Down
6 changes: 2 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ require (
github.com/leodido/go-urn v1.2.4 // indirect
github.com/maratori/testableexamples v1.0.0 // indirect
github.com/nunnatsa/ginkgolinter v0.13.5 // indirect
github.com/onsi/ginkgo v1.16.5 // indirect
github.com/regen-network/cosmos-proto v0.3.1 // indirect
github.com/sagikazarmark/locafero v0.3.0 // indirect
github.com/sagikazarmark/slog-shim v0.1.0 // indirect
Expand Down Expand Up @@ -151,7 +152,7 @@ require (
github.com/dgraph-io/badger/v3 v3.2103.2 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/docker/cli v23.0.1+incompatible // indirect
github.com/docker/docker v23.0.1+incompatible // indirect
github.com/docker/docker v23.0.3+incompatible // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
Expand Down Expand Up @@ -343,9 +344,6 @@ replace (

// N.B. v0.19.5 contains a breaking change to the IAVL API
github.com/cosmos/iavl v0.19.5 => github.com/cosmos/iavl v0.19.7

// Replace PFM with a version that includes https://github.com/cosmos/ibc-apps/pull/105
github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v4 => github.com/osmosis-labs/ibc-apps/middleware/packet-forward-middleware/v4 v4.0.0-20231023074229-4b649a9d9cee
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this intentional?

// use cosmos-compatible protobufs
github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.3-alpha.regen.1

Expand Down
15 changes: 10 additions & 5 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,8 @@ github.com/cosmos/gorocksdb v1.2.0 h1:d0l3jJG8M4hBouIZq0mDUHZ+zjOx044J3nGRskwTb4
github.com/cosmos/gorocksdb v1.2.0/go.mod h1:aaKvKItm514hKfNJpUJXnnOWeBnk2GL4+Qw9NHizILw=
github.com/cosmos/iavl v0.19.7 h1:ij32FaEnwxfEurtK0QKDNhTWFnz6NUmrI5gky/WnoY0=
github.com/cosmos/iavl v0.19.7/go.mod h1:X9PKD3J0iFxdmgNLa7b2LYWdsGd90ToV5cAONApkEPw=
github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v4 v4.1.0 h1:96fDMIUSlSQ6dBcwhdYwTT+uTwFP+EWbSyLORcKSJZ8=
github.com/cosmos/ibc-apps/middleware/packet-forward-middleware/v4 v4.1.0/go.mod h1:Mn/jr9pIYr1ofFuptLEi9N6MjcshTT0cpoOY4ln1DeA=
github.com/cosmos/ibc-apps/modules/async-icq/v4 v4.1.0 h1:1iQ8/rJwkeGJe81fKyZC/ASSajoJP0jEi6IJFiKIr7Y=
github.com/cosmos/ibc-apps/modules/async-icq/v4 v4.1.0/go.mod h1:X/dLZ6QxTImzno7qvD6huLhh6ZZBcRt2URn4YCLcXFY=
github.com/cosmos/ibc-go/v4 v4.5.1 h1:+P73X7aIikGAXBUJ9vP9rEbvdSuekt3KGXmAWCSYets=
Expand Down Expand Up @@ -335,8 +337,8 @@ github.com/docker/cli v23.0.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvM
github.com/docker/distribution v2.8.2+incompatible h1:T3de5rq0dB1j30rp0sA2rER+m322EBzniBPB6ZIzuh8=
github.com/docker/distribution v2.8.2+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v1.4.2-0.20180625184442-8e610b2b55bf/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v23.0.1+incompatible h1:vjgvJZxprTTE1A37nm+CLNAdwu6xZekyoiVlUZEINcY=
github.com/docker/docker v23.0.1+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v23.0.3+incompatible h1:9GhVsShNWz1hO//9BNg/dpMnZW25KydO4wtVxWAIbho=
github.com/docker/docker v23.0.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
Expand Down Expand Up @@ -439,6 +441,7 @@ github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.7.0 h1:ueSltNNllEqE3qcWBTD0iQd3IpL/6U+mJxLkazJ7YPc=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-toolsmith/astcast v1.1.0 h1:+JN9xZV1A+Re+95pgnMgDboWNVnIMMQXwfBwLRPgSC8=
github.com/go-toolsmith/astcast v1.1.0/go.mod h1:qdcuFWeGGS2xX5bLM/c3U9lewg7+Zu4mr+xPwZIB4ZU=
Expand Down Expand Up @@ -926,6 +929,7 @@ github.com/nunnatsa/ginkgolinter v0.13.5 h1:fOsPB4CEZOPkyMqF4B9hoqOpooFWU7vWSVkC
github.com/nunnatsa/ginkgolinter v0.13.5/go.mod h1:OBHy4536xtuX3102NM63XRtOyxqZOO02chsaeDWXVO8=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=
github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
github.com/oklog/oklog v0.3.2/go.mod h1:FCV+B7mhrz4o+ueLpx+KqkyXRGMWOYEvfiXtdGtbWGs=
github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
Expand All @@ -937,7 +941,8 @@ github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+W
github.com/onsi/ginkgo v1.10.2/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc=
github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=
github.com/onsi/ginkgo/v2 v2.11.0 h1:WgqUCUt/lT6yXoQ8Wef0fsNn5cAuMK7+KT9UFRz2tcU=
github.com/onsi/gomega v1.4.1/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
Expand Down Expand Up @@ -969,8 +974,6 @@ github.com/osmosis-labs/cosmos-sdk v0.45.0-rc1.0.20230927020814-2854ac001f06 h1:
github.com/osmosis-labs/cosmos-sdk v0.45.0-rc1.0.20230927020814-2854ac001f06/go.mod h1:nZwthKhHt2cvngoDGFR1kSFsAz+MxCJWAg1hTDiwp8E=
github.com/osmosis-labs/go-mutesting v0.0.0-20221208041716-b43bcd97b3b3 h1:YlmchqTmlwdWSmrRmXKR+PcU96ntOd8u10vTaTZdcNY=
github.com/osmosis-labs/go-mutesting v0.0.0-20221208041716-b43bcd97b3b3/go.mod h1:lV6KnqXYD/ayTe7310MHtM3I2q8Z6bBfMAi+bhwPYtI=
github.com/osmosis-labs/ibc-apps/middleware/packet-forward-middleware/v4 v4.0.0-20231023074229-4b649a9d9cee h1:Ki30YyIgI1DNbGzR9zs9/Q+PxCHGq5VKfbyPZKQToL8=
github.com/osmosis-labs/ibc-apps/middleware/packet-forward-middleware/v4 v4.0.0-20231023074229-4b649a9d9cee/go.mod h1:Mn/jr9pIYr1ofFuptLEi9N6MjcshTT0cpoOY4ln1DeA=
github.com/osmosis-labs/osmosis/osmomath v0.0.7-0.20231014001935-1946419d44eb h1:pXsC6vqGD+pbMGt+fVBHi9XBk/KDQuRZde2fh4s/1+k=
github.com/osmosis-labs/osmosis/osmomath v0.0.7-0.20231014001935-1946419d44eb/go.mod h1:jNZ952fypVNMzOsh31LAUS27JbF9naNJGtELxId6ZCg=
github.com/osmosis-labs/osmosis/osmoutils v0.0.7-0.20231202014129-59ba6328b7ca h1:IfxGWU0UPk7bHHnxGk3zFF/CrdAhyL/wQTr+VKbeqAI=
Expand Down Expand Up @@ -1567,6 +1570,7 @@ golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210225134936-a50acf3fe073/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
Expand Down Expand Up @@ -1697,6 +1701,7 @@ golang.org/x/tools v0.0.0-20201023174141-c8cfbd0f21e6/go.mod h1:emZCQorbCU4vsT4f
golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
Expand Down
4 changes: 3 additions & 1 deletion ingest/ingest_manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@ import (
// Ingest manager handles the processing of blocks and ingesting data into various sinks
// tha are defined by the Ingester interface.
type IngestManager interface {
// RegisterIngester registers an ingester.
// RegisterIngester registers an ingester
// to propagate data into at the end of each block.
RegisterIngester(ingester Ingester)

// ProcessBlock processes the block and ingests data into various sinks.
// Must never panic. If panic occurs, it is silently logged and ignored.
// If the ingester returns an error, it is silently logged and ignored.
Expand Down
27 changes: 13 additions & 14 deletions ingest/sqs/domain/mvc/repository.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,11 @@ import (
"errors"

sdk "github.com/cosmos/cosmos-sdk/types"
"github.com/redis/go-redis/v9"

"github.com/osmosis-labs/osmosis/v20/ingest/sqs/log"

"github.com/redis/go-redis/v9"
)

// AtomicIngester is an interface that defines the methods for the atomic ingester.
// It processes a block by writing data into a transaction.
// The caller must call Exec on the transaction to flush data to sink.
type AtomicIngester interface {
// ProcessBlock processes the block by writing data into a transaction.
// Returns error if fails to process.
// It does not flush data to sink. The caller must call Exec on the transaction
ProcessBlock(ctx sdk.Context, tx Tx) error

SetLogger(log.Logger)
}

// Tx defines an interface for atomic transaction.
type Tx interface {
// Exec executes the transaction.
Expand All @@ -40,6 +27,18 @@ type Tx interface {
ClearAll(ctx context.Context) error
}

// AtomicIngester is an interface that defines the methods for the atomic ingester.
// It processes a block by writing data into a transaction.
// The caller must call Exec on the transaction to flush data to sink.
type AtomicIngester interface {
// ProcessBlock processes the block by writing data into a transaction.
// Returns error if fails to process.
// It does not flush data to sink. The caller must call Exec on the transaction
ProcessBlock(ctx sdk.Context, tx Tx) error

SetLogger(log.Logger)
}

// RedisTx is a redis transaction.
type RedisTx struct {
pipeliner redis.Pipeliner
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import (
"fmt"

"cosmossdk.io/math"
storetypes "github.com/cosmos/cosmos-sdk/store/types"
sdk "github.com/cosmos/cosmos-sdk/types"

"github.com/osmosis-labs/osmosis/osmomath"
Expand Down Expand Up @@ -106,7 +105,7 @@ func (r *routableConcentratedPoolImpl) CalculateTokenOutByTokenIn(tokenIn sdk.Co
}

// Initialize the swap strategy.
swapStrategy := swapstrategy.New(isZeroForOne, osmomath.ZeroBigDec(), &storetypes.KVStoreKey{}, concentratedPool.SpreadFactor)
swapStrategy := swapstrategy.New(isZeroForOne, osmomath.ZeroBigDec(), &sdk.KVStoreKey{}, concentratedPool.SpreadFactor)

var (
// Swap state
Expand Down
2 changes: 1 addition & 1 deletion osmoutils/config_parser.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"fmt"
"strconv"
"strings"

servertypes "github.com/cosmos/cosmos-sdk/server/types"

"github.com/spf13/cast"
Expand Down
Loading