feat: multi zone - plugin & reflection (#8)

* feat: multi zone - plugin & reflection  (#5)

* added plugins feature and reflection features

* fixed minor sdkTx variables issue

* Roll back some changes on config.go

* fixed makefile build command

* improvements over lint

* Added nolint ant changed variable name

* fix .gitworkflow build

* fix .gitworkflow build

* Added makefile build plugin steps

* added nolint on staticcheck on load.go

* added nolint on staticcheck on load.go

* added nolint on staticcheck on load.go

* added nolint on staticcheck on load.go

* gofumpt on project

* skiped a test due to reliance on an experimental feature of cosmos v0.46.0-beta

* build plugin before test

* build plugin before test

* fixed failing test

* fix make test

* added docs

* small refactor on variable names

* Zondax/feature/multi zone fix

* added plugins feature and reflection features

* fixed minor sdkTx variables issue

* Roll back some changes on config.go

* fixed makefile build command

* improvements over lint

* Added nolint ant changed variable name

* fix .gitworkflow build

* fix .gitworkflow build

* Added makefile build plugin steps

* added nolint on staticcheck on load.go

* added nolint on staticcheck on load.go

* added nolint on staticcheck on load.go

* added nolint on staticcheck on load.go

* gofumpt on project

* skiped a test due to reliance on an experimental feature of cosmos v0.46.0-beta

* build plugin before test

* build plugin before test

* fixed failing test

* fix make test

* added docs

* small refactor on variable names

* Updated documentation and added --plugin flag

* updated install step on README

* lint fix

* updated docs

.github/workflows/test.yml

@@ -37,5 +37,5 @@ jobs:
       - name: tests
         if: env.GIT_DIFF
         run: |
-          make test
+          make plugin && make test
           go test -mod=readonly -timeout 30m -coverprofile=coverage.out -covermode=atomic ./...

.gitignore

@@ -65,3 +65,5 @@ debug_container.log
 *.synctex.gz
 /x/genutil/config/priv_validator_key.json
 /x/genutil/data/priv_validator_state.json
+
+/**/*.so

Makefile

@@ -6,10 +6,13 @@ rosetta:
 	go build -mod=readonly ./cmd/rosetta
 
 build:
-	go build ./cmd/rosetta.go
+	go build -mod=readonly ./cmd/rosetta
+
+plugin:
+	cd plugins/cosmos-hub && make plugin
 
 test:
-	go test -mod=readonly -race ./...
+	go test -mod=readonly ./...
 
 ###############################################################################
 ###                                Linting                                  ###

README.md

@@ -1,12 +1,28 @@
 # Rosetta
 
-The `rosetta` package implements Coinbase's [Rosetta API](https://www.rosetta-api.org). This document provides instructions on how to use the Rosetta API integration. For information about the motivation and design choices, refer to [ADR 035](https://docs.cosmos.network/main/architecture/adr-035-rosetta-api-support).
+The `rosetta` project implements Coinbase's [Rosetta API](https://www.rosetta-api.org). This document provides instructions on how to use the Rosetta API integration. For information about the motivation and design choices, refer to [ADR 035](https://docs.cosmos.network/main/architecture/adr-035-rosetta-api-support).
 
-## Add Rosetta Command
+## Installing Rosetta
 
 The Rosetta API server is a stand-alone server that connects to a node of a chain developed with Cosmos SDK.
 
-To enable Rosetta API support, it's required to add the `RosettaCommand` to your application's root command file (e.g. `simd/cmd/root.go`).
+Rosetta can be added to any cosmos chain node. standalone or natively.
+
+### Standalone 
+
+Rosetta can be executed as a standalone service, it connects to the node endpoints and expose the required endpoints.
+
+Install Rosetta standalone server with the following command:
+
+```bash
+go install cosmossdk.io/tools/rosetta
+```
+
+Alternatively, for building from source, simply run `make rosetta`. The binary will be located in the root folder.
+
+### Native - As a node command
+
+To enable Native Rosetta API support, it's required to add the `RosettaCommand` to your application's root command file (e.g. `simd/cmd/root.go`).
 
 Import the `rosettaCmd` package:
 
@@ -38,30 +54,40 @@ An implementation example can be found in `simapp` package.
 
 To run Rosetta in your application CLI, use the following command:
 
+> **Note:** if using the native approach, add your node name before any rosetta comand.
+
 ```shell
-simd rosetta --help
+rosetta --help
 ```
 
 To test and run Rosetta API endpoints for applications that are running and exposed, use the following command:
 
 ```shell
-simd rosetta
+rosetta
      --blockchain "your application name (ex: gaia)"
      --network "your chain identifier (ex: testnet-1)"
      --tendermint "tendermint endpoint (ex: localhost:26657)"
      --grpc "gRPC endpoint (ex: localhost:9090)"
      --addr "rosetta binding address (ex: :8080)"
+     --grpc-types-server (optional) "gRPC endpoint for message descriptor types"
 ```
 
-## Use Rosetta Standalone
+## Plugins - Multi chain connections
 
-To use Rosetta standalone, without having to add it in your application, install it with the following command:
+Rosetta will try to reflect the node types trough reflection over the node gRPC endpoints, there may be cases were this approach is not enough. It is possible to extend or implement the required types easily trough plugins.
 
-```bash
-go install cosmossdk.io/tools/rosetta/cmd/rosetta
-```
+To use Rosetta over any chain, it is required to set up prefixes and registering zone specific interfaces through plugins.
+
+Each plugin is a minimalist implementation of `InitZone` and `RegisterInterfaces` which allow Rosetta to parse chain specific data. There is an example for cosmos-hub chain under `plugins/cosmos-hun/` folder
+- **InitZone**: An empty method that is executed first and defines prefixes, parameters and other settings.
+- **RegisterInterfaces**: This method receives an interface registry which is were the zone specific types and interfaces will be loaded
+
+In order to add a new plugin: 
+1. Create a folder over `plugins` folder with the name of the desired zone
+2. Add a `main.go` file with the mentioned methods above.
+3. Build the code binary through `go build -buildmode=plugin -o main.so main.go` 
 
-Alternatively, for building from source, simply run `make rosetta`. The binary will be located in `tools/rosetta`.
+The plugin folder is selected through the cli `--plugin` flag and loaded into the Rosetta server.
 
 ## Extensions
 

client_offline.go

@@ -79,20 +79,12 @@ func (c *Client) PreprocessOperationsToOptions(_ context.Context, req *types.Con
 	}
 
 	// get the signers
-	signers, err := tx.GetSigners()
-	if err != nil {
-		return nil, err
-	}
-
+	signers := tx.GetSigners()
 	signersStr := make([]string, len(signers))
 	accountIdentifiers := make([]*types.AccountIdentifier, len(signers))
 
 	for i, sig := range signers {
-		addr, err := c.config.InterfaceRegistry.SigningContext().AddressCodec().BytesToString(sig)
-		if err != nil {
-			return nil, err
-		}
-
+		addr := sig.String()
 		signersStr[i] = addr
 		accountIdentifiers[i] = &types.AccountIdentifier{
 			Address: addr,

client_online.go

@@ -32,7 +32,6 @@ import (
 	bank "github.com/cosmos/cosmos-sdk/x/bank/types"
 
 	tmrpc "github.com/cometbft/cometbft/rpc/client"
-
 	"github.com/cosmos/cosmos-sdk/types/query"
 )
 
@@ -72,12 +71,14 @@ func NewClient(cfg *Config) (*Client, error) {
 
 	var supportedOperations []string
 	for _, ii := range cfg.InterfaceRegistry.ListImplementations(sdk.MsgInterfaceProtoName) {
-		_, err := cfg.InterfaceRegistry.Resolve(ii)
+		resolvedMsg, err := cfg.InterfaceRegistry.Resolve(ii)
 		if err != nil {
 			continue
 		}
 
-		supportedOperations = append(supportedOperations, ii)
+		if _, ok := resolvedMsg.(sdk.Msg); ok {
+			supportedOperations = append(supportedOperations, ii)
+		}
 	}
 
 	supportedOperations = append(
@@ -513,11 +514,19 @@ func (c *Client) blockTxs(ctx context.Context, height *int64) (crgtypes.BlockTra
 		panic("block results transactions do now match block transactions")
 	}
 	// process begin and end block txs
-	finalizeBlockTx := &rosettatypes.Transaction{
+	beginBlockTx := &rosettatypes.Transaction{
 		TransactionIdentifier: &rosettatypes.TransactionIdentifier{Hash: c.converter.ToRosetta().BeginBlockTxHash(blockInfo.BlockID.Hash)},
 		Operations: AddOperationIndexes(
 			nil,
-			c.converter.ToRosetta().BalanceOps(StatusTxSuccess, blockResults.FinalizeBlockEvents),
+			c.converter.ToRosetta().BalanceOps(StatusTxSuccess, blockResults.BeginBlockEvents),
+		),
+	}
+
+	endBlockTx := &rosettatypes.Transaction{
+		TransactionIdentifier: &rosettatypes.TransactionIdentifier{Hash: c.converter.ToRosetta().EndBlockTxHash(blockInfo.BlockID.Hash)},
+		Operations: AddOperationIndexes(
+			nil,
+			c.converter.ToRosetta().BalanceOps(StatusTxSuccess, blockResults.EndBlockEvents),
 		),
 	}
 
@@ -532,8 +541,9 @@ func (c *Client) blockTxs(ctx context.Context, height *int64) (crgtypes.BlockTra
 	}
 
 	finalTxs := make([]*rosettatypes.Transaction, 0, 2+len(deliverTx))
+	finalTxs = append(finalTxs, beginBlockTx)
 	finalTxs = append(finalTxs, deliverTx...)
-	finalTxs = append(finalTxs, finalizeBlockTx)
+	finalTxs = append(finalTxs, endBlockTx)
 
 	return crgtypes.BlockTransactionsResponse{
 		BlockResponse: c.converter.ToRosetta().BlockResponse(blockInfo),

cmd/rosetta.go

@@ -28,6 +28,20 @@ func RosettaCommand(ir codectypes.InterfaceRegistry, cdc codec.Codec) *cobra.Com
 			}
 			conf.WithCodec(ir, protoCodec)
 
+			err = rosetta.LoadPlugin(ir, cmd.Flag("plugin").Value.String())
+			if err != nil {
+				fmt.Printf("[Rosetta]- Error while loading cosmos-hub plugin: %s", err.Error())
+				return err
+			}
+
+			if cmd.Flag("grpc-types-server").Value.String() != "" {
+				err = rosetta.ReflectInterfaces(ir, cmd.Flag("grpc-types-server").Value.String())
+				if err != nil {
+					fmt.Printf("[Rosetta]- Error while reflecting from grpc server: %s", err.Error())
+					return err
+				}
+			}
+
 			rosettaSrv, err := rosetta.ServerFromConfig(conf)
 			if err != nil {
 				fmt.Printf("[Rosetta]- Error while creating server: %s", err.Error())

codec.go

@@ -1,37 +1,17 @@
 package rosetta
 
 import (
-	"cosmossdk.io/x/tx/signing"
-	"github.com/cosmos/gogoproto/proto"
-
 	"github.com/cosmos/cosmos-sdk/codec"
-	"github.com/cosmos/cosmos-sdk/codec/address"
 	codectypes "github.com/cosmos/cosmos-sdk/codec/types"
 	cryptocodec "github.com/cosmos/cosmos-sdk/crypto/codec"
-	sdk "github.com/cosmos/cosmos-sdk/types"
 	authcodec "github.com/cosmos/cosmos-sdk/x/auth/types"
 	bankcodec "github.com/cosmos/cosmos-sdk/x/bank/types"
 )
 
 // MakeCodec generates the codec required to interact
 // with the cosmos APIs used by the rosetta gateway
 func MakeCodec() (*codec.ProtoCodec, codectypes.InterfaceRegistry) {
-	ir, err := codectypes.NewInterfaceRegistryWithOptions(
-		codectypes.InterfaceRegistryOptions{
-			ProtoFiles: proto.HybridResolver,
-			SigningOptions: signing.Options{
-				AddressCodec: address.Bech32Codec{
-					Bech32Prefix: sdk.GetConfig().GetBech32AccountAddrPrefix(),
-				},
-				ValidatorAddressCodec: address.Bech32Codec{
-					Bech32Prefix: sdk.GetConfig().GetBech32ValidatorAddrPrefix(),
-				},
-			},
-		},
-	)
-	if err != nil {
-		panic(err)
-	}
+	ir := codectypes.NewInterfaceRegistry()
 	cdc := codec.NewProtoCodec(ir)
 
 	authcodec.RegisterInterfaces(ir)

config.go

@@ -28,6 +28,8 @@ const (
 	DefaultCometEndpoint = "localhost:26657"
 	// DefaultGRPCEndpoint is the default value for the gRPC endpoint
 	DefaultGRPCEndpoint = "localhost:9090"
+	// DefaultGRPCEndpoint is the default value for the gRPC endpoint
+	DefaultGRPCTypesServerEndpoint = ""
 	// DefaultNetwork defines the default network name
 	DefaultNetwork = "network"
 	// DefaultOffline defines the default offline value
@@ -38,21 +40,25 @@ const (
 	DenomToSuggest = "uatom"
 	// DefaultPrices defines the default list of prices to suggest
 	DefaultPrices = "1uatom,1stake"
+	// DefaultPlugin define plugin location for interface and type registry
+	DefaultPlugin = "cosmos-hub"
 )
 
 // configuration flags
 const (
-	FlagBlockchain          = "blockchain"
-	FlagNetwork             = "network"
-	FlagTendermintEndpoint  = "tendermint"
-	FlagGRPCEndpoint        = "grpc"
-	FlagAddr                = "addr"
-	FlagRetries             = "retries"
-	FlagOffline             = "offline"
-	FlagEnableFeeSuggestion = "enable-fee-suggestion"
-	FlagGasToSuggest        = "gas-to-suggest"
-	FlagDenomToSuggest      = "denom-to-suggest"
-	FlagPricesToSuggest     = "prices-to-suggest"
+	FlagBlockchain              = "blockchain"
+	FlagNetwork                 = "network"
+	FlagTendermintEndpoint      = "tendermint"
+	FlagGRPCEndpoint            = "grpc"
+	FlagGRPCTypesServerEndpoint = "grpc-types-server"
+	FlagAddr                    = "addr"
+	FlagRetries                 = "retries"
+	FlagOffline                 = "offline"
+	FlagEnableFeeSuggestion     = "enable-fee-suggestion"
+	FlagGasToSuggest            = "gas-to-suggest"
+	FlagDenomToSuggest          = "denom-to-suggest"
+	FlagPricesToSuggest         = "prices-to-suggest"
+	FlagPlugin                  = "plugin"
 )
 
 // Config defines the configuration of the rosetta server
@@ -259,11 +265,13 @@ func SetFlags(flags *pflag.FlagSet) {
 	flags.String(FlagNetwork, DefaultNetwork, "the network name")
 	flags.String(FlagTendermintEndpoint, DefaultCometEndpoint, "the CometBFT rpc endpoint, without tcp://")
 	flags.String(FlagGRPCEndpoint, DefaultGRPCEndpoint, "the app gRPC endpoint")
+	flags.String(FlagGRPCTypesServerEndpoint, DefaultGRPCTypesServerEndpoint, "the app gRPC Server endpoint for proto messages types and reflection")
 	flags.String(FlagAddr, DefaultAddr, "the address rosetta will bind to")
 	flags.Int(FlagRetries, DefaultRetries, "the number of retries that will be done before quitting")
 	flags.Bool(FlagOffline, DefaultOffline, "run rosetta only with construction API")
 	flags.Bool(FlagEnableFeeSuggestion, DefaultEnableFeeSuggestion, "enable default fee suggestion")
 	flags.Int(FlagGasToSuggest, clientflags.DefaultGasLimit, "default gas for fee suggestion")
 	flags.String(FlagDenomToSuggest, DenomToSuggest, "default denom for fee suggestion")
 	flags.String(FlagPricesToSuggest, DefaultPrices, "default prices for fee suggestion")
+	flags.String(FlagPlugin, DefaultPlugin, "plugin folder name")
 }