Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add support for a ws client & batch processing over ws #1498

Merged
merged 32 commits into from
Apr 26, 2024

Conversation

zivkovicmilos
Copy link
Member

@zivkovicmilos zivkovicmilos commented Jan 5, 2024

Description

Let me start this PR description by explaining what I wanted to accomplish, so you are not discouraged when reading the file changes.

I wanted to:

  • create a WS client from outside the repository (just like we can create http clients)
  • have the WS client support batch requests
  • have the TM2 JSON-RPC server support batch requests / responses over WS
  • not have to rewrite core client / server logic and APIs

It might seem odd, reading the 3rd point and thinking this is not supported. The truth is actually much more troubling.

Our JSON-RPC server (and client!) implementations are not great. HTTP requests are handled and parsed in a completely different flow than WS requests, even though the output of both should be identical (just over different mediums). Lots of logic is coupled, making it hard to extract and simplify. I've broken the WS / HTTP implementation multiple times over the course of this PR, and all of the tests were passing, even though I've critically broken the module at the time.
The client code for the JSON-RPC server requires a response type (of course, for Amino result parsing) to be given at the moment of calling, which is not amazing considering our WS implementation is async, and doesn't have these response types in the result parsing context (these logic flows are handled by different threads).

What I ended up doing:

  • added support for a WS client
    • this was a bigger effort than expected; I extracted and simplified the batching logic, but was still blocked by the lack of batch support in WS request handling
  • added batch support for the TM2 JSON-RPC server
    • I basically mirrored the HTTP batch request handling (hint that these should be identical flows)
  • BREAKING: completely overhauled our JSON-RPC client libraries and actual instances (http / ws), for a couple of reasons:
    • I tried to add support for all previously mentioned items, but it was impossible with the architecture that was in place (baseRPCClient). The slightly tweaked API (for creating HTTP / WS clients, and using batches) is much simpler to use, and actually has error handling.
    • We didn't have nearly enough coverage and good tests for the functionality -- now we have a suite of E2E and unit tests that give confidence.

I will start an effort in the near future for refactoring the JSON-RPC server code from the ground up in a subsequent PR, this time with specs and tests at the forefront.

How to test out the websockets

To test out the WS responses, you can use a tool like websocat.

  1. start a local chain
  2. run websocat ws://127.0.0.1:26657/websocket (this is the default URL)
  3. send a batch request:
[ { "id": 1, "jsonrpc": "2.0", "method": "status", "params": [] }, { "id": 2, "jsonrpc": "2.0", "method": "status", "params": [] } ]

How to test out the updated client code

I created the following snippet for easily testing the functionality updated in this PR:

  • single HTTP / WS requests
  • batch HTTP / WS requests
package main

import (
	"context"
	"fmt"

	"github.com/gnolang/gno/tm2/pkg/bft/rpc/client"
	ctypes "github.com/gnolang/gno/tm2/pkg/bft/rpc/core/types"
)

func main() {
	// HTTP //
	fmt.Printf("\n--- HTTP CLIENT ---\n")

	// Open HTTP connection
	httpClient, err := client.NewHTTPClient("http://127.0.0.1:26657")
	if err != nil {
		panic("unable to start HTTP client")
	}

	// Get a single status
	status, err := httpClient.Status()
	if err != nil {
		fmt.Println("Unable to send single status (HTTP)!")

		panic(err)
	}

	fmt.Printf("\n\nHTTP - Single status: %v\n\n", status)

	// Get a batch of statuses
	httpBatch := httpClient.NewBatch()

	// Add 10 status requests
	for i := 0; i < 10; i++ {
		if err := httpBatch.Status(); err != nil {
			fmt.Println("Unable to add status request to HTTP batch!")

			panic(err)
		}
	}

	// Send the batch
	results, err := httpBatch.Send(context.Background())
	if err != nil {
		fmt.Println("Unable to send HTTP batch!")

		panic(err)
	}

	for index, resultRaw := range results {
		result, ok := resultRaw.(*ctypes.ResultStatus)
		if !ok {
			panic("Invalid status type in batch response!")
		}

		fmt.Printf("\nStatus %d from batch: %v\n", index, result)
	}

	// WS //
	fmt.Printf("\n--- WS CLIENT ---\n")

	// Open WS connection
	wsClient, err := client.NewWSClient("ws://127.0.0.1:26657/websocket")
	if err != nil {
		panic("unable to start WS client")
	}

	// Get a single status
	status, err = wsClient.Status()
	if err != nil {
		fmt.Println("Unable to send single status (WS)!")

		panic(err)
	}

	fmt.Printf("\n\nWS - Single status: %v\n\n", status)

	// Get a batch of statuses
	wsBatch := wsClient.NewBatch()

	// Add 10 status requests
	for i := 0; i < 10; i++ {
		if err := wsBatch.Status(); err != nil {
			fmt.Println("Unable to add status request to WS batch!")

			panic(err)
		}
	}

	// Send the batch
	results, err = wsBatch.Send(context.Background())
	if err != nil {
		fmt.Println("Unable to send WS batch!")

		panic(err)
	}

	for index, resultRaw := range results {
		result, ok := resultRaw.(*ctypes.ResultStatus)
		if !ok {
			panic("Invalid status type in batch response!")
		}

		fmt.Printf("\nStatus %d from batch: %v\n", index, result)
	}

	if err := wsClient.Close(); err != nil {
		fmt.Println("Unable to gracefully close WS client!")

		panic(err)
	}

	fmt.Printf("\n\nGreat success!\n\n")
}

cc @dongwon8247

Contributors' checklist...
  • Added new tests, or not needed, or not feasible
  • Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory
  • Updated the official documentation or not needed
  • No breaking changes were made, or a BREAKING CHANGE: xxx message was included in the description
  • Added references to related issues and PRs
  • Provided any useful hints for running manual tests
  • Added new benchmarks to generated graphs, if any. More info here.

@zivkovicmilos zivkovicmilos added the 📦 🌐 tendermint v2 Issues or PRs tm2 related label Jan 5, 2024
@zivkovicmilos zivkovicmilos self-assigned this Jan 5, 2024
@github-actions github-actions bot added 📦 🤖 gnovm Issues or PRs gnovm related 📦 ⛰️ gno.land Issues or PRs gno.land package related labels Jan 5, 2024
Copy link

codecov bot commented Jan 5, 2024

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 48.36%. Comparing base (fdde3d0) to head (133376e).

Files Patch % Lines
gno.land/pkg/gnoweb/gnoweb.go 33.33% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1498      +/-   ##
==========================================
+ Coverage   46.41%   48.36%   +1.95%     
==========================================
  Files         487      409      -78     
  Lines       68973    62035    -6938     
==========================================
- Hits        32011    30006    -2005     
+ Misses      34313    29531    -4782     
+ Partials     2649     2498     -151     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@thehowl
Copy link
Member

thehowl commented Jan 7, 2024

This is amazing. Thank you Milos. 🎉

Would you like to have a preliminary review, or just have the team try to fool around and break ws batch processing?

@zivkovicmilos zivkovicmilos added the breaking change Functionality that contains breaking changes label Apr 15, 2024
@zivkovicmilos zivkovicmilos marked this pull request as ready for review April 16, 2024 13:25
@zivkovicmilos zivkovicmilos requested a review from ajnavarro April 16, 2024 13:25
@zivkovicmilos
Copy link
Member Author

@gfanton @ajnavarro
What should we do about gnofaucet being removed from the Dockerfile?

@thehowl
Copy link
Member

thehowl commented Apr 18, 2024

From the review meeting:

  • Having gnofaucet in contribs is a good idea, and a faucet implementation should stay in the monorepo (as it is one way to get started)
  • @zivkovicmilos to split out this PR into another one which does the split specifically.

@zivkovicmilos
Copy link
Member Author

From the review meeting:

  • Having gnofaucet in contribs is a good idea, and a faucet implementation should stay in the monorepo (as it is one way to get started)
  • @zivkovicmilos to split out this PR into another one which does the split specifically.

I've merged in master changes from #1955 that fix gnofaucet

Copy link
Member

@gfanton gfanton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! 👍

Just a few suggestions, nothing critical

tm2/pkg/bft/rpc/lib/client/http/client.go Outdated Show resolved Hide resolved
tm2/pkg/bft/rpc/lib/client/ws/client.go Outdated Show resolved Hide resolved
tm2/pkg/bft/rpc/lib/client/ws/client.go Outdated Show resolved Hide resolved
tm2/pkg/bft/rpc/lib/client/ws/client.go Outdated Show resolved Hide resolved
tm2/pkg/bft/rpc/lib/client/ws/client.go Outdated Show resolved Hide resolved
Copy link
Contributor

@ajnavarro ajnavarro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reviewing @gfanton comments, LGTM

@leohhhn
Copy link
Contributor

leohhhn commented Apr 26, 2024

Hey @zivkovicmilos, it seems that your example for testing out the WS batch request is not working correctly:

❯ websocat ws://127.0.0.1:26657/websocket
[ { "id": 1, "jsonrpc": "2.0", "method": "status", "params": [] }, { "id": 2, "jsonrpc": "2.0", "method": "status", "params": [] } ]
{   "jsonrpc": "2.0",   "id": "",   "error": {     "code": -32700,     "message": "Parse error. Invalid JSON",     "data": "json: cannot unmarshal array into Go value of type struct { JSONRPC string \"json:\\\"jsonrpc\\\"\"; ID interface {} \"json:\\\"id\\\"\"; Method string \"json:\\\"method\\\"\"; Params json.RawMessage \"json:\\\"params\\\"\" }"   } }

Not sure if this is something introduced by a recent change.

@zivkovicmilos
Copy link
Member Author

zivkovicmilos commented Apr 26, 2024

Hey @zivkovicmilos, it seems that your example for testing out the WS batch request is not working correctly:

❯ websocat ws://127.0.0.1:26657/websocket
[ { "id": 1, "jsonrpc": "2.0", "method": "status", "params": [] }, { "id": 2, "jsonrpc": "2.0", "method": "status", "params": [] } ]
{   "jsonrpc": "2.0",   "id": "",   "error": {     "code": -32700,     "message": "Parse error. Invalid JSON",     "data": "json: cannot unmarshal array into Go value of type struct { JSONRPC string \"json:\\\"jsonrpc\\\"\"; ID interface {} \"json:\\\"id\\\"\"; Method string \"json:\\\"method\\\"\"; Params json.RawMessage \"json:\\\"params\\\"\" }"   } }

Not sure if this is something introduced by a recent change.

Are you sure you've built the gnoland binary? This is the error you see with the master codebase now
It's working fine on my end, on the latest branch commit 👀

@zivkovicmilos zivkovicmilos merged commit 15ad779 into gnolang:master Apr 26, 2024
199 of 200 checks passed
@zivkovicmilos zivkovicmilos deleted the feat/add-ws-client branch April 26, 2024 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change Functionality that contains breaking changes 📦 🌐 tendermint v2 Issues or PRs tm2 related 📦 ⛰️ gno.land Issues or PRs gno.land package related 📦 🤖 gnovm Issues or PRs gnovm related
Projects
Status: Done
Status: Done
Development

Successfully merging this pull request may close these issues.

5 participants