feat: remove `protoc` pkg and also nodetime helpers `ts-proto` and `sta` #4090

Pantani · 2024-04-20T00:14:35Z

Description

stop using the protoc with the node plugins so we can remove the node time binary. This PR's makes all ts generation using the buf generate command.

Changes

Move buf mode update to buf dep update;
Remove protoc and nodetime;
Add buf.lock dependency digest;

How to test

ignite s chain venus && cd venus && ignite g ts-client

check the ts-client folder into the chain

ilgooz

How do we provide discovered modules' proto files into code generation process?

Pantani · 2024-04-20T10:10:05Z

How do we provide discovered modules' proto files into code generation process?

is almost the same thing as the protoc, we are passing the discovered modules to the include-import flag into the buf generation

clockworkgr · 2024-04-23T09:10:48Z

Tried testing off this branch:

Chain scaffolds ok..

On ignite g ts-client, I get this prompt:

Now ignite supports the `buf.build` (https://buf.build) registry to manage the protobuf dependencies. The embed protoc binary was deprecated and, your blockchain is still using it. Would you like to upgrade and add the `buf.build` config files to `proto/` folder: y/N

Answering No:

✘ Code generation using protoc is only supported by Ignite CLI v0.26.1 or older

Answering Yes:

 buf.build files added: 

/media/clockwork/Projects/Tendermint/venus//buf.gen.ts.yaml
/media/clockwork/Projects/Tendermint/venus//buf.lock
/media/clockwork/Projects/Tendermint/venus//buf.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.gogo.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.pulsar.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.sta.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.swagger.yaml

✗ Ignite proto directory path from the chain config doesn't match the proto directory path from the chain `buf.work.yaml`. Do you want to add the proto path `` to the directories list from the buf work file Y/N:

Answering Yes:

✘ Cannot build app:                                                                                                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                                                                                
error while running command /home/clockwork/go/bin/buf export /media/clockwork/Projects/Tendermint/venus/proto --output=/tmp/includeFolder1117887864: Failure: directory "." listed in /media/clockwork/Projects/Tendermint/venus/buf.work.yaml, it is not valid to have "." as a workspace directory, as this is no different than not having a workspace at all, see https://buf.build/docs/reference/workspaces/#directories for more details
: exit status 1

Answering No:

✘ Cannot build app:                                                                                   
                                                                                                    
invalid Cosmos SDK mod path: [/home/clockwork/go/pkg/mod/github.com/cosmos/[email protected]/proto]

Guessing we need to explicitly supply the proto dir? or the default is not picked up correctly in ignite g commands?

Pantani · 2024-04-23T11:46:37Z

Tried testing off this branch:

Chain scaffolds ok..

On ignite g ts-client, I get this prompt:

Now ignite supports the `buf.build` (https://buf.build) registry to manage the protobuf dependencies. The embed protoc binary was deprecated and, your blockchain is still using it. Would you like to upgrade and add the `buf.build` config files to `proto/` folder: y/N

Answering No:

✘ Code generation using protoc is only supported by Ignite CLI v0.26.1 or older

Answering Yes:

 buf.build files added: 

/media/clockwork/Projects/Tendermint/venus//buf.gen.ts.yaml
/media/clockwork/Projects/Tendermint/venus//buf.lock
/media/clockwork/Projects/Tendermint/venus//buf.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.gogo.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.pulsar.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.sta.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.swagger.yaml

✗ Ignite proto directory path from the chain config doesn't match the proto directory path from the chain `buf.work.yaml`. Do you want to add the proto path `` to the directories list from the buf work file Y/N:

Answering Yes:

✘ Cannot build app:                                                                                                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                                                                                
error while running command /home/clockwork/go/bin/buf export /media/clockwork/Projects/Tendermint/venus/proto --output=/tmp/includeFolder1117887864: Failure: directory "." listed in /media/clockwork/Projects/Tendermint/venus/buf.work.yaml, it is not valid to have "." as a workspace directory, as this is no different than not having a workspace at all, see https://buf.build/docs/reference/workspaces/#directories for more details
: exit status 1

Answering No:

✘ Cannot build app:                                                                                   
                                                                                                    
invalid Cosmos SDK mod path: [/home/clockwork/go/pkg/mod/github.com/cosmos/[email protected]/proto]

Guessing we need to explicitly supply the proto dir? or the default is not picked up correctly in ignite g commands?

Oh, I got it. It's a bug from another PR. I will fix it. Can you test using the chain serve command?

clockworkgr · 2024-04-24T15:15:29Z

Tried testing off this branch:
Chain scaffolds ok..
On ignite g ts-client, I get this prompt:

Now ignite supports the `buf.build` (https://buf.build) registry to manage the protobuf dependencies. The embed protoc binary was deprecated and, your blockchain is still using it. Would you like to upgrade and add the `buf.build` config files to `proto/` folder: y/N

Answering No:

✘ Code generation using protoc is only supported by Ignite CLI v0.26.1 or older

Answering Yes:

 buf.build files added: 

/media/clockwork/Projects/Tendermint/venus//buf.gen.ts.yaml
/media/clockwork/Projects/Tendermint/venus//buf.lock
/media/clockwork/Projects/Tendermint/venus//buf.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.gogo.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.pulsar.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.sta.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.swagger.yaml

✗ Ignite proto directory path from the chain config doesn't match the proto directory path from the chain `buf.work.yaml`. Do you want to add the proto path `` to the directories list from the buf work file Y/N:

Answering Yes:

✘ Cannot build app:                                                                                                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                                                                                
error while running command /home/clockwork/go/bin/buf export /media/clockwork/Projects/Tendermint/venus/proto --output=/tmp/includeFolder1117887864: Failure: directory "." listed in /media/clockwork/Projects/Tendermint/venus/buf.work.yaml, it is not valid to have "." as a workspace directory, as this is no different than not having a workspace at all, see https://buf.build/docs/reference/workspaces/#directories for more details
: exit status 1

Answering No:

✘ Cannot build app:                                                                                   
                                                                                                    
invalid Cosmos SDK mod path: [/home/clockwork/go/pkg/mod/github.com/cosmos/[email protected]/proto]

Guessing we need to explicitly supply the proto dir? or the default is not picked up correctly in ignite g commands?

Oh, I got it. It's a bug from another PR. I will fix it. Can you test using the chain serve command?

not really...the whole idea is to see if the ignite g ts-client command works like before

clockworkgr · 2024-04-26T10:14:05Z

still unable to test this properly to check correctness of ts-client output

faddat · 2024-05-19T16:10:25Z

I was just downloading.... the nearly 1gb main branch here.

This PR is great!

# Conflicts: # ignite/pkg/cosmosbuf/buf.go # ignite/pkg/cosmosgen/generate_go.go # ignite/pkg/cosmosgen/generate_openapi.go

Pantani · 2024-05-23T18:08:23Z

This PR also speeds up the scaffold and sever commands and the CI test

…ta` (#4090) * remove protoc * add changelog * run go mod tidy * remove unused proto folder handling * add ts-client tests * remove unused methods * use import includes instead manual * use buf dep instead buf mod * change buf mod update to comand to buf dep update * improve code readbility * bump buf.build * bump buf into the go.mod * bump protobuf pkgs for buf * check if the folder has proto before update buf dependencies --------- Co-authored-by: Pantani <Pantani>

…ta` (#4090) * remove protoc * add changelog * run go mod tidy * remove unused proto folder handling * add ts-client tests * remove unused methods * use import includes instead manual * use buf dep instead buf mod * change buf mod update to comand to buf dep update * improve code readbility * bump buf.build * bump buf into the go.mod * bump protobuf pkgs for buf * check if the folder has proto before update buf dependencies --------- Co-authored-by: Pantani <Pantani> (cherry picked from commit 8e0937d) # Conflicts: # go.mod # go.sum # ignite/internal/plugin/testdata/execute_fail/go.mod # ignite/internal/plugin/testdata/execute_ok/go.mod # ignite/internal/tools/gen-config-doc/go.mod # ignite/internal/tools/gen-config-doc/go.sum # ignite/internal/tools/gen-mig-diffs/go.mod # ignite/internal/tools/gen-mig-diffs/go.sum # ignite/pkg/cosmosbuf/buf.go # ignite/pkg/cosmosgen/generate_go.go # ignite/pkg/cosmosgen/generate_openapi.go # ignite/pkg/cosmosgen/generate_typescript.go # ignite/pkg/nodetime/nodetime.go # ignite/pkg/nodetime/programs/sta/sta.go # ignite/pkg/nodetime/programs/ts-proto/tsproto.go # ignite/pkg/protoc/protoc.go # ignite/templates/app/files/go.mod.plush # ignite/templates/app/files/proto/buf.lock # integration/doctor/testdata/missing-tools.go.txt # integration/plugin/testdata/example-plugin/go.mod

…ta` (backport #4090) (#4224) * feat: remove `protoc` pkg and also nodetime helpers `ts-proto` and `sta` (#4090) * remove protoc * add changelog * run go mod tidy * remove unused proto folder handling * add ts-client tests * remove unused methods * use import includes instead manual * use buf dep instead buf mod * change buf mod update to comand to buf dep update * improve code readbility * bump buf.build * bump buf into the go.mod * bump protobuf pkgs for buf * check if the folder has proto before update buf dependencies --------- Co-authored-by: Pantani <Pantani> (cherry picked from commit 8e0937d) # Conflicts: # go.mod # go.sum # ignite/internal/plugin/testdata/execute_fail/go.mod # ignite/internal/plugin/testdata/execute_ok/go.mod # ignite/internal/tools/gen-config-doc/go.mod # ignite/internal/tools/gen-config-doc/go.sum # ignite/internal/tools/gen-mig-diffs/go.mod # ignite/internal/tools/gen-mig-diffs/go.sum # ignite/pkg/cosmosbuf/buf.go # ignite/pkg/cosmosgen/generate_go.go # ignite/pkg/cosmosgen/generate_openapi.go # ignite/pkg/cosmosgen/generate_typescript.go # ignite/pkg/nodetime/nodetime.go # ignite/pkg/nodetime/programs/sta/sta.go # ignite/pkg/nodetime/programs/ts-proto/tsproto.go # ignite/pkg/protoc/protoc.go # ignite/templates/app/files/go.mod.plush # ignite/templates/app/files/proto/buf.lock # integration/doctor/testdata/missing-tools.go.txt # integration/plugin/testdata/example-plugin/go.mod * resolve conflicts * fix changelog * remove unused files * fix TestPluginExecute * fix tests for ts generation * rollback go version from tests * fix text namespace version for ts client generation --------- Co-authored-by: Danilo Pantani <[email protected]> Co-authored-by: Julien Robert <[email protected]>

remove protoc

c2b34d5

Pantani self-assigned this Apr 20, 2024

Pantani requested review from ilgooz, jeronimoalbi and julienrbrt as code owners April 20, 2024 00:14

github-actions bot added component:ci CI/CD workflow and automated jobs. component:templates component:configs component:packages labels Apr 20, 2024

add changelog

b9eef1f

Pantani requested review from salmad3 and toschdev as code owners April 20, 2024 00:16

Pantani and others added 2 commits April 20, 2024 02:58

run go mod tidy

d1f1163

Merge branch 'main' into feat/remove-protoc

294dd9f

github-actions bot removed the component:templates label Apr 20, 2024

ilgooz reviewed Apr 20, 2024

View reviewed changes

Merge branch 'main' into feat/remove-protoc

312d2b1

Pantani requested a review from ilgooz April 22, 2024 10:07

Merge branch 'main' into feat/remove-protoc

24eff37

Pantani enabled auto-merge (squash) April 22, 2024 10:49

Pantani requested a review from clockworkgr April 22, 2024 20:56

Pantani and others added 3 commits April 23, 2024 13:46

Merge branch 'main' into feat/remove-protoc

589cf6a

Merge remote-tracking branch 'origin/main' into feat/remove-protoc

de46f32

Merge branch 'main' into feat/remove-protoc

1aa2acd

Merge branch 'main' into feat/remove-protoc

3a62bfd

Pantani added 2 commits May 16, 2024 18:54

Merge branch 'main' into feat/remove-protoc

3250955

Merge branch 'main' into feat/remove-protoc

cb6be1e

Pantani added 2 commits May 21, 2024 20:47

Merge remote-tracking branch 'origin/main' into feat/remove-protoc

6db8f58

# Conflicts: # ignite/pkg/cosmosbuf/buf.go # ignite/pkg/cosmosgen/generate_go.go # ignite/pkg/cosmosgen/generate_openapi.go

use import includes instead manual

c3d8ac0

github-actions bot added the type:internal label May 21, 2024

Pantani added 2 commits May 22, 2024 01:02

use buf dep instead buf mod

ec78d89

change buf mod update to comand to buf dep update

aa62b83

Pantani force-pushed the feat/remove-protoc branch from 8be7c15 to aa62b83 Compare May 22, 2024 10:24

github-actions bot added the component:templates label May 22, 2024

Pantani added 5 commits May 22, 2024 12:26

improve code readbility

ec9804e

bump buf.build

d8a9c42

bump buf into the go.mod

18aa025

bump protobuf pkgs for buf

6dc0147

check if the folder has proto before update buf dependencies

7817f4c

Pantani marked this pull request as ready for review May 22, 2024 15:58

Pantani enabled auto-merge (squash) May 27, 2024 19:00

Pantani added 2 commits May 28, 2024 16:13

Merge branch 'main' into feat/remove-protoc

9247202

Merge branch 'main' into feat/remove-protoc

7d85e0b

julienrbrt approved these changes May 28, 2024

View reviewed changes

Pantani merged commit 8e0937d into main May 28, 2024
46 checks passed

Pantani deleted the feat/remove-protoc branch May 28, 2024 23:21

Pantani added the backport/v28.x.y Backport to v28.x.y label Jul 1, 2024

mergify bot mentioned this pull request Jul 1, 2024

feat: remove protoc pkg and also nodetime helpers ts-proto and sta (backport #4090) #4224

Merged

Pantani removed the backport/v28.x.y Backport to v28.x.y label Jul 4, 2024

BrewTestBot mentioned this pull request Jul 15, 2024

ignite 28.5.0 Homebrew/homebrew-core#177430

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: remove `protoc` pkg and also nodetime helpers `ts-proto` and `sta` #4090

feat: remove `protoc` pkg and also nodetime helpers `ts-proto` and `sta` #4090

Pantani commented Apr 20, 2024 •

edited

Loading

ilgooz left a comment

Pantani commented Apr 20, 2024

clockworkgr commented Apr 23, 2024 •

edited

Loading

Pantani commented Apr 23, 2024

clockworkgr commented Apr 24, 2024

clockworkgr commented Apr 26, 2024

faddat commented May 19, 2024

Pantani commented May 23, 2024

feat: remove protoc pkg and also nodetime helpers ts-proto and sta #4090

feat: remove protoc pkg and also nodetime helpers ts-proto and sta #4090

Conversation

Pantani commented Apr 20, 2024 • edited Loading

Description

Changes

How to test

ilgooz left a comment

Choose a reason for hiding this comment

Pantani commented Apr 20, 2024

clockworkgr commented Apr 23, 2024 • edited Loading

Pantani commented Apr 23, 2024

clockworkgr commented Apr 24, 2024

clockworkgr commented Apr 26, 2024

faddat commented May 19, 2024

Pantani commented May 23, 2024

feat: remove `protoc` pkg and also nodetime helpers `ts-proto` and `sta` #4090

feat: remove `protoc` pkg and also nodetime helpers `ts-proto` and `sta` #4090

Pantani commented Apr 20, 2024 •

edited

Loading

clockworkgr commented Apr 23, 2024 •

edited

Loading