Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Shared Plugin RPC Host #3238

Merged
merged 76 commits into from
Jan 3, 2023
Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
144710c
start to plugin host impl wip
joshLong145 Nov 25, 2022
fe20e03
comments
joshLong145 Nov 25, 2022
7ec89ab
config-add
joshLong145 Nov 27, 2022
07f0e00
minor bug updates
joshLong145 Nov 27, 2022
e5511b9
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Nov 30, 2022
493df1c
updates per plugin config migration
joshLong145 Nov 30, 2022
510f413
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 4, 2022
a5dfc23
small updates to sharedHost plugin option
joshLong145 Dec 4, 2022
d174f8a
update to isolate plugin cache
joshLong145 Dec 5, 2022
3e518b6
rename of utils to cache
joshLong145 Dec 6, 2022
011fc06
print line removing
joshLong145 Dec 6, 2022
5bea32d
tidy
joshLong145 Dec 6, 2022
fa6359a
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 6, 2022
f322a8a
removing print out
joshLong145 Dec 6, 2022
87f2a75
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 6, 2022
4025f98
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 7, 2022
ad31bf6
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 7, 2022
df44738
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 7, 2022
ddff5d7
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 7, 2022
9eaab98
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 8, 2022
bb2577e
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 10, 2022
cd6e1fc
additional comments
joshLong145 Dec 10, 2022
f74eb00
Merge branch 'feat/plugin-host' of github.com:ignite/cli into feat/pl…
joshLong145 Dec 10, 2022
2ce2b44
shared host load test
joshLong145 Dec 10, 2022
2244590
addition of caching tests for plugins
joshLong145 Dec 11, 2022
21e6273
fmt
joshLong145 Dec 11, 2022
15a6eaf
changelog
joshLong145 Dec 11, 2022
66bea2b
additional plugin cache tests
joshLong145 Dec 11, 2022
3b6edab
update shared host tests
joshLong145 Dec 11, 2022
250d15e
review comments
joshLong145 Dec 12, 2022
c0dd25c
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 12, 2022
d20ef29
addition cache test cases
joshLong145 Dec 13, 2022
0a75896
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 13, 2022
c9e7a2f
fix typo
joshLong145 Dec 13, 2022
f2dd80c
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 13, 2022
e752904
fix comment
joshLong145 Dec 13, 2022
2f858d6
Merge branch 'feat/plugin-host' of github.com:ignite/cli into feat/pl…
joshLong145 Dec 13, 2022
5a07b46
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 13, 2022
e4b3670
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 14, 2022
f978a5a
Merge branch 'feat/plugin-host' of github.com:ignite/cli into feat/pl…
joshLong145 Dec 14, 2022
fe8166e
update plugin `KillClient` for shared hosts
joshLong145 Dec 14, 2022
98e16a8
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 15, 2022
5cc9d0b
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 15, 2022
ec3e8b0
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 15, 2022
cf6294a
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 15, 2022
156b49b
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 17, 2022
0632e39
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 17, 2022
2e8cd9b
changelog update
joshLong145 Dec 17, 2022
d6c9bb3
Merge branch 'feat/plugin-host' of github.com:ignite/cli into feat/pl…
joshLong145 Dec 17, 2022
3633dca
Merge branch 'main' into feat/plugin-host
tbruyelle Dec 18, 2022
efd2fb4
fix test
tbruyelle Dec 18, 2022
9460c47
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 18, 2022
742a442
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 19, 2022
f9d3d25
update: migration of `sharedHost` flag to plugin manifest
joshLong145 Dec 20, 2022
651cc3b
scaffolding and plugin host check changes
joshLong145 Dec 20, 2022
0ae8ce2
update tests
joshLong145 Dec 20, 2022
bb36383
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 20, 2022
d96b48f
Merge branch 'feat/plugin-host' of github.com:ignite/cli into feat/pl…
joshLong145 Dec 20, 2022
09f834b
format and lint
joshLong145 Dec 20, 2022
6c47f29
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 21, 2022
930580d
update plugin cache test
joshLong145 Dec 21, 2022
9f92e7b
update property comment
joshLong145 Dec 21, 2022
3d24558
update plugin cache to use full plugin path
joshLong145 Dec 21, 2022
a7b20a5
cleanup of plugin tests
joshLong145 Dec 21, 2022
79c0c1c
Merge branch 'main' of github.com:ignite/cli into feat/plugin-host
joshLong145 Dec 21, 2022
9f66574
Merge branch 'feat/plugin-host' of github.com:ignite/cli into feat/pl…
joshLong145 Dec 21, 2022
0bb2e3a
lint
joshLong145 Dec 21, 2022
5761bcc
update plugin sharedHost tests
joshLong145 Dec 21, 2022
85ae39c
update to plugin docs for `Sharedhost` flag
joshLong145 Dec 22, 2022
66192f9
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 26, 2022
0de72bd
Merge branch 'main' into feat/plugin-host
joshLong145 Dec 28, 2022
c1e9351
test: fix TestPluginLoadSharedHost
tbruyelle Jan 1, 2023
b8fea7d
refac: plugin cach
tbruyelle Jan 1, 2023
c23ab89
Update docs/docs/contributing/01-plugins.md
tbruyelle Jan 1, 2023
66e9463
refac: plugin cache func should be private
tbruyelle Jan 1, 2023
17ef117
test: improve plugin kill assertions
tbruyelle Jan 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

### Features

- [#3238](https://github.com/ignite/cli/pull/3238) Add `Sharedhost` plugin option
- [#3214](https://github.com/ignite/cli/pull/3214) Global plugins config.
- [#3142](https://github.com/ignite/cli/pull/3142) Add `ignite network request param-change` command.
- [#3181](https://github.com/ignite/cli/pull/3181) Addition of `add` `remove` commands for `plugins`
Expand Down
2 changes: 1 addition & 1 deletion ignite/cmd/plugin.go
Original file line number Diff line number Diff line change
Expand Up @@ -561,7 +561,7 @@ func NewPluginScaffold() *cobra.Command {
return err
}
moduleName := args[0]
path, err := plugin.Scaffold(wd, moduleName)
path, err := plugin.Scaffold(wd, moduleName, false)
if err != nil {
return err
}
Expand Down
143 changes: 143 additions & 0 deletions ignite/services/plugin/cache.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
package plugin

import (
"encoding/gob"
"fmt"
"net"
"path"
"path/filepath"

hplugin "github.com/hashicorp/go-plugin"

chainconfig "github.com/ignite/cli/ignite/config"
"github.com/ignite/cli/ignite/pkg/cache"
)

const (
cacheFileName = "ignite_plugin_cache.db"
cacheNamespace = "plugin.rpc.context"
)

var (
storage *cache.Storage
storageCache *cache.Cache[ConfigContext]
)

func init() {
gob.Register(hplugin.ReattachConfig{})
gob.Register(net.UnixAddr{})
}

type ConfigContext struct {
Plugin hplugin.ReattachConfig
Addr net.UnixAddr
}

func WritePluginConfigCache(pluginPath string, conf hplugin.ReattachConfig) error {
name := path.Base(pluginPath)

if name == "." {
return fmt.Errorf("provided path is invalid: %s", pluginPath)
}

if conf.Addr == nil {
return fmt.Errorf("plugin Address info cannot be empty")
}

confCont := ConfigContext{}
// TODO: figure out a better way of resolving the type of network connection is established between plugin server and host
// currently this will always be a unix network socket. but this might not be the case moving forward.
ua, err := net.ResolveUnixAddr(conf.Addr.Network(), conf.Addr.String())
if err != nil {
return err
}

confCont.Addr = *ua
conf.Addr = nil
confCont.Plugin = conf

cache, err := newCache()
if err != nil {
return err
}

cache.Put(name, confCont)

return err
}

func ReadPluginConfigCache(pluginPath string, ref *hplugin.ReattachConfig) error {
name := path.Base(pluginPath)
joshLong145 marked this conversation as resolved.
Show resolved Hide resolved

if name == "." {
return fmt.Errorf("provided path is invalid: %s", pluginPath)
}

cache, err := newCache()
if err != nil {
return err
}

confCont, err := cache.Get(name)
if err != nil {
return err
}

*ref = confCont.Plugin
ref.Addr = &confCont.Addr

return nil
}

func CheckPluginConfCache(pluginPath string) bool {
name := path.Base(pluginPath)

if name == "." {
return false
}

cache, err := newCache()
if err != nil {
return false
}
if _, err := cache.Get(name); err != nil {
return false
}
return true
}

func DeletePluginConfCache(pluginPath string) error {
name := path.Base(pluginPath)

if name == "." {
return fmt.Errorf("provided path is invalid: %s", pluginPath)
}
cache, err := newCache()
if err != nil {
return err
}

if err := cache.Delete(name); err != nil {
return err
}

return nil
}

func newCache() (*cache.Cache[ConfigContext], error) {
cacheRootDir, err := chainconfig.DirPath()
if err != nil {
return nil, err
}
if storage == nil {
storageTmp, err := cache.NewStorage(filepath.Join(cacheRootDir, cacheFileName))
if err != nil {
return nil, err
}
storage = &storageTmp
cacheTmp := cache.New[ConfigContext](*storage, cacheNamespace)
storageCache = &cacheTmp
}

return storageCache, nil
}
110 changes: 110 additions & 0 deletions ignite/services/plugin/cache_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
package plugin

import (
"net"
"testing"

hplugin "github.com/hashicorp/go-plugin"
"github.com/stretchr/testify/require"
)

func TestPluginCacheAdd(t *testing.T) {
joshLong145 marked this conversation as resolved.
Show resolved Hide resolved
t.Run("Should cache plugin config and read from cache", func(t *testing.T) {
const path = "/path/to/awesome/plugin"
unixFD, _ := net.ResolveUnixAddr("unix", "/var/folders/5k/sv4bxrs102n_6rr7430jc7j80000gn/T/plugin193424090")

rc := hplugin.ReattachConfig{
Protocol: hplugin.ProtocolNetRPC,
ProtocolVersion: hplugin.CoreProtocolVersion,
Addr: unixFD,
Pid: 24464,
}

err := WritePluginConfigCache(path, rc)
require.NoError(t, err)

c := hplugin.ReattachConfig{}
err = ReadPluginConfigCache(path, &c)
require.NoError(t, err)
joshLong145 marked this conversation as resolved.
Show resolved Hide resolved
})

t.Run("Should error writing bad plugin config to cache", func(t *testing.T) {
const path = "/path/to/awesome/plugin"
rc := hplugin.ReattachConfig{
Protocol: hplugin.ProtocolNetRPC,
ProtocolVersion: hplugin.CoreProtocolVersion,
Addr: nil,
Pid: 24464,
}

err := WritePluginConfigCache(path, rc)
require.Error(t, err)
})

t.Run("Should error with invalid plugin path", func(t *testing.T) {
const path = ""
rc := hplugin.ReattachConfig{
Protocol: hplugin.ProtocolNetRPC,
ProtocolVersion: hplugin.CoreProtocolVersion,
Addr: nil,
Pid: 24464,
}

err := WritePluginConfigCache(path, rc)
require.Error(t, err)
})
}

func TestPluginCacheDelete(t *testing.T) {
t.Run("Delete plugin config after write to cache should remove from cache", func(t *testing.T) {
const path = "/path/to/awesome/plugin"
unixFD, _ := net.ResolveUnixAddr("unix", "/var/folders/5k/sv4bxrs102n_6rr7430jc7j80000gn/T/plugin193424090")

rc := hplugin.ReattachConfig{
Protocol: hplugin.ProtocolNetRPC,
ProtocolVersion: hplugin.CoreProtocolVersion,
Addr: unixFD,
Pid: 24464,
}

err := WritePluginConfigCache(path, rc)
require.NoError(t, err)

err = DeletePluginConfCache(path)
require.NoError(t, err)

c := hplugin.ReattachConfig{}
// there should be an error after deleting the config from the cache
err = ReadPluginConfigCache(path, &c)
require.Error(t, err)
})

t.Run("Delete plugin config should return error given empty path", func(t *testing.T) {
const path = ""
err := DeletePluginConfCache(path)
require.Error(t, err)
})
}

func TestPluginCacheCheck(t *testing.T) {
const path = "/path/to/awesome/plugin"
unixFD, _ := net.ResolveUnixAddr("unix", "/var/folders/5k/sv4bxrs102n_6rr7430jc7j80000gn/T/plugin193424090")

rc := hplugin.ReattachConfig{
Protocol: hplugin.ProtocolNetRPC,
ProtocolVersion: hplugin.CoreProtocolVersion,
Addr: unixFD,
Pid: 24464,
}

t.Run("Cache should be hydrated", func(t *testing.T) {
err := WritePluginConfigCache(path, rc)
require.NoError(t, err)
require.Equal(t, true, CheckPluginConfCache(path))
})

t.Run("Cache should be empty", func(t *testing.T) {
_ = DeletePluginConfCache(path)
require.Equal(t, false, CheckPluginConfCache(path))
})
}
10 changes: 10 additions & 0 deletions ignite/services/plugin/interface.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,16 @@ type Manifest struct {
// Hooks contains the hooks that will be attached to the existing ignite
// commands.
Hooks []Hook
// SharedHost enables sharing a single plugin server across all running instances
// of a plugin. Useful if a plugin adds or extends long running commands
//
// example: `ignite chain serve`
//
// When enabled, all plugins of the same `Path` loaded from the same configuration will
// attach it's rpc client to a an existing rpc server.
//
// If a plugin instance has no other running plugin servers, it will create one and it will be the host.
joshLong145 marked this conversation as resolved.
Show resolved Hide resolved
SharedHost bool `yaml:"shared_host"`
}

// Command represents a plugin command.
Expand Down
Loading