Skip to content

Latest commit

 

History

History
107 lines (80 loc) · 3.41 KB

ExternalPlugins.md

File metadata and controls

107 lines (80 loc) · 3.41 KB
Raw

External Plugins

Conduit supports external plugins, but not in the way you may be used to in other pluggable systems. In order to limit adding dependencies, external plugins are enabled with a custom build that imports exactly the plugins you would like to deploy.

Over time this process can be automated, but for now it is manual and requires a go development environment and a little bit of code.

Plugin list

This is where external and third party plugins will be listed. If you create a plugin, please make a PR adding a bullet for it below.

Configuring a Deployment that Includes External Plugins

As an example, we'll create a custom deployment that combines a subset of the built-in Conduit plugins and the plugins from conduit-plugin-template.

Initial Setup

To get started, initialize a new directory and setup the project. We'll add two dependencies. To include other third party plugins, they would be added in an analogous way.

go mod init mywebsite.com/my_custom_conduit
go get github.com/algorand/conduit@latest
go get github.com/algorand/conduit-plugin-template@latest
go mod tidy
go mod download

Main File

To install a plugin, it must be imported. This is done using the following pattern:

package main

import (
  "fmt"
  "os"

  // Import some of the built-in plugins.
  _ "github.com/algorand/conduit/conduit/plugins/exporters/noop"
  _ "github.com/algorand/conduit/conduit/plugins/exporters/postgresql"

  // Import from the conduit-plugin-template repo
  _ "github.com/algorand/conduit-plugin-template/plugin/exporter"
  _ "github.com/algorand/conduit-plugin-template/plugin/importer"
  _ "github.com/algorand/conduit-plugin-template/plugin/processor"

  // The cli package is used to start Conduit
  "github.com/algorand/conduit/pkg/cli"
)

func main() {
  conduitCmd := cli.MakeConduitCmdWithUtilities()
  if err := conduitCmd.Execute(); err != nil {
    fmt.Fprintf(os.Stderr, "%v\n", err)
    os.Exit(1)
  }
  os.Exit(0)
}

Build and Test

For a simple test, run main.go directly to list the plugins. The algod importer and file_writer exporter are imported by default, which is why you see them included in the list:

$ go run main.go list
importers:
  algod             - Importer for fetching blocks from an algod REST API.
  importer_template - Example importer.

processors:
  processor_template - Example processor.

exporters:
  exporter_template - Example exporter.
  file_writer       - Exporter for writing data to a file.
  noop              - noop exporter
  postgresql        - Exporter for writing data to a postgresql instance.

Build a binary:

go build .

Cross-compile to different operating systems or architectures:

GOOS=linux GOARCH=amd64 go build -o my_custom_conduit-linux-amd64 main.go
GOOS=linux GOARCH=arm64 go build -o my_custom_conduit-linux-arm64 main.go
GOOS=darwin GOARCH=amd64 go build -o my_custom_conduit-darwin-amd64 main.go
GOOS=darwin GOARCH=arm64 go build -o my_custom_conduit-darwin-arm64 main.go
GOOS=windows GOARCH=amd64 go build -o my_custom_conduit-windows-amd64.exe main.go
GOOS=windows GOARCH=arm64 go build -o my_custom_conduit-windows-arm64.exe main.go