-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[receiver/libhoney] New receiver for libhoney (#36706)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Creates a new receiver that accepts libhoney traffic as either logs or traces. This PR doesn't do the conversion yet. Just contains configurations and some factory components. <!-- Issue number (e.g. #1234) or full URL to issue, if applicable. --> #### Link to tracking issue #36693
- Loading branch information
Showing
19 changed files
with
812 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# Use this changelog template to create an entry for release notes. | ||
|
||
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' | ||
change_type: new_component | ||
|
||
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver) | ||
component: libhoneyreceiver | ||
|
||
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). | ||
note: Introduce the scaffolding of a new component, libhoneyreceiver | ||
|
||
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. | ||
issues: [36693] | ||
|
||
# (Optional) One or more lines of additional information to render under the primary note. | ||
# These lines will be padded with 2 spaces and then inserted directly into the document. | ||
# Use pipe (|) for multiline entries. | ||
subtext: | ||
|
||
# If your change doesn't affect end users or the exported elements of any package, | ||
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label. | ||
# Optional: The change log or logs in which this entry should be included. | ||
# e.g. '[user]' or '[user, api]' | ||
# Include 'user' if the change is relevant to end users. | ||
# Include 'api' if there is a change to a library API. | ||
# Default: '[user]' | ||
change_logs: [] |
Validating CODEOWNERS rules …
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
include ../../Makefile.Common | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
# Libhoney Receiver | ||
<!-- status autogenerated section --> | ||
| Status | | | ||
| ------------- |-----------| | ||
| Stability | [development]: traces, logs | | ||
| Distributions | [] | | ||
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Areceiver%2Flibhoney%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Flibhoney) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Areceiver%2Flibhoney%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Flibhoney) | | ||
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@TylerHelmuth](https://www.github.com/TylerHelmuth) | | ||
|
||
[development]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#development | ||
<!-- end autogenerated section --> | ||
|
||
### The purpose and use-cases of the new component | ||
|
||
The Libhoney receiver will accept data for either Trace or Logs signals that are emitted from applications that were | ||
instrumented using [Libhoney](https://docs.honeycomb.io/send-data/logs/structured/libhoney/) libraries. | ||
|
||
## Configuration | ||
|
||
The configuration has 2 parts, One is the HTTP receiver configuration and the rest is about mapping attributes from the | ||
freeform libhoney format into the more structured OpenTelemetry objects. | ||
|
||
### Example configuration for the component | ||
|
||
The following settings are required: | ||
|
||
- `http` | ||
- `endpoint` must set an endpoint. Defaults to `127.0.0.1:8080` | ||
- `resources`: if the `service.name` field is different, map it here. | ||
- `scopes`: to get the `library.name` and `library.version` set in the scope section, set them here. | ||
- `attributes`: if the other trace-related data have different keys, map them here, defaults are otlp-like field names. | ||
|
||
The following setting is required for refinery traffic since: | ||
|
||
- `auth_api`: should be set to `https://api.honeycomb.io` or a proxy that forwards to that host. | ||
Some libhoney software checks `/1/auth` to get environment names so it needs to be passed through. | ||
|
||
|
||
```yaml | ||
libhoney: | ||
http: | ||
endpoint: 0.0.0.0:8088 | ||
traces_url_paths: | ||
- "/1/events" | ||
- "/1/batch" | ||
include_metadata: true | ||
auth_api: https://api.honeycomb.io | ||
resources: | ||
service_name: service_name | ||
scopes: | ||
library_name: library.name | ||
library_version: library.version | ||
attributes: | ||
trace_id: trace_id | ||
parent_id: parent_id | ||
span_id: span_id | ||
name: name | ||
error: error | ||
spankind: span.kind | ||
durationFields: | ||
- duration_ms | ||
``` | ||
### Telemetry data types supported | ||
It will subscribe to the Traces and Logs signals but accept traffic destined for either pipeline using one http receiver | ||
component. Libhoney doesnot differentiate between the two so the receiver will identify which pipeline to deliver the | ||
spans or log records to. | ||
No support for metrics since they'd look just like logs. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package libhoneyreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/libhoneyreceiver" | ||
|
||
import ( | ||
"errors" | ||
"fmt" | ||
"net/url" | ||
"path" | ||
|
||
"go.opentelemetry.io/collector/config/confighttp" | ||
"go.opentelemetry.io/collector/confmap" | ||
) | ||
|
||
// Config represents the receiver config settings within the collector's config.yaml | ||
type Config struct { | ||
HTTP *HTTPConfig `mapstructure:"http"` | ||
AuthAPI string `mapstructure:"auth_api"` | ||
Wrapper string `mapstructure:"wrapper"` | ||
Resources ResourcesConfig `mapstructure:"resources"` | ||
Scopes ScopesConfig `mapstructure:"scopes"` | ||
Attributes AttributesConfig `mapstructure:"attributes"` | ||
} | ||
|
||
type HTTPConfig struct { | ||
*confighttp.ServerConfig `mapstructure:",squash"` | ||
|
||
// The URL path to receive traces on. If omitted "/" will be used. | ||
TracesURLPaths []string `mapstructure:"traces_url_paths,omitempty"` | ||
} | ||
|
||
type ResourcesConfig struct { | ||
ServiceName string `mapstructure:"service_name"` | ||
} | ||
|
||
type ScopesConfig struct { | ||
LibraryName string `mapstructure:"library_name"` | ||
LibraryVersion string `mapstructure:"library_version"` | ||
} | ||
|
||
type AttributesConfig struct { | ||
TraceID string `mapstructure:"trace_id"` | ||
ParentID string `mapstructure:"parent_id"` | ||
SpanID string `mapstructure:"span_id"` | ||
Name string `mapstructure:"name"` | ||
Error string `mapstructure:"error"` | ||
SpanKind string `mapstructure:"spankind"` | ||
DurationFields []string `mapstructure:"durationFields"` | ||
} | ||
|
||
func (cfg *Config) Validate() error { | ||
if cfg.HTTP == nil { | ||
return errors.New("must specify at least one protocol when using the arbitrary JSON receiver") | ||
} | ||
return nil | ||
} | ||
|
||
func (cfg *Config) Unmarshal(conf *confmap.Conf) error { | ||
// first load the config normally | ||
err := conf.Unmarshal(cfg) | ||
if err != nil { | ||
return err | ||
} | ||
|
||
if !conf.IsSet("http") { | ||
cfg.HTTP = nil | ||
} else { | ||
var err error | ||
|
||
for idx := range cfg.HTTP.TracesURLPaths { | ||
if cfg.HTTP.TracesURLPaths[idx], err = sanitizeURLPath(cfg.HTTP.TracesURLPaths[idx]); err != nil { | ||
return err | ||
} | ||
} | ||
} | ||
if cleanURL, err := url.Parse(cfg.AuthAPI); err != nil { | ||
cfg.AuthAPI = cleanURL.String() | ||
} else { | ||
return err | ||
} | ||
|
||
return nil | ||
} | ||
|
||
func sanitizeURLPath(urlPath string) (string, error) { | ||
u, err := url.Parse(urlPath) | ||
if err != nil { | ||
return "", fmt.Errorf("invalid HTTP URL path set for signal: %w", err) | ||
} | ||
|
||
if !path.IsAbs(u.Path) { | ||
u.Path = "/" + u.Path | ||
} | ||
return u.Path, nil | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
//go:generate mdatagen metadata.yaml | ||
|
||
package libhoneyreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/libhoneyreceiver" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package libhoneyreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/libhoneyreceiver" | ||
|
||
import ( | ||
"context" | ||
"fmt" | ||
|
||
"go.opentelemetry.io/collector/component" | ||
"go.opentelemetry.io/collector/config/confighttp" | ||
"go.opentelemetry.io/collector/consumer" | ||
"go.opentelemetry.io/collector/receiver" | ||
|
||
"github.com/open-telemetry/opentelemetry-collector-contrib/internal/sharedcomponent" | ||
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/libhoneyreceiver/internal/metadata" | ||
) | ||
|
||
const ( | ||
httpPort = 8080 | ||
) | ||
|
||
var defaultTracesURLPaths = []string{"/events", "/event", "/batch"} | ||
|
||
// NewFactory creates a new OTLP receiver factory. | ||
func NewFactory() receiver.Factory { | ||
return receiver.NewFactory( | ||
metadata.Type, | ||
createDefaultConfig, | ||
receiver.WithTraces(createTraces, metadata.TracesStability), | ||
receiver.WithLogs(createLogs, metadata.LogsStability), | ||
) | ||
} | ||
|
||
// createDefaultConfig creates the default configuration for receiver. | ||
func createDefaultConfig() component.Config { | ||
durationFieldsArr := []string{"duration_ms"} | ||
endpointStr := fmt.Sprintf("localhost:%d", httpPort) | ||
return &Config{ | ||
HTTP: &HTTPConfig{ | ||
ServerConfig: &confighttp.ServerConfig{ | ||
Endpoint: endpointStr, | ||
}, | ||
TracesURLPaths: defaultTracesURLPaths, | ||
}, | ||
AuthAPI: "", | ||
Resources: ResourcesConfig{ | ||
ServiceName: "service.name", | ||
}, | ||
Scopes: ScopesConfig{ | ||
LibraryName: "library.name", | ||
LibraryVersion: "library.version", | ||
}, | ||
Attributes: AttributesConfig{ | ||
TraceID: "trace.trace_id", | ||
SpanID: "trace.span_id", | ||
ParentID: "trace.parent_id", | ||
Name: "name", | ||
Error: "error", | ||
SpanKind: "span.kind", | ||
DurationFields: durationFieldsArr, | ||
}, | ||
} | ||
} | ||
|
||
func createLogs( | ||
_ context.Context, | ||
set receiver.Settings, | ||
cfg component.Config, | ||
nextConsumer consumer.Logs, | ||
) (receiver.Logs, error) { | ||
oCfg := cfg.(*Config) | ||
var err error | ||
r := receivers.GetOrAdd( | ||
oCfg, | ||
func() (lh component.Component) { | ||
lh, err = newLibhoneyReceiver(oCfg, &set) | ||
return lh | ||
}, | ||
) | ||
|
||
if err != nil { | ||
return nil, err | ||
} | ||
|
||
r.Unwrap().(*libhoneyReceiver).registerLogConsumer(nextConsumer) | ||
return r, nil | ||
} | ||
|
||
// createTraces creates a trace receiver based on provided config. | ||
func createTraces( | ||
_ context.Context, | ||
set receiver.Settings, | ||
cfg component.Config, | ||
nextConsumer consumer.Traces, | ||
) (receiver.Traces, error) { | ||
oCfg := cfg.(*Config) | ||
var err error | ||
r := receivers.GetOrAdd( | ||
oCfg, | ||
func() (lh component.Component) { | ||
lh, err = newLibhoneyReceiver(oCfg, &set) | ||
return lh | ||
}, | ||
) | ||
if err != nil { | ||
return nil, err | ||
} | ||
|
||
r.Unwrap().(*libhoneyReceiver).registerTraceConsumer(nextConsumer) | ||
return r, nil | ||
} | ||
|
||
var receivers = sharedcomponent.NewSharedComponents() |
Oops, something went wrong.