Skip to content

Commit

Permalink
[confmap/provider] feat: Add confmap.Provider that supports AES decry…
Browse files Browse the repository at this point in the history
…ption of values (#35549)

**Description:**


This package provides a `confmap.Provider` implementation for symmetric
AES encryption of credentials (and other sensitive values) in
configurations. It relies on the environment variable
`OTEL_CREDENTIAL_PROVIDER` with the value of the AES key, base64
encoded. 16, 24, or 32 byte keys are supported, selecting AES-128,
AES-192, or AES-256 respectively.

An AES 32-byte (AES-256) key can be generated using the following
command:

```shell
openssl rand -base64 32
```

Configurations can now use placeholders with the following pattern
`${credential:<encrypted & base64-encoded value>}`. The value will be
decrypted using the AES key provided in the environment variable
`OTEL_CREDENTIAL_PROVIDER`

> For example:
> 
> ```shell
> export
OTEL_CREDENTIAL_PROVIDER="GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac="
> ```
> 
> ```yaml
> password: ${aes:RsEf6cTWrssi8tlssfs1AJs2bRMrVm2Ce5TaWPY=}
> ```
> 
> will resolve to:
> ```yaml
> password: '1'
> ```

Since AES is a symmetric encryption algorithm, the same key must be used
to encrypt and decrypt the values. If the key is exchanged between the
collector and a server, it should be done over a secure connection.

When the collector persists its configuration to disk, storing the key
in the environment prevents compromising secrets in the configuration.
It still presents a vulnerability if the attacker has access to the
collector's memory or the environment's configuration, but increases
security over plaintext configurations.


**Testing:**

Unit tests with 93.0% coverage, built agent and configured with
`${credential:<encrypted value>}` values.

**Documentation:**

`README.md` reflecting this PR description.

**Issue:**


#35550
  • Loading branch information
shazlehu authored Oct 22, 2024
1 parent e788e31 commit b0cbb61
Show file tree
Hide file tree
Showing 14 changed files with 348 additions and 0 deletions.
16 changes: 16 additions & 0 deletions .chloggen/aes-credential-provider.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: new_component

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: confmap/aesprovider

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Initial aes encryption provider. Allows configurations to decrypt secrets using AES encryption.

# One or more tracking issues related to the change
issues: [35550]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ cmd/telemetrygen/ @open-teleme

confmap/provider/s3provider/ @open-telemetry/collector-contrib-approvers @Aneurysm9
confmap/provider/secretsmanagerprovider/ @open-telemetry/collector-contrib-approvers @driverpt @atoulme
confmap/provider/aesprovider/ @open-telemetry/collector-contrib-approvers @djaglowski @shazlehu

connector/countconnector/ @open-telemetry/collector-contrib-approvers @djaglowski @jpkrohling
connector/datadogconnector/ @open-telemetry/collector-contrib-approvers @mx-psi @dineshg13 @ankitpatel96
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ body:
- cmd/otelcontribcol
- cmd/oteltestbedcol
- cmd/telemetrygen
- confmap/provider/aesprovider
- confmap/provider/s3provider
- confmap/provider/secretsmanagerprovider
- connector/count
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ body:
- cmd/otelcontribcol
- cmd/oteltestbedcol
- cmd/telemetrygen
- confmap/provider/aesprovider
- confmap/provider/s3provider
- confmap/provider/secretsmanagerprovider
- connector/count
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/other.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ body:
- cmd/otelcontribcol
- cmd/oteltestbedcol
- cmd/telemetrygen
- confmap/provider/aesprovider
- confmap/provider/s3provider
- confmap/provider/secretsmanagerprovider
- connector/count
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/unmaintained.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ body:
- cmd/otelcontribcol
- cmd/oteltestbedcol
- cmd/telemetrygen
- confmap/provider/aesprovider
- confmap/provider/s3provider
- confmap/provider/secretsmanagerprovider
- connector/count
Expand Down
1 change: 1 addition & 0 deletions confmap/provider/aesprovider/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../../Makefile.Common
33 changes: 33 additions & 0 deletions confmap/provider/aesprovider/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
## Summary

This package provides a `confmap.Provider` implementation for symmetric AES encryption of credentials (and other sensitive values) in configurations. It relies on the environment variable `OTEL_AES_CREDENTIAL_PROVIDER` set to the value of the AES key, base64 encoded. 16, 24, or 32 byte keys are supported, selecting AES-128, AES-192, or AES-256 respectively.

An AES 32-byte (AES-256) key can be generated using the following command:

```shell
openssl rand -base64 32
```

## How it works
Use placeholders with the following pattern `${aes:<encrypted & base64-encoded value>}` in a configuration. The value will be decrypted using the AES key provided in the environment variable `OTEL_AES_CREDENTIAL_PROVIDER`

> For example:
>
> ```shell
> export OTEL_AES_CREDENTIAL_PROVIDER="GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac="
> ```
>
> ```yaml
> password: ${aes:RsEf6cTWrssi8tlssfs1AJs2bRMrVm2Ce5TaWPY=}
> ```
>
> will resolve to:
> ```yaml
> password: '1'
> ```
## Caveats
Since AES is a symmetric encryption algorithm, the same key must be used to encrypt and decrypt the values. If the key needs to be exchanged between the collector and a server, it should be done over a secure connection.
When the collector persists its configuration to disk, storing the key in the environment prevents compromising secrets in the configuration. It still presents a vulnerability if the attacker has access to the collector's memory or the environment's configuration, but increases security over plaintext configurations.
23 changes: 23 additions & 0 deletions confmap/provider/aesprovider/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
module github.com/open-telemetry/opentelemetry-collector-contrib/confmap/provider/aesprovider

go 1.22.0

require (
github.com/stretchr/testify v1.9.0
go.opentelemetry.io/collector/confmap v1.17.1-0.20241008154146-ea48c09c31ae
go.uber.org/zap v1.27.0

)

require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/knadh/koanf/maps v0.1.1 // indirect
github.com/knadh/koanf/providers/confmap v0.1.0 // indirect
github.com/knadh/koanf/v2 v2.1.1 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
37 changes: 37 additions & 0 deletions confmap/provider/aesprovider/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions confmap/provider/aesprovider/metadata.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
status:
codeowners:
active: [djaglowski, shazlehu]
108 changes: 108 additions & 0 deletions confmap/provider/aesprovider/provider.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package aesprovider // import "github.com/open-telemetry/opentelemetry-collector-contrib/confmap/provider/aesprovider"

import (
"context"
"crypto/aes"
"crypto/cipher"
"encoding/base64"
"fmt"
"os"
"strings"

"go.opentelemetry.io/collector/confmap"
"go.uber.org/zap"
)

const (
schemaName = "aes"
// This environment variable holds a base64-encoded AES key, either 16, 24, or 32 bytes to select AES-128, AES-192, or AES-256.
keyEnvVar = "OTEL_AES_CREDENTIAL_PROVIDER"
)

type provider struct {
logger *zap.Logger
key []byte
}

// NewFactory creates a new provider factory
func NewFactory() confmap.ProviderFactory {
return confmap.NewProviderFactory(
func(settings confmap.ProviderSettings) confmap.Provider {
return &provider{
logger: settings.Logger,
}
})
}

func (*provider) Scheme() string {
return schemaName
}

func (*provider) Shutdown(context.Context) error {
return nil
}

func (p *provider) Retrieve(_ context.Context, uri string, _ confmap.WatcherFunc) (*confmap.Retrieved, error) {

if !strings.HasPrefix(uri, schemaName+":") {
return nil, fmt.Errorf("%q uri is not supported by %q provider", uri, schemaName)
}

if p.key == nil {
// base64 decode env var
base64Key, ok := os.LookupEnv(keyEnvVar)
if !ok {
return nil, fmt.Errorf("env var %q not set, required for %q provider", keyEnvVar, schemaName)
}
key, err := base64.StdEncoding.DecodeString(base64Key)
if err != nil {
return nil, fmt.Errorf("%q provider uri failed to base64 decode key: %w", schemaName, err)
}
p.key = key
}

// Remove schemaName
cipherText := strings.Replace(uri, schemaName+":", "", 1)

clearText, err := p.decrypt(cipherText)
if err != nil {
return nil, fmt.Errorf("%q provider failed to decrypt value: %w", schemaName, err)
}

return confmap.NewRetrieved(clearText)
}

func (p *provider) decrypt(cipherText string) (string, error) {

cipherBytes, err := base64.StdEncoding.DecodeString(cipherText)
if err != nil {
return "", err
}

block, err := aes.NewCipher(p.key)
if err != nil {
return "", err
}

aesGCM, err := cipher.NewGCM(block)
if err != nil {
return "", err
}

nonceSize := aesGCM.NonceSize()
if len(cipherBytes) < nonceSize {
return "", fmt.Errorf("ciphertext too short")
}

nonce, cipherBytes := cipherBytes[:nonceSize], cipherBytes[nonceSize:]

clearBytes, err := aesGCM.Open(nil, nonce, cipherBytes, nil)
if err != nil {
return "", err
}

return string(clearBytes), nil
}
121 changes: 121 additions & 0 deletions confmap/provider/aesprovider/provider_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package aesprovider

import (
"context"
"os"
"testing"

"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/confmap"
)

func TestAESCredentialProvider(t *testing.T) {

tests := []struct {
name string
configValue string
expectedValue string
expectedError string
envVars map[string]string
}{
{
name: "Valid type, key, JSON value",
configValue: `aes:RsEf6cTWrssi8tls+5Tlk3H8AQBVT2uVDcJ88zj3Sph1iSN8z97yGlxNrWf7oUHSyS4lISEcuqrQq2sPsvcl480rEajM0DIWNVoPi2ApKhADCygGoRxbykm4Tuce+rx0aWIPj1zDkuBM6NIYI1E9H2gxrH+P89Hlwmwq26S3HkqIvtW/BFAHJ8Z08iIg95NFwdbZBu4bMwY93r0KWhL0Y4xk9PGmCZhHk5jaaCgd5YREhJCH8ahenZ/t71yGdu71gSIVbg1xwGZNDKf9wRpOCW4oVQ+OWORDrgKX+58QZMOu0zjRxIkeFQV66xMIEziLYERW0xnT/GthH2+FdO/OlrUlzTBnQBbgqpp86cW1xKPFE1XeFJAIHXca7sYIjl/bz6csiUGSXKVyzApsz6fQEhaFwPGw7YKg/hN4+AEu7zYwHlpiPZyQZVE8xPEgwAy1ZKKk7nJ391ujXohXEsmc7u40bOdQmvktyjemf3KcYSCSoVqvdm49K8/ZfKgG1LMCSnHMk0HWdVGyO8jjBj3RnhpT1/dQ/aMudwPMsPFE+85GYEPTe9Sjq8erkjLRfGomDYWJhSpMESMovVRKpyjv9W+3xS0fHracj1AIx8iRQ9KNq7YKSX9n+wtE7LXMr6CUxwDs0wm1FCdVpc/JfH7BghbAh9qNSZg7qeNWQO9BG9vVa31EpRTDHOOogzGOQU2APtjc0qU65we6lpShBl2HU6S/5SgjB5m9ZnCsJSqlCQTf/e/Riuvx8l5LBlv1JNbHnLD6LO7xarpEKzR2Nc2N2+6pP86SvVB/ZqxGug06SUckjQbrmVrjU5X0RFWQAb4ZdPUobxk2xOXGhxUxEB/pDv5DcuDaEry97XsYBgzYpCtVZr8uQc5kd5jPcMsVgIYo78t+v+2yvCdYtRSHOrAcrOyBbrXCo1yI4UA9qAmfBE1PWC7km9xdhtlIAA5Szei+2oRxCwSvVO0TeYCwByDmYDolL0Tv5jtdgsPbcgnZsL/b9KRBAUU4wXKVm55mzw3AiOehX/bms84XLnRWZaxN06tJ/DiMbMcatTQP0pxk4zoemVD66wo7dA8U0nrnfP8AMfQmFQ==`,
expectedValue: `{ "type": "service_account", "project_id": "my-test-project-12345", "private_key_id": "abcdef1234567890abcdef1234567890abcdef12", "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCAmMwggJfAgEAAoGBALKlO+j9ALhlg5po\nfakePrivateKeyValue+/abc/def/123/fakeData==\n-----END PRIVATE KEY-----\n", "client_email": "test-service-account@my-test-project-12345.iam.gserviceaccount.com", "client_id": "123456789012345678901", "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://oauth2.googleapis.com/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test-service-account%40my-test-project-12345.iam.gserviceaccount.com" }`,

envVars: map[string]string{
"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
},
},

{
name: "Invalid base64 key",
configValue: `aes:RsEf6cTWrssi8tlssfs1AJs2bRMrVm2Ce5TaWPY=`,
expectedError: `"aes" provider uri failed to base64 decode key: illegal base64 data at input byte 25`,

envVars: map[string]string{
"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8lN8bVU2k0TAKtzwJzac=",
},
},
{
name: "Invalid AES key",
configValue: `aes:RsEf6cTWrssi8tls+5Tlk3H8AQBVT2uVDcJ88zj3Sph1iSN8z97yGlxNrWf7oUHSyS4lISEcuqrQq2sPsvcl480rEajM0DIWNVoPi2ApKhADCygGoRxbykm4Tuce+rx0aWIPj1zDkuBM6NIYI1E9H2gxrH+P89Hlwmwq26S3HkqIvtW/BFAHJ8Z08iIg95NFwdbZBu4bMwY93r0KWhL0Y4xk9PGmCZhHk5jaaCgd5YREhJCH8ahenZ/t71yGdu71gSIVbg1xwGZNDKf9wRpOCW4oVQ+OWORDrgKX+58QZMOu0zjRxIkeFQV66xMIEziLYERW0xnT/GthH2+FdO/OlrUlzTBnQBbgqpp86cW1xKPFE1XeFJAIHXca7sYIjl/bz6csiUGSXKVyzApsz6fQEhaFwPGw7YKg/hN4+AEu7zYwHlpiPZyQZVE8xPEgwAy1ZKKk7nJ391ujXohXEsmc7u40bOdQmvktyjemf3KcYSCSoVqvdm49K8/ZfKgG1LMCSnHMk0HWdVGyO8jjBj3RnhpT1/dQ/aMudwPMsPFE+85GYEPTe9Sjq8erkjLRfGomDYWJhSpMESMovVRKpyjv9W+3xS0fHracj1AIx8iRQ9KNq7YKSX9n+wtE7LXMr6CUxwDs0wm1FCdVpc/JfH7BghbAh9qNSZg7qeNWQO9BG9vVa31EpRTDHOOogzGOQU2APtjc0qU65we6lpShBl2HU6S/5SgjB5m9ZnCsJSqlCQTf/e/Riuvx8l5LBlv1JNbHnLD6LO7xarpEKzR2Nc2N2+6pP86SvVB/ZqxGug06SUckjQbrmVrjU5X0RFWQAb4ZdPUobxk2xOXGhxUxEB/pDv5DcuDaEry97XsYBgzYpCtVZr8uQc5kd5jPcMsVgIYo78t+v+2yvCdYtRSHOrAcrOyBbrXCo1yI4UA9qAmfBE1PWC7km9xdhtlIAA5Szei+2oRxCwSvVO0TeYCwByDmYDolL0Tv5jtdgsPbcgnZsL/b9KRBAUU4wXKVm55mzw3AiOehX/bms84XLnRWZaxN06tJ/DiMbMcatTQP0pxk4zoemVD66wo7dA8U0nrnfP8AMfQmFQ==`,
expectedError: `"aes" provider failed to decrypt value: crypto/aes: invalid key size 1`,

envVars: map[string]string{
"OTEL_AES_CREDENTIAL_PROVIDER": "MQ==",
},
},

{
name: "simple message",
configValue: "aes:RsEf6cTWrssi8tlssfs1AJs2bRMrVm2Ce5TaWPY=",
expectedValue: "1",
envVars: map[string]string{
"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
},
},
{
name: "empty message",
configValue: "aes:RsEf6cTWrssi8tlsZ4V68iFEJRFI8o71+QoYYw==",
expectedValue: "",
envVars: map[string]string{
"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
},
},
{
name: "Truncated base64 value",
configValue: "aes:R=",
expectedError: `"aes" provider failed to decrypt value: illegal base64 data at input byte 1`,
envVars: map[string]string{
"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
},
},
{
name: "Truncated encryted text",
configValue: "aes:MQ==",
expectedError: `"aes" provider failed to decrypt value: ciphertext too short`,
envVars: map[string]string{
"OTEL_AES_CREDENTIAL_PROVIDER": "GQi+Y8HwOYzs8lAOjHUqB7vXlN8bVU2k0TAKtzwJzac=",
},
},
{
name: "Wrong schema",
configValue: "foo:MQ==",
expectedError: `"foo:MQ==" uri is not supported by "aes" provider`,
},
{
name: "No env vars",
configValue: "aes:MQ==",
expectedError: `env var "OTEL_AES_CREDENTIAL_PROVIDER" not set, required for "aes" provider`,
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
os.Clearenv()
for k, v := range tt.envVars {
if err := os.Setenv(k, v); err != nil {
t.Fatalf("Failed to set env var %s: %v", k, err)
}
}

p := NewFactory().Create(confmap.ProviderSettings{})
retrieved, err := p.Retrieve(context.Background(), tt.configValue, nil)
if tt.expectedError == "" {
require.NoError(t, err)
} else {
require.Error(t, err)
require.Equal(t, tt.expectedError, err.Error())
return
}
require.NotNil(t, retrieved)
stringValue, err := retrieved.AsString()
require.NoError(t, err)
require.Equal(t, tt.expectedValue, stringValue)
})
}
}
Loading

0 comments on commit b0cbb61

Please sign in to comment.