Split secrets from configuration files

[kellerza]

Description:
Option to split secrets from configuration files. Secrets can be stored in (one or both):

• secrets.yaml (in same folder as configuration.yaml)
• keyring (optional, only if dep installed)**

This uses the loaded configuration.yaml and inserts secrets where applicable:

• It is completely dependent on configuration.yaml. So the user can choose to remove secrets (if at all).
• Placing a (-) in the value will resolve the secret.
• If a username is present, and no password the password will be resolved. The username can also contain a (-)
• If password present it will be left as is. Or resolved if it contains (-)
• If a secret can't be resolved a WARNING will be logged.
• Stale secrets in secrets.yaml raises a warning
• Using a list of the resolved secrets, a UI interface can be created to update secrets in secrets.yaml / keyring
  • Until a UI exists editing secrets.yaml would be the best way to store secrets
  • Easiest way to populate keyring would be a command line option like --passwords?

Related issue (if applicable): fixes #

Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#

Example entry for configuration.yaml:

sensor:
   api_key: (-)
http:
  api_password: (-)
mycomponent:
  username: (-)
mycomponent2:
  username: un02
  password: (-)

resolves to:

sensor:
  api_key: ABCDEF
http:
  api_password: JKLMNOP
mycomponent:
  username: un01
  password: pw01
mycomponent 2:
  username: un02
  password: pw02

using secrets.yaml file:

/http/api_password: JKLMNOP
/sensor/api_key: ABCDEF  
/mycomponent/username: un01
/mycomponent/password: pw01
/mycomponent_2/password: pw02

Checklist:

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated in home-assistant.io

If the code does not interact with devices:

• Local tests with tox run successfully. Your PR cannot be merged unless tests pass
• Tests have been added to verify that the new code works.

[balloob]

How does it work with lists?

[kellerza]

It will iterate over all items in lists. and handle all OrderedDicts in the lists

[kellerza]

Added some tests, but will have to add the array index to new_path to truly separate items in lists

[balloob]

what's with the pylint disables ? The imports are ordered built-in, 3rd party, home assistant.

[kellerza]

pylint complained about the try catch around the import if it is before the home assistant imports. Not sure if there's a better way to do it?

And keyring should probably be optional

[balloob]

I see an issue with lists. It is very common for people to have a list of platforms that contain duplicate platform types and that could totally be unrelated usernames/passwords. That would not work right now as the path to it would be the same.

And that brings me to the next issue: I think that this is getting way too complex. The goal is to be able to extract sensitive information from the main config and optionally offer an extra way of encrypting this in a keyring.

I think that instead of doing like this, we should go way simpler. Instead of auto-generating namespaced names, let's just allow the user to do it. Base it on the existing !env_var constructor for YAML but instead of looking it up in the environment variables, look it up in a dict. Simple, one to one mapping.

nest:
  username: !yaml_var nest_username
  password: !yaml_var nest_password

I want to try to avoid as much magic as possible when it comes to the config because it is already confusing as is. If we start doing things like auto-resolving password fields if we find username the user will not know how to even start debugging if a problem occurs.

[kellerza]

Ok, that is a good idea, will simplify things a bit

Do you prefer !yaml_var or !secret? Or is the idea to hook into existing envvar code for secrets?

Or !yaml_secret AND !keyring for different storage locations? At the moment it starts with secrets.yaml and then keyring

[balloob]

For secret, let's use !yaml_var. We can always do keyring later.

[kellerza]

!secret currently supports both in keyring2 branch.

!yaml_var might be a bit confusing, since these vars can only come from secret.yaml at the moment and that is not configurable. If you have a suggestion to make it configurable I can look into it, but since the !secrets are substituted during loading of configuration.yaml, there is no guarantee these vars have been loaded by configuration.yaml before being used for substitution.