bitovi · arm4b · Feb 20, 2023 · Feb 17, 2023 · Feb 17, 2023 · Feb 17, 2023
diff --git a/docs/configuration-base.md b/docs/configuration-base.md
@@ -17,8 +17,13 @@ $tool
 * `cli` - object that contains CLI arguments
 * `options` - an object that offers additional control over how a tool executes
 
+## Environemnt Variables Defaulting
+Plugin environment variables are automatically mapped to the `bitops.config.yaml` keys. This means that you can use the same environment variable names as in the plugin config. For example, if you want to override the `ansible.cli.skip-tags` value, you can use the `BITOPS_ANSIBLE_SKIP_TAGS` environment variable. In this case, `BITOPS` is the prefix, `ANSIBLE` is the plugin name, and `SKIP_TAGS` is the key name (note hypens are replaced with underscores).
+
+The precedence order is: `ENV` vars > `bitops.config.yaml` values > `bitops.config.schema.yaml` defaults. This way, ENV variables specified by user are taking highest precedence over config values and defaults. We recommend using them dynamically in CI/CD pipelines to control the deployment based on condition (PR run, branch, etc) or manually passing to the the BitOps docker container.
+
 ## Arbitrary Environment Variables
-During the docker run command, you can specify an ENV var and it will be accessible during all processing stages of BitOps. 
+During the docker run command, you can specify an ENV var and it will be accessible during all processing stages of BitOps. This is useful for passing in secrets or other custom values used in the workflows.
 
 ## Common Configuration
 There are some global configuration options that are shared among all tools and cloud providers during a BitOps run. These are set via environment variables

diff --git a/docs/development/local-plugin-creation.md b/docs/development/local-plugin-creation.md
@@ -20,14 +20,35 @@ To start with, create a file called `bitops.schema.yaml` and add the following c
 duplicate-environment:
     type: object
     properties:
+      # CLI properties will composed into a CLI string and exported as "${BITOPS_MY_PLUGIN_CLI}"
+      cli:
+        type: object
+        properties:
+          # positional argument, if no `parameter` set
+          # ex: "run"
+          command:
+            type: string
+            default: run
+            required: true
+          # cli argument
+          # ex: "--key=value"
+          key:
+            parameter: key
+            type: string
+          # boolean flag, set if value is `true`
+          # ex: "--bar"
+          bar:
+            parameter: bar
+            type: boolean
+            default: true
       options:
         type: object
         properties:
           foo:
             type: string
             export_env: DUPLICATE_ENVIRONMENT_FOO
             required: true
-            default: bar
+            default: foo_value
 ```
 
 Next, create a simple `deploy.sh` script.  This script does some checks and shows how to use some of the available environment variables then outputs a configuration value defined by the `bitops.schema.yaml`.
@@ -38,12 +59,9 @@ set -ex
 echo "Running Duplicate Environment Plugin deployment script..."
 
 # vars
-export DUPLICATE_ENVIRONMENT_ROOT_SCRIPTS="$BITOPS_PLUGIN_DIR"
-export DUPLICATE_ENVIRONMENT_ROOT_OPERATIONS="$BITOPS_OPSREPO_ENVIRONMENT_DIR"
-export BITOPS_SCHEMA_ENV_FILE="$DUPLICATE_ENVIRONMENT_ROOT_OPERATIONS/ENV_FILE"
-export SCRIPTS_DIR="$DUPLICATE_ENVIRONMENT_ROOT_SCRIPTS/scripts"
+export BITOPS_SCHEMA_ENV_FILE="$BITOPS_OPSREPO_ENVIRONMENT_DIR/ENV_FILE"
 
-if [ ! -d "$DUPLICATE_ENVIRONMENT_ROOT_OPERATIONS" ]; then
+if [ ! -d "$BITOPS_OPSREPO_ENVIRONMENT_DIR" ]; then
   echo "No duplicate-environment directory.  Skipping."
   exit 0
 fi
@@ -56,32 +74,25 @@ else
   source "$BITOPS_SCHEMA_ENV_FILE"
 fi
 
-bash $SCRIPTS_DIR/validate_env.sh
-
-cd $DUPLICATE_ENVIRONMENT_ROOT_OPERATIONS
+cd $BITOPS_OPSREPO_ENVIRONMENT_DIR
 
-echo "Listing contents of duplicate-environment Root: $DUPLICATE_ENVIRONMENT_ROOT_OPERATIONS"
+echo "Listing contents of duplicate-environment Root: $BITOPS_OPSREPO_ENVIRONMENT_DIR"
 ls -al .
 
-echo "settings:"
+
+echo "Running the plugin CLI:"
+# plugin_command run --key=value --bar
+echo plugin_command "${BITOPS_MY_PLUGIN_CLI}"
+
+echo "Options:"
 echo "DUPLICATE_ENVIRONMENT_FOO"
 echo "$DUPLICATE_ENVIRONMENT_FOO"
+
 ```
 > **Note:** Much of the above is best practice boilerplate and is not strictly necessary.
 
 > **Important:** Be sure to `chmod +x deploy.sh`
 
-Create `scripts/validate.sh` as follows.  This script simply checks any necessary preconditions before continuing.
-```
-#!/bin/bash
-set -e 
-
-if [ -z "$BITOPS_ENVIRONMENT" ]; then
-  echo "environment variable (ENVIRONMENT) not set"
-  exit 1
-fi
-```
-
 Finally, create a `plugin.config.yaml` to configure how BitOps uses the plugin.
 ```
 plugin:
@@ -108,7 +119,11 @@ Populate the tool's `bitops.config.yaml` based on the schema defined above:
 `/path/to/ops-repo/plugin-no-install/duplicate-environment/bitops.config.yaml`
 ```
 duplicate-environment:
-  properties:
+  cli:
+    command: run
+    key: value
+    bar: true
+  options:
     foo: baz
 ```
 
@@ -290,4 +305,4 @@ bitops:
       plugin: duplicate-environment
 ```
 
-Then, you can follow the steps in #5 above (sans the `copy-plugins.sh` script).
+Then, you can follow the steps in #5 above (sans the `copy-plugins.sh` script).
diff --git a/docs/examples/plugin-examples/plugin-install/duplicate-environment/bitops.config.yaml b/docs/examples/plugin-examples/plugin-install/duplicate-environment/bitops.config.yaml
@@ -1,2 +1,7 @@
 duplicate-environment:
-  foo: baz
+  cli:
+    command: run
+    key: value
+    bar: true
+  options:
+    foo: baz
diff --git a/docs/lifecycle.md b/docs/lifecycle.md
@@ -9,6 +9,7 @@ This is a useful way to extend the functionality of BitOps. A popular use case w
 
 ![lifecycle diagram](assets/images/lifecycle.png)
 
+### Main Execution Flow
 A single run of BitOps will:
 
 1. Copy the contents of `/opt/bitops_deployment` to a temporary working directory
@@ -27,7 +28,7 @@ A single run of BitOps will:
     * Run any `bitops.before-deploy.d/*.sh` scripts
     * Load `bitops.config.yaml` and set environment
     * Merge contents with [Default environment](default-environment.md) - [TODO](https://github.com/bitovi/bitops/issues/18)
-    * Run `ansible-playbook $playbook` for each `*.yaml` or `*.yml` file in `$env/ansible/` 
+    * Run `ansible-playbook playbook.yaml` in `$env/ansible/` 
     * Run any `bitops.after-deploy.d/*.sh` scripts
 4. If a `helm/` directory exists within the selected environment:
     * Run the following for `$env/helm/$ENVIRONMENT_HELM_SUBDIRECTORY/` or for all charts in `$env/helm/`
@@ -49,7 +50,9 @@ A single run of BitOps will:
     * Create or delete cfn stack. Wait for completion
     * Run any `bitops.after-deploy.d/*.sh` scripts
 
-### Environment Variables
-Plugins can export the environment when a value is specified in an ops_repo level `bitops.config.yaml`. These environment variables are prefixed with `BITOPS_` and their plugin name. 
+### Imported Environment Variables
+The plugin config values and defaults are overriden by user environment variables passed to BitOps by prefixing them with `BITOPS_`. For example, `BITOPS_ANSIBLE_SKIP_TAGS=tag1,tag2` will set the plugin's config `ansible.cli.skip-tags` value to `tag1,tag2`.
+See [Environemnt Variables Defaulting](configuration-base.md#environemnt-variables-defaulting) for more information.
 
-So for example, if the terraform plugin exported the environment variable `BUTTER_FLAG`, it would be accessible in the lifecycle hooks by referencing `BITOPS_TERRAFORM_BUTTER_FLAG`.
+### Exported Environment Variables
+BitOps exports the environment variables to the plugin when a ENV var name is specified in `bitops.schema.yaml` via `export_env`. This is useful for passing values to lifecycle hooks, custom scripts, or directly to the plugin executable.
diff --git a/docs/migration.md b/docs/migration.md
@@ -1,5 +1,10 @@
 Here is the list of upgrade notes for major breaking changes that you need to be aware of when migrating between the BitOps versions.
 
+## v2.4
+* `ANSIBLE_ROOT` ENV var in Ansible plugin was removed. Use `BITOPS_OPSREPO_ENVIRONMENT_DIR` instead, which is consistent across all plugins.
+* Plugin environment variables are now automatically mapped to the plugin schema. This means that you can now use the same environment variable names as in the plugin schema. For example, if you want to override the `ansible.cli.skip-tags` value, you can now use the `BITOPS_ANSIBLE_SKIP_TAGS` environment variable. This is true for all the plugins. The precedence order is: `ENV` vars > `bitops.config.yaml` values > `bitops.config.schema.yaml` defaults.
+Keep the new convention in mind when upgrading to `v2.4` as it may clash with your existing environment variables.
+
 ## v2.2
 * Terraform plugin `stack-action` was moved from `options` to `cli` section in `bitops.config.yaml`.
 You need to update your configuration from old:

diff --git a/docs/roadmap.md b/docs/roadmap.md
@@ -22,6 +22,13 @@ Create an Issue, open a Discussion, join our Community and we'll be happy to wor
 Check the [`main`](https://github.com/bitovi/bitops) repository branch to see the ongoing development.
 
 ## Release History
+### Done in v2.4.0
+  - **Plugins:** ENV variable mapping based on plugin schema
+  - **Plugins:** Plugin CLI command generation based on schema (beta)
+  - **Website:** Revamp with new video and list of BitOps values
+  - **Community:** Switch from Slack to Discord
+  - **Community:** Add non-stop user survey
+
 ### Done in v2.3.0
   - **Security:** Secrets masking
   - **Core:** Local plugin install via `file://`

diff --git a/docs/tool-configuration/configuration-ansible.md b/docs/tool-configuration/configuration-ansible.md
@@ -1,9 +1,9 @@
-> ⚠️ Note from the developers: We are currently in the process of moving our documentation and so the below documentation is only partially correct. For more information on this tool please checkout our [plugin documentation](https://github.com/bitops-plugins/ansible).
+> ⚠️ For more information on this tool please checkout [Ansible plugin repository](https://github.com/bitops-plugins/ansible).
 
 # Ansible
 
 ### Example `bitops.config.yaml`, minimum required: 
-```
+```yaml
 ansible:
     cli: {}
     options: {}
@@ -20,43 +20,33 @@ ansible:
     forks: 20
     inventory: beta
     skip-tags: ignore-this-tag
-    tags: run-this-tag
-    vault-id: [dev@dev-passwordfile, prod@prod-passwordfile]
-    vault-password-file: $TEMPDIR/secrets/password_file
-  options:
+    tags: run-with-this-tag
     dryrun: false
-    verbosity: 4
+  options:
+    verbosity: 0
+    skip-deploy: false
 ```
 
-## CLI Configuration
-
-| Property            | Environmental Variable             | Description                                                  | Default | Required |
-| ------------------- | ---------------------------------- | ------------------------------------------------------------ | ------- | -------- |
-| main-playbook       | BITOPS_ANSIBLE_MAIN_SCRIPT         | Specify an entry playbook to run ansible-playbook with. |   `playbook.yaml` | Yes |
-| extra-vars          | BITOPS_ANSIBLE_EXTRA_VARS          | Add additional ansible playbook parameters directly or load via JSON/YAML file. |         |          |
-| flush-cache         | BITOS_ANSIBLE_FLUSH_CACHE          | Clear the fact cache for every host in the inventory.        |         |          |
-| force-handlers      | BITOPS_ANSIBLE_FORCE_HANDLERS      | Clear the fact cache for every host in the inventory.        |         |          |
-| forks               | BITOPS_ANSIBLE_FORKS               | Specify the number of parallel processes to use              | 5       |          |
-| inventory           | BITOPS_ANSIBLE_INVENTORY           | Specify inventory host path or comma-separated host list.    |         |          |
-| skip-tags           | BITOPS_ANSIBLE_SKIP_TAGS           | Only run plays and tasks whose tags do not match these values. |         |          |
-| tags                | BITOPS_ANSIBLE_TAGS                | Only run plays and tasks tagged with these values.           |         |          |
-| vault-id            | BITOPS_ANSIBLE_VAULT_ID            | This is a list.  Specify Ansible vault-id `[dev@dev-passwordfile]` or multiple `[dev@dev-passwordfile, prod@prod-passwordfile]` or password client script `[[email protected]]`. Cannot be used with `@prompt` for equivalent `--ask-vault-pass` functionality |         |          |
-| vault-password-file | BITOPS_ANSIBLE_VAULT_PASSWORD_FILE | Specify Ansible vault password file for decryption.          |         |          |
+## CLI configuration
+CLI configuration is used to pass in CLI parameters to the ansible-playbook command.
 
-## Options Configuration
-
-| Property  | Environmental Variable | Description                                                  | Default | Required |
-| --------- | ---------------------- | ------------------------------------------------------------ | ------- | -------- |
-| dryrun    | BITOPS_ANSIBLE_DRYRUN  | Will run `--list-tasks` but won't actually execute playbook(s) |  `false` |          |
-| verbosity | ANSIBLE_VERBOSITY      | Acceptable values `0|1|2|3|4`. Equivalent to adding `-verbose` or repeating `-v` flags. Will override a pre-existing `ANSIBLE_VERBOSITY` environmental variable or `[default]` `verbosity=` setting in ansible.cfg. | N/A     |          |
-
-## Additional Environment Variable Configuration
-Although not captured in `bitops.config.yaml`, the following environment variables can be set to further customize behavior.
-
-| Environmental Variable | Description                                                  |
-| ---------------------- | ------------------------------------------------------------ |
-| EXTRA_ENV              | Before Ansible playbook execution, BitOps will look for an `extra_env` file containing additional environment parameters (`FOO=val1`) in the Ansible plugin directory. If found, the values will be exported to the BitOps environment. |
-| ANSIBLE_SKIP_DEPLOY    | Will skip all ansible executions. This supersedes all other configurations. |
+| **Parameter** | **Environment Variable** | **Type** | **Required** | **Default** | **Description** |
+|   |  |   |   |   |   |
+| `main-playbook`  | `BITOPS_ANSIBLE_MAIN_PLAYBOOK`    | _string_  | _yes_ | `playbook.yaml` | Specify which playbook to run ansible-playbook with |
+| `extra-vars`     | `BITOPS_ANSIBLE_EXTRA_VARS`     | _string_  |   |  | Add additional ansible playbook parameters directly or load via JSON/YAML file. |
+| `flush-cache`    | `BITOPS_ANSIBLE_FLUSH_CACHE`    | _boolean_ |   |  | Clear the fact cache for every host in inventory. |
+| `force-handlers` | `BITOPS_ANSIBLE_FORCE_HANDLERS` | _boolean_ |   |  | Clear the fact cache for every host in inventory. |
+| `forks`          | `BITOPS_ANSIBLE_FORKS`          | _integer_ |   |  | Specify number of parallel processes to use. |
+| `inventory`      | `BITOPS_ANSIBLE_INVENTORY`      | _string_  |   |  | Specify inventory host path or comma separated host list. |
+| `skip-tags`      | `BITOPS_ANSIBLE_SKIP_TAGS`      | _string_  |   |  | Only run plays and tasks whose tags do not match these values. |
+| `tags`           | `BITOPS_ANSIBLE_TAGS`           | _string_  |   |  | Only run plays and tasks tagged with these values. |
+| `dryrun`         | `BITOPS_ANSIBLE_DRYRUN`         | _boolean_ |   |  | Don't make any changes; instead, try to predict some of the changes that may occur. |
 
 
+## Options Configuration
+Options configurations are used to export variables without using the CLI generation or for any advanced logic that is not supported by the Ansible CLI.
 
+| **Parameter** | **Environment Variable** | **Type** | **Required** | **Default** | **Description** | 
+|  |  |   |   |   |   |
+| `skip-deploy` | `ANSIBLE_SKIP_DEPLOY`      | _boolean_ |  |  | If set to "true", regardless of the stack-action, deployment actions will be skipped. | 
+| `verbosity`   | `BITOPS_ANSIBLE_VERBOSITY` | _integer_ |  |  | Equivalent to adding `-verbose` or repeating `-v` flags. Will override `[default]` `verbosity=` setting in ansible.cfg. Acceptable values `0\|1\|2\|3\|4`. |