Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to create custom types. #27365

Open
adamdonahue opened this issue Dec 24, 2020 · 16 comments
Open

Ability to create custom types. #27365

adamdonahue opened this issue Dec 24, 2020 · 16 comments
Labels
enhancement new new issue not yet triaged

Comments

@adamdonahue
Copy link

adamdonahue commented Dec 24, 2020

Instead of something like this:

module-inputs.tf

variable "configuration" {
  type = object({
      x = string
      y = string
      z = list(string)
   })
}

infra-inputs.tf

variable "named-configurations" {
  type = map(
    object({
      x = string
      y = string
      z = list(string)
   })
  )
}

which, here and especially for multiple top-level modules, involves a ridiculous amount of boilerplate, we'd like to see something along the lines of:

types.tf

type "configuration" {
  definition = map(
    object({
      x = string
      y = string
      z = list(string)
   })
  )
} 

and then

module-inputs.tf

variable "configurtion" {
  type = types.configuration
}

infra-inputs.tf

variable "named-configurations" {
  type = map(types.configuration)
}
@adamdonahue adamdonahue added enhancement new new issue not yet triaged labels Dec 24, 2020
@adamdonahue
Copy link
Author

Just wanted to check on the status of this -- it's very useful.

It'd actually be useful to somehow be able to export a type definition from a module, as well. So we can define the input type(s) for a module and reuse them in other places.

@JohannesMoersch
Copy link

Has there been any discussion or consideration of this enhancement? I think this would be super valuable, and would be one of the best things that could be done to improve consistency and code quality within terraform.

@binte
Copy link

binte commented Sep 16, 2021

This would indeed be very helpful. Any chance we might be able to see this at some point?

@juarezr
Copy link

juarezr commented Oct 28, 2021

Complementing this feature request, it would be pretty useful to also have the same features as input variable in custom type definitions:

Below an example:

type "db_password" {  
  description = "Database administrator password"
  type        = string  
  sensitive   = true
}

type "timestamp" {
  description = "Example of how to validate a type a formula defined by an expression"
  type        = string
  validation {
    condition     = can(formatdate("", type.timestamp))
    error_message = "value for timestamp type requires a valid RFC 3339 timestamp"
  }
  default = timestamp()
}

type "result" {
  description = "Example of validation for enum/option type"
  type        = string
  validation {
    condition = anytrue([
      type.result == "good",
      type.result == "bad",
      type.result == "ugly"
    ])
    error_message = "value for variable of type result must be 'good', 'bad', or 'ugly'"
  }
  default = "good"
}

local {
  ip_address_regex = "^(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$"
}

type "ip_address" {
  description = "Example of using a local constant for type validation"
  type = string
  validation {
   condition = can(regex(local.ip_address_regex, type.value))
        error_message = "Invalid IP address provided."
  }
}

type "configuration" {
  description = "Example of using other types for type definition"
  type = map(
    object({
      x = type.timestamp
      y = type.result
      z = list(type.ip_address)
   })
  )
}

variable "configuration" {
  type = type.configuration
}

variable "multiple-configurations" {
  type = list(type.configuration)
}

@Vetrenik
Copy link

Truely interested in this feature!

@julienbeaussier
Copy link

That feature would be indeed very useful and a great complement of this optional var improvement coming in v1.3.

@ImadYIdrissi
Copy link

ImadYIdrissi commented Sep 28, 2022

I adhere to the declarative paradigm when it is available, because it adds visibility and versionning to previously unused branches (by me at least) such as infrastructure build and evolution, and it also describes "what we want" instead of "how we obtain" it... Unfortunately, it is still a pain today as many basic coding tools that exist in the imperative paradigm are quite far behind in Terraform.

I'm quite new to contributing to opensource, I've seen in the contribution guide for Terraform that a new feature must be discussed and receive feedback before implementations are even considered. I'd say this issue is it, and would love to dip my toe in building a solution. As such, I'd love some pointers on where to start.

Kudos for this feature request.

@icicimov
Copy link

icicimov commented Oct 5, 2022

I wished this existed more than once in the past. Every time a have to declare multiple variables of the same structure I wish I could declare a custom type once and use it multiple times declaratively instead of replicating the same structure over and over again in each var.

@JonRoma
Copy link

JonRoma commented Nov 17, 2022

This would be an incredibly useful feature, particularly if there is a mechanism to reuse definitions of complex types across multiple Terraform modules.

If this goes forward, please avoid repeating the moderately annoying fandango with inconsistent nameing – e.g., the locals {} blocks are declared as a plural, while references to local variable uses the singular form as local.foo.

@bwinter
Copy link

bwinter commented Nov 29, 2022

It would also be useful to specify another module as a type. So that a sub field of your config could be held to that modules interface, rather than having to redefine it. e.g.

module/infra.tf

variable "configuration" {
  type = object({
      x = string
      y = string
      z = list(string)
   })
}

module/consumer.tf

variable "configurations" {
  type = object({
    clusterName = string
    configs = list(module.infra)
  }
}

This way you could have nested relationships / dependencies; but, wouldn't have to redefine the interface to ensure the correct variables were passed.

@dylanturn
Copy link

It would also be useful to specify another module as a type. So that a sub field of your config could be held to that modules interface, rather than having to redefine it. e.g.

module/infra.tf

variable "configuration" {
  type = object({
      x = string
      y = string
      z = list(string)
   })
}

module/consumer.tf

variable "configurations" {
  type = object({
    clusterName = string
    configs = list(module.infra)
  }
}

This way you could have nested relationships / dependencies; but, wouldn't have to redefine the interface to ensure the correct variables were passed.

I like the idea of some of the more complex suggestions, but honestly even just this would be nice. Often times simple resource modules are used in more complex module compositions and being able to pass complex variable types forward would make life so much easier.

I think a great example of this is found in the AWS EKS module. As it currently stands the terraform-aws-eks module defines a variable of eks_managed_node_groups with a type of any. This means that when you're using your IDE to try deploy an EKS cluster the Terraform language server is powerless to give you any suggestions. Being able to pass the entirety of the eks_managed_node_group as a type would be a HUGE improvement to the developer trying to use it.

Example of what it could look like:

module "eks_managed_node_group" {
  source = "./modules/eks-managed-node-group"

  for_each = { for k, v in var.eks_managed_node_groups : k => v if var.create && !local.create_outposts_local_cluster }

  ... rest of the config ...
}

variable "eks_managed_node_groups" {
  description = "Map of EKS managed node group definitions to create"
  type        = map(module.eks_managed_node_group)
  default     = {}
}

Additionally, what would also be nice is something like a data source that could be used to lookup and reference a non-local modules variable type. Something like this:

# This data source gets the source modules variable types.
data "terraform_remote_types" "example_module" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.0"
}

variable "module_as_variable" {
  type = data.terraform_remote_types.example_module
}

Building on this idea, perhaps we could even reference the type of a single module variable too:

variable "module_subset_as_variable" {
  type = data.terraform_remote_types.example_module.self_managed_node_group_defaults
}

@maroux
Copy link

maroux commented May 4, 2023

One workaround I've found for this:

variables.tf

variable "config_a" {}

typed_variables.tf

module "config_a" {
  source = "./modules/type-config"
  input    = var.config_a
}

./modules/type-config/variables.tf

variable "input" {
  type = object({
      x = string
      y = string
      z = list(string)
  })
}

./modules/type-config/outputs.tf

output "value" {
  value = var.input
}

Now you can use module.config_a.value as a typed value.

@juarezr
Copy link

juarezr commented May 4, 2023

One workaround I've found for this:
...

Now you can use module.config_a.value as a typed value.

Nice! I still miss a shared type for variables instead of workarounds. Maybe someday.

Another (ugly) workaround is using symlinks for variable definitions files:

Pros:

  • Avoids the need to sync the copy-n-paste of common variable definitions into distinct folders/states/modules.
  • Allows splitting huge remote states into smaller, easier-to-migrate and evolve, separated folders/states/modules.
  • Allow sharing between remote states/modules the same tfvars and tfbackend files corresponding to environmental parameters, cloud configuration, application settings, typed modules, etc...

Cons:

  • Workarounds with symlinks don't play fair when using Windows.
  • It requires that you spend some additional effort to ensure that you are doing correctly:
    1. Clone the Git repository running as administrator.
    2. Using the git clone --config core.symlinks=true <repos-url> flag while cloning the repository
    3. Paying attention while committing new symlinks files to the repository, pull and merge
  • Sometimes it doesn't worth the extra effort

Example:

In ../shared/vars/vars-env.tf

variable "env" {
  description = "Environment Variables"
  type = object({
    name    = string
    project = string
    region  = string
    zone    = string
    tags = map(string)
  })

In ../shared/config/vars-env-dev.tfvars

env = {
  name    = "dev"
  project = "POC-project-123456"
  region  = "us-east4"
  zone    = "us-east4-a"
  tags = {
    "costs": "owned",
    "env": "dev",
  }
}

In ../shared/config/tfstate-dev.tfbackend one can put the remote state configuration like:

bucket  = "tfstate-dev"

In ../tf/module1/tf-providers-module1.tf one can use the var.env reference:

terraform {
  required_version = ">= 1.3.3, < 1.5.0"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">= 4.61.0, < 5.0.0"
    }
  }
}

provider "google" {
  project = var.env.project
  region  = var.env.region
  zone    = var.env.zone
}

terraform {
  backend "gcs" {
    prefix = "tf/module1"
  }
}

In ./tf/module1/ one should run the following commands to link to the shared files:

$ ls -s ../../shared/vars/env.tf env.tf
$ ls -s ../../shared/config config

After this, one can run terraform as usual. In ./tf/module1/ it would be:

$ terraform init -backend-config=./config/tfstate-dev.tfbackend
$ terraform plan  -var-file=./config/vars-env-aws-dev.tfvars

TLDR: A lot of effort to share some types.

@aaziz993
Copy link

Are there any official solutions about this issue nowadays.

@fardarter
Copy link

Coming here to wish for this.

@ketzacoatl
Copy link
Contributor

Custom data types is one of the most powerful features in Haskell, and it opens a lot of doors (pattern matching on sum types, for example). I would love to see a similar capability in Terraform! It would also improve our R&D feedback cycle, where terraform plan would be able to find issues with those types not matching or lining up the way they need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement new new issue not yet triaged
Projects
None yet
Development

No branches or pull requests