inspec-gcp not working in kitchen-terraform v5.0 #351

tdsacilowski · 2019-07-03T19:19:21Z

Hi all. I'm working on building out some tests against resources being built by Terraform in GCP. I had been using KT v4.9.0 and I was able to make use of inspec-gcp:

My Gemfile:

source "https://rubygems.org/" do
  gem "kitchen-terraform", "~> 4.0"
end

My kitchen.yml file:

---
driver:
  name: terraform

  variables:
    gcp_project: teddy-testing
    tfe_instance_zone: us-east4-a
    tfe_instance_name: tfe

provisioner:
  name: terraform

verifier:
  name: terraform

platforms:
  - name: terraform

suites:
  - name: root-example

    driver:
      root_module_directory: examples/root-example

    verifier:
      systems:
        - name: inspec-gcp
          backend: gcp
          controls:
            - gcp_compute_instance_exists

        - name: local
          backend: local
          controls:
            - tfe_ports_reachable

My inspec.yml file:

---
name: root-example

depends:
  - name: inspec-gcp
    git: https://github.com/inspec/inspec-gcp.git
    tag: v0.10.0
#supports:
#  - platform: gcp

attributes:
  - name: input_gcp_project
    required: true
    type: string

  - name: input_tfe_instance_zone
    required: true
    type: string

  - name: input_tfe_instance_name
    required: true
    type: string

  - name: output_tfe_instance_public_ip
    required: true
    type: string

My results:

❯ bundle exec kitchen verify -D
-----> Starting Kitchen (v1.25.0)
-----> Verifying <root-example-terraform>...
$$$$$$ Running command `terraform workspace select kitchen-terraform-root-example-terraform` in directory /Users/teddy/Dev/tfe-demo/terraform-gcp/examples/root-example
$$$$$$ Running command `terraform output -json` in directory /Users/teddy/Dev/tfe-demo/terraform-gcp/examples/root-example
inspec-gcp: Verifying

Profile: root-example
Version: (not specified)
Target:  gcp://[email protected]

  ✔  gcp_compute_instance_exists: Instance tfe
     ✔  Instance tfe status should eq "RUNNING"
     ✔  Instance tfe first_network_interface_nat_ip_exists should equal true
     ✔  Instance tfe first_network_interface_name should eq "external-nat"
     ✔  Instance tfe first_network_interface_type should eq "one_to_one_nat"


Profile: Google Cloud Platform Resource Pack (inspec-gcp)
Version: 0.9.0
Target:  gcp://[email protected]

     No tests executed.

Profile Summary: 1 successful control, 0 control failures, 0 controls skipped
Test Summary: 4 successful, 0 failures, 0 skipped
local: Verifying
Skipping profile: 'inspec-gcp' on unsupported platform: 'mac_os_x/18.6.0'.

Profile: root-example
Version: (not specified)
Target:  local://

     No tests executed.

Test Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <root-example-terraform> (0m5.96s).
-----> Kitchen is finished. (0m8.59s)

However, my local tests were being skipped due to the host resource seemingly not being supported on MacOS X. Digging through inspec updates, seemed that host is supported on MacOS X, but in version 4, so I decided to try to upgrade kitchen-terraform to v5.0 since it updated inspec to version 4.

My gemfile:

source "https://rubygems.org/" do
  gem "kitchen-terraform", "~> 5.0"
end

No changes to kitchen.yml and inspec.yml.

My results:

❯ bundle exec kitchen verify -D
-----> Starting Kitchen (v2.2.5)
-----> Verifying <root-example-terraform>...
$$$$$$ Running command `terraform workspace select kitchen-terraform-root-example-terraform` in directory /Users/teddy/Dev/tfe-demo/terraform-gcp/examples/root-example
$$$$$$ Running command `terraform output -json` in directory /Users/teddy/Dev/tfe-demo/terraform-gcp/examples/root-example
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Verify failed on instance <root-example-terraform>.  Please see .kitchen/logs/root-example-terraform.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Log output:

cat .kitchen/logs/root-example-terraform.log                                        
I, [2019-07-03T15:13:01.409279 #81601]  INFO -- root-example-terraform: -----> Verifying <root-example-terraform>...
W, [2019-07-03T15:13:01.409750 #81601]  WARN -- root-example-terraform: Running command `terraform workspace select kitchen-terraform-root-example-terraform` in directory /Users/teddy/Dev/tfe-demo/terraform-gcp/examples/root-example
E, [2019-07-03T15:13:02.020103 #81601] ERROR -- root-example-terraform: Verify failed on instance <root-example-terraform>.
E, [2019-07-03T15:13:02.020244 #81601] ERROR -- root-example-terraform: ------Exception-------
E, [2019-07-03T15:13:02.020270 #81601] ERROR -- root-example-terraform: Class: Kitchen::ActionFailed
E, [2019-07-03T15:13:02.020287 #81601] ERROR -- root-example-terraform: Message: inspec-gcp: Can't find train plugin gcp. Please install it first.
E, [2019-07-03T15:13:02.020303 #81601] ERROR -- root-example-terraform: ----------------------
E, [2019-07-03T15:13:02.020318 #81601] ERROR -- root-example-terraform: ------Backtrace-------
E, [2019-07-03T15:13:02.020363 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/kitchen-terraform-5.0.0/lib/kitchen/verifier/terraform.rb:98:in `rescue in call'
E, [2019-07-03T15:13:02.020389 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/kitchen-terraform-5.0.0/lib/kitchen/verifier/terraform.rb:91:in `call'
E, [2019-07-03T15:13:02.020407 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:459:in `block in verify_action'
E, [2019-07-03T15:13:02.020430 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:552:in `synchronize_or_call'
E, [2019-07-03T15:13:02.020496 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:514:in `block in action'
E, [2019-07-03T15:13:02.020532 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/2.6.0/benchmark.rb:293:in `measure'
E, [2019-07-03T15:13:02.020568 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:513:in `action'
E, [2019-07-03T15:13:02.020598 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:451:in `verify_action'
E, [2019-07-03T15:13:02.020629 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:382:in `block (2 levels) in transition_to'
E, [2019-07-03T15:13:02.020646 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/lifecycle_hooks.rb:45:in `run_with_hooks'
E, [2019-07-03T15:13:02.020663 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:381:in `block in transition_to'
E, [2019-07-03T15:13:02.020712 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:380:in `each'
E, [2019-07-03T15:13:02.020748 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:380:in `transition_to'
E, [2019-07-03T15:13:02.020790 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:162:in `verify'
E, [2019-07-03T15:13:02.020816 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/command.rb:197:in `public_send'
E, [2019-07-03T15:13:02.020834 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/command.rb:197:in `run_action_in_thread'
E, [2019-07-03T15:13:02.020868 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
E, [2019-07-03T15:13:02.020884 #81601] ERROR -- root-example-terraform: ----End Backtrace-----
E, [2019-07-03T15:13:02.020923 #81601] ERROR -- root-example-terraform: ---Nested Exception---
E, [2019-07-03T15:13:02.020957 #81601] ERROR -- root-example-terraform: Class: Kitchen::Terraform::Error
E, [2019-07-03T15:13:02.020994 #81601] ERROR -- root-example-terraform: Message: inspec-gcp: Can't find train plugin gcp. Please install it first.
E, [2019-07-03T15:13:02.021029 #81601] ERROR -- root-example-terraform: ----------------------
E, [2019-07-03T15:13:02.021060 #81601] ERROR -- root-example-terraform: ------Backtrace-------
E, [2019-07-03T15:13:02.021096 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/kitchen-terraform-5.0.0/lib/kitchen/verifier/terraform.rb:98:in `rescue in call'
E, [2019-07-03T15:13:02.021130 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/kitchen-terraform-5.0.0/lib/kitchen/verifier/terraform.rb:91:in `call'
E, [2019-07-03T15:13:02.021166 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:459:in `block in verify_action'
E, [2019-07-03T15:13:02.021199 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:552:in `synchronize_or_call'
E, [2019-07-03T15:13:02.021235 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:514:in `block in action'
E, [2019-07-03T15:13:02.021270 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/2.6.0/benchmark.rb:293:in `measure'
E, [2019-07-03T15:13:02.021305 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:513:in `action'
E, [2019-07-03T15:13:02.021334 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:451:in `verify_action'
E, [2019-07-03T15:13:02.021375 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:382:in `block (2 levels) in transition_to'
E, [2019-07-03T15:13:02.021406 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/lifecycle_hooks.rb:45:in `run_with_hooks'
E, [2019-07-03T15:13:02.021433 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:381:in `block in transition_to'
E, [2019-07-03T15:13:02.021473 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:380:in `each'
E, [2019-07-03T15:13:02.021508 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:380:in `transition_to'
E, [2019-07-03T15:13:02.021536 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/instance.rb:162:in `verify'
E, [2019-07-03T15:13:02.021578 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/command.rb:197:in `public_send'
E, [2019-07-03T15:13:02.021606 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/command.rb:197:in `run_action_in_thread'
E, [2019-07-03T15:13:02.021649 #81601] ERROR -- root-example-terraform: /Users/teddy/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/test-kitchen-2.2.5/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
E, [2019-07-03T15:13:02.021680 #81601] ERROR -- root-example-terraform: ----End Backtrace-----

Not sure if this is a bug or if I need to alter my configuration. Can't seem to figure out the root issue here.

As a side note, updating to KT v5.0 (inspec v4) doesn't seem to help with my host resource issue so I'll likely need to spin up a Vagrant box running Linux to test this:

❯ bundle exec kitchen verify -D
-----> Starting Kitchen (v2.2.5)
-----> Verifying <root-example-terraform>...
$$$$$$ Running command `terraform workspace select kitchen-terraform-root-example-terraform` in directory /Users/teddy/Dev/tfe-demo/terraform-gcp/examples/root-example
$$$$$$ Running command `terraform output -json` in directory /Users/teddy/Dev/tfe-demo/terraform-gcp/examples/root-example
local: Verifying
Skipping profile: 'inspec-gcp' on unsupported platform: 'mac_os_x/18.6.0'.

Profile: root-example
Version: (not specified)
Target:  local://

     No tests executed.

Test Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <root-example-terraform> (0m1.76s).
-----> Kitchen is finished. (0m4.56s)

The text was updated successfully, but these errors were encountered:

aaron-lane · 2019-07-04T13:35:00Z

Hello again, @tdsacilowski!

How did you detemine that the lack of InSpec 3's support for host on MacOS was causing the local system to not execute any tests? I'm surprised that all tests would be skipped because of that issue. I have seen that behaviour before when a control file had an error which was being silently swallowed by InSpec. If you're not certain about the cause, feel free to share the control file and we can dig in to it.

With respect to InSpec 4 and inspec-gcp, inspec-gcp is apparently not yet compatible. 😢 I've opened inspec/inspec-gcp#153 to request support for InSpec 4.

tdsacilowski · 2019-07-04T13:49:46Z

Hi Aaron, wrt the host issue I was seeing, I did a bit more digging and thing I found the cause. I outlined what I found over at the inspec project: inspec/inspec#4253.

And to clarify, I only had one local test, using the host resource and it was skipped.

And thanks for the heads-up on inspec-gcp not being yet compatible with inspec 4. Wasn’t sure how to tell if it was or wasn’t. (And now, reviewing inspec/inspec-gcp#153, I see that it’s pinned to inspec 3)

For now I’ll move back to kitchen-terraform 4.9.0. Thanks!

bgeesaman · 2019-08-23T02:35:52Z

I think I have things pretty well pinned down (hah!)

FROM ruby:alpine

RUN mkdir -p /share
RUN apk add --no-cache --update build-base~=0.5-r1 libxml2-dev~=2.9.9-r1 \
    libffi-dev~=3.2.1-r6 git~=2.20.1-r0 openssh-client~=7.9_p1-r5 && \
    gem install inspec:3.9.3 kitchen-inspec:1.2.0 kitchen-terraform:5.1.0 --no-document 

# See: https://releases.hashicorp.com/terraform/ TERRAFORM_SHA256SUM="6544eb55b3e916affeea0a46fe785329c36de1ba1bdb51ca5239d3567101876f"
ENV TERRAFORM_VERSION="0.11.14" \
    TERRAFORM_SHA256SUM="9b9a4492738c69077b079e595f5b2a9ef1bc4e8fb5596610f69a6f322a8af8dd"

WORKDIR /tmp

# Terraform
RUN curl -sSL https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip \
      > terraform.zip && \
    echo "${TERRAFORM_SHA256SUM}  terraform.zip" > shasums && \
    sha256sum -c shasums && \
    rm shasums && \
    unzip -o terraform.zip -d /usr/local/bin && \
    chmod +x /usr/local/bin/terraform && \
    rm -f terraform_${TERRAFORM_VERSION}_linux_amd64.zip

CMD ["inspec"]
VOLUME ["/share"]
WORKDIR /share

Enter the container

docker run --rm -it -v `pwd`:/share -v ~/.config:/root/.config myimage:latest ash

/share # inspec exec test/integration/example -t gcp:// --attrs test/integration/attributes.yml
Can't find train plugin gcp. Please install it first.
/share# gem which rbnacl
/usr/local/bundle/gems/rbnacl-4.0.2/lib/rbnacl.rb
/share# exit

Edit the Dockerfile to move to kitchen-terraform 4.8.1:

   gem install inspec:3.9.3 kitchen-inspec:1.2.0 kitchen-terraform:4.8.1 --no-document

Enter the new container:

docker run --rm -it -v `pwd`:/share -v ~/.config:/root/.config myimage:latest ash

/share # inspec exec test/integration/example -t gcp:// --attrs test/integration/attributes.yml
...snip...
Profile Summary: 1 successful control, 0 control failures, 0 controls skipped
Test Summary: 20 successful, 0 failures, 0 skipped

/share # gem which rbnacl
ERROR:  Can't find Ruby library file or shared library rbnacl

/share # gem install rbnacl:4.0.2
Fetching rbnacl-4.0.2.gem
Successfully installed rbnacl-4.0.2
1 gem installed

/share # inspec exec test/integration/example -t gcp:// --attrs test/integration/attributes.yml
Can't find train plugin gcp. Please install it first.

/share # gem uninstall rbnacl
Successfully uninstalled rbnacl-4.0.2

/share # inspec exec test/integration/example -t gcp:// --attrs test/integration/attributes.yml
...snip...
Profile Summary: 1 successful control, 0 control failures, 0 controls skipped
Test Summary: 20 successful, 0 failures, 0 skipped

/share # exit

The only downside to this workaround is that kitchen-terraform 4.8.1 works with terraform 0.11.4+ and I'd like to use 0.12+ and I believe 5.x+ supports that. Thanks for reading!

aaron-lane · 2019-08-24T15:27:19Z

@bgeesaman thank you for the investigation! This definitely helps to clarify the problem.

aaron-lane · 2019-08-24T16:13:17Z

I believe I have identified the issue. While rbnacl is intsalled, require "train/transports/gcp" fails with the following error:

LoadError: Could not open library 'sodium': Error loading shared library sodium: No such file or directory.
Could not open library 'libsodium.so': Error loading shared library libsodium.so: No such file or directory
from /usr/local/bundle/gems/ffi-1.11.1/lib/ffi/library.rb:145:in `block in ffi_lib'

After installing libsodium-dev in the container, inspec exec no longer fails due to the missing Train GCP plugin. We may have to remove the dependency on rbnacl and document it as an optional requirement if support for ed25519 keys is required.

aaron-lane · 2019-08-24T16:36:06Z

Can some one please verify if this branch solves the issue? #358

bgeesaman · 2019-08-26T14:15:18Z

I can confirm both solutions work for me:

Installing libsodium-dev in the container and installing KT 5.1.0 OR
Not installing libsodium-dev and just Installing KT from the branch in Remove rbnacl from gem dependencies #358

Thank you, Aaron!

aaron-lane mentioned this issue Jul 4, 2019

Add support for InSpec 4 inspec/inspec-gcp#153

Closed

aaron-lane mentioned this issue Jul 6, 2019

Reintroduce Support for InSpec 3 #352

Merged

aaron-lane closed this as completed in #352 Jul 6, 2019

tdsacilowski mentioned this issue Jul 9, 2019

Work towards updating to Kitchen-Terraform 5 tdsacilowski/tfe-demo-modules#5

Merged

aaron-lane reopened this Aug 22, 2019

aaron-lane mentioned this issue Aug 24, 2019

Remove rbnacl from gem dependencies #358

Merged

aaron-lane closed this as completed in #358 Aug 29, 2019

aaron-lane mentioned this issue Feb 12, 2020

Kitchen verify errors out for Inspec 4.18 and Kitchen-terraform - Can't find train plugin #383

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

inspec-gcp not working in kitchen-terraform v5.0 #351

inspec-gcp not working in kitchen-terraform v5.0 #351

tdsacilowski commented Jul 3, 2019

aaron-lane commented Jul 4, 2019

tdsacilowski commented Jul 4, 2019 •

edited

Loading

bgeesaman commented Aug 23, 2019 •

edited

Loading

aaron-lane commented Aug 24, 2019

aaron-lane commented Aug 24, 2019

aaron-lane commented Aug 24, 2019

bgeesaman commented Aug 26, 2019

inspec-gcp not working in kitchen-terraform v5.0 #351

inspec-gcp not working in kitchen-terraform v5.0 #351

Comments

tdsacilowski commented Jul 3, 2019

aaron-lane commented Jul 4, 2019

tdsacilowski commented Jul 4, 2019 • edited Loading

bgeesaman commented Aug 23, 2019 • edited Loading

aaron-lane commented Aug 24, 2019

aaron-lane commented Aug 24, 2019

aaron-lane commented Aug 24, 2019

bgeesaman commented Aug 26, 2019

tdsacilowski commented Jul 4, 2019 •

edited

Loading

bgeesaman commented Aug 23, 2019 •

edited

Loading