Ansible Tower and Hashicorp Vault – Looking Up Multiple Secrets

A question that I’ve been approached about several times recently is how to lookup multiple Hashicorp Vault Secrets and assign them to a single Ansible Tower Credential for use in a Playbook. A while ago I looked at the process of integrating Hashicorp Vault with Ansible Tower (a less that perfect process in the first place) but this has repeatedly led to the same question about multiple Secrets, so in this post we’ll dive in to that scenario as well as mapping secrets from multiple Vault Secret Engines.

Please take a look at the previous article HERE before diving in otherwise this might not be very clear.

Tower Isn’t Very Flexible Here!

The input specification for a Tower Custom Credential is incredibly rigid (it’s defined here if you want to see it) and limits your input type to only strings and booleans. This means sending our secrets as a list or dictionary is out which would let us operate with some grace.

The only real way I’ve found to get around this and maintain some flexibility is to instead define multiple inputs in your Custom Credential Input Configuration and make only the first mandatory (using the Required key). This will give us a Credential type which offers us the ability to look up multiple secrets but only requires that we lookup a minimum of one. Below is an example of an input configuration we can use:

--# INPUT CONFIGURATION

--# These keys define the GUI fields which will be presented in Tower. One is offered
--# FOR EACH SECRET LOOKUP. Strings  are named Secret1, Secret2 etc. and will hold
--# the Vault Secret data initially.
--# Each has a unique id (lookupsecret1, lookupsecret2 etc.) and their "type" is "secret"
--# meaning they will be obscured on input. As we will always need to work with a minimum
--# of a single secret; we have also set the first secret as "required" meaning it will
--a become a mandatory field in the GUI
fields:
  - id: lookupsecret1
    type: string
    label: Secret1
    secret: true
  - id: lookupsecret2
    type: string
    label: Secret2
    secret: true
  - id: lookupsecret3
    type: string
    label: Secret3
    secret: true
  - id: lookupsecret4
    type: string
    label: Secret4
    secret: true
  - id: lookupsecret5
    type: string
    label: Secret5
    secret: true
required:
  - lookupsecret1

The Injector Configuration‘s extra_vars key is prone to exactly the same rigidity and allows only the output of strings, so out goes the idea of constructing a list or dictionary of new vars to pass directly to a Playbook here either, we can however send each to it’s own variable. Inside the Playbook these will just be empty values if no secret has been passed.

--# INJECTOR CONFIGURATION

--# Here the "lookupsecret" values are converted to a new variables named
--# vault_secret1, vault_secret2 etc., these values can be read in to
--# playbooks
extra_vars:
  vault_secret1: '{{ lookupsecret1 }}'
  vault_secret2: '{{ lookupsecret2 }}'
  vault_secret3: '{{ lookupsecret3 }}'
  vault_secret4: '{{ lookupsecret4 }}'
  vault_secret5: '{{ lookupsecret5 }}'

Looking Up Multiple Secrets in Ansible Tower

As with our previous example we can now load a new Credential type in to Tower, however now we have the option to load multiple secrets in to a single Credential:

For clarity, we’ll be using the below secret in Vault our target:

We’re going to perform lookups exactly as in the previous article, but only for SECRET1, SECRET2 and SECRET3, leaving the other two Secret Inputs empty (to ensure that this won’t cause any issues), then attach our new Credential to a Template as usual:

…we’re then going to run a very primitive Playbook as a Sanity Test to confirm that the secrets are being read by Ansible:

---
- name: credentialdebug
  hosts: localhost
  connection: local
  gather_facts: false
  any_errors_fatal: true
  vars:
    vault_lookups:
      - '{{ vault_secret1 }}'
      - '{{ vault_secret2 }}'
      - '{{ vault_secret3 }}'
      - '{{ vault_secret4 }}'
      - '{{ vault_secret5 }}'
  tasks:
    - name: print credentials
      debug:
        msg: | 
          Secret1 is "{{ vault_lookups[0] }}".
          Secret2 is "{{ vault_lookups[1] }}".
          Secret3 is "{{ vault_lookups[2] }}".
          Secret4 is "{{ vault_lookups[3] }}".
          Secret5 is "{{ vault_lookups[4] }}".
...

Sure enough, we can see the secrets are being read and that defining empty variables hasn’t caused us any problems:

Multiple Secret Engines?

This should be implicit, but it’s good to know that this solution also works if we want to look up Secrets from two different Secret Engines in the same Vault deployment and the functionality was vastly enhanced in Tower v3.6 (on which this guide is based).

Below we can see Secrets in two different Secret Engines as an example:

When populating our Credential Metadata we can simply specify two different “backends” for each of the Secrets, as shown below:

If you haven’t upgraded to Tower 3.6 or later the Backend field is not present, simply entering the full path will work I.E. /password/welsh/pass

Preferences Over Variables

If you happened to see my thoughts on the integration with Azure KeyVault then you’ll know that I think this process is already ugly enough. I don’t much care for handling the secrets inside Playbooks as 5 separate variables and prefer to convert them to a list, this however is down to your own preference and use cases (it’s just a little disappointing the option doesn’t exist out of the gate in the Injection Configuration, if anyone finds out a way let me know). A scenario you might want to consider before you rush in to an implementation however is looking up secrets from multiple KeyVaults in your Templates and how that’s going to work for your long term strategy.

If you want to convert your variables in to a list later in your workflow you can do so, working with our simple example we can use:

vars:
  vault_lookups:
    - {{ vault_secret1 }}
    - {{ vault_secret2 }}
    - {{ vault_secret3 }}
    - {{ vault_secret4 }}
    - {{ vault_secret5 }}

…which will allow for the referencing of your secrets at the play level as:

---
- name: credentialdebug
  hosts: localhost
  connection: local
  gather_facts: false
  any_errors_fatal: true
  tasks:
    - name: print credentials
      debug:
        msg: | 
          Secret1 is "{{ vault_lookups[0] }}".
          Secret2 is "{{ vault_lookups[1] }}".
          Secret3 is "{{ vault_lookups[2] }}".
          Secret4 is "{{ vault_lookups[3] }}".
          Secret5 is "{{ vault_lookups[4] }}".
...

Semi-Painless!