Vault
Deploy HCP Vault Dedicated performance replication with Terraform
Note
Performance replication is available for the HCP Vault Dedicated Plus tier on both AWS and Azure.
This tutorial will demonstrate how to deploy a HashiCorp Cloud Platform (HCP) Vault cluster with performance replication enabled.
Because Vault provides the critical services of identity management, secrets storage, and policy management, this functionality is expected to be highly available. With HCP Vault you can deploy a performance replica in a separate region to achieve high availability.
If you are not familiar with HCP Vault Dedicated, please review the Getting Started with HCP Vault Dedicated tutorials before you begin.
This tutorial builds off the concepts introduced Deploy HCP Vault Dedicated with Terraform, Codify Management of HCP Vault Dedicated, and HCP Vault Dedicated Performance Replication.
Scenario introduction
In this tutorial, you are going to create an Vault Dedicated primary cluster in the US East region, and the secondary cluster in the US West region.
![Vault Dedicated Performance Replication](/_next/image?url=https%3A%2F%2Fcontent.hashicorp.com%2Fapi%2Fassets%3Fproduct%3Dtutorials%26version%3Dmain%26asset%3Dpublic%252Fimg%252Fvault%252Fcloud%252Fhcp-perf-replication.png%26width%3D2107%26height%3D1007&w=3840&q=75)
Prerequisites
An HCP account with appropriate permissions to initiate this change.
Install git on your local machine.
Install Vault on your local machine.
Install Terraform on your local machine.
Create service principal and key
For Terraform to interact with HCP, a service principal must be created with a role of Contributor, and have a key associated with it.
Launch the HCP portal and login.
From the navigation menu, click Access control (IAM).
Click Service principals, and then click Create service principal.
Enter the name you prefer in the Name field.
For example,
learn-hcp-vault
for this tutorial.Select
Contributor
from the Role select field.Click Save. The Service principal is created and displays the overview page.
Click Create service principal key.
A dialog like the following example appears:
Copy the Client ID.
In a terminal, export the variable
HCP_CLIENT_ID
to the Client ID.$ export HCP_CLIENT_ID=<client id value previously copied>
Terraform authenticates with HCP using this ID.
Copy the Client secret.
In a terminal, export the variable
HCP_CLIENT_SECRET
to the Client secret.$ export HCP_CLIENT_SECRET=<client secret value previously copied>
Terraform authenticates with HCP using this secret.
Click <- Back to Overview to return to the Cloud dashboard page.
Clone the tutorial repository
Clone the learn-hcp-vault-replication-terraform repository to get the necessary Terraform configuration for the example scenarios.
Ensure that you are in a directory that you wish to work through this tutorial in, and clone the repository.
$ git clone https://github.com/hashicorp/learn-hcp-vault-replication-terraform
Change into the directory containing the base example scenario Terraform configuration.
$ cd learn-hcp-vault-replication-terraform
Verify that you are in the correct directory before proceeding.
$ ls -1 README.md deploy-hcp-vault configure-hcp-vault
The
deploy-hcp-vault
directory has the minimum configuration necessary to deploy Vault Dedicated using the Terraform HCP provider as an example for this tutorial.The
configure-hcp-vault
directory includes an example configuration to configure Vault Dedicated using the Terraform HCP provider for this tutorial.You will need to significantly expand on these examples to build a more advanced configurations for actual use cases.
Review deploy-hcp-vault configuration
Since the Terraform Vault provider relies on a Vault cluster to be available before it can be configured, you will first review the Terraform configuration needed to deploy an Vault Dedicated cluster with performance replication.
Review variables.tf
Switch to the deploy-hcp-vault directory.
$ cd deploy-hcp-vault
Examine the
main.tf
file. This file defines required providers and any specific provider configuration.main.tf
terraform { required_providers { hcp = { source = "hashicorp/hcp" version = ">=0.26.0" } } } provider "hcp" { # Configuration options }
Since performance replication was introduced in version
0.26.0
of the HCP provider, this is the minimum version required. By including the "greater-than or equal to operator">=
before the version, this will ensure Terraform uses the latest version of the provider, with a minimum version of at least0.26.0
.Examine the
variables.tf
file. This file defines the variables used by the Terraform HCP provider which can be referenced in various configurations.variables.tf
variable "primary_cluster_hvn" { description = "The ID of the HCP HVN." type = string default = "hvn-us-east-1" } variable "primary_cluster_hvn_cidr" { description = "The ID of the HCP HVN." type = string default = "172.25.16.0/20" } variable "secondary_cluster_hvn" { description = "The ID of the HCP HVN." type = string default = "hvn-us-west-2" } variable "secondary_cluster_hvn_cidr" { description = "The ID of the HCP HVN." type = string default = "172.24.16.0/20" } variable "primary_cluster_id" { description = "The ID of the Vault Dedicated cluster." type = string default = "vault-cluster-primary" } variable "secondary_cluster_id" { description = "The ID of the Vault Dedicated cluster." type = string default = "vault-cluster-secondary" } variable "primary_region" { description = "The region of the primary cluster HCP HVN and Vault cluster." type = string default = "us-east-1" } variable "secondary_region" { description = "The region of the secondary cluster HCP HVN and Vault cluster." type = string default = "us-west-2" } variable "cloud_provider" { description = "The cloud provider of the HCP HVN and Vault cluster." type = string default = "aws" } variable "tier" { description = "Tier of the Vault Dedicated cluster. Valid options for tiers." type = string default = "plus_small" }
Lines 1-23 includes the string and CIDR for each region's HVN. HVN CIDR ranges cannot overlap.
Lines 25-35 define the name of each of the clusters that will be created by Terraform.
Lines 37-53 define the cloud provider and regions from the cloud provider that will be used.
Lines 55-59 specify the cluster tier and size. Performance replication is available on any size within the Plus tier. The primary and secondary clusters must be the same tier and size. A list of available tiers and sizes is available in the Terraform registry.
HCP project considerations
By default, Terraform will create the HVN defined in the default or oldest HCP project. If you are using multiple projects in your HCP account, you can control where Terraform will create the HVN and cluster using the
project_id
parameter.For more information on how to work with multiple projects, refer to the HCP provider documentation.
Review hcpvault.tf
Examine the
hcpvault.tf
file. This file defines the configuration used by the Terraform HCP provider to deploy the HCP HVN and Vault clusters. Values here are referenced from thevariables.tf
file you reviewed earlier in this tutorial.hcvault.tf
resource "hcp_hvn" "primary_cluster_hvn" { hvn_id = var.primary_cluster_hvn cloud_provider = var.cloud_provider region = var.primary_region cidr_block = var.primary_cluster_hvn_cidr } resource "hcp_vault_cluster" "primary_cluster" { hvn_id = hcp_hvn.primary_cluster_hvn.hvn_id cluster_id = var.primary_cluster_id tier = var.tier public_endpoint = true } resource "hcp_hvn" "secondary_cluster_hvn" { hvn_id = var.secondary_cluster_hvn cloud_provider = var.cloud_provider region = var.secondary_region cidr_block = var.secondary_cluster_hvn_cidr } resource "hcp_vault_cluster" "secondary_cluster" { hvn_id = hcp_hvn.secondary_cluster_hvn.hvn_id cluster_id = var.secondary_cluster_id tier = var.tier primary_link = hcp_vault_cluster.primary_cluster.self_link paths_filter = ["do-not-replicate-namespace", "replicate-namespace/do-not-replicate-secrets"] public_endpoint = true }
Lines 1-6 create the HVN for the primary cluster in the region defined in
variables.tf
.Lines 8-13 create the cluster, sets the tier defined in
variables.tf
, and sets the public address totrue
for purposes of completing this tutorial.Lines 15-20 create the HNV for the secondary cluster in the region defined in
variables.tf
.Lines 22-29 create the secondary cluster. The
primary_link
parameter here is required to enable performance replication. This parameter identifies which cluster is the primary cluster in the replication topology. Line 27 includes thepaths_filter
parameter. This optional parameter allows you to define which paths will not be replicated from the primary cluster to the secondary cluster.
Deploy HCP Vault Dedicated
Now that you have reviewed the Terraform configuration for Vault Dedicated, you are ready to deploy the Vault clusters.
Initialize Terraform.
$ terraform init Initializing the backend... Initializing provider plugins... ...snip... Terraform has been successfully initialized!
After Terraform is initialized, you can verify that the resources will be created using
terraform plan
.$ terraform plan Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: ...snip... Plan: 4 to add, 0 to change, 0 to destroy.
Review the planned changes from Terraform and deploy the clusters using
terraform apply
.$ terraform apply ...snip... Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value:
Confirm the run by entering
yes
.Once you confirm, it will take up to 10 minutes for each cluster to complete the deployment. If the deployment was successful, you should observe output at the end resembling this example.
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Review the HCP Vault Dedicated deployment
Switch back to the the HCP Portal.
From the navigation menu, click Vault.
You will see two clusters: vault-cluster-primary and vault-cluster-secondary. The arrow to the left of vault-cluster-secondary signifies it is a replica of the cluster above.
Click vault-cluster-primary then click Replication from the navigation menu.
From the Replication page you can manage replication settings such deleting the secondary cluster or editing paths filters.
Click the ellipses to the right of the vault-cluster-secondary card and select Edit path filters.
The path filters listed in the
paths_filter
parameter of thehcpvault.tf
Terraform configuration are listed.Click Cancel to return to the Replication page.
Using the supplied Terraform configuration, you were able to create two HVNs, two Vault Dedicated clusters, and configure performance replication by linking the designated secondary cluster to the primary cluster.
Review configure-hcp-vault configuration
Now that you have an Vault Dedicated cluster up and running, you can now use Terraform to configure your cluster with namespaces, policies, auth methods, and secrets engines.
To demonstrate the replication capabilities of Vault Dedicated you will create the following resources:
Type | Name | Description |
---|---|---|
namespace | replicate-namespace | A namespace with path: admin/replicate-namespace |
namespace | do-not-replicate-namespace | A namespace with path: `admin/do-not-replicate-namespace |
secrets engine | do-not-replicate-secrets | Enable kv-v2 secrets engine at do-not-replicate-secrets in replicate-namespace |
To learn more about managing additional Vault configurations such as auth methods or policies, try the Codify Management of HCP Vault Dedicated tutorial.
Review main.tf
Switch to the
configure-hcp-vault
directory.$ cd ../configure-hcp-vault
Examine the
main.tf
file. This file defines required providers and any specific provider configuration.main.tf
terraform { required_providers { hcp = { source = "hashicorp/hcp" version = ">=0.26.0" } vault = { source = "hashicorp/vault" version = ">=3.4.1" } } } provider "vault" { # Configuration options # Configuration options alias = "admin" namespace = "admin" }
To configure Vault Dedicated clusters, you use the Vault terraform provider. Because all Vault Dedicated clusters operate from the
admin
namespace, you define this in thevault
provider configuration options section.
Review namespace.tf
Examine the
namespace.tf
file. This file is used to define the namespaces Terraform will create.namespace.tf
resource "vault_namespace" "replicate-namespace" { provider = vault.admin path = "replicate-namespace" } resource "vault_namespace" "do-not-replicate-namespace" { provider = vault.admin path = "do-not-replicate-namespace" }
The resource block defines the provider to be used, and the path where the namespace will be created.
Review secrets.tf
Examine the
secrets.tf
file. This file is used to define the secrets engines, the path, and namespace Terraform will create them in.secrets.tf
resource "vault_mount" "kv-v2" { depends_on = [vault_namespace.replicate-namespace] provider = vault.replicate-namespace path = "do-not-replicate-secrets" type = "kv-v2" }
The
vault_mount
resource allows you to define configuration for the desired secrets engine. Because you will create this in a yet to be created namespace, you can use thedepends_on
parameter. This parameter lets Terraform know it must complete the configuration listed - in this example creating thereplicate-namespace
namespace, before taking any action on this resource.
Configure HCP Vault Dedicated
Now that you have reviewed the Terraform configuration for Vault Dedicated, you are ready
to configure the primary Vault cluster. The secondary cluster will have the configuration
replicated from the primary, without the paths defined in the path_filter
parameter.
Switch back to the HCP Portal.
Click vault-cluster-primary.
Under Cluster URLs, click the Public Cluster URL.
In a terminal, set the
VAULT_ADDR
environment variable to the copied address.$ export VAULT_ADDR=<Public_Cluster_URL>
Return to the Overview page and click Generate token.
Within a few moments, a new token will be generated.
Copy the Admin Token.
In a terminal, set the
VAULT_TOKEN
environment variable to store the token value.$ export VAULT_TOKEN=<Admin_Token>
The
admin
namespace is the top-level namespace automatically created by HCP Vault. All CLI operations default to use the namespace defined in this environment variable.Set the
VAULT_NAMESPACE
environment variable toadmin
.$ export VAULT_NAMESPACE="admin"
Verify that no namespaces exist.
$ vault namespace list No namespaces found
Initialize Terraform.
$ terraform init Initializing the backend... Initializing provider plugins... ...snip... Terraform has been successfully initialized!
After Terraform is initialized, you can verify that the resources will be created using
terraform plan
.$ terraform plan Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: ...snip... Plan: 3 to add, 0 to change, 0 to destroy.
Review the planned changes from Terraform and deploy the clusters using
terraform apply
.$ terraform apply ...snip... Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value:
Confirm the run by entering
yes
.Once you confirm, it will take a few seconds to complete the configuration. If the configuration was successful, you should observe output at the end resembling this example.
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Validation
You already have your shell session configured to authenticate with the primary Vault cluster. You can now validate that the namespaces and secrets engines were created.
Run
vault namespace list
to verify the namespaces were created.$ vault namespace list Keys ---- do-not-replicate-namespace/ replicate-namespace/
Both the
do-not-replicate-namespace
andreplicate-namespace
namespaces were created.View the secrets engines available in the
replicate-namespace
namespace.$ vault secrets list -namespace="admin/replicate-namespace" Path Type Accessor Description ---- ---- -------- ----------- cubbyhole/ ns_cubbyhole ns_cubbyhole_773fac29 per-token private secret storage do-not-replicate-secrets/ kv kv_056e611d n/a identity/ ns_identity ns_identity_92d21333 identity store sys/ ns_system ns_system_a2b7452a system endpoints used for control, policy and debugging
They KV v2 secret engine was created with a path of
do-not-replicate-secrets
.Switch back to the HCP portal and click Replication in the navigation menu.
Click vault-cluster-secondary.
Under Cluster URLs, click the Public Cluster URL.
In a terminal, set the
VAULT_ADDR
environment variable to the copied address.$ export VAULT_ADDR=<Public_Cluster_URL>
Return to the Overview page and click Generate token.
Within a few moments, a new token will be generated.
Copy the Admin Token.
In a terminal, set the
VAULT_TOKEN
environment variable to store the token value.$ export VAULT_TOKEN=<Admin_Token>
Your shell session is now configured to authenticate with the secondary cluster.
View the namespaces available on the secondary cluster.
$ vault namespace list Keys ---- replicate-namespace/
The
do-not-replicate-namespace
namespace was not replicated because you added the path to thepath_filter
parameter when deploying Vault Dedicated.View the secrets engines available in the
replicate-namespace
namespace.$ vault secrets list -namespace="admin/replicate-namespace" Path Type Accessor Description ---- ---- -------- ----------- cubbyhole/ ns_cubbyhole ns_cubbyhole_773fac29 per-token private secret storage identity/ ns_identity ns_identity_92d21333 identity store sys/ ns_system ns_system_a2b7452a system endpoints used for control, policy and debugging
The
do-not-replicate-secrets
K/V secret engine was not replicated, even though the namespace was replicated because you added the secret engine path to thepath_filter
parameter when deploying Vault Dedicated.
Clean up
Switch to the
deploy-hcp-vault
directory.$ cd ../deploy-hcp-vault
Run
terraform destroy
to delete the clusters.$ terraform destroy ...snip... Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: ...snip... Plan: 0 to add, 0 to change, 4 to destroy. Do you really want to destroy all resources? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value:
Confirm the destroy by entering
yes
.Once you confirm, it will take a few minutes to complete the the destroy. When successful, you should observe output resembling the following example.
Destroy complete! Resources: 4 destroyed.
Remove the state files.
$ rm *.tfstate.*
Remove the state files under the
configure-hcp-vault
folder.$ rm ../configure-hcp-vault/*.tfstate.*
Unset the
VAULT_TOKEN
andVAULT_ADDR
environment variables.$ unset VAULT_TOKEN VAULT_ADDR
In this tutorial you learned how to automate the deployment of an Vault Dedicated cluster with performance replication and path filters using the Terraform HCP provider. You also learned about some of the resources available with the provider.
Help and reference
- Terraform Vault Provider documentation page
- Terraform Provider GitHub repository
- Learn Terraform