Deploy Cloudera using Terraform

Step 1: Clone the repository

The cdp-tf-quickstarts repository contains Terraform resource files to quickly deploy Cloudera on cloud and associated pre-requisite cloud resources. It uses the Cloudera Terraform Modules provided by Cloudera to do this.

Clone this repository and navigate to the local directory with the cloned repository:

git clone https://github.com/cloudera-labs/cdp-tf-quickstarts.git
cd cdp-tf-quickstarts

Step 2: Edit the configuration file for the required cloud provider

In the cloned repository, change to the required cloud provider directory - AWS, Azure, or GCP.

Edit the input variables in the configuration file as required:

cd aws
mv terraform.tfvars.template terraform.tfvars
vi terraform.tfvars

cd azure
mv terraform.tfvars.template terraform.tfvars
vi terraform.tfvars

cd gcp
mv terraform.tfvars.template terraform.tfvars
vi terraform.tfvars

Following is a sample configuration file indicating the values to be changed. The variables are explained after the sample. You should review and update all the variables.

# ------- Global settings -------
env_prefix = "<ENTER_VALUE>" # Required name prefix for cloud and Cloudera resources, e.g. cldr1

# ------- Cloud Settings -------
aws_region = "<ENTER_VALUE>" # Change this to specify Cloud Provider region, e.g. eu-west-1

# ------- Cloudera Environment Deployment -------
deployment_template = "<ENTER_VALUE>"  # Specify the deployment pattern below. Options are public, semi-private or private

# ------- Global settings -------
env_prefix = "<ENTER_VALUE>" # Required name prefix for cloud and Cloudera resources, e.g. cldr1

# ------- Cloud Settings -------
azure_region = "<ENTER_VALUE>" # Change this to specify Cloud Provider region, e.g. eastus

# ------- Cloudera Environment Deployment -------
deployment_template = "<ENTER_VALUE>"  # Specify the deployment pattern below. Options are public, semi-private or private

# ------- Global settings -------
env_prefix = "<ENTER_VALUE>" # Required name prefix for cloud and Cloudera resources, e.g. cldr1

# ------- Cloud Settings -------
gcp_project = "<ENTER_VALUE>" # Change this to specify the GCP Project ID
gcp_region = "<ENTER_VALUE>" # Change this to specify Cloud Provider region, e.g. europe-west2

# ------- Cloudera Environment Deployment -------
deployment_template = "<ENTER_VALUE>"  # Specify the deployment pattern below. Options are public, semi-private or private

As a result of this step, your configuration file should look similar to the following:

# ------- Global settings -------
env_prefix = "test-env" # Required name prefix for cloud and Cloudera resources, e.g. cldr1

# ------- Cloud Settings -------
aws_region = "eu-west-1" # Change this to specify Cloud Provider region, e.g. eu-west-1

# ------- Cloudera Environment Deployment -------
deployment_template = "public"  # Specify the deployment pattern below. Options are public, semi-private or private

# ------- Global settings -------
env_prefix = "test-env" # Required name prefix for cloud and Cloudera resources, e.g. cldr1

# ------- Cloud Settings -------
azure_region = "westeurope" # Change this to specify Cloud Provider region, e.g. eastus

# ------- Cloudera Environment Deployment -------
deployment_template = "public"  # Specify the deployment pattern below. Options are public, semi-private or private

# ------- Global settings -------
env_prefix = "cdp-demo" # Required name prefix for cloud and Cloudera resources, e.g. cldr1

# ------- Cloud Settings -------
gcp_project = "my-gcp-project-id" # Change this to specify the GCP Project ID
gcp_region = "us-central1" # Change this to specify Cloud Provider region, e.g. europe-west2

# ------- Cloudera Environment Deployment -------
deployment_template = "public"  # Specify the deployment pattern below. Options are public, semi-private or private

The following tables explain the mandatory inputs that need to be provided in the configuration file.

Table 1: Mandatory inputs

The following tables explain the optional inputs that can be added to the configuration file. While the mandatory input attributes are included in the configuration file and only their values need to be entered, optional attributes and values must be added manually.

Table 2: Optional inputs

env_tags = {
  owner   = "<ENTER_VALUE>"
  project = "<ENTER_VALUE>"
  enddate = "<ENTER_VALUE>"
}

env_tags = {
  owner   = "<ENTER_VALUE>"
  project = "<ENTER_VALUE>"
  enddate = "<ENTER_VALUE>"
}

env_tags = {
  owner   = "<ENTER_VALUE>"
  project = "<ENTER_VALUE>"
  enddate = "<ENTER_VALUE>"
}

Step 3: Launch the deployment

terraform init
terraform apply

Terraform displays a plan with the list of cloud provider and Cloudera resources that will be created.

When you are prompted, type yes to instruct Terraform to perform the deployment. Typically, this takes about 60 minutes. Once the deployment is complete, Cloudera will print output similar to the following:

Apply complete! Resources: 46 added, 0 changed, 0 destroyed.

You can navigate to the Cloudera web interface at https://cdp.cloudera.com/ and see your deployment progress. Once the deployment completes, you can create Cloudera Data Hub clusters and data services.

Clean up the Cloudera environment and infrastructure

If you no longer need the infrastructure provisioned by Terraform, run the following command to remove the deployment infrastructure and terminate all resources:

terraform destroy