Using terraform to create a kubernetes cluster and a nginx ingress

In this tutorial, we are going to build a kubernetes cluster using terraform and EKS. And we also we will setup a nginx ingress that will be responsible to handle all the incoming requests and proxy to pods inside the cluster.

When you finish this tutorial, you will have this resources created on your AWS account:

  • EKS cluster
  • VPC, Subnets, Nat gateway, etc
  • IAM policy
  • 2 kubernetes pods for tests
  • 1 AWS Load balancer
  • 1 ingress-nginx to receive requests and proxy for pods

All the resources needed to execute this tutorial are on my github: https://github.com/renandeandradevaz/terraform-kubernetes

First, we need to clone the repo and init terraform:

git clone git@github.com:renandeandradevaz/terraform-kubernetes
cd terraform-kubernetes
terraform init

To keep it simple, I decided to keep the AWS region inside the file main.tf. But, probably, in a production environment, it could be organized in a separated file. If you want to use another AWS region, you must change the main.tf file.

Then, we have to apply the resources on the cloud:

terraform apply

This step may take a few minutes.

While your cluster is being created, let’s take a look at the terraform file main.tf

provider "aws" {
region = "us-west-1"
}

data "aws_eks_cluster" "cluster" {
name = module.eks.cluster_id
}

data "aws_eks_cluster_auth" "cluster" {
name = module.eks.cluster_id
}

provider "kubernetes" {
host = data.aws_eks_cluster.cluster.endpoint
cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
token = data.aws_eks_cluster_auth.cluster.token
load_config_file = false
version = "~> 1.11"
}

data "aws_availability_zones" "available" {
}

locals {
cluster_name = "my-cluster"
}

module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "2.47.0"

name = "k8s-vpc"
cidr = "172.16.0.0/16"
azs = data.aws_availability_zones.available.names
private_subnets = ["172.16.1.0/24", "172.16.2.0/24", "172.16.3.0/24"]
public_subnets = ["172.16.4.0/24", "172.16.5.0/24", "172.16.6.0/24"]
enable_nat_gateway = true
single_nat_gateway = true
enable_dns_hostnames = true

public_subnet_tags = {
"kubernetes.io/cluster/${local.cluster_name}" = "shared"
"kubernetes.io/role/elb" = "1"
}

private_subnet_tags = {
"kubernetes.io/cluster/${local.cluster_name}" = "shared"
"kubernetes.io/role/internal-elb" = "1"
}
}

module "eks" {
source = "terraform-aws-modules/eks/aws"
version = "12.2.0"

cluster_name = "${local.cluster_name}"
cluster_version = "1.17"
subnets = module.vpc.private_subnets

vpc_id = module.vpc.vpc_id

node_groups = {
first = {
desired_capacity = 2
max_capacity = 3
min_capacity = 1

instance_type = "m5.large"
}
}

write_kubeconfig = true
config_output_path = "./"

workers_additional_policies = [aws_iam_policy.worker_policy.arn]
}

resource "aws_iam_policy" "worker_policy" {
name = "iam-worker-policy"
description = "Worker policy"

policy = file("iam-policy.json")
}

In this file above, we are going to create 1 vpc, some subnets, nat gateway, iam policy and also the EKS cluster. The eks cluster is using EC2 machines with instance type = m5.large. The minimum number of machines is 1 and the max number is 3.

After the cluster created, a file called kubeconfig_my-cluster was created as well. This file contains the information about our cluster on EKS. And we use it to connect to our cluster.

Now, let’s create an environment variable to keep kubeconfig data:

export KUBECONFIG=${PWD}/kubeconfig_my-cluster

We are going to use now kubectl to deploy some resources to our cluster.

Create the nginx ingress inside the cluster:

kubectl apply -f ingress-nginx.yaml

And then, let’s create 2 pods to test our configuration. If you want, you can use another image/container

kubectl apply -f banana.yaml
kubectl apply -f apple.yaml

The pod’s yaml file is very simple:

kind: Pod
apiVersion: v1
metadata:
name: apple-app
labels:
app: apple
spec:
containers:
- name: apple-app
image: hashicorp/http-echo
args:
- "-text=apple"

---

kind: Service
apiVersion: v1
metadata:
name: apple-service
spec:
selector:
app: apple
ports:
- port: 5678

In this file above, we have: pod, service, image, port. The other file is pretty similiar. But the args is different, is banana instead of apple. It’s just to differentiate one container from another.

Now, we must deploy the routes:

kubectl apply -f routes.yaml

The routes file is where we are going to create our routes based on the path.

- http:
paths:
- path: /apple
backend:
serviceName: apple-service
servicePort: 5678
- path: /banana
backend:
serviceName: banana-service
servicePort: 5678

(Optional) And to finish, if you want, you can activate the auto scaler on your eks cluster:

kubectl apply -f auto-scaler.yaml

PS: The cluster’s name is inside of the file above. The name is “my-cluster”. If you choose a different name for the cluster, you must rename the cluster inside the file.

After all resources created. Now it’s time to test. Go to https://console.aws.amazon.com/ec2/v2/home#LoadBalancers and copy the DNS name of the balancer that was created recently.

And test if application returns something:

curl {AWS_LOAD_BALANCER_URL}/banana

and

curl {AWS_LOAD_BALANCER_URL}/apple

And, in the end, you can remove all created resources to save money:

terraform destroy

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Top 9 Statistics To Measure Software Quality

Typical test report in donut style

Laravel Basic | Laravel 7/6 Resource Route and Controller Tutorial

How flex-box has made positioning of website elements easier.

Visual Studio Code on iPadOS, ChromeOS or Windows 11 SE — in the Browser

Android Modularization

Modularization (my point of view)

SaaS insights this week: The Cloud Paradox and The impact of covid-19 on Kuberentes adoption

Django: Template Inheritance and Reusability

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Renan Vaz

Renan Vaz

More from Medium

Provision EKS cluster with Terraform and Cloudify

Setting up a Kubernetes cluster with Kops

Kubernetes kOps: Step-By-Step Example & Alternatives

Backup an entire Kubernetes cluster using Velero to AWS S3