How to setup kubernetes jenkins pipeline on AWS?

Hello guys It is been quite some time I was planning to write a blog post on How to set up your Kubernetes CI CD Jenkins pipeline on AWS step by step? It took some time for me to figure out all the details for setting up the continuous pipeline but at last once I was able to deploy it successfully on AWS, it was a sigh of relief for me.

I have listed all the steps which you can find inside the table of content and also I will also record a session and will upload on youtube.

(If you are interested in setting up similar pipeline on premise or using virtual machine the checkout this post )

Table of Content

1. Setup an AWS EC2 Instance
2. Connect to EC2 Instance
3. Install JDK on AWS EC2 Instance
4. Install and Setup Jenkins
5. Update visudo and assign administrative privileges to Jenkins user
6. Install Docker
7. Install and Setup AWS CLI
8. Install and Setup Kubectl
9. Install and Setup eksctl
10. Create eks cluster using eksctl
11. Add Docker and GitHub Credentials into Jenkins
12. Add jenkins stages
13. Build, deploy and test CI CD pipeline

1. Setup an AWS EC2 Instance

The first step would be for us to set up an EC2 instance and on this instance, we will be installing -

1. JDK
2. Jenkins
3. eksctl
4. kubectl

1.1 Launch EC2 instance

1. But first let's head over to AWS and in the search box type in ec2.

2. Click on the EC2 and after that, you need to look for the Launch Insance option -

3. Now select the image type for the EC2 instance(For this article we are going to select Ubuntu Server 20.04)

4. Choose an Instance Type.

For this tutorial, we are going to use t2.medium because we will be installing Jenkins and t2.micro will not be sufficient enough to setup Jenkins

5. Configure Instance Details - you can simply verify the detail and proceed to add the storage part.

6. Add storage - In general 8 Gib of memory is sufficient enough for setting up Jenkins

7. Add tags - This part is optional but you can add some meaningful tag names to your EC2 instance.

8. Configure Security group - This is an important step because here we need to add Custom TCP Port 8080, if you do not add this port then you will not be able to access Jenkins using the public IP address of the AWS EC2 instance.

9. Finally click on review and launch

10. But before you launch your EC2 instance you need to create and download the key pair(private key and public key)

11. Type in the key pair name and then click on the Download Key Pair.

Your ec2 instance should be up and running.

2. Connect to EC2 Instance

Before you connect to your EC2 instance you must start your EC2 instance.

Goto your AWS EC2 dashboard and click on EC2 after that click on Instances(running).

Remove the running filter and you should see your EC2 instance which you set up in Step 1.

Now we need to start the EC2 instance and it can be done by first selecting the instance and then

Goto-> Instance Start -> Start Instance

Once the instance state is Running you can select the instance and click on Connect

We will connect using SSH Client -

We will use the jenkins-ec2.pem file to connect, so carefully copy the ssh command. (Following command will be different for you because the IP address of EC2 instance will always be different for you and also you need to supply your server pem file)

1ssh -i "my-key-pair-1.pem" ubuntu@ec2-18-185-59-154.eu-central-1.compute.amazonaws.com

After successful login, you should see something similar on your terminal -

3. Install JDK on AWS EC2 Instance

The next requirement is we need to install JAVA(JDK) on the EC2 instance.

In the previous step we have seen how to connect and ssh into the EC2 instance.

Now before we do the JDK installation lets first update the package manager of the virtual machine -

1sudo apt-get update  

Check if you have java already installed onto your EC2 machine by running the following command -

1java -version 

In case if you do not have java installed then you will see the following message -

1Command 'java' not found, but can be installed with:
2
3sudo apt install openjdk-11-jre-headless  # version 11.0.11+9-0ubuntu2~20.04, or
4sudo apt install default-jre              # version 2:1.11-72
5sudo apt install openjdk-13-jre-headless  # version 13.0.7+5-0ubuntu1~20.04
6sudo apt install openjdk-16-jre-headless  # version 16.0.1+9-1~20.04
7sudo apt install openjdk-8-jre-headless   # version 8u292-b10-0ubuntu1~20.04
8sudo apt install openjdk-14-jre-headless  # version 14.0.2+12-1~20.04 

But you can install java by running the following command

1sudo apt install openjdk-11-jre-headless

If you see the following message then you have installed java successfully -

1openjdk version "11.0.11" 2021-04-20
2OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04)
3OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.20.04, mixed mode, sharing) 

4. Install and Setup Jenkins

The next step would be to install the Jenkins. You can follow the official Jenkins Installation guide also. But here I have listed down the steps for installing the Jenkins on the EC2 instance.

First, we need to add the Jenkins repository to the package manager -

1wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -

1sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'  

After adding the repository link of Jenkins update the package manager

1sudo apt-get update 

Then finally install Jenkins using the following command

1sudo apt-get install jenkins

On successful installation, you should see Active Status

1sudo service jenkins status 

1● jenkins.service - LSB: Start Jenkins at boot time
2     Loaded: loaded (/etc/init.d/jenkins; generated)
3     Active: active (exited) since Tue 2021-06-22 20:31:18 UTC; 37s ago
4       Docs: man:systemd-sysv-generator(8)
5    Process: 16297 ExecStart=/etc/init.d/jenkins start (code=exited, status=0/SUCCESS)

4.1 Setup jenkins

After installing jenkins lets go back to AWS dashboard -> EC2 -> Instances(running)

Click on the instance ID as mentioned in the above image.

Now we need to find the public IP address of the EC2 machine so that we can access the Jenkins.

Once you click on the instance ID you should see the following page with lots of information about the EC2 instance.

We need to look for Public IPv4 address

Alright now we know the public IP address of the EC2 machine, so now we can access the Jenkins from the browser using the public IP address followed by the port 8080

If you are installing the Jenkins for the first time then you need to supply the initialAdminPassword and you can obtain it from -

1sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the password and paste it into the initial page of the Jenkins. After that, Jenkins will prompt you for installing the plugins.

Opt for install suggested plugin -

After completing the installation of the suggested plugin you need to set the First Admin User for Jenkins

Also, check the instance configuration because it will be used for accessing the Jenkins

And now your Jenkins is ready for use

4.2 Setup Gradle

In this lab session, we will be using the Spring Boot Application, so we need to use a build tool for compilation and building the JAR file and for that, we will be using Gradle as the preferred tool.

For setting up the gradle Goto -> Manage Jenkins -> Global Tool Configuration -> Gradle

Here is the screenshot for your reference -

5. Update visudo and assign administration privileges to jenkins user

Now we have installed the Jenkins on the EC2 instance. To interact with the Kubernetes cluster Jenkins will be executing the shell script with the Jenkins user, so the Jenkins user should have an administration(superuser) role assigned forehand.

Let's add jenkins user as an administrator and also ass NOPASSWD so that during the pipeline run it will not ask for root password.

Open the file /etc/sudoers in vi mode

1sudo vi /etc/sudoers 

Add the following line at the end of the file

1jenkins ALL=(ALL) NOPASSWD: ALL 

After adding the line save and quit the file.

Now we can use Jenkins as root user and for that run the following command -

1sudo su - jenkins  

6. Install Docker

Now we need to install the docker after installing the Jenkins.

The docker installation will be done by the Jenkins user because now it has root user privileges.

Use the following command for installing the docker -

1sudo apt install docker.io

After installing the docker you can verify it by simply typing the docker --version onto the terminal

It should return you with the latest version of the docker

1Docker version 20.10.2, build 20.10.2-0ubuntu1~20.04.2

6.1 Add jenkins user to Docker group

Jenkins will be accessing the Docker for building the application Docker images, so we need to add the Jenkins user to the docker group.

1sudo usermod -aG docker jenkins 

7. Install and Setup AWS CLI

Okay so now we have our EC2 machine and Jenkins installed. Now we need to set up the AWS CLI on the EC2 machine so that we can use eksctl in the later stages

Let us get the installation done for AWS CLI

1sudo apt install awscli 

Verify your AWS CLI installation by running the following command -

1aws --version 

It should return you with the version of CLI

1aws-cli/1.18.69 Python/3.8.5 Linux/5.4.0-1045-aws botocore/1.16.19 

7.1 Configure AWS CLI

Okay now after installing the AWS CLI, let's configure the AWS CLI so that it can authenticate and communicate with the AWS environment.

To configure the AWS the first command we are going to run is -

1aws configure 

Once you execute the above command it will ask for the following information -

1. AWS Access Key ID [None]:
2. AWS Secret Access Key [None]:
3. Default region name [None]:
4. Default output format [None]:

You can find this information by going into AWS -> My Security Credentials

Then navigate to Access Keys (access key ID and secret access key)

You can click on the Create New Access Key and it will let you generate - AWS Access Key ID, AWS Secret Access Key.

(Note: - Always remember you can only download your access id and secret once, if you misplace the secret and access then you need to recreate the keys again)

Default region name - You can find it from the menu

Alright now we have installed and set up AWS CLI.

8. Install and Setup Kubectl

Moving forward now we need to set up the kubectl also onto the EC2 instance where we set up the Jenkins in the previous steps.

Here is the command for installing kubectl

1curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"

1chmod +x ./kubectl 

1sudo mv ./kubectl /usr/local/bin

Verify the kubectl installation

Verify the kubectl installation by running the command kubectl version and you should see the following output

1Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
2Error from server (Forbidden): <html><head><meta http-equiv='refresh' content='1;url=/login?from=%2Fversion%3Ftimeout%3D32s'/><script>window.location.replace('/login?from=%2Fversion%3Ftimeout%3D32s');</script></head><body style='background-color:white; color:white;'> 

9. Install and Setup eksctl

The next thing which we are gonna do is to install the eksctl, which we will be using to create AWS EKS Clusters.

Okay, the first command which we are gonna run to install the eksctl

1 curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp

1sudo mv /tmp/eksctl /usr/local/bin 

Verify the installation by running the command -

1eksctl version

And it will return you with the version -

10.52.0 

So at the time of installation, I had 0.52.0

Installing eksctl on other OS

For Mac OS

1brew tap weaveworks/tap
2brew install weaveworks/tap/eksctl

For Windows

1 chocolatey install eksctl

or Scoop

1 scoop install eksctl

10. Create eks cluster using eksctl

In all the previous 9 steps we were preparing our AWS environment. Now in this step, we are going to create EKS cluster using eksctl

You need the following in order to run the eksctl command

1. Name of the cluster : --name jhooq-test-cluster
2. Version of Kubernetes : --version 1.17
3. Region : --name eu-central-1
4. Nodegroup name/worker nodes : worker-nodes
5. Node Type : t2.micro
6. Number of nodes: -nodes 2

Here is the eksctl command -

1eksctl create cluster --name jhooq-test-cluster --version 1.17 --region eu-central-1 --nodegroup-name worker-nodes --node-type t2.micro --nodes 2

(*Note - Be patient with the above command because it may take 20-30 minutes to complete)

For me it almost took 20 minutes, here are the timestamps

10.1 Verify the EKS kubernetes cluster from AWS

You can go back to your AWS dashboard and look for Elastic Kubernetes Service -> Clusters

Click on the Cluster Name to verify the worker nodes -

11. Add Docker and GitHub Credentials into Jenkins

As we know Kubernetes is a container orchestration tool and container management we are using docker.

(In case if you haven't set up Docker Hub Account then please create a DockerHub Account because we are gonna need it.)

Alright so if you are reading this line then I am assuming you have a DockerHub Account and GitHub Account.

Here is the link of GitHub Repository for this project

11.1 Setup Docker Hub Secret Text in Jenkins

You can set the docker credentials by going into -

Goto -> Jenkins -> Manage Jenkins -> Manage Credentials -> Stored scoped to jenkins -> global -> Add Credentials

11.2 Setup GitHub Username and password into Jenkins

Now we add one more username and password for GitHub.

Goto -> Jenkins -> Manage Jenkins -> Manage Credentials -> Stored scoped to jenkins -> global -> Add Credentials

12. Add jenkins stages

Okay, now we can start writing out the Jenkins pipeline for deploying the Spring Boot Application into the Kubernetes Cluster.

12.1 Jenkins stage-1 : Checkout the GitHub Repository

Add the following Jenkins script for checking out the GitHub Repository -

1     stage("Git Clone"){
2
3        git credentialsId: 'GIT_HUB_CREDENTIALS', url: 'https://github.com/rahulwagh/k8s-jenkins-aws'
4    }

12.2 Jenkins stage-2 : Gradle compilation and build

Now after checking out the repository let compile and build the application using Gradle

1stage('Gradle Build') {
2    sh './gradlew build'
3}

12.3 Jenkins stage-3 : Create Docker Container and push to Docker Hub

After successful compilation and build let's create a Docker image and push to the docker hub

 1stage("Docker build"){
 2    sh 'docker version'
 3    sh 'docker build -t jhooq-docker-demo .'
 4    sh 'docker image list'
 5    sh 'docker tag jhooq-docker-demo rahulwagh17/jhooq-docker-demo:jhooq-docker-demo'
 6}
 7
 8stage("Push Image to Docker Hub"){
 9        sh 'docker push  rahulwagh17/jhooq-docker-demo:jhooq-docker-demo'
10}

12.4 Jenkins stage-4 : Kubernetes deployment

Finally, do the Kubernetes deployment

1stage("kubernetes deployment"){
2  sh 'kubectl apply -f k8s-spring-boot-deployment.yml'
3}

Here is the complete final script for Jenkins pipeline -

 1node {
 2
 3    stage("Git Clone"){
 4
 5        git credentialsId: 'GIT_HUB_CREDENTIALS', url: 'https://github.com/rahulwagh/k8s-jenkins-aws'
 6    }
 7
 8     stage('Gradle Build') {
 9
10       sh './gradlew build'
11
12    }
13
14    stage("Docker build"){
15        sh 'docker version'
16        sh 'docker build -t jhooq-docker-demo .'
17        sh 'docker image list'
18        sh 'docker tag jhooq-docker-demo rahulwagh17/jhooq-docker-demo:jhooq-docker-demo'
19    }
20
21    withCredentials([string(credentialsId: 'DOCKER_HUB_PASSWORD', variable: 'PASSWORD')]) {
22        sh 'docker login -u rahulwagh17 -p $PASSWORD'
23    }
24
25    stage("Push Image to Docker Hub"){
26        sh 'docker push  rahulwagh17/jhooq-docker-demo:jhooq-docker-demo'
27    }
28    
29    stage("kubernetes deployment"){
30        sh 'kubectl apply -f k8s-spring-boot-deployment.yml'
31    }
32} 

13. Build, deploy and test CI/CD pipeline

Create new Pipeline: Goto Jenkins Dashboard or Jenkins home page click on New Item

Pipeline Name: Now enter Jenkins pipeline name and select Pipeline

Add pipeline script: Goto -> Configure and then pipeline section.

Copy the Jenkins script from Step 12 and paste it there.

Build and Run Pipeline: Now goto pipeline and click on build now

Verify the build status:

Verify using kubectl commands

You can also verify the Kubernetes deployment and service with kubectl command .e.g kubectl get deployments, kubectl get service

You can access the rest end point from browser using the EXTERNAL-IP address

I hope you enjoyed this article. For other devops topic like Terraform, Helm Chart you can check the category and my YouTube Channel

Learn more On Kubernetes -

1. Setup kubernetes on Ubuntu
2. Setup Kubernetes on CentOs
3. Setup HA Kubernetes Cluster with Kubespray
4. Setup HA Kubernetes with Minikube
5. Setup Kubernetes Dashboard for local kubernetes cluster
6. Setup Kubernetes Dashboard On GCP(Google Cloud Platform)
7. How to use Persistent Volume and Persistent Volume Claims in Kubernetes
8. Deploy Spring Boot Microservice on local Kubernetes cluster
9. Deploy Spring Boot Microservice on Cloud Platform(GCP)
10. Setting up Ingress controller NGINX along with HAproxy inside Kubernetes cluster
11. CI/CD Kubernetes | Setting up CI/CD Jenkins pipeline for kubernetes
12. kubectl export YAML | Get YAML for deployed kubernetes resources(service, deployment, PV, PVC....)
13. How to setup kubernetes jenkins pipeline on AWS?
14. Implementing Kubernetes liveness, Readiness and Startup probes with Spring Boot Microservice Application?
15. How to fix kubernetes pods getting recreated?
16. How to delete all kubernetes PODS?
17. How to use Kubernetes secrets?
18. Share kubernetes secrets between namespaces?
19. How to Delete PV(Persistent Volume) and PVC(Persistent Volume Claim) stuck in terminating state?
20. Delete Kubernetes POD stuck in terminating state?