Jenkins CI/CD Series

Goal

Understand how to use environment variables in Jenkins pipelines

• Define variables at:

  • Global level

  • Pipeline level

  • Stage level

Use environment variables to control pipeline behavior dynamically

Learn how to manage sensitive data securely

Purpose

The purpose of this part is to make your pipelines configurable, reusable, and secure.

So far, you have:

• Static pipelines

• Parameterized inputs

But real-world pipelines require:

• Managing configuration like URLs, ports, and credentials

• Handling different environments (Dev, UAT, Prod)

• Avoiding hardcoding sensitive data in scripts

Environment variables help you:

• Centralize configuration

• Reuse pipelines across environments

• Improve maintainability and readability

This is a core DevOps practice used in:

• CI/CD pipelines

• Kubernetes deployments

• Cloud-native applications

Prerequisite

Before starting this part, ensure:

• Ready to use Host and the directory structure to run Dockerfiles and docker-compose.yml (Refer to Part 1)

Step-by-step implementation

• Prepare the code

pipeline {
    agent any

    environment {
        def myString = "Hello World"
        def myNumber = 10
        def myBool = true
    }

    stages{
        stage("Demo") {
            steps {
                echo "myString: ${myString}"
                echo "myNumber: ${myNumber}"
                echo "myBool: ${myBool}"
            }
        }
    }
}

• Create pipeline "variable-jenkins."

• Paste the code into the pipeline script

• Build Now

• Now we can see the pipeline using the variables given in the environment block

Done!!!

Conclusion

In this part, you upgraded your pipeline to a fully configurable and production-ready CI/CD system

You learned how to:

• Use environment variables effectively in pipelines

• Avoid hardcoding values

• Manage configurations across environments

• Improve pipeline reusability and security

This is how modern DevOps pipelines are designed:

• Flexible across environments

• Secure with proper secret handling

• Easy to maintain and scale

🔚 Final Note (Series Completion)

🎉 Congratulations! we have completed the Jenkins CI/CD Mastery Series

We have built:

• End-to-end CI/CD pipelines

• Scalable automation workflows

• Production-ready DevOps practices

🔗 Continue the Series

⬅️ Previous Article: Part 9 Parameterized Pipelines
➡️ Next Article: Home

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

Jenkins CI/CD Series

Goal

Create a Jenkins pipeline named parameter-pipeline that accepts user inputs through parameters to control a deployment configuration. The pipeline should allow the user to specify a custom deployment name, choose the target AWS Availability Zone, and confirm the deployment before execution.

Purpose

The purpose of this part is to make your Jenkins pipelines interactive, flexible, and production-ready.

So far, your pipelines were:

• Static

• Hardcoded

• Same behavior for every run

In real-world DevOps:

• Deployments vary by environment (Dev, UAT, Prod)

• Infrastructure changes by region/AZ

• Releases need manual confirmation or approvals

Parameterized pipelines solve this by:

• Allowing user input during execution

• Enabling controlled deployments

• Reducing the need to modify pipeline code

This is a critical DevOps practice used in:

• Multi-environment deployments

• Release approvals

• Dynamic infrastructure provisioning

Prerequisite

Before starting this part, ensure:

• Ready to use Host and the directory structure to run Dockerfiles and docker-compose.yml (Refer to Part 1)

Step-by-step implementation

Here is the code that will be used

pipeline {
    agent any

    parameters {
        string(
            name: "deploymentName",
            defaultValue: "",
            description: "Deployment Name?"
        )

        choice(
            name: "azDeploy",
            choices: ["ap-southeast-1a", "ap-southeast-1b", "ap-southeast-1c"],
            description: "What AZ?"
        )

        booleanParam(
            name: "confirmDeploy",
            defaultValue: false,
            description: "CONFIRM DEPLOYMENT?"
        )
    }

    stages {
        stage("Deploy") {
            steps {
                echo "Deployment Name: ${params.deploymentName}"
                echo "AZ Selected: ${params.azDeploy}"
                echo "Deployment Confirmation: ${params.confirmDeploy}"
            }
        }
    }
}

• Create Pipeline parameter-pipeline"

• Add the script to the pipeline script

• Build Now

• Provide inputs and click build with parameters

• Expected console output

• Pipeline status

Done!!!

Conclusion

In this part, you transformed your pipeline into a dynamic, user-driven CI/CD system

We learned how to:

• Accept user inputs directly from Jenkins UI

• Control pipeline execution using parameters

• Make deployments flexible and environment-aware

• Reduce hardcoding in pipeline scripts

This is how real-world pipelines operate in production:

• Same pipeline, multiple use cases

• Controlled deployments with user validation

• Better flexibility without changing code

🔗 Continue the Series

⬅️ Previous Article: Part 8 Multibranch Pipelines
➡️ Next Article: Part 10 Environment Variables in Pipelines

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

Jenkins CI/CD Series

Goal

• Implement a Multibranch Pipeline in Jenkins

• Automatically detect and build multiple Git branches

• Enable Jenkins to run pipelines per branch dynamically

• Understand how code changes in different branches trigger separate builds

Purpose

• The purpose of this part is to move from a single pipeline setup to a scalable, branch-aware CI/CD system.

  In real-world DevOps environments:

  • Teams work on multiple branches (feature, dev, release, main)

  • Each branch may have different code and pipeline behavior

  • Manual pipeline creation per branch is not scalable

  Multibranch Pipelines solve this by:

  • Automatically discovering branches from GitHub

  • Running pipelines based on Jenkinsfile inside each branch

  • Allowing parallel and independent builds per branch

  This is a key step toward production-grade CI/CD pipelines.

Prerequisites

Before starting this part, ensure:

Infrastructure Setup

• Ready to use Host and the directory structure to run Dockerfiles and docker-compose.yml (Refer to Part 1)

• GitHub repository demo-app

• Jenkins has:

  • Git plugin installed

  • Access to GitHub repository (public or credentials configured)

Step-by-step implementation

• Go to the repo and add another branch.

• Scan the Multibranch pipeline now, and it will populate the branches

• Let's make some changes to the repo and commit to the new branch."devbranch"

Here, I removed a small block from the Jenkinsfile and merged it to devbranch

• Let's check the last Build from devbranch

Done!!!

Conclusion

In this part, you successfully evolved your CI/CD pipeline to a branch-aware automation system

You learned how to:

• Automatically detect new branches in GitHub

• Run pipelines dynamically per branch

• Manage different code versions using Jenkins pipelines

• Enable scalable CI/CD workflows for team-based development

This is exactly how modern DevOps teams handle:

• Feature development

• Parallel testing

🔗 Continue the Series

⬅️ Previous Article: Part 7 Control Jenkins Pipeline through GitHub
➡️ Next Article: Part 9 Parameterized Pipelines

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

This setup is perfect for DevOps engineers, SREs, and Kubernetes learners who want a production-like environment on their laptop.

Tips -

Follow/Read this article first to get base idea and then deploy as per your requirement by using cka-practice-code repo for specfic commands

Lab Architecture

Mac (Host Machine)
        │
        ▼
VirtualBox Hypervisor
        │
        ▼
Vagrant Managed VMs
        │
        ├── controlplane
        ├── node01
        └── node02

Environment Setup

Tools used in the lab:

• Oracle VM VirtualBox – VM hypervisor

• Vagrant – Infrastructure automation

• Kubernetes – Cluster platform

• Project Calico – CNI networking plugin

VM Resources

Configure laptop and build VMs

Install the following tools on Windows or macOS x86.

Download and install Oracle VM VirtualBox

https://www.virtualbox.org/wiki/Downloads

• Go to the App click VirtualBox to complete the installation

Go to Terminal and Install Vagrant

brew tap hashicorp/tap
brew install hashicorp/tap/hashicorp-vagrant

• Verify Installation with vagrant --version

Clone the cka-practice-code locally

• Navigate to the cloned directory

cka-practice-code/kubeadm-clusters/virtualbox and type vagrant up to start creating virtual machines (Controlplane, node01 & node02) through a script

Note the IPs after completion, required for SSH

We can check the status in Oracle Virtual Manager as well

• Access VMs by SSH to install required tools/components

Use the commands below to play with the environment

Check Status

vagrant status

Stop all VMs:

vagrant halt

Destroy environment:

vagrant destroy

Show VM's Names

VBoxManage list vms | awk -F\" '{print $2}'

Install Multipass on Mac M series (Apple Silicon)

https://canonical.com/multipass/install

• Navigate to the Apple-Silicon directory and run the script ./deploy-virtual-machines.sh

• Destroyed by the script ./destroy-virtual-machines.sh

Primary setup: Run these commands on all three nodes Controlplane, node01 & node3

Tips - Kubernetes document for reference

Go to the repo 04-node-setup.md and start running commands one by one

Connect all three by SSH with the below credentials.

username: vagrant
password: vagrant

Note- Run all these commands on all three (Controlplane, node01, node2)

Boot the controlplane

Configure Controlplane by following 05-Controlplane.md commands

Note- Run this only on Controlplane

Refer CNI documents but required commands provided in repo itself

Remember to copy the kubeadm join command and follow the remaining steps until

• Verify Controlplane is Ready

kubectl get pods -n kube-system

Join the workers nodes to Controlplane by using step 06-workers.md

Hope you have the below ready-to-use command copied from Controlplane; otherwise, it can be retrieved by using kubeadm token create --print-join-command

kubeadm join 192.168.0.185:6443 --token 2lccuf.ha2rxb9dclxzkvrp \
        --discovery-token-ca-cert-hash sha256:26d681f9cd77e66a71bbe5e4cd903fae34458660ce2304ff7b73f168fcf9c4c2

• Verify on Controlplane if nodes are joined as Workers

Test the cluster, follow 07-test.md

• Test worker node01

• Test worker node02

• Since we have installed the cluster with bridge networking (the default), we can view NodePort services with your browser.

Run the following command on controlplane to get the browser address, then copy the output to your browser:

echo "http://\((dig +short node01):\)PORT_NUMBER"

Verify from the browser

Key Learnings

Building Kubernetes manually with kubeadm provides a deeper understanding of:

• Control plane components

• Node registration

• Cluster networking

• Kubernetes troubleshooting

• Real production cluster architecture

Final Result

✔ Multi-node Kubernetes cluster running locally
✔ Networking configured with Calico
✔ Worker nodes successfully joined
✔ Nginx application deployed and accessible from the browser

This environment closely simulates a real production Kubernetes setup and is extremely useful for learning Kubernetes architecture and troubleshooting.

Thanks for your time and effort to get your hands dirty.

Happy Learning!

Abhinandan Chougule

Goal

Configure Jenkins to use a Jenkinsfile stored in a GitHub repository, enabling pipelines to be version-controlled, automatically updated, and executed directly from source code.

Purpose

The purpose of this part is to introduce Pipeline as Code using GitHub, which is a core DevOps practice.

By the end of this setup, you will:

• Store Jenkins pipeline logic inside GitHub

• Configure Jenkins to pull pipeline code from SCM

• Automatically detect changes in the repository

• Build pipelines that are version-controlled and reusable

This approach ensures that your CI/CD pipelines are:

• Traceable (tracked via Git commits)

• Collaborative (teams can contribute)

• Consistent (same pipeline across environments)

Prerequisites

Before starting this part, ensure you have:

• Ready to use Host and the directory structure to run Dockerfiles and docker-compose.yml (Refer to Part 1)

• A GitHub account

• Access to a demo-app repository (fork or create)

Step-by-step implementation

• Jenkinsfile is placed at the location below in the repo, and will be used

• Create a new Job maven-github-pipeline

• Select options as below

Definition: Pipeline script from SCM
SCM: Git
Repository URL : Forked repo HTTP URL
Branch Specifier (blank for 'any'): */main
Script Path: jenkins/Jenkinsfile

Build now

• Go through the console output logs

• Build now one more time, and go to Jobs Console output and compare

This is not cloning the repo again; instead, it's checking if there are any latest commits and changes

Done!!!

Conclusion

In this part, you successfully moved your Jenkins pipeline to GitHub, adopting the powerful concept of Pipeline as Code.

You have:

• Connected Jenkins to a GitHub repository

• Configured pipeline jobs to fetch the Jenkinsfile from SCM

• Executed builds directly from the repository code

• Observed how Jenkins detects changes without re-cloning

• Introduced version-controlled CI/CD workflows

This is a critical step toward modern DevOps practices, where pipelines are no longer managed manually in Jenkins but are fully integrated with source control systems.

With this setup, your pipeline becomes:

• Scalable

• Maintainable

• Production-ready

🔗 Continue the Series

⬅️ Previous Article: Part 6 Running Ansible from Jenkins
➡️ Next Article: Part 8 Multibranch Pipelines

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

Goal

Enable Jenkins to execute Ansible automation tasks by integrating Ansible into the Jenkins container, allowing centralized orchestration of remote infrastructure using playbooks.

Purpose

The purpose of this part is to extend your CI/CD pipeline capabilities by introducing configuration management with Ansible.

By the end of this setup, you will:

• Run Ansible commands directly from Jenkins

• Manage remote systems using Ansible inventory + playbooks

• Automate infrastructure tasks via Jenkins jobs

• Establish a foundation for Infrastructure as Code (IaC)

This bridges the gap between CI/CD pipelines and infrastructure automation, which is a core DevOps skill.

Prerequisites

Before starting this part, ensure you have:

• Ready to use Host and the directory structure to run Dockerfiles and docker-compose.yml (Refer to Part 1)

Step-by-step implementation

Install Ansible on Docker (on an existing Jenkins Container)

Create another folder /home/jenkins/jenkins_home/jenkins-ansible$

• Create a customized Dockerfile for Jenkins to create a Jenkins container to support Ansible

FROM jenkins/jenkins
USER root

RUN apt-get update && apt-get install python3-pip -y

# New lines to set up a virtual environment
####
ENV ANSIBLE_VENV=/ansible_venv
RUN mkdir $ANSIBLE_VENV && \
    chown jenkins:jenkins $ANSIBLE_VENV && \
    apt-get install python3-venv -y
####

USER jenkins

# Activate the virtual environment
RUN python3 -m venv $ANSIBLE_VENV

# Use the venv to install Ansible
RUN $ANSIBLE_VENV/bin/pip3 install ansible

# Ensure the Ansible binary is accessible
ENV PATH=\(PATH:\)ANSIBLE_VENV/bin

• Go back to one more directory, where we have our Docker Compose file, and update the content to fetch details from the Dockerfile we created in the earlier step.

/home/jenkins/jenkins_home/docker-compose.yml

services:
  jenkins:
    container_name: jenkins
    image: jenkins-ansible
    build:
      context: jenkins-ansible
    ports:
      - "8080:8080"
    volumes:
      - $PWD/jenkins_home:/var/jenkins_home
    networks:
      - net

  remote_host:
    container_name: remote-host
    image: remote-host
    build:
      context: centos7
    volumes:
      - "$PWD/aws-s3.sh:/tmp/script.sh"
    networks:
      - net

  db_host:
    container_name: db
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: 1234
    volumes:
      - $PWD/db_data:/var/lib/mysql
    networks:
      - net

networks:
  net:

Tips - Update Jenkins block (image: jenkins-ansible, build: context: jenkins-ansible)

docker compose build

Note- Always run the docker compose from the directory where we have docker-compose.yml

• The new Jenkins Image has been built

• Create a Jenkins container by using this Image

#Below commands used to vrify Jenkins container has ansible installed

docker compose up -d
docker ps
docker exec -it jenkins bash
ansible

Note- Typing ansible will show avaible options to run any command means, its installed

Created an Ansible folder for the Ansible home directory

/home/jenkins/jenkins_home/ansible

Note- /jenkins_home is acting as an volumes for our jenkins container since we have given same context in docker-compose.yml

• Copy the remote key to the Ansible folder directly

home/jenkins/jenkins_home$ sudo cp centos7/remote-key jenkins_home/ansible/

Since we're moving around the remote-key file, it's always a good idea to ensure that Jenkins has the correct permissions for it.

• Run the following command to set the right permissions:

docker exec # Asks Docker to run a command 
-u root # Using the root user 
jenkins # On the container named jenkins 
bash # Using bash 
-c "chmod 400 /var/jenkins_home/ansible/remote-key && chown 1000:1000 /var/jenkins_home/ansible -R" # Set proper permissions

• What does this do?

chmod 400 /tmp/remote-key → Sets read-only permissions for the key file.

chown 1000:1000 /tmp/remote-key → Ensures Jenkins (user ID 1000) owns the key.

This step prevents permission issues when using the key inside the container.

• Create Inventory in Ansible

Create a file named hosts on /home/jenkins/jenkins_home/jenkins-ansible and add the below content

[all:vars]

ansible_connection - ssh

[test] 
test1 ansible_host=remote_host ansible_user=remote_user ansible_private_key_file=/var/jenkins_home/ansible/remote-key

• Copy the hosts file to the Jenkins container in jenkins_home/ansible folder

cp hosts ../jenkins_home/ansible

• Log in to the Jenkins container and run the command

docker exec -it jenkins bash

• Verify files

• Run the inventory

ansible -i hosts -m ping test1

Note - Run ansible command from where is the inventory file located

any issues to connect, it must be SSH key permission issue or when connecting to from jenkins to remote_host

ssh -i /var/jenkins_home/ansible/remote-key remote_user@remote_host

Create a Playbook play.yml at the host directory /home/jenkins/jenkins_home/jenkins-ansible

- name: Test playbook 
  hosts: test1 
  tasks:
    - name: Create file using shell 
      shell: echo Hello World > /tmp/ansible-file

• Copy play.yml to the Jenkins container

cp play.yml ../jenkins_home/ansible/

• Log in to the Jenkins container

• Check if the file is present

• Run the play.yml

ansible-playbook -i hosts play.yml

• Check where the file is generated?

Here it is on the remote-host

Jenkins and Ansible Integration

• Log in to the Jenkins console and install the Ansible plugin.

• Create Freestyle pipeline ansible-test

• Click on Build Now and Console Output

• Verify output

Done!!!

Conclusion

In this part, you successfully integrated Ansible with Jenkins, transforming your CI/CD pipeline into a powerful automation engine capable of managing infrastructure.

You have:

• Installed Ansible inside a custom Jenkins container

• Configured secure SSH access using private keys

• Created and tested Ansible inventory and playbooks

• Executed automation tasks on remote systems

• Integrated Ansible execution into Jenkins jobs

This is a major milestone in your DevOps journey — you are no longer just building pipelines, but automating infrastructure at scale.

This setup reflects real-world practices where Jenkins acts as an orchestrator, and Ansible handles configuration management and deployment.

🔗 Continue the Series

⬅️ Previous Article: Part 5 Making Jenkins Automation Scalable
➡️ Next Article: Part 7 Jenkins Pipeline from GitHub

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

Goal

The goal of this project is to transform the existing Jenkins automation into a scalable and reusable solution by externalizing scripts and ensuring persistence across container restarts.

In this exercise, we will move the backup script outside the container and mount it using Docker volumes, allowing it to persist even if the container is recreated. Additionally, we will extend the setup to support multiple databases and multiple S3 buckets, making the automation flexible for real-world use cases.

By the end of this project, you will have a Jenkins-driven automation setup that is resilient, reusable, and scalable across multiple environments.

Purpose

The purpose of this exercise is to demonstrate how to design Jenkins automation in a way that is production-ready and maintainable.

In earlier parts, the automation worked, but it was tightly coupled to the container lifecycle. In real-world DevOps environments, automation must be:

• Persistent – survives container restarts or failures

• Reusable – can be applied across multiple use cases

• Scalable – supports multiple applications, databases, or environments

By externalizing the script and using Docker volume mapping, we ensure that critical automation logic is not lost when containers are recreated.

Extending the solution to handle multiple databases and S3 buckets introduces flexibility, which is essential for managing enterprise-scale infrastructure and data workflows.

This approach reflects real DevOps practices where pipelines are designed to be modular, reusable, and environment-agnostic.

Preresuisite

• Ready to use Host and the directory structure to run Dockerfiles and docker-compose.yml (Refer to Part 1)

Step-by-step implementation

• How to create this script permanently on a container?

• Go to the home folder and create an aws-s3.sh and copy the content of the script.sh

/home/jenkins/jenkins_home$vi aws-s3.sh

• Go to docker-compose.yml and update the volumes block for remote_host which will copy this script to the remote host in case of a container crash

services:
  jenkins:
    container_name: jenkins
    image: jenkins/jenkins:lts
    ports:
      - "8080:8080"
    volumes:
      - $PWD/jenkins_home:/var/jenkins_home
    networks:
      - net

  remote_host:
    container_name: remote-host
    image: remote-host
    build:
      context: centos7
    volumes:
      - "$PWD/aws-s3.sh:/tmp/script.sh"
    networks:
      - net

  db_host:
    container_name: db
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: 1234
    volumes:
      - $PWD/db_data:/var/lib/mysql
    networks:
      - net

networks:
  net:

• Test it by deleting the remote-host

• What if we have multiple databases and the backup has to be uploaded to different S3 buckets?

• Create another S3 "jenkins-mysql-backup-mylab-2"

• Create another database testdb2

• Now, let's try to take this testdb2 backup

• Check logs

• Verify upload

Done!!!

Conclusion

In this part of the series, we enhanced our Jenkins automation to make it scalable, reusable, and production-ready.

We achieved this by:

• Externalizing the backup script to ensure persistence

• Using Docker volumes to maintain scripts across container restarts

• Testing resilience by recreating containers

• Extending the solution to support multiple databases and S3 buckets

This transformation moves our setup from a simple working automation to a robust DevOps solution capable of handling real-world scenarios.

Designing scalable automation is a key skill for DevOps engineers, as it ensures systems can grow and adapt without requiring constant reconfiguration.

🔗 Continue the Series

⬅️ Previous Article: Part 4 Automating MySQL Backup to AWS S3
➡️ Next Article: Part 6 Running Ansible from Jenkins

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

Goal

The goal of this project is to build a real-world DevOps automation workflow where Jenkins automatically creates a MySQL database backup and uploads it to Amazon S3.

In this exercise, Jenkins will connect to the remote server created in Part 2, execute a script that performs a mysqldump, and securely upload the backup file to an S3 bucket using AWS CLI.

By the end of this project, you will have a fully automated database backup pipeline controlled by Jenkins.

We can make use of this server to take a MySQL DB backup and upload it to a remote location (S3)

Purpose

The purpose of this exercise is to demonstrate how Jenkins can be used to automate infrastructure maintenance tasks, such as database backups and secure cloud storage operations.

In production environments, database backups are critical for:

• Disaster recovery

• Data protection

• Compliance requirements

• Operational resilience

Instead of manually running backup scripts, DevOps teams typically automate this process using CI/CD tools like Jenkins.

Through this project, readers will learn how to:

• Use Jenkins to trigger automation scripts

• Execute scripts on remote infrastructure

• Securely manage credentials

• Integrate Jenkins with AWS services

This project simulates a real-world backup automation pipeline commonly used in enterprise infrastructure.

Prerequisites

Before starting this project, ensure the following components are already set up.

• Ready to use Host and the directory structure to run Dockerfiles a docker-compose.yml (Refer to Part 1)

• An AWS account

Step-by-step implementation

Before we create the "db" container, we need to install useful tools, mainly the MySQL server, client and AWS CLI, on the remote-host container

Pro-Tip: Read Dockerfile carefully

• Go to /home/jenkins/jenkins_home/centos7$ and Modify Dockerfile

FROM rockylinux:9

# Install required packages
RUN dnf install -y \
    openssh-server \
    passwd \
    sudo \
    mysql \
    unzip \
    less \
    groff && \
    dnf clean all

# Install AWS CLI v2
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
    unzip awscliv2.zip && \
    ./aws/install && \
    ./aws/install --update && \
    rm -rf aws awscliv2.zip

# Create user
RUN useradd -m remote_user && \
    echo "remote_user:1234" | chpasswd

# Configure SSH directory
RUN mkdir -p /home/remote_user/.ssh

# Copy public SSH key
COPY remote-key.pub /home/remote_user/.ssh/authorized_keys

# Fix SSH permissions
RUN chown -R remote_user:remote_user /home/remote_user/.ssh && \
    chmod 700 /home/remote_user/.ssh && \
    chmod 600 /home/remote_user/.ssh/authorized_keys

# Enable password authentication for SSH
RUN sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config

# Generate SSH host keys
RUN ssh-keygen -A

# Expose SSH port
EXPOSE 22

# Start SSH daemon
CMD ["/usr/sbin/sshd","-D"]

• Update docker-compose.yml to create a db container (We have it on \jenkins_home\)
  Create a container named "db" by pulling image 8.0, its password will be "1234", and volumes will be stored at the path given, and will be using the same existing network to communicate with other containers

services:
  jenkins:
    container_name: jenkins
    image: jenkins/jenkins:lts
    ports:
      - "8080:8080"
    volumes:
      - $PWD/jenkins_home:/var/jenkins_home
    networks:
      - net

  remote_host:
    container_name: remote-host
    image: remote-host
    build:
      context: centos7
    networks:
      - net

  db_host:
    container_name: db
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: 1234
    volumes:
      - $PWD/db_data:/var/lib/mysql
    networks:
      - net

networks:
  net:

• Build a new image for the container remote-host (SSH server)

docker compose build 

• Create db container

docker compose up -d

• Verify the logs, if MySQL is ready

docker logs -f db

• Look at the message shown below

docker ps

• Log in to the db container and verify

# login to container
docker exec -it db bash

# login to mysql
mysql -u root -p

# password is given in docker compose
1234

# check databases
show databases;

• These results should look like this:

• Log in to the remote-host and check if the basic MySQL and AWS CLI tools are working

Note - Error are expected, means its installed.

• Go back to db container and log in docker exec -ti db bash

• Create a Database by using the below SQL queries

# Create DB
mysql> create databse testdb;

# Change db
mysql> use testdb

# Create DB table
mysql> create table info (name varchar(20), lastname varchar(20), age int(2));

# Add some fake info
mysql> insert into info values ('abhi', 'chougule', 30);

# Vrify
mysql> select * from info

• Look Like below

• Insert some details

Next AWS S3

Create an S3 bucket by logging into AWS

• Create an IAM user with S3 access

• Create a secret access key with the Command Line Interface

• Manual upload: Go back to the remote-host container and take a backup of MySQL

[IMP] Note - We are taking backup from remote-host bash terminal

mysqldump -u root -h remote-host -p testdb > /tmp/db.sql

• Verify backup

• Configure AWS CLI on the remote-host container to S3 and upload

• Verify on S3

• Let's automate this process of creating a dump and uploading it using Jenkins

Create a script.sh on remote-host under /tmp/script.sh

#!/bin/bash

DATE=$(date +%H-%M-%S)
BACKUP=db-$DATE.sql

DB_HOST=$1
DB_PASSWORD=$2
DB_NAME=$3
AWS_SECRET=$4
BUCKET_NAME=$5

mysqldump -u root -h "\(DB_HOST" -p"\)DB_PASSWORD" "\(DB_NAME" > /tmp/\)BACKUP && \
export AWS_ACCESS_KEY_ID=AKIAQE537I5GMUA722C6 && \
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET && \
echo "uploading your $BACKUP backup" && \
aws s3 cp /tmp/db-\(DATE.sql s3://\)BUCKET_NAME/$BACKUP

Description

DATE: The variable for date will be saved
BACKUP: Name will be derived for SQL dump
Numbering in front of variables: Will decide the script sequence
Where the actions will be performed
Export Access ID is provided, but the Access key will be retrieved from Jenkins credentials
Copy-Paste command

• Go to the Jenkins console and create a job to trigger this process

• Create secret credentials

• Go to Manage Jenkins--> Credentials-->Add credentials--> Secret text
  AWS_SECRET_ACCESS_KEY MYSQL_PASSWORD

• Create backup-mysql freestyle job and add these details

• Select: This Project is parameterized
  String parameter: MYSQL_HOST
  Default Value: db_host

  String Parameter: DATABASE_NAME
  Default Value: testdb

  String Parameter: AWS_BUCKET_NAME
  Default Value: jenkins-mysql-backup-mylab

  String Parameter: DATABASE_NAME
  Default Value: testdb

• Environments:
  secret text: MYSQL_PASSWORD
  Credentials: <MYSQL_PASSWORD>

  secret text: AWS_SECRET
  Credentials: <AWS_SECRET_ACCESS_KEY>

• Build Steps:
  Select: Execute a shell script on a remote host using SSH
  SSH site: remote_user@remote_host:22
  Command: /tmp/script.sh \(MYSQL_HOST \)MYSQL_PASSWORD \(DATABASE_NAME \)AWS_SECRET $AWS_BUCKET_NAME

Pro-Tip: The command: sequnce is based on script numbering to call each block from the setup

Note 1 - This step we created SSH site at start of the article
Go to Manage Jenkins --> SSH Server and add details of our SSH server

Note 2- Form the command based on the sequence in script.sh for all the above parameters created here and use the "$" prefix

Here is what I set up

• Run the Build with Parameters

• Logs

• Verify upload

Done!!!

Conclusion:

In this project, we built a fully automated database backup pipeline using Jenkins.

The workflow included:

• Jenkins triggering automation tasks

• Remote server executing the backup script

• MySQL database dump creation

• Secure upload to AWS S3

This setup demonstrates a practical DevOps automation pattern used in production environments, where CI/CD tools manage infrastructure operations such as backups, deployments, and system maintenance.

By automating database backups through Jenkins, organizations can ensure reliable, repeatable, and secure data protection workflows.

🔗 Continue the Series

⬅️ Previous Article: Part 3 Jenkins + SSH Remote Execution
➡️ Next Article: Part 5 Making Jenkins Automation Scalable

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

Goal

The goal of this project is to create and run the first Jenkins Pipeline that automates the process of building and testing an application.

In this exercise, we will configure Jenkins to pull application code from a Git repository, execute build commands using Maven, and run automated tests. This introduces the fundamental structure of a Jenkins Declarative Pipeline, including stages such as workspace cleanup, source code checkout, build execution, and testing.

By the end of this project, you will understand how Jenkins pipelines automate repetitive development tasks and form the foundation of modern CI/CD workflows.

Purpose

The purpose of this exercise is to demonstrate how Jenkins pipelines automate the software development lifecycle by defining a structured workflow for building and testing applications.

In traditional development environments, developers manually run build and test commands. Jenkins pipelines eliminate these manual steps by automatically executing them whenever a job is triggered.

Through this project, you will learn how to:

• Define a Declarative Jenkins Pipeline

• Organize automation tasks using pipeline stages

• Integrate Jenkins with Git repositories

• Execute build and test commands automatically

• Understand how CI pipelines validate code changes

This hands-on exercise introduces the core concept of Continuous Integration, where code is automatically built and verified before moving further in the deployment pipeline.

Prerequisites

Before starting this project, ensure the following requirements are completed:

• Ready to use Host and the directory structure to run Dockerfiles and docker-compose.yml (Refer to Part 1)

• Demo-app repo: Fork the application repository used for the pipeline demonstration:

Step-by-step Implementation

/home/jenkins/jenkins_home/docker-compose.yml

docker-compose.yml

services:
  jenkins:
    container_name: jenkins
    image: jenkins/jenkins
    ports:
      - "8080:8080"
    volumes:
      - $PWD/jenkins_home:/var/jenkins_home
    networks:
      - net
networks:
  net:

Note- No need to use customized Dockerfile since this will download a jenkins image from Docker Hub which has enough packages required for this project

docker compose up -d

• Install Maven

# Login
docker exec -it -u root jenkins bash

# Install maven
apt-get install maven

Note- Since I have Maven already installed, it shows it exists

• Create Jenkins Job I names it maven-pipeline

• Create Jenkinsfile seperatly and paste it into Pipleine script area

• Demo application repo can be forked

pipeline {
    agent any

    stages {

        stage('Clean Workspace') {
            steps {
                deleteDir()
            }
        }

        stage('Checkout Code') {
            steps {
                git url: 'https://github.com/abhinandan-chougule/demo-app-java-maven.git', branch: 'main'
            }
        }

        stage('Build with Maven') {
            steps {
                sh 'mvn clean install'
            }
        }

        stage('Run Tests') {
            steps {
                sh 'mvn test'
            }
        }
    }
}

• Click on Build Now

• Console output Logs

Done!!!

Project 4: Jenkinsfile controlled through version control GitHub

• Goal:
  Create Pipeline Job
  Provide where to refer the Jenkinsfile
  Run the Build

Note- We will make use of Jenkins is from above project

• Fork repo

• Jenkinsfile placed at below locaion in repo and will be used

• Create new Job maven-github-pipeline

• Select options as below

Definition: Pipeline script from SCM
SCM: Git
Repository URL : Forked repo HTTP URL
Branch Specifier (blank for 'any'): */main
Script Path: jenkins/Jenkinsfile

Build now

• Go through Console output logs

• Build now one more time and go to Jobs Console output and compare

This is not cloning the repo again, instead it's checking if there are any latest commits and changes

Done!!!

Project: 5 Multibranch Pipeline

• Go to the repo and add another branch.

• Scan the Multibranch pipeline now, and it will populate the branches

• Let's make some changes to the repo and commit to the new branch."devbranch"

• I am deleting the last stage

Done!!!

Project 6: Parameterize to give the option in the Jenkins Job to run on a specific AZ

Goal

Create a Jenkins pipeline named parameter-pipeline that accepts user inputs through parameters to control a deployment configuration. The pipeline should allow the user to specify a custom deployment name, choose the target AWS Availability Zone, and confirm the deployment before execution.

Purpose

The purpose of this Configuration is to demonstrate how parameterized Jenkins pipelines work. By defining parameters, the pipeline becomes more flexible and interactive, allowing users to provide deployment-specific inputs at runtime instead of hardcoding values in the pipeline.

This exercise introduces three commonly used Jenkins parameter types:

• String Parameter – to provide a custom deployment name.

• Choice Parameter – to select the AWS Availability Zone (EU-WEST-2A, EU-WEST-2B, EU-WEST-2C).

• Boolean Parameter – to confirm whether the deployment should proceed.

Using parameterized pipelines is a common practice in DevOps CI/CD workflows, enabling controlled deployments across different environments or regions while minimizing manual changes to pipeline code.

Here is the code that will be used

pipeline {
    agent any

    parameters {
        string(
            name: "deploymentName",
            defaultValue: "",
            description: "Deployment Name?"
        )

        choice(
            name: "azDeploy",
            choices: ["ap-southeast-1a", "ap-southeast-1b", "ap-southeast-1c"],
            description: "What AZ?"
        )

        booleanParam(
            name: "confirmDeploy",
            defaultValue: false,
            description: "CONFIRM DEPLOYMENT?"
        )
    }

    stages {
        stage("Deploy") {
            steps {
                echo "Deployment Name: ${params.deploymentName}"
                echo "AZ Selected: ${params.azDeploy}"
                echo "Deployment Confirmation: ${params.confirmDeploy}"
            }
        }
    }
}

• Create Pipeline parameter-pipeline"

• Add the script to the pipeline script

• Build Now

• Provide inputs and click build with parameters

• Expected console output

• Pipeline status

Done!!!

Project: How to virtualize the Jenkins jobs so it would be reusable

Goal

Create a Jenkins pipeline that demonstrates how to define and use environment variables within a Jenkins Declarative Pipeline. The pipeline initializes different types of variables (string, number, and boolean) and verifies that they can be accessed and printed during pipeline execution.

Purpose

The purpose of this task is to understand how environment variables work inside a Jenkins pipeline and how they can be referenced within stages and steps. By defining variables such as a string, a number, and a boolean inside the environment block, the pipeline provides a simple demonstration of how configuration values can be stored and reused across pipeline stages.

This approach helps standardize pipeline configurations and allows DevOps engineers to manage commonly used values in a centralized location within the pipeline. Environment variables are widely used in CI/CD workflows for tasks such as configuring application settings, defining build parameters, and passing values between different stages of a pipeline.

Through this exercise, users learn how Jenkins pipelines can manage and reference variables efficiently, which is a foundational concept when building more advanced automation and deployment pipelines.

• Prepare the code

pipeline {
    agent any

    environment {
        def myString = "Hello World"
        def myNumber = 10
        def myBool = true
    }

    stages{
        stage("Demo") {
            steps {
                echo "myString: ${myString}"
                echo "myNumber: ${myNumber}"
                echo "myBool: ${myBool}"
            }
        }
    }
}

• Create pipeline "variable-jenkins."

• Paste the code into the pipeline script

• Build Now

• Now we can see the pipeline using the variables given in the environment block

Done!!!

Conclusion

In this part of the Jenkins CI/CD series, we created and executed our first Jenkins pipeline. The pipeline automated a basic CI workflow that includes cleaning the workspace, cloning source code from Git, building the application using Maven, and running automated tests.

This project introduced the fundamental structure of Jenkins pipelines and demonstrated how build and test processes can be automated through a simple pipeline script.

Understanding how to create and execute pipelines is a critical step toward building scalable CI/CD workflows. Jenkins pipelines allow teams to standardize build processes, reduce manual intervention, and ensure consistent application validation.

In the next part of the series, we will expand Jenkins automation capabilities by enabling it to execute commands on remote servers using SSH, which is a common requirement in deployment and infrastructure automation workflows.

🔗 Continue the Series

⬅️ Previous Article: Part 1 Install Jenkins with Docker
➡️ Next Article: Part 3 Jenkins SSH Remote Execution

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

Goal

The goal of this project is to extend the Jenkins environment created in Part 1 by integrating it with a remote server through SSH (Secure Shell). In this exercise, we will create a container that acts as a remote host with an SSH server and configure Jenkins to securely connect to it.

By the end of this project, Jenkins will be able to execute commands and scripts on a remote machine, enabling automation tasks such as running scripts, managing servers, and performing deployment-related operations.

Purpose

The purpose of this exercise is to demonstrate how Jenkins can interact with external systems through SSH. In real-world DevOps environments, Jenkins often needs to run commands on remote machines to perform operations such as application deployment, server configuration, and maintenance tasks.

By creating a dedicated SSH-enabled container and integrating it with Jenkins, we simulate a typical automation scenario where Jenkins acts as the orchestration tool while executing commands on remote infrastructure.

This setup helps us to understand how Jenkins communicates with remote systems securely using SSH keys and credentials, which is a fundamental concept in building automated CI/CD pipelines.

Prerequisites

Before starting this project, ensure the following requirements are completed:

• Ready to use Host and the directory structure to run Dockerfiles and docker-compose.yml (Refer to Part 1)

• Access to the Jenkins dashboard with administrative privileges.

Step-by-Step Implementation

The purpose of the SSH server is to run shell commands or scripts on other servers

The directory should look like this:

Note- Since we are using Docker, we will use Dockerfiles or images available on Docker Hub.

Create Containers

Jenkins - will download the latest image from Docker Hub
SSH server - Customized Dockerfile (name: remote-host)

• Create a Dockerfile in the directory jenkins/jenkins_home/centos7

Note- centos7 is just random directory name I used

Dockerfile

FROM rockylinux:9

RUN dnf install -y openssh-server passwd sudo && dnf clean all

RUN useradd -m remote_user && \
    echo "remote_user:1234" | chpasswd

RUN mkdir -p /home/remote_user/.ssh

COPY remote-key.pub /home/remote_user/.ssh/authorized_keys

RUN chown -R remote_user:remote_user /home/remote_user/.ssh && \
    chmod 700 /home/remote_user/.ssh && \
    chmod 600 /home/remote_user/.ssh/authorized_keys

RUN sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config

RUN ssh-keygen -A

EXPOSE 22

CMD ["/usr/sbin/sshd","-D"]

• The YAML will create two containers, and I have placed it at jenkins/jenkins_home/

docker-compose.yml

services:
  jenkins:
    container_name: jenkins
    image: jenkins/jenkins
    ports:
      - "8080:8080"
    volumes:
      - $PWD/jenkins_home:/var/jenkins_home
    networks:
      - net
  remote_host:
    container_name: remote-host
    image: remote-host
    build:
      context: centos7
    networks:
      - net
networks:
  net:

• Build the remote-host image for the SSH server

docker compose build

• Verify Image

docker images

• Create containers by using the docker-compose.yml file from jenkins/jenkins_home/

docker compose up -d

• Verify containers with docker ps

• Log in to Jenkins

Note - host_ip:8080

By using the credentials you created (Refer First part of this article if needed)

• Go to Manage Jenkins--> Plugins and install the SSH plugin

• Go to Manage Jenkins --> Credentials --> SSH Username with Private key

• Add username remote-user and copy the private key as a remote key below

Note- This is created part of docker compose and placed in container context (centos: it will act like persistant volume for remote-host container)

• Go to Manage Jenkins --> SSH Servere and add details of our SSH server and Check connection

here: Integration completed

• Let's test by creating a Freestyle project and testing

• Go to Build Steps and add Execute shell script on remote host using SSH

• Select our user details from the drop-down,

• Verify if this Job Runs the command on the remote-host and creates a file

Click Build now and check console output from Job number

• Logs

• Verify if the file has been created or not on the remote-host

Done!!!

Conclusion

In this part of the series, we successfully created a remote server container running an SSH service and integrated it with Jenkins. We configured Jenkins to securely connect to the remote host using SSH credentials and executed commands on the remote system through a Jenkins job.

This setup demonstrates how Jenkins can control external infrastructure and automate tasks beyond its own environment. Remote command execution is widely used in DevOps pipelines for tasks such as application deployment, server maintenance, configuration management, and system monitoring.

With this integration in place, Jenkins is now capable of orchestrating tasks across multiple systems, which is a critical capability for modern CI/CD automation.

In the next part of the series, we will build on this foundation by creating a real-world automation workflow where Jenkins performs a MySQL database backup and uploads it to AWS S3 automatically.

🔗 Continue the Series

⬅️ Previous Article: Part 2 Creating Your First Jenkins Pipeline
➡️ Next Article: Part 4 Automating MySQL Backup to AWS S3

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

The Gateway API introduces a more powerful and expressive model for managing traffic in Kubernetes clusters.

In this guide, we will build a modern traffic routing architecture using KGateway and the Gateway API on Kubernetes.

Advantages:

• Better than traditional Ingress

• Standard Kubernetes networking API

• Flexible routing rules

• Multi-cluster support

• Advanced traffic management

We will cover:

• Installing Gateway API CRDs

• Deploying the KGateway controller

• Creating a Gateway

• Deploying an application

• Routing traffic using HTTPRoute

Environment Setup

Create an EC2 Instance with t2-medium and 30gb gp3 disk, it will be used as a jump host to manage the cluster

sudo apt update

• Connect to EC2 by SSH

• Clone the repo

This will create all basic configurations for EKS

• Install AWS CLI

#!/bin/bash

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt install unzip
unzip awscliv2.zip
sudo ./aws/install
aws configure

• Create an Access key and secret key if its first time using the AWS console, and Enter into the console when it asks while installing AWS CLI

• Install Terraform

#!/bin/bash

#Update system package
sudo apt-get update

#Install GNU software properties and curl packages
sudo apt-get install -y gnupg software-properties-common -y

#Install the HashiCorp GPG key
wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg > /dev/null

#Verify the key's fingerprint
gpg --no-default-keyring \
--keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg \
--fingerprint

echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
sudo tee /etc/apt/sources.list.d/hashicorp.list

sudo apt update
sudo apt-get install terraform -y

terraform --version

• Install Kubectl

#!/bin/bash

curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin
kubectl version --short --client

• aws configure to connect with the AWS account

• aws sts get-caller-identity to Verify Account

• cd to the directory and Terraform init

• terraform plan

• Apply the changes to deploy EKS

terraform apply --auto-approve

• Install helm

sudo apt update & sudo apt upgrade -y
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

• Verify if the cluster is accessible

aws eks --region ap-southeast-1 update-kubeconfig --name askabhi-cluster

kubectl get nodes

• Install CRD for kgateway by helm upgrade

Note: Prerequisites

Before you begin, ensure you have:

• A Kubernetes cluster.

• kubectl installed and configured to communicate with your cluster.

• helm installed, the package manager for Kubernetes.

Install the Kubernetes kgateway CRD and Gateway API CRDs (Custom Resource Definitions)
Kgateway is an implementation of the Kubernetes Gateway API, which requires specific CRDs to be present in your cluster. The official documentation recommends installing the CRDs separately from the main kgateway chart.

helm upgrade -i --create-namespace --namespace kgateway-system kgateway-crds oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds --version v2.3.0-main

(Note: The --version tag may need to be updated to the latest release.)

Install the Kgateway Controller
Once the CRDs are in place, install the main kgateway controller.

helm upgrade -i --namespace kgateway-system kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway --version v2.3.0-main

Verify the Installation
After the installation is complete, verify that the kgateway pods are running in the kgateway-system namespace.

kubectl get pods -n kgateway-system

You should see pods with a Running status. You can also verify that the GatewayClass resource named kgateway has been created.

We need to decide which controller needs to be used

kubectl get gatewayclass kgateway

• Install the Gateway API CRDs

First, apply the standard CRDs from the official Kubernetes sigs GitHub repository to your cluster. The standard channel is recommended for most users as it includes stable (GA or beta) resources:

kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.0/standard-install.yaml

To verify the installation, you can run:

Verify

kubectl get crds | grep gateway

Install the httpbin Application

The httpbin sample application for kgateway (formerly part of Gloo) can be deployed in Kubernetes using the following command to test routing rules:

kubectl apply -f https://raw.githubusercontent.com/kgateway-dev/kgateway/refs/heads/main/examples/httpbin.yaml

Purpose: This YAML deploys a httpbin service in a namespace (typically httpbin), which is used to verify that the Gateway proxy is correctly routing external traffic to backend services.

Verify what the namespaces we have are

Check the route for the gateway

kubectl get gateway

Traffic is going to which service

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kgateway-system
spec:
  gatewayClassName: kgateway
  listeners:
  - name: http
    protocol: HTTP
    port: 80
    # Optional: restricts which namespaces can attach routes to this listener
    allowedRoutes:
      namespaces:
        from: All # Other options: Same, Selector
EOF

Check gateway Created

kubectl get gateway

The address shown in the above snap is Cloud Load Balancer a243521ba1b904868a7d7b8707db3b7c-2045018130.ap-southeast-1.elb.amazonaws.com

• Copy this and go to GoDaddy and add a CNAME

• Go back to the server and do nslookup for the Cloud Load Balancer and get the IP.

Note- we should see the IP's like below else wait

• Add an A record with the IP you see

Route the traffic to the Domain

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin
  namespace: httpbin
spec:
  parentRefs:
  - name: kgateway-system
    namespace: default
  hostnames:
  - "mytestproject.shop"
  rules:
  - backendRefs:
    - name: httpbin
      port: 8000
EOF

• What did we do? We pointed out the following service/httpbin to route through the domain

• Now you can try to access the Domain

Conclusion:

Kubernetes networking continues to evolve, and the Gateway API represents the next step forward in traffic management.

By combining Gateway API and KGateway, we can build scalable and flexible networking architectures that are easier to manage than traditional ingress-based setups.

This architecture enables modern DevOps teams to implement advanced routing patterns while keeping Kubernetes networking standardized and maintainable.

This Part 1 base requirement for rest of the series parts

Goal

The goal of this project is to install Jenkins using Docker and create a fully functional Jenkins environment that can be used for building CI/CD pipelines. By the end of this exercise, Jenkins will be running inside a Docker container and accessible through a web browser.

This setup provides a clean and reproducible environment for Jenkins, making it easy to deploy, manage, and upgrade without directly installing Jenkins on the host operating system.

Purpose

The purpose of this exercise is to introduce the foundation of Jenkins-based CI/CD environments by containerizing Jenkins using Docker. Running Jenkins inside a container ensures portability, easier management, and better isolation from the host system.

This approach is widely used in modern DevOps environments because it allows teams to quickly spin up Jenkins instances, maintain persistent data using volumes, and scale automation infrastructure efficiently.

Through this project, we will:

• Install Docker and configure the Docker repository

• Pull the official Jenkins image from Docker Hub

• Deploy Jenkins using Docker Compose

• Configure persistent storage for Jenkins data

• Access the Jenkins web interface and complete the initial setup

This forms the starting point for the Jenkins CI/CD Mastery series, where the Jenkins instance created in this step will be used for building pipelines, automating tasks, and integrating with other tools in later projects.

Prerequisites

Prepare Host Linux machine locally on Windows/Mac (Refer), or on any Cloud

Pro-Tip: Recommended on AWS:EC2, Stop the EC2 instance whenever finish the task

Step-by-Step Implementation

Run the following command to uninstall all conflicting packages in case you are using any existing Linux system

sudo apt remove $(dpkg --get-selections docker.io docker-compose docker-compose-v2 docker-doc podman-docker containerd runc | cut -f1)

Install using the apt repository

Before you install Docker Engine for the first time on a new host machine, you need to set up the Docker apt repository. Afterward, you can install and update Docker from the repository.

• Set up Docker's apt repository.

Add Docker's official GPG key:

sudo apt update
sudo apt install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc

Add the repository to Apt sources:

sudo tee /etc/apt/sources.list.d/docker.sources <<EOF Types: deb URIs: https://download.docker.com/linux/ubuntu Suites: \((. /etc/os-release && echo "\){UBUNTU_CODENAME:-$VERSION_CODENAME}") Components: stable Signed-By: /etc/apt/keyrings/docker.asc EOF

sudo apt update

Install the Docker packages.

To install the latest version, run:

sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Note

The Docker service starts automatically after installation. To verify that Docker is running, use:

sudo systemctl status docker

Some systems may have this behavior disabled and will require a manual start:

 sudo systemctl start docker

Download Jenkins Image

It will have all the required configurations

sudo docker pull jenkins/jenkins

• Navigating/creating folder structure home/jenkins/jenkins_home#

Create docker-compose.yml

services:
  jenkins:
    container_name: jenkins
    image: jenkins/jenkins
    ports:
      - "8080:8080"
    volumes:
      - $PWD/jenkins_home:/var/jenkins_home
    networks:
      - net
networks:
  net:

• Create another folder that is mentioned in the YAML above home/jenkins/jenkins_home

Give the permissions

sudo chown 1000:1000 jenkins_home -R
docker compose up -d
docker ps

Note - Alwasy run docker compose from directory where docker-compose.yml resides

Pro Tips - Check the logs. docker logs -f jenkins

• Access by using "host_IP:8080."

• The password will be stored at /var/jenkins_home/secrets/initialAdminPassword

Install Suggested Plugins

• Create an admin user and go to the Home page

Done!!!

Conclusion:

In this first part of the Jenkins CI/CD Mastery Series, we successfully set up Jenkins using Docker, providing a clean and portable environment for running Jenkins without installing it directly on the host operating system. By containerizing Jenkins, we ensured that the setup is easy to manage, reproducible, and scalable for future CI/CD workloads.

We installed Docker, configured the Docker repository, pulled the official Jenkins image, and deployed Jenkins using Docker Compose with persistent storage. Finally, we accessed the Jenkins web interface and completed the initial setup by installing the suggested plugins and creating the administrator account.

This Jenkins instance will serve as the foundation for the rest of the series, where we will build real-world automation and CI/CD workflows.

In the next part, we will extend this environment by creating an SSH-enabled remote server container and integrating it with Jenkins. This will allow Jenkins to execute commands on remote systems — a key capability used in many DevOps automation pipelines.

🔗 Continue the Series

⬅️ Previous Article: Home
➡️ Next Article: Part 2 Creating Your First Jenkins Pipeline

⭐ If you found this article useful, follow https://ask-abhi.com for more DevOps tutorials.

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

• Add Helm Stable Charts for Your Local Client

helm repo add stable https://charts.helm.sh/stable

• Add Prometheus Helm Repository

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

• Create Prometheus Namespace

kubectl create namespace prometheus
kubectl get ns

• Install Prometheus using Helm

helm install stable prometheus-community/kube-prometheus-stack -n prometheus

• Verify Prometheus installation

kubectl get pods -n prometheus

• Check the services file (svc) of the Prometheus

kubectl get svc -n prometheus

• Expose Prometheus and Grafana to the external world through Node Port

[Note] change it from Cluster IP to NodePort after changing make sure you save the file and open the assigned nodeport to the service.

kubectl edit svc/stable-kube-prometheus-sta-prometheus -n prometheus

Verify service

kubectl get svc -n prometheus

• Now, let’s change the SVC file of Grafana and expose it to the outer world

kubectl edit svc stable-grafana -n prometheus

• Check the Grafana service

kubectl get svc -n prometheus

• Access the Grafana with <public-ip-of-master>:32108 and Prometheus with <public-ip-of-master>:31521

Note - First port-forward through the below command:

kubectl port-forward svc/stable-grafana 32108:80 -n prometheus --address 0.0.0.0 &

kubectl port-forward svc/stable-kube-prometheus-sta-prometheus 31521:9090 -n prometheus --address 0.0.0.0 &

• Prometheus:

• Get a password for Grafana

[Note] Username: admin

kubectl get secret --namespace prometheus stable-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

• Now, view the Dashboard in Grafana

• Click on Dashboards

• Put the Data Source: Prometheus and Namespace: wanderlust and run for the last 5 minutes.

• Custom dashboard by searching on google.com

• Select anything you liked! and copy the ID

• Import

Finish!!!

If you managed to follow all the parts and implemented all the steps sucessfully then it's time to celebrate because its most demanding skills in Market

argocd login 65.2.186.181:31797 --username admin

[Note] 65.2.186.181:31797 --> This should be your argocd url and after login enter your password

• Check how many clusters are available in argocd

argocd cluster list

• Get your cluster name

kubectl config get-contexts

• Prepare the command to add your cluster to argocd and Enter

argocd cluster add iam-root-account@wanderlust.ap-southeast-1.eksctl.io --name wanderlust-eks-cluster

Note: Unknown possible due to sync issues, miss config or may takes some time, troubleshoot accordingly, continue to next and try later by readding again after connecting to repo in next step

Go to Settings --> Repositories and click on Connect your forked repo

[Note] Connection should be successful

• Now, go to Applications and click on New App

[Important] Make sure to click on the Auto-Create Namespace option while creating argocd application

Congratulations, your application is deployed on the AWS EKS Cluster

• Open ports 31000 and 31100 on the nodes of the EKS Cluster, and access it on a browser

<worker-public-ip>:31000

Email Notification

Next...

The Jenkins Pipeline script you will get in my GitHub —> Jenkinsfile, before putting the pipeline script in Jenkins, do the below steps:

• Fork the Repository

• Go to Jenkinsfile

• Make Sure you have forked and configured the shared library repository for jenkinsfile which uses groovy syntax.

• Name the Jenkins slave node by Node only.

• The SonarQube tool, which Jenkins will install, should be named as Sonar as per the pipeline script.

• Make Sure you change the repository URL and branch as per your GitHub URL.

• Change the image name and username.

Note- In terms of sucessful pipleine run we need to setup all customized parameters as per our environment variables

• In place of fullstack-backend-beta, Put the name of docker image that you want, and in place of abhic25, put your Dockerhub username so that it gets tagged with it.

Create one more pipeline fullstack-cd

• The Jenkins Pipeline script you will get in my GitHub —> GitOps → Jenkinsfile, before putting the pipeline script in Jenkins, change all the customized parameters with respect to environments

Ex. What we did for the CI Job earlier

• Go to Kubernetes → Change the username in backend.yaml and frontend.yaml as per your Dockerhub username.

After all of this, Our Jobs are now ready create one more pipeline, fullstack-CD

• Provide permission to docker socket so that Docker build and push commands do not fail (On the Jenkins Slave)

• Sudo chmod 777 /var/run/docker.sock
  

• Now Click on fullstack-ci and build, and then when it gets completed, fullstack-cd will automatically trigger and get built.

• Note- We have provide new Tag in ci job each time so CD will detect cahnge in Image and copletes job sucessfully

• 

• CI Job

• CD Job

Next...

• OWASP (Dependency Check)
• SonarQube Scanner (Quality)
• Git (Code)
• Docker (Containerrization)
• Docker Pipeline
• Pipeline: Stage View
• Pipeline: Job
• Blue Ocean

Fork GitHub Repo: fullstack and share-library

• Configure OWASP, move to Manage Jenkins --> Plugins --> Available plugins

• After the OWASP plugin is installed, move to Manage Jenkins --> Tools

• Log in to the SonarQube server and create the credentials for Jenkins to integrate with SonarQube
  Navigate to Administration --> Security --> Users --> Token

• Now, go to Manage Jenkins --> credentials
  SonarQube: Integration
  GitHub: Pull shared library and application source code
  Docker Hub: to push the image
  Create an API credentials for OWASP

• GitHub user name and PAT (as a password)

• Create and use an NVD API Key so OWASP can download with the API
  Generate API Key. Go to: https://nvd.nist.gov/developers/request-an-api-key
  Takes ~10 seconds. Key arrives by email.

• Add Key to Jenkins Credentials In Jenkins: Manage Jenkins → Credentials → Global → Add Credentials
  Type: Secret Text
  ID: NVD_API_KEY
  Secret: Fix Your API Key: 397ff9xxxxxxxxxxxxxx7b7d1c37

• Go to Manage Jenkins --> Tools and search for SonarQube Scanner installations:

• Now again, go to Manage Jenkins --> System and search for Global Trusted Pipeline Libraries:

Note: Before putting this shared library, Fork GitHub shared repository, and put the project repository URL of your repository.

• Log in to the SonarQube server, go to Administration --> Webhook, and click on create

Public Ip is of Jenkins CI (Master node) and port 8080 because SonarQube is running on it only.

http://public-ip-of-master:8080/sonarqube-webhook

• Now, go to the fullstack GitHub repository (hope you forked already) and under the Automations directory, update the instance-id field on both the updatefrontendnew.sh updatebackendnew.sh with the EKS worker's node instance id

Next...

• Jenkins Master

• Jenkins Slave

• EKS

• SonarQube

• ArgoCD

• Trivy

[Note] This project will be implemented in the Singapore region (ap-southeast-1).

Setting up the Jenkins Master

• Spinup AWS EC2 instances with 2CPU, 8GB of RAM (t2.large), and 30 GB of storage, and install Docker on them.

• Open the ports below in the security group of the master machine, and also attach the same security group to the Jenkins worker node (We will create the worker node shortly)

• Install Java

sudo apt update
sudo apt install openjdk-21-jre-headless
java -version

openjdk 21.0.8 2025-07-15
OpenJDK Runtime Environment (build 21.0.8+9-Debian-1)
OpenJDK 64-Bit Server VM (build 21.0.8+9-Debian-1, mixed mode, sharing)

• Install and configure Jenkins (Master machine)

sudo wget -O /etc/yum.repos.d/jenkins.repo \
    https://pkg.jenkins.io/rpm-stable/jenkins.repo
sudo dnf upgrade

# Add required dependencies for the Jenkins package
sudo dnf install jenkins
sudo systemctl daemon-reload 

• Start Jenkins. You can enable the Jenkins service to

#start at boot with 
sudo systemctl enable jenkins 

#start the Jenkins service 
sudo systemctl start jenkins 

#check the status of the Jenkins service
sudo systemctl status Jenkins

Get your password from

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

• Now, access the Jenkins Master in your browser using the EC2 public IP on port 8080 and configure it.

• Install suggested plugins

Note - If we install suggested pugins then it may slows down Jenkins UI

• Create an admin user to manage Jenkins

• Create AWS IAM-->user-->Create user and then create Access Key

Ex.
Access ID: AKIAZDZTBOHGUD5KKPX3
Secret access key • uPBtoXTLltBRhxoMs8IdfUzQM42LAASsm9nzOgQa

• AWS CLI and configure on Jenkins Master (Setup AWSCLI)

"https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
    sudo apt install unzip
    unzip awscliv2.zip
    sudo ./aws/install
    aws configure

• Check AWS connectivity

aws sts get-caller-identity

• Install kubectl (Master machine) (Setup kubectl )

    curl -o kubectl https://amazon-eks.s3.us-west- 2.amazonaws.com/1.19.6/2021-01-05/bin/linux/amd64/kubectl
    chmod +x ./kubectl
    sudo mv ./kubectl /usr/local/bin
    kubectl version --short --client

• Install eksctl (Master machine) (Setup eksctl)

    curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
    sudo mv /tmp/eksctl /usr/local/bin
    eksctl version

• Create EKS Cluster (Master machine)

• Note It may take several minutes to create a new cluster

    eksctl create cluster --name=wandercluster \
    --region=ap-south-1 \
    --version=1.30 \
    --without-nodegroup

• Associate IAM OIDC Provider (Master machine)

    eksctl utils associate-iam-oidc-provider \
    --region ap-south-1 \
    --cluster wandercluster \
    --approve

• Create Nodegroup (On Jenkins Master)

    eksctl create nodegroup --cluster=wandercluster \
    --region=ap-southeast-1 \
    --name=wandercluster \
    --node-type=t2.large \
    --nodes=2 \
    --nodes-min=2 \
    --nodes-max=2 \
    --node-volume-size=29 \
    --ssh-access \
    --ssh-public-key=my-devops-key

Install and configure SonarQube (On the Master node)

docker run -itd --name SonarQube-Server -p 9000:9000 sonarqube:lts-community

• Access the Sonarqube server on: public-ip-of-jenkins-master:9000

Note- Initial username= admin and password= admin

Install and Configure ArgoCD (On the Jenkins Master)

• Create argocd namespace

kubectl create namespace argocd

• Apply the argocd manifest

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

• Make sure all pods are running in the argocd namespace

watch kubectl get pods -n argocd

• Install argocd CLI

sudo curl --silent --location -o /usr/local/bin/argocd https://github.com/argoproj/argo cd/releases/download/v2.4.7/argocd-linux-amd64

• Provide executable permission

sudo chmod +x /usr/local/bin/argocd

• Verify argocd

• Check argocd services

• Change the argocd server's service from ClusterIP to NodePort

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}'

Or you can manually edit the manifest file also by going inside it, using this command:

• Confirm service is patched or not

kubectl get svc -n argocd

• Check the port where the ArgoCD server is running and expose it on the security groups of a worker node

port: 31797

• Access it on the browser, click on advance, and proceed with public-ip-master-node:317979

Note: If you find an error like this in the above image, then run the command below:

kubectl port-forward svc/argocd-server -n argocd 31797:80 --address 0.0.0.0 &

• Page

• Fetch the initial password of the argocd server from Jenkins-ci-master

 kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

Username: admin •
Now, go to User Info and update your argocd password wnx0ZrUozW5oZeDq

Setting up Jenkins Slave node

• Create a new EC2 instance (Jenkins Slave) with 2CPU, 8GB of RAM (t2.large), and 30 GB of storage, and install Java on it

sudo apt update
sudo apt install openjdk-21-jre-headless
java -version

• Attach the IAM role with administrator access

Create an IAM role with administrator access, and attach it to the Jenkins Slave node. Select Jenkins slave EC2 instance --> Actions --> Security --> Modify IAM role

• Configure AWSCLI (Setup AWSCLI)

"https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
    sudo apt install unzip
    unzip awscliv2.zip
    sudo ./aws/install
    aws configure

• **Generate ssh keys (on the master) to set up the Jenkins master-slave relation
  **ssh-keygen

• On the Jenkins Slave node, move to the directory(.ssh) where your ssh keys are generated, and copy the content of the public key and paste to authorized_keys file.

• Now, go to the Jenkins master and navigate to Manage Jenkins --> Nodes, and click on New node

Name: Node • Type: permanent agent
Number of executors: 2
Remote root directory: /home/ubuntu
Labels: Node
Usage: Only build jobs with label expressions matching this node
Launch method: Via ssh
Host: public ip of Jenkins slave

Manage Jenkins---> Credentials: Add --> Kind: ssh username with private key --> ID: Worker --> Description: Worker --> Username: ubuntu --> Private key: Enter directly --> Add Private key

Host Key Verification Strategy: Non-Verifying Verification Strategy
Availability: Keep this agent online as much as possible

Note: the public IP should be the master’s instance-generated private, which is stored in id_ed25519.

• And your Jenkins slave node is added

If any error occurs, open the settings icon of the Node and make changes, then launch the agent again.

• Install Docker (Jenkins slave)

sudo apt install docker.io -y
sudo usermod -aG docker ubuntu && newgrp docker

Install Trivy (On the Jenkins slave)

sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update -y
sudo apt-get install trivy -y

Steps to add email notification

• If not yet, then -> Go to your Jenkins Master EC2 instance and allow for SMTPS

• Now, we need to generate an application password from our Gmail account to authenticate with Jenkins

• Open Gmail and go to Manage your Google Account --> Security

Note- [Important] Make sure 2-step verification is on

• Search for the App password and create an app password for Jenkins

Note- App password will be in this form "atqw gvjh ifrr wdkf" make sure when you enter in jenkins it should have no space between them like "atqwgvjhifrrwdkf"

• Once the app password is created, go back to Jenkins, Manage Jenkins --> Credentials to add username and password for email notification

• Go back to Manage Jenkins --> System and search for Extended E-mail Notification and set up email notification

• [Important] Enter your Gmail password, which we copied recently, in the password field E-mail Notification --> Advanced

• Verify if the test email was received (Jenkins to Gmail setup successful)

The next configuration is in Part 3

This blog is my space to share practical DevOps, Cloud, and Security insights — not just theory, but step-by-step guides, reusable templates, and lessons learned from real-world projects.

🌐 What You’ll Find Here

• Cloud Migration Playbooks — structured comparisons, decision matrices, and automation scripts.

• CI/CD Pipeline Optimization — best practices for traceable Docker tagging, robust error handling, and secure automation.

• Security & Compliance — IAM, PAM, vulnerability management, and embedding security into DevOps workflows.

• Portfolio Projects — curated case studies and technical walkthroughs that recruiters and peers can explore

🎯 My Goal

I want this blog to be:

• A knowledge hub for engineers tackling complex migrations and automation challenges.

• A portfolio showcase for my projects, highlighting both technical depth and professional polish.

• A community resource where ideas are accessible, actionable, and future-proof.

🚀 What’s Next

If you’re a recruiter, engineer, or tech enthusiast — welcome! This is where I share my journey and the systems I build.

• Why CI/CD pipelines are critical for modern cloud deployments.

• The importance of integrating security checks, automation, and monitoring.

• A quick overview of the tools used: Jenkins, GitHub, OWASP, SonarQube, Trivy, Docker, Argo CD, Prometheus, and Grafana.

🔹 Architecture Overview

• Developer → GitHub → Jenkins CI/CD → AWS EKS.

• CI pipeline handles code quality, security scans, and image builds.

• CD pipeline handles deployment automation and monitoring.

• Monitoring stack (Prometheus + Grafana) ensures visibility.

• Notifications via email keep stakeholders updated.

🔹 CI Pipeline Stages

1. Code Checkout → Jenkins pulls code from GitHub.

2. Dependency Scanning (OWASP) → Detects vulnerable libraries.

3. Code Quality (SonarQube) → Enforces coding standards and quality gates.

4. Security Scan (Trivy) → Scans filesystem and Docker images for vulnerabilities.

5. Docker Build & Push → Builds container images and pushes to Docker Hub.

6. Version Update → Updates Kubernetes manifests with new image tags.

7. Commit Back to GitHub → Ensures GitOps workflow consistency.

🔹 CD Pipeline Stages

1. Trigger from CI → Jenkins CD job starts.

2. Argo CD Deployment → Syncs manifests from GitHub to AWS EKS.

3. Kubernetes Execution → Deploys updated pods/services.

4. Monitoring:

   • Prometheus → Collects metrics.

   • Grafana → Visualizes dashboards.

5. Notifications → Email alerts on pipeline completion.

🔹 Tools & Integrations

• Jenkins → CI/CD automation.

• GitHub → Source control + GitOps repo.

• OWASP Dependency Check → Security scanning.

• SonarQube → Code quality analysis.

• Trivy → Vulnerability scanning.

• Docker Hub → Image registry.

• Argo CD → GitOps deployment.

• AWS EKS → Managed Kubernetes cluster.

• Prometheus & Grafana → Monitoring and visualization.

• Email (SMTP) → Notifications.

🔹 What have I done in This Series?

• Part 1: What is expected from this series?
  Why CI/CD matters, an overview of tools, and an architecture diagram.

• Part 2: Install Jenkins Master, Slave & Tools
  Jenkins jobs, GitHub webhooks, credentials.

• Part 3: Tools Integration with Jenkins
  Install plugins, set up credentials, and configure

• Part 4: CI Job for Security and Quality Gate
  CI Pipeline setup

• Part 5: Set up ArgoCD
  EKS and Application repo

• Part 6: Observability stack setup to monitor
  Setting up dashboards and alerts

Ansible is one of the most powerful automation tools used by DevOps engineers to manage infrastructure, configure systems, and deploy applications across hundreds or thousands of servers.

With its agentless architecture and simple YAML-based playbooks, Ansible makes infrastructure automation easy and scalable.

In this guide, we will cover the core concepts of Ansible, including:

• Ansible architecture

• Inventory

• Modules

• Playbooks

• Roles

• Variables

• Handlers

• Vault

• Real-world DevOps use cases

Table of Contents

1. What is Ansible

2. Why DevOps Engineers Use Ansible

3. Ansible Architecture

4. Core Components of Ansible

5. Ansible Inventory

6. Ansible Modules

7. Ansible Playbooks

8. Ansible Roles

9. Ansible Variables

10. Ansible Handlers

11. Ansible Vault

12. Real-World DevOps Use Case

13. Conclusion

What is Ansible?

Ansible is an open-source automation tool used for:

• Configuration management

• Infrastructure provisioning

• Application deployment

• Workflow orchestration

Unlike traditional configuration management tools, Ansible does not require agents on target machines. Instead, it communicates using SSH for Linux systems and WinRM for Windows systems.

This makes Ansible lightweight, secure, and easy to maintain.

Why DevOps Engineers Use Ansible

Ansible has become one of the most popular automation tools in DevOps due to its simplicity and flexibility.

Key Benefits

✔ Agentless architecture
✔ Easy YAML syntax
✔ Powerful automation modules
✔ Supports multi-cloud environments
✔ Scalable infrastructure management

Organizations use Ansible to automate:

• Server configuration

• Software installation

• Security hardening

• Application deployments

• Cloud infrastructure provisioning

Ansible Architecture

Ansible follows a simple control architecture.

Control Node

The control node is the machine where Ansible is installed. It executes playbooks and manages automation tasks.

Managed Nodes

Managed nodes are the servers that Ansible configures and manages.

Ansible connects to these nodes using SSH.

Architecture Overview

DevOps Engineer
      │
      ▼
Control Node (Ansible)
      │
 ┌────┴──────┐
 ▼           ▼
Web Server   Database Server

The control node sends instructions to managed nodes through Ansible playbooks and modules.

Core Components of Ansible

Understanding Ansible requires knowing its core components.

Key Components

• Inventory

• Modules

• Playbooks

• Roles

• Tasks

• Variables

• Handlers

• Templates

Each component plays an important role in automation workflows.

Ansible Inventory

The inventory defines the list of servers that Ansible manages.

It can be a simple text file that organizes hosts into groups.

Example Inventory

[webservers]
web1.example.com
web2.example.com

[dbservers]
db1.example.com

Inventories can be:

• Static inventory (manual server lists)

• Dynamic inventory (cloud-based infrastructure)

Dynamic inventories are commonly used with cloud providers such as AWS and Azure.

Ansible Modules

Modules are the building blocks of Ansible automation.

They perform specific tasks such as installing packages, managing services, or copying files.

Common Modules

• package

• apt

• yum

• service

• copy

• file

• command

• shell

Example Module Usage

- name: Install nginx
  apt:
    name: nginx
    state: present

Ansible includes thousands of modules for managing different systems and services.

Ansible Playbooks

Playbooks define the automation tasks executed by Ansible.

They are written in YAML format and describe the desired system state.

Example Playbook

- hosts: webservers
  become: yes

  tasks:
    - name: Install nginx
      apt:
        name: nginx
        state: present

This playbook installs Nginx on all web servers.

Running a Playbook

ansible-playbook install-nginx.yml

Playbooks are used for:

• Software deployment

• Server configuration

• Infrastructure setup

Ansible Roles

Roles allow you to organize automation code into reusable components.

Instead of writing large playbooks, you can break automation tasks into structured roles.

Role Directory Structure

roles/
 └ nginx/
     ├ tasks/
     ├ handlers/
     ├ templates/
     ├ vars/
     ├ defaults/

Roles improve:

• Code organization

• Reusability

• Scalability

Many roles are available through Ansible Galaxy.

Ansible Variables

Variables make playbooks dynamic and flexible.

They allow values to change based on the environment.

Example Variable

vars:
  package_name: nginx

Variables can be defined in:

• playbooks

• inventory

• group variables

• host variables

Ansible Handlers

Handlers are special tasks that run only when triggered by other tasks.

They are commonly used to restart services.

Example Handler

handlers:
  - name: restart nginx
    service:
      name: nginx
      state: restarted

Handlers run only when notified, which helps avoid unnecessary actions.

Ansible Vault

Ansible Vault is used to secure sensitive information such as passwords and API keys.

Encrypt a File

ansible-vault encrypt secrets.yml

Edit Encrypted File

ansible-vault edit secrets.yml

This ensures sensitive data is protected inside your automation workflows.

Real-World DevOps Use Case

Ansible is commonly used inside CI/CD pipelines to automate infrastructure configuration.

Example Workflow

Git Commit
     │
     ▼
CI/CD Pipeline
     │
     ▼
Ansible Playbook Execution
     │
     ▼
Application deployed to servers

For example:

• Terraform creates infrastructure

• Ansible configures servers

• Jenkins deploys applications

This approach is widely used in cloud-native DevOps environments.

Conclusion

Ansible is a powerful automation tool that simplifies infrastructure management and application deployment.

With its agentless architecture, YAML-based playbooks, and extensive module ecosystem, Ansible enables DevOps teams to automate complex operations efficiently.

By understanding key concepts like inventory, modules, playbooks, roles, and vault, engineers can build scalable and reliable automation pipelines.

Ansible remains one of the most important tools in the modern DevOps toolkit.