Introduction:

DevOps is like a team sport for making software better and faster. It's all about teamwork, using cool tools, and working together to make awesome apps and games quickly and with fewer mistakes. In short, DevOps is like teamwork between software developers (who write code) and operations teams (who manage infrastructure). It's a way of working that helps teams build and deliver software more quickly and smoothly. Before starting this project it will be better for us to understand a little some about related terminology and their history.

Before DevOps: ( The Evolution of Application Development )

In the past, developing and delivering software was slow. Teams worked separately, causing delays and communication issues. Everything was done manually, which led to mistakes. Improvements were rare because teams didn't collaborate much, and feedback took a long time to make any changes. This process followed a predefined approach known as SDLC or Software Development Life Cycle.

DevOps is also a part of the Software Development Life Cycle (SDLC). It focuses on optimizing and improving the processes within the SDLC, especially in terms of collaboration, automation, and continuous delivery. DevOps practices are integrated into different stages of the SDLC to enhance efficiency, quality, and speed of software development, ensuring a more streamlined and effective development process overall.

Before DevOps, several software development models were commonly used within the broader Software Development Life Cycle (SDLC).:

1. Waterfall Model: A linear and sequential approach where each phase (requirements, design, implementation, testing, deployment) is completed before moving to the next. Progression flows in one direction, making it challenging to accommodate changes once a phase is finished.

   💡

   Once the water starts flowing over the edge of the cliff, it starts falling down the mountain and the water cannot go back up.

   

2. Spiral Model: An iterative model combining elements of both waterfall and prototyping models. The Spiral Model is a crucial part of the Software Development Life Cycle (SDLC), focusing on managing risks. It resembles a spiral with undefined loops, each loop being a development phase. It involves a series of cycles, each encompassing risk analysis, planning, engineering, and evaluation. It allows for flexibility in accommodating changes during the development process.

   

3. Agile Model: Agile methodologies, such as Scrum or Kanban, emphasize iterative and incremental development. They focus on adaptability, collaboration, and responding to change throughout the development process, allowing for quicker releases and customer feedback incorporation.

   In more simple definition the Agile Model is like building something piece by piece, quickly and flexibly. It focuses on teamwork, adapting to changes, and delivering small parts of the project often. Unlike the Waterfall or Spiral Models, Agile doesn't wait until the end to show progress; it's all about quick adjustments and regular updates.

4. V-Model (Verification and Validation Model): Similar to the waterfall model but with a focus on validation and verification at each stage. It pairs each development stage with a testing phase, emphasizing testing and validation alongside development activities.

5. Rapid Application Development (RAD): Emphasizes rapid prototyping and iterative development. It involves close interaction between developers and end-users to quickly build and refine software based on continuous feedback.

How to choose : ( Different ways to make software )

1. Step-by-Step Recipe (Waterfall): Good when you know exactly what you want, like following steps in order.

2. Adjustable Building Blocks (Agile): Like playing with Legos, building bit by bit and changing as needed.

3. Risk Management Circles (Spiral): Handling risks in loops, especially for uncertain or risky projects.

4. Prototypes (Prototype): Make a sample first, refining it based on feedback before building the whole thing.

5. Little by Little (Incremental): Working on small parts, adding and improving gradually.

6. Multiple Cycles (Iterative): Repeating steps to improve with each cycle.

7. All-at-Once (Big Bang): Building everything together, best for smaller or vague projects.

Each way suits different projects depending on how clear the plan is and how much it might change.

DevOps Engineer: (The Role and Responsibilities)

A DevOps engineer is like a bridge builder between software development and IT operations. They work to make sure that the processes for building, testing, and delivering software are smooth and efficient. Their job involves automating tasks, managing tools, and improving collaboration among different teams to speed up software delivery while maintaining quality and reliability. They focus on making sure that the software development and deployment run smoothly and that everyone involved works together effectively.

They rely on a range of tools to accomplish these tasks, including Jenkins Continuous Integration/Continuous Deployment (CI/CD) tools, version control systems like Git, configuration management tools such as Ansible or Puppet, containerization tools like Docker or Kubernetes, monitoring and logging solutions like Prometheus or grafana, collaboration platforms like Slack or Jira, cloud services from providers such as AWS or Azure, as well as testing tools like Selenium or JUnit. This expansion includes a mention of various tools commonly used by DevOps engineers to support their responsibilities in software development and deployment.

Key Tools Utilized in this Project

Certainly!

1. Linux: An open-source operating system used widely due to its stability, security, and flexibility. It's prevalent in server environments and for software development.

2. Git: A version control system that tracks changes in code, enabling collaboration among developers, and managing codebase versions.

3. GitHub: A platform built around Git, providing hosting for software development and collaboration using Git repositories.

4. Docker: Allows the creation, deployment, and running of applications in containers, ensuring consistency across various environments.

5. Kubernetes: An orchestration tool for managing containerized applications, automating deployment, scaling, and management.

6. DockerHub: A cloud-based repository where Docker users store and share container images.

7. Jenkins: An automation server used for Continuous Integration and Continuous Deployment (CI/CD) to automate the software development process.

8. AWS (Amazon Web Services): A comprehensive cloud computing platform offering various services like computing power, storage, and databases.

9. MobaXterm: A terminal tool with many features like remote connectivity, X11 server, and tabbed SSH client, enhancing remote working capabilities.

Progressing Methodically: ( Our Step-by-Step Approach in this Project )

Throughout this comprehensive DevOps project journey, we will progress through the following steps methodically, one by one.

1. Set Up GitHub Repository: Create a GitHub account, establish a code repository, and push the code to it.

2. Provision Jenkins Server: Set up and configure the Jenkins server for continuous integration and deployment.

3. Deploy Kubernetes Cluster: Create and configure a Kubernetes cluster for application deployment and management.

4. Build Jenkins CI/CD Pipeline: Develop a comprehensive CI/CD pipeline using Jenkins for seamless code integration and deployment.

5. Deploy WP-MySQL Galera Multinode: Implement a robust Galera-based MySQL setup ensuring database security and failover, monitored using Grafana.

6. Configure AWS Load Balancer: Set up an AWS Application Load Balancer to efficiently distribute traffic between application instances.

7. Domain Name Configuration: Add a domain name and configure access to the application through it for user-friendly access.

8. Code Verification: Modify the code and verify the changes to ensure proper functionality.

9. Application Scaling: Scale the application up and down to handle variable loads efficiently.

10. Rolling Updates & Rollbacks: Perform rolling updates for seamless application upgrades and have a rollback plan in case of issues.

11. Backup and Restore Strategy: Implement a backup and restoration process to ensure data security and reliability.

Step-1: (Set Up GitHub Repository)

Go to the GitHub account login page and log in with your GitHub account authentication. Navigate to new and create a new repository. Assign a Repository name and Description (optional). Choose between the account options: public repositories, visible to the world, and private repositories, requiring payment and authentication for access.

Check to Add a README file That helps write the description for your project. Nevigate to <> Code Local and copy this HTTPS code, actually this link is crucial for pushing new code to your GitHub account repository from git. In my case my repository name is grras and HTTPS link is https://github.com/rakesh08061994/grras.git

Further, we will set webhook also for integration with Jenkins for auto build facility in a CI/CD pipeline.

When you navigate to Add webhook option, you will see a page that needs some configuration to enter manually.

Step-2: (Provision Jenkins Server)

Either you can go threw this official Jenkins link to configure Jenkins on Linux or you can set up your own. For this, we have created a Rhel Linux instance and got SSH through MobaXterm on Windows localhost for further configuration and setup. I am using Rhel Linux instance with 4 GB RAM & 2 Core CPU configuration. So let's start.

• Follow link https://www.jenkins.io/doc/book/installing/linux/#red-hat-centos

  But first change hostname for jenkins machine

# hostnamectl set-hostname jenkins
# exec bash

• Open the 8080 port on the machine security firewall. You can see 8080 port is listened to by JAVA.

# firewall-cmd --permanent --add-port=8080/tcp
# firewall-cmd --reload
# firewall-cmd --list-all
# # netstat -tulpn | grep 8080
tcp6       0      0 :::8080                 :::*                    LISTEN      3648/java

• Import a Jenkins repository under /etc/yum.repos.d and key from the Jenkins server repository which helps to authenticate that Jenkins should be downloaded or installed from an authenticated source.

# wget -O /etc/yum.repos.d/jenkins.repo \
    https://pkg.jenkins.io/redhat-stable/jenkins.repo
# rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key

• Upgrade the system

# sudo yum upgrade

• Install pre-required Jenkins dependencies packages and java sdk, which ismust for Jenkins. Because Jenkins is developed by Oracle and created in java.

# yum install fontconfig java-17-openjdk

• Install Jenkins in the system

# yum install jenkins

• Start and enable Jenkins service.

•       # systemctl enable --now jenkins
        # systemctl status jenkins
  
        ● jenkins.service - Jenkins Continuous Integration Server
             Loaded: loaded (/usr/lib/systemd/system/jenkins.service; enabled; vendor preset: disabled)
             Active: active (running) since Tue 2023-12-12 12:07:18 IST; 35min ago
           Main PID: 3648 (java)
              Tasks: 39 (limit: 10756)
             Memory: 359.2M
                CPU: 1min 25.363s
             CGroup: /system.slice/jenkins.service
                     └─3648 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/java/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
        Dec 12 12:07:20 jenkins jenkins[3648]: 2023-12-12 06:37:20.235+0000 [id=49]        INFO        hudson.util.Retrier#start: Performed the action check updates server>
        ...
  

#  echo "192.168.199.139 www.jenkins.com" >> /etc/hosts

• Now go to the web browser and paste exactly this.

http://www.jenkins.com:8080

• Unlock Jenkins through Jenkins secret password, which is saved on the location Jenkins home directory "/var/lib/jenkins/secrets/initialAdminPassword" copy it and paste into the browser input bar.

• Click on Default Install Suggested plugins option, and wait to complete the installation plugins packages.

# cat  /var/lib/jenkins/secrets/initialAdminPassword
557545ee5a8045858f61a3b3ab0ef686

• http://www.jenkins.com:8080/ is your web address to access Jenkins through your local area network. Click Save and Finish.

• Congratulations! Your Jenkins is ready to work.

--------> jenkins user is an application user in os check UID:977
# cat /etc/passwd | grep "jenkins"
jenkins:x:977:977:Jenkins Automation Server:/var/lib/jenkins:/bin/false

---------> These following files come with this Jenkins package.
# rpm -ql jenkins
/usr/bin/jenkins
/usr/lib/systemd/system/jenkins.service
/usr/share/java/jenkins.war
/usr/share/jenkins/migrate
/var/cache/jenkins
/var/lib/jenkins

--------> Find all files related to user jenkins and list 
# find / -name jenkins -type d -exec ls -ld {} \;
# find / -name jenkins -type f -exec ls -al {} \;

-------> Check jenkins related files & data
# ls /var/lib/jenkins
config.xml                      jenkins.install.InstallUtil.installingPlugins   jenkins.telemetry.Correlator.xml  plugins                   updates
hudson.model.UpdateCenter.xml   jenkins.install.InstallUtil.lastExecVersion     jobs                              secret.key                userContent
hudson.plugins.git.GitTool.xml  jenkins.install.UpgradeWizard.state             nodeMonitors.xml                  secret.key.not-so-secret  users
identity.key.enc                jenkins.model.JenkinsLocationConfiguration.xml  nodes                             secrets

NOTE: If you want to change Jenkins's default configuration service then you can modify it with the below command. Follow this official link: https://www.jenkins.io/doc/book/installing/initial-settings/

# YOURPORT=8080
# PERM="--permanent"
# SERV="$PERM --service=jenkins"

# firewall-cmd $PERM --new-service=jenkins
# firewall-cmd $SERV --set-short="Jenkins ports"
# firewall-cmd $SERV --set-description="Jenkins port exceptions"
# firewall-cmd $SERV --add-port=$YOURPORT/tcp
# firewall-cmd $PERM --add-service=jenkins
# firewall-cmd --zone=public --add-service=http --permanent
# firewall-cmd --reload

Step-3: (Set Up Kubernetes Cluster Deployment Environment**)**

• If you are using Cloud Machines like AWS, go with this article: https://rakeshkumarjangid.hashnode.dev/how-to-setup-kubernetes-cluster-over-cloud-on-ubuntu-os

• Firewall and SELinux should be disabled

• swap memory should be removed and off

Result of Kubernetes two nodes cluster:

Lots of Congratulations...

Step-4: (Build Jenkins CI/CD Pipeline)

Your cluster is ready, your Jenkins server is running properly. you have created a GitHub repository account and you are ready to push the code so now we are ready to create a Jenkins pipeline for the application.

Step4.1: (Push Code on GitHub & add GitHub Jenkins server address on webhook)

• A developer writes code using the Windows software application VS Code and pushes it using Git.

• Open or edit code through VS CODE and push code through git.

• Add config global information to tell who is actually working and push the code on GitHub repository server.

$ git config --global user.email "jangidrakesh71@gmail.com"
$ git config --global user.name "Rakesh"

$ git config user.name
Rakesh

$ git config user.email
jangidrakesh71@gmail.com

• Create and move the all data at once using git checkout -b <new_branch>

$ git checkout -b stage
Switched to a new branch 'stage'

• Add GitHub remote repository and assign with an alias name origin .

$ git status 
$ git remote add origin https://github.com/rakesh08061994/grras.git

• Check whether remote repository are added or not

$ git remote -v
origin  https://github.com/rakesh08061994/grras.git (fetch)
origin  https://github.com/rakesh08061994/grras.git (push)

💡

We are creating a password less public-key based GitHub authentication

$ ssh-keygen.exe
$ cat /c/Users/Rakesh-PC/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwOtiIb9Yctu2MJgnS9Avol93dTYGlgPPCaRxTCxz+xxlOb+sZJsz29+4GFkvcZZDbhsmjKRE4ByqlKXBIM+Jv1LlgL8i2hsNFUqdij6Y41kFzbSQIVINNDnQHBOxjMUjz7HRkqteLQQRHRF3roG61sv1kFWC/Xza0DmsHeE6TrRhl+Kj2OEDT7AZ3oQuTp2yGgVTrAk/wOdQnxjvK0eY3EewlVhbJStkCZ6Fac7q05oBh24eY6egf5othVNvUeRq0W5OureB7dyRvobB98rHcmwaLxnzOwZZVH/YHHIgM1PAxCfW3iam+EI9Sm0C++wBZA6/51zzjjekzZaf3Ir04kZe7qZXQoS9hfTDgaW0tanDZvGOl3GdUH9XxnRur5MCNmivcPv8IjlsIleoGM7vLDLXBMTgIEFAZ/++b2MBww59gGQZ6+oCfNgIXBaoGl3FWapbKKx9mbEI1kDxEs11qHvUxTw4kQInacEO6abxvJeI5XFXktvUGokKhBW1P8q8= Rakesh-PC@RakeshPC

• We will paste this public key to the GitHub for public key-based authentication.

• Go to https://github.com/settings/keys and click on "New ssh key" and paste your public key here.

Your attached key should reflected like this output, otherwise check your public key and steps carefully.

Now open your git on windows, and type below command to check whether we are ready to work with GitHub passwordless.

• Ensure SSH agent is running on your windows machine

$ eval "$(ssh-agent -s)"
Agent pid 1217

• Add the private key to the SSH agent and check the connection with GitHub.

$  ssh-add ~/.ssh/id_rsa
$ ssh -T git@github.com

• Commit the codes with a message, and push them to stage branch on grras GitHub repository. (This time github will not ask the username and password)

$ git status
$ git commit -m "This is my first commit"
$ git push origin stage

Output should be

• Check the data in GitHub repository stage branch .

• Now add Jenkins to the GitHub webhook for trigger option.

Jenkins-PublicIP/github-webhook/

---

Step-5: (Deploy WP-MySQL Galera Multinode)

Step-6: (Configure AWS Load Balancer)

Step-7: (Domain Name Configuration)

Step-8: (Code Verification)

Step-9: (Application Scaling)

Step-10: (Rolling Updates & Rollbacks)

Step-11: (Backup and Restore Strategy)

Reference:

• https://www.makeuseof.com/using-ssh-for-github-passwordless-authentication/