1. Initialization and Cloning:

• git init: Initializes a new Git repository in the current directory.

• git clone <repository>: Clones an existing Git repository to your local machine.

2. Basic Workflow:

• git add <file>: Adds a file or changes to the staging area.

• git commit -m "<message>": Commits the staged changes with a descriptive message.

• git status: Shows the current status of the repository.

• git log: Displays the commit history.

3. Branching and Merging:

• git branch: Lists all branches in the repository.

• git branch <branch-name>: Creates a new branch.

• git checkout <branch-name>: Switches to the specified branch.

• git merge <branch-name>: Merges changes from one branch into the current branch.

• git rebase <branch-name>: Incorporates changes from one branch onto another.

4. Remote Repository:

• git remote add <name> <url>: Adds a remote repository.

• git remote -v: Lists all remote repositories.

• git pull <remote> <branch>: Fetches changes from a remote repository and merges them into the current branch.

• git push <remote> <branch>: Pushes local commits to a remote repository.

5. Undoing Changes:

• git revert <commit>: Reverts a commit by creating a new commit that undoes the changes.

• git reset <commit>: Resets the repository to a specific commit, discarding subsequent commits.

• git checkout -- <file>: Discards changes made to a file and restores the last committed version.

• git stash: Temporarily saves changes that are not ready to be committed.

6. Tagging:

• git tag: Lists all tags in the repository.

• git tag <tag-name>: Creates a lightweight tag at the current commit.

• git tag -a <tag-name> -m "<message>": Creates an annotated tag with a message.

• git push --tags: Pushes all local tags to the remote repository.

7. Collaboration:

• git fetch: Downloads changes from a remote repository but does not merge them.

• git branch -r: Lists remote branches.

• git diff <commit1> <commit2>: Shows differences between two commits.

• git blame <file>: Shows who last modified each line of a file.

These are some of the commonly used Git commands at both basic and advanced levels. Git provides a wide range of commands to handle various scenarios and workflows. For more detailed information about each command and their options, you can refer to the Git documentation or use git --help or git <command> --help for specific command usage.

Thanks,

Anil Kushwaha

Create the first freestyle build-type job in the name of "Project1-build"

Create a second freestyle deploy job.

Next, we install the plugin "build pipeline" to create a simple pipeline.

Now after installing the plugins just restart and re-login Jenkins.

Next, we have to configure the "Project1 build"

When our first project i.e. "Project1 build" is once done then will execute the second project so we have to choose the "Build another project" option from "Add post-build action" now.

Now our pipeline is created and next to visualize the pipeline to perform the below steps.

Our pipeline is created under the parent & child concept so here have given the job name i.e. "project1-build.

Click on the "Run" button to visualize the jobs below.

The "Project1-build" job pulls the code from SCM and creates or generates a build and once the bill is generated then it will be triggered to the 2nd build job i.e. "Project1-test-deploy".

Now here you can see the first job is done and then 2nd job will be triggered automatically.

The Build Pipeline plugin allows you to create a graphical representation of your pipeline and easily track the progress of each stage. You can also add manual approval steps and integrate with other Jenkins plugins to enhance the functionality of your pipeline.

That's it, we created a simple CICD pipeline.

Thanks,

Anil Kushwaha

• Prometheus is an open-source monitoring and alerting toolkit, originally built at SoundCloud. It is designed for collecting, storing, and querying time-series data.

• Grafana is an open-source data visualization and monitoring platform. It allows you to create dashboards and panels to display your Prometheus metrics and other data sources.

Use Cases:

• Prometheus can be used to monitor various types of systems, including servers, databases, and applications. It can help you detect and diagnose issues before they become critical.

• Grafana can be used to visualize and explore your Prometheus metrics. You can create dashboards and panels to display data in real-time, track trends over time, and set up alerts based on specific thresholds or conditions.

Workflow:

1. Prometheus scrapes data from targets (servers, applications, databases, etc.) using various collectors.

2. The scraped data is stored in a time-series database.

3. Grafana connects to the Prometheus data source and allows you to create dashboards and panels to display your metrics.

4. You can use Grafana to set up alerts based on specific thresholds or conditions.

5. When an alert is triggered, you can use Grafana to investigate the issue further and take corrective actions as needed.

Overall, Prometheus and Grafana are powerful tools for monitoring and visualizing your systems' performance. They can help you identify and troubleshoot issues quickly, leading to improved reliability and performance.

For beginner guide.

1. Installation of Prometheus.

   -Prometheus public IP: 13.213.57.83

2. How to create and Install Node exporter.

   -Node exporter public: 13.213.57.83

   -Node exporter public: 13.213.37.30 (Remote machine)

3. Installing the Grafana.

   -Grafana dashboard public IP: 13.213.57.83

4. Setting up the Grafana Dashboard.

5. Setting up Prometheus and Grafana for Kubernetes on AWS (Advance Use-Case)

6. Advantages of Prometheus

7. Advantages of Grafana

8. Conclusion

1. Installation of Prometheus.

• Goto https://prometheus.io/download/ and select the prometheus-x.xx.x.linux-amd64.tar.gz file for download. (Note- Here x.xx.x is version number)

• Extract the download binary file. (Note- Replace x.xx.x with the downloaded version of Prometheus)

• root@prometheus:~# mkdir prometheus

• root@prometheus:~# cd prometheus

• root@prometheus:~/prometheus# wget https://github.com/prometheus/prometheus/releases/download/v2.43.0/prometheus-2.43.0.linux-amd64.tar.gz

root@prometheus:~/prometheus# tar -xvf prometheus-2.43.0.linux-amd64.tar.gz

root@prometheus:~/prometheus# cd prometheus-2.43.0.linux-amd64/

Start the prometheus server with the command and by default prometheus port is 9090.

#./prometheus

The above command has our Prometheus server up and running.

• The Prometheus server should start on port 9090.

• You can access the Prometheus graph UI by visint http://PublicIP.com:9090/graph

• You can access the Prometheus metrics UI by visint http://PublicIP.com:9090/metrics

2. How to create and Install Node exporter.

Node exporter is responsible for fetching the statistics from various hardware and virtual resources in the format which Prometheus can understand and with the help of the prometheus server those statistics can be exposed on port 9100.

2.1 How to Install Node exporter.

After installing the Prometheus in the previous step the next package we are going to install is Node Exporter in remote machine. Node exported is used for collecting various hardware and kernel-level metrics of your machine.

Here are the download and installation steps to remote machine -

1. Download the binary of Node exporter based on the operating system.(In my case I am using Ubuntu linux machine of prometheus server.)

#mkdir nodeexporter

#cd nodeexporter

#wget https://github.com/prometheus/node_exporter/releases/download/v1.5.0/node_exporter-1.5.0.linux-amd64.tar.gz

#tar -xvf node_exporter-1.5.0.linux-amd64.tar.gz

#cd node_exporter-1.5.0.linux-amd64

#./node_exporter

#cp -r node_exporter /usr/local/bin/

Create a node_exporter service file in the /etc/systemd/system directory.

#vim /etc/systemd/system/node_exporter.service

Paste the below content in your service file.

[Unit]

Description=Node Exporter

[Service]

User=root

Group=root

Type=simple

ExecStart=/usr/local/bin/node_exporter --web.listen-address=:9090

[Install]

WantedBy=multi-user.target

Set User permission and ownership for the node_exporter service file.

#chown -R root:root /etc/systemd/system/node_exporter.service 15

#chmod -R 644 /etc/systemd/system/node_exporter.service

Now let's start and enable the node_exporter service using the below commands

# sudo systemctl daemon-reload

# sudo systemctl start node_exporter

# sudo systemctl enable node_exporter

#sudo systemctl status node_exporter

Let's Check the Status of Node Exporter:

Node exporter is started and running by executing the above command and now it will collect metrics from the installation machine and push to the Prometheus server.

1. URL - http://publicIP:9100 and accesses node exporter metrics on a browser using port 9100 by default.

2.2 Adding node exporter scrap_configs to prometheus as a YAML Configuration.

In the previous node export installation, we had Prometheus and node exporter running on the same server. But if you want to add node exporter for any remote server then you need to define scrap_configs with target hosts inside the YAML configuration.

Here is an example of scrap_configs -

(\Note - In the below configuration you have replaced IP address 13.213.57.83 with your host IP or Hostname)*

global:

scrape_interval: 15s

scrape_configs:

- job_name: node

static_configs:

- targets: ['localhost:9100','IP:9100']

For the above, we go to Prometheus server and press ctrl+c and clear the screen.

1. As you can see I have added two hosts localhost, 13.213.57.83 inside the scrap_config.

2. Save the above configuration as YAML(ex. exporter-config.yaml)

3. Start the Prometheus server bypassing scrap_config with the help of --config.file flag.

Example -

#./prometheus --config.file=exporter-config.yml

#touch exporter-config.yml

#ls -larth

Here, it's needed to add remote machine IP with port 9100.

#vi exporter-config.yml

root@prometheus:~/prometheus/prometheus-2.43.0.linux-amd64# cat exporter-config.yml

global:

scrape_interval: 15s

scrape_configs:

- job_name: node

static_configs:

- targets: ['localhost:9100','13.213.57.83:9100']

root@prometheus:~/prometheus/prometheus-2.43.0.linux-amd64#

#./prometheus --config.file=exporter-config.yml

It will execute the below syntax to create an exporter config file in Prometheus server.

Verify the Host status-

After adding the remote host into the node exporter configuration verify it by accessing the Prometheus target URL 13.213.57.83:9090/targets

http://13.213.57.83:9090/

Next, we need to log in Prometheus server and add remote machine IP in the target field.

#pwd

/prometheus/prometheus-2.43.0.linux-amd64

# ls

LICENSE NOTICE console_libraries consoles data exporter-config.yml prometheus prometheus.yml prometheus.yml-backup promtool

Configure the Node Exporter as a Prometheus target.

Go to the end of the prometheus.yml file and add this entry at the bottom of the line and then restart the Prometheus server.

#vim prometheus.yml

static_configs:

- targets: ["Remote_machine_IP:9090"]

After changing in the config file you need to restart to Prometheus

Now restart the Prometheus Service

#sudo systemctl restart prometheus

or

#./prometheus --config.file=exporter-config.yml

Logged in to Remote machine to install node exporter:-

To install the node exporter on the remote machine will use the number of the above-mentioned steps "2.1 How to Install Node exporter".

http://13.213.37.30:9100/metrics

Node exporter is installed & configured on the remote machine now and able to see its metrics.

Next, we'll go to the Prometheus server (13.213.57.83) and press ctrl+C to stop the Prometheus server and to collect remote machine metrics add our remote machine public IP (13.213.37.30) in exporter-config.yml file.

Logged in to Prometheus server:-

Prometheus using the syntax, restart the server and on Prometheus login and check that all instances are collecting metrics from the UP server.

#./prometheus --config.file=exporter-config.yml

http://13.213.57.83:9090/targets?search=&scrapePool=node

2.3 Enable and disable the collector

If you have more than one collector configured then node_exporter allows you to enable and disable some of the collectors with run time arguments -

Here is an example command for starting the node_exporter with disable flag -

#node_exporter --no-collector.ENTER-THE-NAME-OF-COLLECTOR

For enabling the collector use the following command -

#node_exporter --collector.ENTER-THE-NAME-OF-COLLECTOR

Now we will install and configure the same steps for node exporter in our another new remote machine to collect data from remote machine (13.213.37.30) and push data to prometheus server.

3. Installing the Grafana

After installing the Prometheus server and node exporter let's install the Grafana for setting up visualization using Grafana dashboard. Here is the official download link of Grafana https://grafana.com/docs/grafana/latest/setup-grafana/installation/, choose the correct Grafana binary based on your operating system. For this blog post, I am going to use ubuntu Linux binary.

Steps for Installing Grafana-

In this installation, we are going to install the latest enterprise edition Grafana on the same prometheus server.

1. Update the package info -

   #sudo apt-get install -y apt-transport-https

   #sudo apt-get install -y software-properties-common wget

   #wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -

Add stable repository of Grafana -

1. #echo "deb https://packages.grafana.com/enterprise/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

2. Update repository and Install Grafana

   #sudo apt-get update

   #sudo apt-get install grafana-enterprise

3. Start the Grafana Server -

   #sudo systemctl daemon-reload

   #sudo systemctl start grafana-server

   #sudo systemctl status grafana-server

4. Configure Grafana to run at boot time-

   #sudo systemctl enable grafana-server.service

   

5. Access the Grafana Dashboard using http://13.213.57.83:3000/login

6. The default login Username: Password for accessing the Grafana dashboard is admin: admin

7. Change the default admin password after the login.

   

4. Setting up the Grafana Dashboard

The next step after installing the Grafana Server would be to set up the Grafana Dashboard. Till now, we have installed the Prometheus Server, Node Exporter as well as Grafana Server but there is a connection between Prometheus and Grafana.

4.1 Let's create a Prometheus Datasource inside Grafana Dashboard

1. Goto your Grafana Dashboard and click on Gear(Settings Icon)->Data Sources

Grafana Dashboard create data source

1. Click on Add Data Source

Grafana Add Data source

1. Select the Prometheus as preferred data source -

Prometheus as preferred data source

1. Enter the hostname or IP address of the prometheus server

Enter the hostname or ip address of prometheus server e.g. http://localhost:9090

1. Save and test data source

Save and test prometheus datasource

4.2 Import Grafana Dashboard from Grafana Labs

Now after settings the data source we can import pre-existing opensource dashboard from Grafana Labs using the Dashboard ID.

1. Goto Grafana Dashboard(https://grafana.com/grafana/dashboards/)and search for Node Exporter Full

Grafana lab dashboard 1860 Node Exporter Full

1. Click on the result and then copy the board ID .e.g 1860 or 9894.

Or.

Grafana lab dashboard 1860 Node Exporter Full

1. Goto Import and enter the board ID .e.g. 1860

Grafana dashboard import

Import Grafana dashboard with dashboard id 1860.

1. Click load and then select the data source as Prometheus

Select prometheus as preferred datasource

1. Finally Click on Import

2. You should be able to see the memory dashboard

   http://13.213.57.83:3000

Grafana dashboard

5. Advantages of Prometheus

Now I have talked about how to install and use Prometheus along with Grafana, let's see what are the advantages of using Prometheus in the landscape of DevOps and SREs -

1. Prometheus is a very independent framework that does not impact your existing cloud or in-premise infrastructure.

2. It can be customized to suit your infrastructure needs

3. To use Prometheus and Node exporter a developer does not have to install all the node exporters available in the market. A Developer can only install the required node exporter and to avoid extra overhead on the hardware resources

4. Prometheus node exporter can be customized to suit your infrastructure need, if you do not find a suitable node exporter in the library then Prometheus allows you to build a custom node exporter.

5. Prometheus comes along with PromQL and TSDB which make it a more powerful tool for debugging troubleshooting.

6. Prometheus has a built in database and you do not have to install the database separately.

7. The pull mechanism of Prometheus is very flexible, and it allows you to schedule variable scrappers at different intervals.

8. Also, Prometheus is being open-source, it has a wide variety of community support available for developers.

6. Advantages of Grafana

1. If you compare Grafana with traditional tools Zabbix and LibreNMS then Grafana allows you to create multiple datasource whereas Zabbix and LibreNMS allow you only to create a single Datasource.

2. In the Grafana dashboard you can customize various panels based on your requirement, also you can have integration with multiple data sources in the same Dashboard which is by far the biggest merit of using Grafana.

3. In Grafana open-source community you can share your dashboard settings with others, and similarly, you can use other's dashboards by searching it on Grafana Community, so you do not have to re-invent the wheel again.

4. Grafana dashboard's user experience is very pleasing and also it allows a developer to put PNG images on the dashboard to make it more appealing.

5. Grafana developer community is building a lot of features and various datasource support which makes Grafana more adaptable for various infrastructure monitoring needs.

7. Conclusion

No doubt Grafana and Prometheus are a really popular frameworks for monitoring, alerting, and data visualization although there is a bit of learning because Prometheus has custom query language PromQL which requires little of practice before you call yourself an expert in the PromQL.

But when you look from DevOps and SREs end you would always like to have Prometheus and Grafana in your arsenal because of various third-party Node exporters, setting up various data sources, customizable dashboards, and panels. I hope this beginners guide will help you to get you started with Prometheus, Grafana, and Node exporter setup, and if you are interested in some advance setup of Prometheus and Grafana then I would recommend this guide on - How to setup Prometheus, Grafana dashboard for Kubernetes monitoring on AWS

All done.

Thanks,

Anil Kushwaha

1. Install Java: Jenkins is a Java-based application, so you need to install Java on your Ubuntu system. You can install Java using the following command:

   #sudo apt update

   #sudo apt install openjdk-11-jre

   #sudo java -version

   

2. Install Jenkins: You can install Jenkins using the following commands:

   This is the Debian package repository of Jenkins to automate installation and upgrade. To use this repository, first add the key to your system:

   #curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \ /usr/share/keyrings/jenkins-keyring.asc > /dev/null

   #echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \

   https://pkg.jenkins.io/debian-stable binary/ | sudo tee \

   /etc/apt/sources.list.d/jenkins.list > /dev/null

   #sudo apt-get update

   #sudo apt-get install jenkins

   

   #sudo systemctl enable jenkins --now

   #sudo systemctl status jenkins

3. Access Jenkins: Jenkins runs on port 8080 by default, so you can access it using your server's IP address and port 8080, before accessing, 8080 port needs to allow in AWS Inbound rule of a security group. (e.g. http://your-server-ip-address:8080/). You will be prompted to enter an initial admin password which can be found in the file /var/lib/jenkins/secrets/initialAdminPassword.

   http://13.235.81.85:8080

   

   

4. Install Plugins: After logging in to Jenkins, you need to install the required plugins. You can install plugins by clicking on "Manage Jenkins" > "Manage Plugins" > "Available" and then searching for and selecting the desired plugins.

   Follow the screenshots on the setup page to complete the installation and set up your Jenkins instance.

   

   

   

   

   

   Or

   You can download the latest version of Jenkins from the official website (https://www.jenkins.io/download/) and follow the installation instructions.

   That's it! You have successfully installed Jenkins on your AWS Ubuntu instance.