Chapter 12 — Hierarchical Clustering
1. Definition
Hierarchical Clustering is an Unsupervised Machine Learning Algorithm that groups similar data into clusters by creating a hierarchical tree structure called a Dendrogram.
Unlike K-Means, Hierarchical Clustering does not require us to specify the number of clusters (K) before training. It first builds a hierarchy of clusters, and later we can decide where to cut the tree to form the final clusters.
In simple words:
Hierarchical Clustering groups similar data step by step and organizes it like a family tree.
Example
Suppose a company has customer data.
| Customer | Age | Monthly Income |
|---|---|---|
| C1 | 22 | 25000 |
| C2 | 24 | 27000 |
| C3 | 45 | 70000 |
| C4 | 47 | 72000 |
| C5 | 50 | 76000 |
Initially, every customer is considered as a separate cluster.
C1 C2 C3 C4 C5
The algorithm finds the two most similar customers and merges them.
(C1,C2) C3 C4 C5
Then it continues merging the closest clusters until all customers become part of one large hierarchy.
All Customers
/ \
Cluster A Cluster B
/ \ / \
C1,C2 C3 C4,C5
This hierarchical structure is called a Dendrogram.
2. Purpose
The main purpose of Hierarchical Clustering is to discover natural groups in data while showing the relationship between those groups.
Unlike K-Means, it allows us to visualize how clusters are formed at different levels.
Businesses use Hierarchical Clustering for:
Customer Segmentation
Product Categorization
Gene Analysis
Medical Research
Document Grouping
Image Classification
Market Analysis
It is especially useful when we do not know the ideal number of clusters beforehand.
3. When to Use
Use Hierarchical Clustering when:
There is no target column.
You do not know how many clusters should exist.
You want to visualize relationships between groups.
The dataset is small or medium-sized.
You want a hierarchical view of the data.
Real-Life Applications
Customer Segmentation
Product Grouping
Biological Classification
DNA Analysis
Employee Grouping
Medical Diagnosis
Research Analysis
Market Segmentation
4. Complete Python Code (Using Pandas DataFrame)
import pandas as pd
from sklearn.cluster import AgglomerativeClustering
# ---------------------------------
# Create Dataset
# ---------------------------------
data = {
"Age": [
22,24,23,25,27,
45,46,48,50,52,
30,31,33,35,36,
55,57,58,60,62
],
"MonthlyIncome": [
25000,27000,26000,28000,30000,
70000,72000,73000,76000,78000,
42000,44000,45000,47000,49000,
85000,87000,89000,91000,93000
]
}
df = pd.DataFrame(data)
print("="*50)
print("DATASET")
print("="*50)
print(df)
# ---------------------------------
# Select Features
# ---------------------------------
X = df[
[
"Age",
"MonthlyIncome"
]
]
# ---------------------------------
# Create Model
# ---------------------------------
model = AgglomerativeClustering(
n_clusters=3
)
# ---------------------------------
# Train Model
# ---------------------------------
clusters = model.fit_predict(X)
# ---------------------------------
# Add Cluster Column
# ---------------------------------
df["Cluster"] = clusters
print("\n")
print("="*50)
print("CUSTOMERS WITH CLUSTERS")
print("="*50)
print(df)
# ---------------------------------
# Show Each Cluster
# ---------------------------------
print("\n")
print("="*50)
print("CLUSTER 0")
print("="*50)
print(df[df["Cluster"] == 0])
print("\n")
print("="*50)
print("CLUSTER 1")
print("="*50)
print(df[df["Cluster"] == 1])
print("\n")
print("="*50)
print("CLUSTER 2")
print("="*50)
print(df[df["Cluster"] == 2])
Sample Output
CUSTOMERS WITH CLUSTERS
Age MonthlyIncome Cluster
0 22 25000 2
1 24 27000 2
2 23 26000 2
3 25 28000 2
4 27 30000 2
5 45 70000 1
6 46 72000 1
7 48 73000 1
8 50 76000 1
9 52 78000 1
10 30 42000 0
11 31 44000 0
12 33 45000 0
13 35 47000 0
14 36 49000 0
Cluster 0
Medium Income Customers
Cluster 1
High Income Customers
Cluster 2
Low Income Customers
Summary
| Section | Description |
|---|---|
| Definition | Hierarchical Clustering is an Unsupervised Learning algorithm that groups similar data by building a hierarchical tree (Dendrogram). |
| Purpose | To discover natural groups and show the relationship between them without requiring the number of clusters in advance. |
| When to Use | Use when there is no target column, the number of clusters is unknown, and you want to visualize how clusters are formed. |
| Code Example | Uses a Pandas DataFrame, AgglomerativeClustering, cluster assignment, and displays each cluster separately. |
Chapter 13 — DBSCAN (Density-Based Spatial Clustering of Applications with Noise)
1. Definition
DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is an Unsupervised Machine Learning Algorithm that groups data points based on density.
Unlike K-Means and Hierarchical Clustering, DBSCAN does not require you to specify the number of clusters before training.
It can also identify outliers (noise), which are data points that do not belong to any cluster.
In simple words:
DBSCAN creates clusters by finding areas where many data points are close together and labels isolated points as noise (outliers).
Example
Suppose a shopping company has customer data.
| Customer | Age | Monthly Income |
|---|---|---|
| C1 | 22 | 25000 |
| C2 | 23 | 25500 |
| C3 | 24 | 26000 |
| C4 | 45 | 70000 |
| C5 | 46 | 71000 |
| C6 | 90 | 300000 |
Customers C1, C2, and C3 are very close to each other.
Customers C4 and C5 are also close.
Customer C6 is far away from everyone else.
DBSCAN may produce:
Cluster 0
C1
C2
C3
Cluster 1
C4
C5
Noise
C6
Notice that customer C6 is not forced into any cluster.
2. Purpose
The main purpose of DBSCAN is to:
Find clusters based on data density.
Detect outliers (noise).
Handle irregularly shaped clusters.
Work without knowing the number of clusters beforehand.
Businesses use DBSCAN for:
Fraud Detection
Customer Segmentation
GPS Location Analysis
Network Intrusion Detection
Earthquake Detection
Image Processing
Medical Data Analysis
One of its biggest advantages is that it automatically identifies unusual records, which many other clustering algorithms cannot do.
3. When to Use
Use DBSCAN when:
There is no target column.
The number of clusters is unknown.
Your data contains outliers.
Clusters may have irregular shapes.
You want to detect abnormal records.
Real-Life Applications
Credit Card Fraud Detection
Banking Transaction Analysis
Customer Segmentation
GPS Tracking
Crime Hotspot Detection
Disease Outbreak Detection
Sensor Data Analysis
Network Security
4. Complete Python Code (Using Pandas DataFrame)
import pandas as pd
from sklearn.cluster import DBSCAN
# ---------------------------------
# Create Dataset
# ---------------------------------
data = {
"Age": [
22,23,24,25,26,
45,46,47,48,49,
30,31,32,33,34,
90
],
"MonthlyIncome": [
25000,25500,26000,27000,28000,
70000,71000,72000,73000,74000,
42000,43000,44000,45000,46000,
300000
]
}
df = pd.DataFrame(data)
print("="*50)
print("DATASET")
print("="*50)
print(df)
# ---------------------------------
# Select Features
# ---------------------------------
X = df[
[
"Age",
"MonthlyIncome"
]
]
# ---------------------------------
# Create Model
# ---------------------------------
model = DBSCAN(
eps=5000,
min_samples=2
)
# ---------------------------------
# Train Model
# ---------------------------------
clusters = model.fit_predict(X)
# ---------------------------------
# Add Cluster Column
# ---------------------------------
df["Cluster"] = clusters
print("\n")
print("="*50)
print("CUSTOMERS WITH CLUSTERS")
print("="*50)
print(df)
# ---------------------------------
# Display Noise Points
# ---------------------------------
print("\n")
print("="*50)
print("NOISE (OUTLIERS)")
print("="*50)
print(df[df["Cluster"] == -1])
# ---------------------------------
# Display Cluster 0
# ---------------------------------
print("\n")
print("="*50)
print("CLUSTER 0")
print("="*50)
print(df[df["Cluster"] == 0])
# ---------------------------------
# Display Cluster 1
# ---------------------------------
print("\n")
print("="*50)
print("CLUSTER 1")
print("="*50)
print(df[df["Cluster"] == 1])
# ---------------------------------
# Display Cluster 2
# ---------------------------------
print("\n")
print("="*50)
print("CLUSTER 2")
print("="*50)
print(df[df["Cluster"] == 2])
Sample Output
CUSTOMERS WITH CLUSTERS
Age MonthlyIncome Cluster
22 25000 0
23 25500 0
24 26000 0
25 27000 0
26 28000 0
30 42000 1
31 43000 1
32 44000 1
33 45000 1
34 46000 1
45 70000 2
46 71000 2
47 72000 2
48 73000 2
49 74000 2
90 300000 -1
Cluster 0
Low Income Customers
Cluster 1
Medium Income Customers
Cluster 2
High Income Customers
Noise
Age = 90
Income = 300000
Cluster = -1
This customer is very different from everyone else, so DBSCAN identifies it as an outlier (noise) instead of forcing it into a cluster.
Summary
| Section | Description |
|---|---|
| Definition | DBSCAN is an Unsupervised Learning algorithm that creates clusters based on data density and identifies outliers (noise). |
| Purpose | To find natural clusters, detect outliers, and handle irregularly shaped data without specifying the number of clusters beforehand. |
| When to Use | Use when there is no target column, the number of clusters is unknown, or you need to detect anomalies in the data. |
| Code Example | Uses a Pandas DataFrame, DBSCAN, cluster assignment, and displays both clusters and noise points. |
Principal Component Analysis (PCA)
It will follow the same format:
Definition
Purpose
When to Use
Complete Python Code (using a Pandas DataFrame with dummy data)
0 Comments