Chapter 8 — Random Forest
Random Forest is one of the most powerful and widely used supervised Machine Learning algorithms.
It works for both classification (predicting categories such as Yes/No, Pass/Fail) and regression (predicting continuous values such as sales or house prices).
It is popular because it usually provides high accuracy, handles noisy data well, and reduces the overfitting problem often seen in Decision Trees.
What is Random Forest?
A Random Forest is a collection of many Decision Trees.
Instead of relying on a single tree, Random Forest creates many trees and combines their predictions.
Think of it like asking several experts for advice instead of asking only one.
Real-Life Example
Imagine you want to buy a laptop.
You ask only one friend.
Friend A says:
Buy Laptop X.
Would you trust only one opinion?
Probably not.
Instead, you ask:
Friend A
Friend B
Friend C
Friend D
Friend E
Suppose their answers are:
Laptop X
Laptop X
Laptop Y
Laptop X
Laptop X
Most people recommend Laptop X.
You choose Laptop X.
This is exactly how Random Forest works.
Decision Tree vs Random Forest
Decision Tree
One model makes the decision.
Tree
|
Prediction
If the tree makes a mistake, the final prediction is wrong.
Random Forest
Many trees make predictions.
Tree 1
Tree 2
Tree 3
Tree 4
Tree 5
↓
Final Decision
Mistakes made by one tree can be corrected by the others.
Why is it called "Random Forest"?
The name has two parts:
Random
Each tree is built using:
Random samples of the data.
Random subsets of features.
This makes each tree different.
Forest
Many Decision Trees together form a forest.
Tree
Tree
Tree
Tree
Tree
Together they are called a Random Forest.
Why not use one Decision Tree?
Suppose you have one Decision Tree.
It learns the training data perfectly.
But when new data arrives, it performs poorly.
This is called:
Overfitting
Random Forest reduces this problem because many trees work together.
Main Idea of Random Forest
Instead of asking:
"What does one tree think?"
Random Forest asks:
"What do all trees think?"
The final prediction is based on the group's decision.
Components of Random Forest
A Random Forest consists of:
Training Dataset
Bootstrap Sampling
Multiple Decision Trees
Random Feature Selection
Voting (Classification)
Averaging (Regression)
Final Prediction
Let's understand each one.
1. Training Dataset
Suppose we have this customer purchase dataset:
| Customer | Income | Visits | Purchased |
|---|---|---|---|
| 1 | 30000 | 3 | No |
| 2 | 45000 | 8 | Yes |
| 3 | 28000 | 2 | No |
| 4 | 60000 | 10 | Yes |
| 5 | 52000 | 9 | Yes |
This dataset is used to build many trees.
2. Bootstrap Sampling
This is one of the most important concepts.
What is Bootstrap Sampling?
Instead of giving every tree the entire dataset, Random Forest creates a new dataset for each tree by randomly selecting records with replacement.
What does "with replacement" mean?
Suppose the original dataset is:
| Record |
|---|
| A |
| B |
| C |
| D |
| E |
One bootstrap sample might be:
| Sample |
|---|
| A |
| C |
| C |
| D |
| E |
Notice:
Cappears twice.Bis missing.
This is normal.
Another tree may receive:
| Sample |
|---|
| B |
| D |
| A |
| E |
| E |
Each tree gets a different version of the data.
Why Bootstrap Sampling?
If every tree used the exact same dataset, all trees would become very similar.
Different samples create different trees.
Different trees lead to better overall predictions.
3. Multiple Decision Trees
Suppose we create:
100 Trees
Each tree:
Uses a different bootstrap sample.
Learns different patterns.
Each tree makes its own prediction.
Example:
| Tree | Prediction |
|---|---|
| Tree 1 | Yes |
| Tree 2 | No |
| Tree 3 | Yes |
| Tree 4 | Yes |
| Tree 5 | Yes |
4. Random Feature Selection
Random Forest does not allow every tree to examine all features at every split.
Suppose your dataset has:
Income
Age
Website Visits
Reviews
Discount
Membership
One tree may consider:
Income
Reviews
Discount
Another tree may consider:
Age
Website Visits
Membership
This randomness makes the trees less alike and improves the strength of the forest.
Why Random Features?
If every tree always used the same strongest feature first, the trees would look almost identical.
Random feature selection forces diversity.
5. Majority Voting (Classification)
Suppose 9 trees make predictions:
| Tree | Prediction |
|---|---|
| 1 | Yes |
| 2 | Yes |
| 3 | No |
| 4 | Yes |
| 5 | Yes |
| 6 | No |
| 7 | Yes |
| 8 | Yes |
| 9 | Yes |
Votes:
Yes = 7
No = 2
Final prediction:
Yes
This process is called Majority Voting.
6. Averaging (Regression)
For regression, trees predict numbers.
Example:
| Tree | Predicted Sales |
|---|---|
| 1 | 25000 |
| 2 | 25500 |
| 3 | 24800 |
| 4 | 25200 |
| 5 | 25050 |
Average:
(25000 + 25500 + 24800 + 25200 + 25050) / 5
= 25110
Final prediction:
25110
Complete Workflow
Training Data
│
▼
Bootstrap Sampling
│
▼
Create Many Decision Trees
│
▼
Each Tree Makes Prediction
│
▼
Combine Predictions
│
▼
Majority Vote (Classification)
or
Average (Regression)
│
▼
Final Prediction
Advantages of Random Forest
High prediction accuracy.
Reduces overfitting compared to a single Decision Tree.
Works well with large datasets.
Handles many input features.
Can estimate feature importance.
Robust to noisy data.
Can solve both classification and regression problems.
Disadvantages of Random Forest
Slower to train than a single Decision Tree.
Uses more memory because many trees are stored.
Harder to interpret than one Decision Tree.
Large forests can increase prediction time.
Common Applications
Random Forest is widely used for:
Product purchase prediction
Customer churn prediction
Loan approval
Fraud detection
Disease prediction
Employee attrition prediction
Credit risk analysis
House price prediction (regression)
Sales prediction (regression)
Important Terms to Remember
| Term | Meaning |
|---|---|
| Random Forest | A collection of many Decision Trees |
| Tree | One Decision Tree inside the forest |
| Bootstrap Sampling | Randomly selecting training records with replacement |
| Random Feature Selection | Using only a random subset of features when splitting |
| Ensemble Learning | Combining multiple models into one stronger model |
| Bagging | Training many models independently on different bootstrap samples |
| Majority Voting | Final prediction for classification based on the most common class |
| Averaging | Final prediction for regression based on the average of all tree outputs |
| Feature Importance | A score indicating how useful each feature is for prediction |
n_estimators | Number of trees in the forest |
| Overfitting | Model memorizes training data and performs poorly on new data |
Chapter 8.5 — Random Forest in Python (Complete Practical Implementation)
Now it's time to implement everything you've learned.
In this chapter, we'll build a complete Random Forest Classification project from scratch.
Just as you previously learned Logistic Regression and Decision Tree with a product-based dataset, we'll use a similar business example.
Learning Objectives
By the end of this chapter, you will be able to:
Import the required libraries
Create a dataset using a Python dictionary and Pandas
Prepare features and target
Split data into training and testing sets
Train a
RandomForestClassifierMake predictions
Evaluate the model
Interpret the output
Predict new records
Understand feature importance
Project
Product Purchase Prediction
A company wants to predict whether a customer will purchase a product.
Each customer has:
| Feature | Meaning |
|---|---|
| Age | Customer age |
| MonthlyIncome | Monthly income |
| WebsiteVisits | Number of website visits |
| PreviousPurchases | Number of previous purchases |
| Purchased | Target (0 = No, 1 = Yes) |
Step 1 — Import Libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import (
confusion_matrix,
accuracy_score,
precision_score,
recall_score,
f1_score
)
Step 2 — Create Dataset
data = {
"Age": [
22,25,28,30,35,40,45,50,29,33,
26,31,38,42,47,52,24,27,36,41,
23,34,39,44,48,53,21,32,37,46
],
"MonthlyIncome": [
25000,30000,35000,42000,50000,
55000,60000,65000,38000,45000,
32000,43000,52000,57000,62000,
68000,28000,34000,51000,56000,
26000,47000,54000,59000,64000,
70000,24000,41000,53000,61000
],
"WebsiteVisits": [
2,3,4,5,7,
8,9,10,4,6,
3,5,7,8,9,
10,2,4,7,8,
2,6,8,9,10,
11,1,5,7,9
],
"PreviousPurchases": [
0,1,1,2,3,
4,5,6,1,2,
1,2,3,4,5,
6,0,1,3,4,
0,2,3,4,5,
6,0,2,3,5
],
"Purchased": [
0,0,0,0,1,
1,1,1,0,1,
0,0,1,1,1,
1,0,0,1,1,
0,1,1,1,1,
1,0,0,1,1
]
}
df = pd.DataFrame(data)
Step 3 — Display Dataset
print(df.head())
Output:
| Age | MonthlyIncome | WebsiteVisits | PreviousPurchases | Purchased |
|---|---|---|---|---|
| 22 | 25000 | 2 | 0 | 0 |
| 25 | 30000 | 3 | 1 | 0 |
| 28 | 35000 | 4 | 1 | 0 |
| 30 | 42000 | 5 | 2 | 0 |
| 35 | 50000 | 7 | 3 | 1 |
Step 4 — Select Features
X = df[
[
"Age",
"MonthlyIncome",
"WebsiteVisits",
"PreviousPurchases"
]
]
These columns are the inputs to the model.
Step 5 — Select Target
y = df["Purchased"]
This is the value the model will learn to predict.
Step 6 — Split Dataset
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.30,
random_state=42
)
Here:
70% of the data is used for training.
30% is used for testing.
random_state=42ensures reproducible results.
Step 7 — Create Random Forest Model
model = RandomForestClassifier(
n_estimators=100,
random_state=42
)
Meaning:
Create 100 Decision Trees.
Use the same random seed every time.
Step 8 — Train the Model
model.fit(X_train, y_train)
The model learns patterns from the training data.
Step 9 — Make Predictions
y_pred = model.predict(X_test)
The model predicts whether each customer in the test set will purchase the product.
Step 10 — Prediction Probabilities
probabilities = model.predict_proba(X_test)
print(probabilities)
Example output:
[[0.94 0.06]
[0.15 0.85]
[0.82 0.18]]
Interpretation:
First customer:
94% chance of No Purchase
6% chance of Purchase
Second customer:
15% chance of No Purchase
85% chance of Purchase
Step 11 — Evaluate Model
cm = confusion_matrix(y_test, y_pred)
accuracy = accuracy_score(y_test, y_pred)
precision = precision_score(y_test, y_pred)
recall = recall_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred)
Step 12 — Print Results
print(cm)
print("Accuracy :", accuracy)
print("Precision:", precision)
print("Recall :", recall)
print("F1 Score :", f1)
Possible output:
[[4 1]
[0 4]]
Accuracy : 0.89
Precision: 0.80
Recall : 1.00
F1 Score : 0.89
Step 13 — Feature Importance
One of the biggest advantages of Random Forest is that it estimates how important each feature is.
importance = model.feature_importances_
for feature, score in zip(X.columns, importance):
print(feature, ":", round(score, 4))
Example output:
Age : 0.12
MonthlyIncome : 0.46
WebsiteVisits : 0.28
PreviousPurchases : 0.14
Interpretation:
MonthlyIncomeis the most influential feature.WebsiteVisitsis the second most important.Agehas the least influence among these features.
The importance scores usually add up to approximately 1.0.
Step 14 — Predict a New Customer
new_customer = [[35, 55000, 8, 3]]
prediction = model.predict(new_customer)
probability = model.predict_proba(new_customer)
Step 15 — Display Prediction
if prediction[0] == 1:
print("Customer Will Purchase")
else:
print("Customer Will Not Purchase")
print("No Purchase Probability:",
round(probability[0][0]*100,2), "%")
print("Purchase Probability:",
round(probability[0][1]*100,2), "%")
Possible output:
Customer Will Purchase
No Purchase Probability: 12.00 %
Purchase Probability: 88.00 %
What Happens Internally?
When you execute:
model.fit(X_train, y_train)
Random Forest performs these steps:
Training Data
│
▼
Bootstrap Sampling
│
▼
100 Different Training Sets
│
▼
Train 100 Decision Trees
│
▼
Random Feature Selection
│
▼
Forest Ready
When you execute:
model.predict(new_customer)
The new customer is sent to every tree.
Example:
Tree 1 → Purchase
Tree 2 → Purchase
Tree 3 → No Purchase
Tree 4 → Purchase
...
Tree 100 → Purchase
The majority vote becomes the final prediction.
Important Methods
| Method | Purpose |
|---|---|
fit() | Train the model |
predict() | Predict class labels |
predict_proba() | Return probabilities for each class |
score() | Return the model's accuracy on a dataset |
feature_importances_ | Show how important each feature is |
Random Forest Classifier vs Random Forest Regressor
| Classifier | Regressor |
|---|---|
| Predicts categories | Predicts numbers |
| Majority voting | Averaging |
| Output: Yes/No, 0/1 | Output: 25,000, 50.5, etc. |
| Uses classification metrics | Uses regression metrics like MAE, RMSE, and R² |
Common Interview Questions
1. Why is Random Forest better than a Decision Tree?
Because it combines many Decision Trees, reducing overfitting and usually improving accuracy.
2. Why does Random Forest use Bootstrap Sampling?
To train each tree on a different version of the dataset, increasing diversity.
3. Why does Random Forest use Random Feature Selection?
To prevent all trees from becoming too similar by forcing them to consider different subsets of features at each split.
4. What is the purpose of n_estimators?
It specifies the number of Decision Trees in the forest.
5. Can Random Forest perform Regression?
Yes.
Use:
from sklearn.ensemble import RandomForestRegressor
0 Comments