Introduction to K-Nearest Neighbors (KNN)

What is K-Nearest Neighbors (KNN)?

K-Nearest Neighbors (KNN) is a Supervised Machine Learning algorithm used to make predictions based on the similarity between data points.

It predicts the output of a new data point by looking at the K nearest (closest) data points in the training dataset.

In simple words,

KNN says, "Find the most similar data points and make a prediction based on them."

Why is KNN Needed?

Suppose you want to predict whether a customer will purchase a product.

Instead of creating a mathematical formula, you simply look at customers who are most similar to the new customer.

If most similar customers purchased the product, then the new customer is also likely to purchase it.

This idea is exactly how KNN works.

What Does KNN Stand For?

KNN consists of three words:

K

"K" represents the number of nearest neighbors considered while making a prediction.

Example:

K = 3

The algorithm will use the 3 closest data points.

K = 5

it will use the 5 closest data points.

Nearest

Nearest means closest.

The closeness is measured using a mathematical distance formula.

Imagine you want to find your nearest restaurant.

You will choose the restaurant that is physically closest to your location.

Similarly, KNN finds the closest data points.

Neighbor

A neighbor is simply another data point already present in the dataset.

Example:

Customer	Age	Income	Purchased
A	22	25000	No
B	24	28000	No
C	30	45000	Yes

If a new customer is similar to Customer B, then Customer B becomes one of its nearest neighbors.

Understanding KNN with a Real-Life Example

Imagine you have shifted to a new neighborhood.

You don't know whether most people in the area are:

Families
Students

Instead of asking everyone, you ask only your five nearest neighbors.

Their answers are:

Neighbor	Type
1	Family
2	Family
3	Student
4	Family
5	Family

Count the answers:

Family = 4

Student = 1

Since most nearby people are families, you conclude:

This is mostly a family neighborhood.

KNN follows exactly this logic.

Machine Learning Example

Suppose we have the following training data.

Customer	Age	Income	Purchased
A	22	25000	No
B	24	28000	No
C	30	50000	Yes
D	35	65000	Yes
E	32	55000	Yes

A new customer arrives.

Age	Income
29	48000

KNN compares this customer with every customer in the dataset.

Suppose the nearest three customers are:

Customer	Purchased
C	Yes
D	Yes
B	No

Count the votes:

Yes = 2

No = 1

Prediction:

Purchased = Yes

The majority of nearby customers purchased the product.

How KNN Makes a Prediction

KNN always follows the same sequence of steps.

Step 1

Store the training dataset.

↓

Step 2

A new data point arrives.

↓

Step 3

Measure the distance between the new data point and every training record.

↓

Step 4

Find the K nearest data points.

↓

Step 5

For classification, count the votes.

For regression, calculate the average.

↓

Step 6

Return the final prediction.

KNN for Classification

Classification means predicting a category.

Example:

Customer	Purchased
A	Yes
B	No
C	Yes

Prediction:

Yes

No

KNN for Regression

Regression means predicting a numerical value.

Example:

Area	House Price
1000	40 Lakhs
1200	48 Lakhs
1400	55 Lakhs

Prediction:

52 Lakhs

Does KNN Learn During Training?

No.

This is one of the biggest differences between KNN and many other algorithms.

During training, KNN simply stores the dataset.

It does not:

build an equation,
create a decision tree,
calculate coefficients, or
learn weights.

Most of the work happens only when a prediction is requested.

What Happens When K Changes?

Suppose

K = 1

Only one nearest neighbor decides the prediction.

This makes the model very sensitive to unusual or noisy data.

Suppose

K = 3

Three nearest neighbors vote.

The prediction becomes more stable.

Suppose

K = 7

Seven neighbors vote.

The prediction becomes smoother, but choosing a very large K may ignore important local patterns.

Choosing the correct value of K is very important and will be discussed later.

Real-Life Applications of KNN

Customer Purchase Prediction

Predict whether a customer will purchase a product.

House Price Prediction

Predict the price of a house by comparing it with similar houses.

Medical Diagnosis

Predict whether a patient has a disease by comparing with similar patients.

Movie Recommendation

Recommend movies liked by users with similar interests.

Loan Approval

Predict whether a customer is likely to repay a loan based on similar customers.

Spam Email Detection

Classify emails as:

Spam
Not Spam

Handwriting Recognition

Recognize handwritten numbers by comparing them with previously stored handwritten samples.

Advantages of KNN

Very easy to understand.
Simple to implement.
No complicated training process.
Works well for small datasets.
Can solve both classification and regression problems.
Learns from real examples instead of creating complex equations.

Disadvantages of KNN

Prediction becomes slow for large datasets because every new record must be compared with all training records.
Requires more memory because the entire training dataset must be stored.
Sensitive to irrelevant features.
Sensitive to different feature scales, so feature scaling is often required.
Selecting the correct value of K is important.

KNN vs Logistic Regression

KNN	Logistic Regression
Uses nearest neighbors to predict.	Uses a mathematical equation to predict.
No equation is learned.	Learns coefficients during training.
Prediction depends on nearby records.	Prediction depends on learned weights.
Works well for complex local patterns.	Works well when the relationship is approximately linear.

KNN vs Decision Tree

KNN	Decision Tree
Uses distance between data points.	Uses decision rules to split the data.
Stores all training records.	Builds a tree during training.
Prediction is based on neighbors.	Prediction is based on decision paths.
Prediction is slower for large datasets.	Prediction is generally faster after training.

Chapter 2 — Distance Calculation in K-Nearest Neighbors (KNN)

KNN makes predictions by finding the nearest (closest) data points.

But how does KNN know which data point is the closest?

The answer is Distance Calculation.

What is Distance?

Distance tells us how far one data point is from another data point.

In everyday life, we calculate the distance between two cities.

Example:

Delhi → Agra = 230 km

Delhi → Jaipur = 280 km

Since Agra is closer than Jaipur, we say Agra is the nearest city.

KNN works in exactly the same way.

Instead of cities, it calculates the distance between data records.

Example

Suppose we have the following training data.

Customer	Age	Income	Purchased
A	22	25000	No
B	25	30000	No
C	30	50000	Yes
D	35	65000	Yes

A new customer arrives.

Age	Income
28	45000

KNN must decide whether this customer will purchase a product.

To do that, it first calculates the distance between the new customer and every customer in the training data.

Why is Distance Important?

Suppose there are two customers.

Customer	Age	Income
A	27	44000
B	50	90000

New Customer:

Age = 28

Income = 45000

Clearly,

Customer A

is much more similar than

Customer B

Therefore, Customer A should have more influence on the prediction.

Distance helps KNN identify similar records.

Types of Distance

There are several methods for calculating distance.

The most common ones are:

Euclidean Distance
Manhattan Distance
Minkowski Distance
Hamming Distance (used mainly for categorical/binary data)

The most popular method is Euclidean Distance.

Euclidean Distance

Euclidean Distance is the straight-line distance between two points.

Imagine two houses.

House A •---------------------• House B

The straight line between them is the Euclidean Distance.

This is the default distance used in many KNN implementations.

Euclidean Distance Formula

For two features:

genui{"geometry_measurement_learning_block":{"type_id":"DISTANCE_FORMULA","content":"d=\sqrt{(x_2-x_1)^2+(y_2-y_1)^2}"}}

Where:

d = Distance
x₁, y₁ = First data point
x₂, y₂ = Second data point

Example 1

Suppose

Customer A

Age	Income
25	30000

New Customer

Age	Income
28	45000

Using the formula:

d = √((28 − 25)² + (45000 − 30000)²)

d = √(3² + 15000²)

d = √(9 + 225000000)

≈ 15000

Notice something interesting.

Although the age differs by only 3 years, the income differs by 15,000.

The income completely dominates the distance.

This is why feature scaling is important (we will study it later).

Example 2

Suppose all features are already scaled.

Customer A

Height	Weight
170	65

New Person

Height	Weight
175	70

Distance:

d = √((175−170)² + (70−65)²)

d = √(25 + 25)

d = √50

≈ 7.07

Manhattan Distance

Instead of moving in a straight line, imagine you are driving through city roads.

You cannot drive through buildings.

You move:

left
right
up
down

This is called Manhattan Distance.

Formula:

d = |x₂ − x₁| + |y₂ − y₁|

Example

Customer

Height	Weight
170	65

New Person

Height	Weight
175	70

Distance:

= |175−170| + |70−65|

= 5 + 5

= 10

Notice that the answer is different from Euclidean Distance.

Euclidean vs Manhattan

Suppose you want to reach a shop.

Euclidean

You can walk directly.

Home ●──────────────● Shop

Shortest straight-line path.

Manhattan

You must follow the roads.

Home ●───┐
         │
         │
         ● Shop

Longer path.

Which Distance Does KNN Use?

By default, most KNN implementations use:

Euclidean Distance

However, you can also use:

Manhattan Distance
Minkowski Distance
Chebyshev Distance

depending on your problem.

Which Customer is Nearest?

Suppose

Customer	Distance
A	4
B	8
C	2
D	6

The nearest customer is:

Customer C

because

is the smallest distance.

KNN always chooses the smallest distances first.

What Happens After Distance Calculation?

Suppose

K = 3

Distances are:

Customer	Distance	Purchased
A	2	Yes
B	4	No
C	6	Yes
D	10	No

The three nearest customers are:

Votes:

Yes = 2

No = 1

Final Prediction:

Purchased = Yes

Python Example

from sklearn.neighbors import KNeighborsClassifier

model = KNeighborsClassifier(
    n_neighbors=3
)

model.fit(X_train, y_train)

prediction = model.predict(X_test)

By default,

metric="minkowski"
p=2

which is equivalent to Euclidean Distance.

To use Manhattan Distance:

model = KNeighborsClassifier(
    n_neighbors=3,
    metric="manhattan"
)

Advantages of Euclidean Distance

Simple to calculate.
Fast.
Works well for continuous numerical data.
Default choice for many KNN problems.

Disadvantages of Euclidean Distance

Sensitive to different feature scales.
Large numerical values can dominate the distance.
Works best after feature scaling.

Interview Questions

What is distance in KNN?

Answer:
Distance measures how close or far two data points are from each other. KNN uses this distance to find the nearest neighbors.

Which distance is used by default in KNN?

Answer:
Euclidean Distance.

Why is distance calculation important?

Answer:
Because KNN predicts using the nearest data points. Without calculating distance, it cannot determine which records are most similar.

Which distance is shorter?

Straight Line

Answer: Euclidean Distance.

Which distance follows roads?

Answer: Manhattan Distance.

Choosing the Value of K in K-Nearest Neighbors (KNN)

One of the most important decisions in KNN is selecting the value of K.

The performance of the model depends heavily on this value.

If K is chosen incorrectly, the model may produce poor predictions.

What is K?

K represents the number of nearest neighbors used to make a prediction.

Suppose

K = 3

The algorithm will look at the 3 nearest data points.

K = 5

The algorithm will consider the 5 nearest data points.

How K Works

Suppose we have the following training data.

Customer	Purchased
A	Yes
B	Yes
C	No
D	Yes
E	No

A new customer arrives.

Case 1: K = 1

Only the nearest customer is considered.

Nearest Customer:

Customer	Purchased
A	Yes

Prediction:

Yes

Only one vote is taken.

Case 2: K = 3

The three nearest customers are:

Customer	Purchased
A	Yes
B	Yes
C	No

Votes:

Yes = 2

No = 1

Prediction:

Yes

Case 3: K = 5

Nearest customers:

Customer	Purchased
A	Yes
B	Yes
C	No
D	Yes
E	No

Votes:

Yes = 3

No = 2

Prediction:

Yes

What Happens if K is Too Small?

Suppose

K = 1

Only one neighbor decides the prediction.

Example:

Customer	Purchased
A	No

Prediction:

No

Even if all other nearby customers purchased the product, the prediction will still be No.

This makes the model sensitive to unusual data (noise).

Example of Noise

Suppose

Customer	Purchased
A	No
B	Yes
C	Yes
D	Yes

K = 1

Prediction:

No

because only Customer A is considered.

This prediction is not reliable because one unusual record has influenced the result.

What Happens if K is Too Large?

Suppose there are 100 customers.

K = 95

then the model will consider almost the entire dataset.

Even if the nearest customers suggest Yes, many distant customers may vote No.

As a result, the prediction may ignore local patterns.

Small K vs Large K

Small K

Advantages

Learns local patterns.
Sensitive to small changes.

Disadvantages

Easily affected by noise.
Can overfit the training data.

Large K

Advantages

More stable predictions.
Less affected by noise.

Disadvantages

May ignore nearby important data.
Can underfit the data.

Example

Suppose we want to classify a fruit.

Nearest fruits are:

Fruit	Type
Apple	Apple
Apple	Apple
Mango	Mango
Apple	Apple
Mango	Mango

K = 1

Prediction:

Apple

K = 3

Votes:

Apple = 2

Mango = 1

Prediction:

Apple

K = 5

Votes:

Apple = 3

Mango = 2

Prediction:

Apple

Choosing an Odd Value of K

For binary classification problems (Yes/No), it is common to use an odd value of K.

Examples:

K = 3

K = 5

K = 7

K = 9

Why?

Because odd values reduce the chance of a tie.

Tie Problem

Suppose

K = 4

Nearest neighbors:

Purchased
Yes
Yes
No
No

Votes:

Yes = 2

No = 2

The algorithm faces a tie.

Different implementations may resolve ties differently, but using an odd value of K often avoids this issue.

How to Find the Best Value of K

There is no fixed rule such as:

Always use K = 5

The best value depends on the dataset.

The common approach is:

Train the model with different K values.
Calculate the accuracy for each value.
Choose the K that gives the highest accuracy on validation or test data.

Example:

K	Accuracy
1	88%
3	92%
5	95%
7	94%
9	91%

Best choice:

K = 5

because it gives the highest accuracy.

Python Example

from sklearn.neighbors import KNeighborsClassifier

model = KNeighborsClassifier(
    n_neighbors=5
)

model.fit(X_train, y_train)

prediction = model.predict(X_test)

Changing K:

model = KNeighborsClassifier(
    n_neighbors=3
)

model = KNeighborsClassifier(
    n_neighbors=7
)

Only the n_neighbors value changes.

Real-Life Example

Imagine you want to decide whether a new restaurant is good.

K = 1

You ask only one friend.

If that friend dislikes the restaurant, you conclude it is bad.

This may not be a fair decision.

K = 5

You ask five friends.

If four of them recommend it, you are more confident that the restaurant is good.

This is similar to how KNN uses multiple neighbors to make a more reliable prediction.

Choosing K in Regression

For KNN Regression, the process is the same.

The difference is that instead of taking a majority vote, KNN calculates the average value of the K nearest neighbors.

Example:

Nearest house prices:

House	Price (₹)
A	40,00,000
B	42,00,000
C	44,00,000

Prediction:

Average Price

= (40 + 42 + 44) / 3

= 42 Lakhs

What does K represent in KNN?

Answer:
K represents the number of nearest neighbors considered when making a prediction.

What happens if K is too small?

Answer:
The model becomes sensitive to noise and may overfit the training data.

What happens if K is too large?

Answer:
The model may ignore important local patterns and underfit the data.

Why do we often use an odd value of K?

Answer:
To reduce the chances of a tie in binary classification problems.

Is there a fixed value of K for every dataset?

Answer:
No. The best value of K depends on the dataset and is usually selected by testing different values and comparing model performance.

Feature Scaling in K-Nearest Neighbors (KNN)

Many beginners train a KNN model without scaling the data and get poor results.

Unlike Decision Trees, KNN is highly dependent on the values of the features because it calculates distances.

If one feature has much larger values than another, it can dominate the distance calculation and lead to incorrect predictions.

What is Feature Scaling?

Feature Scaling is the process of bringing all numerical features to a similar range.

In simple words,

Feature Scaling makes sure that every feature gets a fair chance to influence the prediction.

Why Do We Need Feature Scaling?

Suppose we have the following customer data.

Customer	Age	Monthly Income
A	22	25,000
B	25	30,000
C	30	50,000

A new customer is:

Age	Monthly Income
28	45,000

Notice the values.

Age:
22
25
30

Income:
25000
30000
50000

Age values are between 20 and 30.

Income values are between 25,000 and 50,000.

Income is much larger than Age.

What Happens Without Scaling?

Suppose we calculate the Euclidean Distance.

Age Difference

28 − 25 = 3

Income Difference

45000 − 30000 = 15000

Distance

√(3² + 15000²)

The value 3 is almost ignored.

The value 15000 dominates the distance.

As a result,

KNN mainly compares customers based on Income, while Age has almost no effect.

This may lead to poor predictions.

Real-Life Example

Suppose a college wants to select students based on:

Age
Percentage

Example:

Student	Age	Percentage
A	18	92
B	19	95
C	20	90

Here,

Age ranges from 18 to 20.

Percentage ranges from 90 to 95.

Both are on a similar scale.

No problem occurs.

Now consider another feature.

Student	Age	Annual Family Income
A	18	250000
B	19	500000
C	20	800000

Income values are much larger.

Without scaling, Income dominates the distance calculation.

What Does Scaling Do?

Scaling converts all features into a similar range.

Example

Before Scaling

Age	Income
22	25000
30	50000

After Scaling

Age	Income
0.20	0.18
0.85	0.90

Now both features have similar importance.

Feature Scaling Example

Original Data

Height (cm)	Weight (kg)
170	65
180	75
190	90

After Scaling

Height	Weight
0.0	0.0
0.5	0.4
1.0	1.0

Now both features contribute fairly to the distance calculation.

Common Feature Scaling Techniques

The two most commonly used methods are:

StandardScaler
MinMaxScaler

1. StandardScaler

StandardScaler transforms the data so that:

Mean becomes 0
Standard Deviation becomes 1

The transformed values usually lie around:

-3 to +3

Example

Before Scaling

Income
25000
50000
75000

After StandardScaler

Income
-1.20
0.00
1.20

Notice that the values are now much smaller and centered around zero.

Python Example

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)

X_test = scaler.transform(X_test)

Notice:

fit_transform()

is used only for the training data.

For the testing data we use:

transform()

Why?

Because the test data must be scaled using the same scaling parameters learned from the training data.

2. MinMaxScaler

MinMaxScaler converts all values into a range between:

0 and 1

Formula

New Value

=

(Current Value − Minimum)

/

(Maximum − Minimum)

Example

Before Scaling

Age
20
25
30

After Scaling

Age
0.0
0.5
1.0

Python Example

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()

X_train = scaler.fit_transform(X_train)

X_test = scaler.transform(X_test)

StandardScaler vs MinMaxScaler

StandardScaler	MinMaxScaler
Mean becomes 0	Values are between 0 and 1
Standard deviation becomes 1	Preserves the relative position within the range
Can produce negative values	No negative values (unless original data is outside the fitted range)
Commonly used for many ML algorithms	Useful when a fixed range is preferred

Which Algorithms Need Feature Scaling?

Algorithm	Scaling Required?
KNN	✅ Yes
Logistic Regression	✅ Yes (recommended)
SVM	✅ Yes
K-Means	✅ Yes
PCA	✅ Yes
Linear Regression	Recommended, but not always required
Decision Tree	❌ No
Random Forest	❌ No

Why Doesn't a Decision Tree Need Scaling?

Decision Trees make decisions using conditions such as:

Income > 50000

Age < 30

They do not calculate distances.

Therefore, the scale of the features does not affect how the tree splits the data.

Why Does KNN Need Scaling?

KNN compares data points using distance.

If one feature has much larger values than another, it will dominate the distance calculation.

Scaling ensures that all features contribute fairly.

Complete Workflow with Scaling

Load Dataset

↓

Data Cleaning

↓

Encoding (if needed)

↓

Split Dataset

↓

Feature Scaling

↓

Train KNN Model

↓

Predict

↓

Evaluate Model

Python Example

import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier

df = pd.read_csv("customer_data.csv")

X = df.drop("Purchased", axis=1)
y = df["Purchased"]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)

X_test = scaler.transform(X_test)

model = KNeighborsClassifier(n_neighbors=5)

model.fit(X_train, y_train)

prediction = model.predict(X_test)

print(prediction)

Interview Questions

What is Feature Scaling?

Answer:
Feature Scaling is the process of converting numerical features to a similar range so that no single feature dominates the model.

Why is Feature Scaling important in KNN?

Answer:
KNN uses distance calculations. Without scaling, features with larger numerical values dominate the distance and reduce prediction quality.

Name two Feature Scaling techniques.

Answer:

StandardScaler
MinMaxScaler

Which scaler converts values between 0 and 1?

Answer:
MinMaxScaler.

Which scaler makes the mean equal to 0?

Answer:
StandardScaler.

Why do we use `fit_transform()` on the training data and `transform()` on the test data?

Answer:
The scaler learns the scaling parameters (such as mean, standard deviation, minimum, and maximum) only from the training data. The test data is transformed using those same parameters to avoid data leakage.

Chapter 5 — Implementing K-Nearest Neighbors (KNN) Classification in Python

Now that you understand the theory behind KNN, let's learn how to implement it in Python using Scikit-learn.

We will go through every step exactly as you would in a real Machine Learning project.

Step 1 — Import Libraries

First, import all the required libraries.

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import StandardScaler

from sklearn.neighbors import KNeighborsClassifier

from sklearn.metrics import accuracy_score

Explanation

pandas → Read the dataset.
train_test_split → Split data into training and testing.
StandardScaler → Scale numerical features.
KNeighborsClassifier → Create the KNN model.
accuracy_score → Measure model accuracy.

Step 2 — Load the Dataset

df = pd.read_csv("customer_data.csv")

Example Dataset

Age	Income	WebsiteVisits	Purchased
22	25000	2	0
24	28000	3	0
26	32000	4	1
30	40000	6	1

Step 3 — Select Features and Target

Features (Input)

X = df[
    [
        "Age",
        "Income",
        "WebsiteVisits"
    ]
]

Target (Output)

y = df["Purchased"]

Here,

X contains input features.
y contains the value we want to predict.

Step 4 — Split the Dataset

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

Suppose the dataset has 100 records.

Dataset	Records
Training	80
Testing	20

The model learns from the training data and is evaluated on the testing data.

Step 5 — Scale the Features

Since KNN uses distance calculations, feature scaling is necessary.

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)

X_test = scaler.transform(X_test)

Why?

Without scaling,

Income = 50000

Age = 25

Income would dominate the distance calculation.

Scaling ensures that all features contribute fairly.

Step 6 — Create the KNN Model

model = KNeighborsClassifier(
    n_neighbors=5
)

Here,

K = 5

The model will consider the 5 nearest neighbors for every prediction.

Step 7 — Train the Model

model.fit(
    X_train,
    y_train
)

During training,

KNN stores the training data.

It does not build an equation or a decision tree.

Step 8 — Predict the Test Data

y_pred = model.predict(X_test)

The model predicts the class for every record in the testing dataset.

Example

Actual Values

Predicted Values

Step 9 — Calculate Accuracy

accuracy = accuracy_score(
    y_test,
    y_pred
)

print("Accuracy:", accuracy)

Example Output

Accuracy: 0.95

Meaning

95%

The model predicted 95% of the testing records correctly.

Complete Program

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.preprocessing import StandardScaler

from sklearn.neighbors import KNeighborsClassifier

from sklearn.metrics import accuracy_score

df = pd.read_csv("customer_data.csv")

X = df[
    [
        "Age",
        "Income",
        "WebsiteVisits"
    ]
]

y = df["Purchased"]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)

X_test = scaler.transform(X_test)

model = KNeighborsClassifier(
    n_neighbors=5
)

model.fit(
    X_train,
    y_train
)

y_pred = model.predict(X_test)

accuracy = accuracy_score(
    y_test,
    y_pred
)

print("Accuracy:", accuracy)

Predicting a New Customer

Suppose a new customer has:

Age	Income	Website Visits
28	36000	5

First, create the new record.

new_customer = [
    [28, 36000, 5]
]

Scale it using the same scaler.

new_customer = scaler.transform(
    new_customer
)

Predict the result.

prediction = model.predict(
    new_customer
)

print(prediction)

Example Output

[1]

Meaning

Customer will purchase the product.

Understanding `n_neighbors`

Suppose

model = KNeighborsClassifier(
    n_neighbors=3
)

The model uses 3 nearest neighbors.

model = KNeighborsClassifier(
    n_neighbors=7
)

The model uses 7 nearest neighbors.

Changing only this parameter changes how the model makes predictions.

Important Parameters of `KNeighborsClassifier`

`n_neighbors`

Number of nearest neighbors.

Example

n_neighbors=5

`metric`

Distance calculation method.

Default

metric="minkowski"

Equivalent to Euclidean Distance when p=2.

Example

model = KNeighborsClassifier(
    n_neighbors=5,
    metric="euclidean"
)

model = KNeighborsClassifier(
    n_neighbors=5,
    metric="manhattan"
)

`weights`

Determines how neighbors vote.

Uniform (Default)

Every neighbor gets equal importance.

weights="uniform"

Distance

Closer neighbors get more importance.

weights="distance"

Common Mistakes

Mistake 1

Training without scaling.

model.fit(
    X_train,
    y_train
)

on unscaled data.

Result:

Poor accuracy.

Mistake 2

Using

fit_transform()

on the testing data.

Wrong

X_test = scaler.fit_transform(
    X_test
)

Correct

X_test = scaler.transform(
    X_test
)

Mistake 3

Choosing a very small value of K.

Example

n_neighbors=1

The model becomes sensitive to noise.

Mistake 4

Choosing a very large value of K.

Example

n_neighbors=50

The model may ignore important local patterns.

Real-Life Example

Imagine you are buying a house.

You compare the new house with five nearby houses having similar:

Area
Number of bedrooms
Location
Age of the building

If most similar houses are expensive, the new house is also likely to be expensive.

This is exactly how KNN works.

Interview Questions

Which class is used to implement KNN in Scikit-learn?

Answer

KNeighborsClassifier

Why do we scale the data before KNN?

Answer

Because KNN uses distance calculations, and scaling ensures that all features contribute fairly.

Which method trains the KNN model?

Answer

model.fit()

Which method predicts new data?

Answer

model.predict()

Which method evaluates the model's accuracy?

Answer

accuracy_score()

Does KNN learn an equation during training?

Answer

No. KNN stores the training data and performs most calculations during prediction.

LetSkillsUp

Menu

KNN in machine learning

Introduction to K-Nearest Neighbors (KNN)

What is K-Nearest Neighbors (KNN)?

Why is KNN Needed?

What Does KNN Stand For?

K

Nearest

Neighbor

Understanding KNN with a Real-Life Example

Machine Learning Example

How KNN Makes a Prediction

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

KNN for Classification

KNN for Regression

Does KNN Learn During Training?

What Happens When K Changes?

Real-Life Applications of KNN

Customer Purchase Prediction

House Price Prediction

Medical Diagnosis

Movie Recommendation

Loan Approval

Spam Email Detection

Handwriting Recognition

Advantages of KNN

Disadvantages of KNN

KNN vs Logistic Regression

KNN vs Decision Tree

Chapter 2 — Distance Calculation in K-Nearest Neighbors (KNN)

What is Distance?

Example

Why is Distance Important?

Types of Distance

Euclidean Distance

Euclidean Distance Formula

Example 1

Example 2

Manhattan Distance

Example

Euclidean vs Manhattan

Euclidean

Manhattan

Which Distance Does KNN Use?

Which Customer is Nearest?

What Happens After Distance Calculation?

Python Example

Advantages of Euclidean Distance

Disadvantages of Euclidean Distance

Interview Questions

What is distance in KNN?

Which distance is used by default in KNN?

Why is distance calculation important?

Which distance is shorter?

Which distance follows roads?

Choosing the Value of K in K-Nearest Neighbors (KNN)

What is K?

How K Works

Case 1: K = 1

Case 2: K = 3

Case 3: K = 5

What Happens if K is Too Small?

Example of Noise

What Happens if K is Too Large?

Small K vs Large K

Small K

Large K

Example

K = 1

K = 3

K = 5

Choosing an Odd Value of K

Tie Problem

How to Find the Best Value of K

Why do we use `fit_transform()` on the training data and `transform()` on the test data?

Understanding `n_neighbors`

Important Parameters of `KNeighborsClassifier`

`n_neighbors`

`metric`

`weights`