Analyzing the Performance of SVM and KNN in Supervised Learning for Classification

Abstract

Machine Learning is one of the fastest growing areas of computer science, with far-reaching applications. In these paper, Supervised Learning is one of the tasks most frequently carried out by the Intelligent systems. Supervised Learning Algorithms as well as determines the most efficient classification algorithm based on the data set, the number of enhances and features. The main goal of this paper the overall concept of SVM and KNN algorithms using Supervised Learning Techniques.

Introduction

The text provides an overview of Machine Learning (ML), its methods, and key supervised learning algorithms, focusing especially on Support Vector Machines (SVM) and K-Nearest Neighbors (KNN).

1. Introduction to Machine Learning

• Machine Learning is a subset of Artificial Intelligence (AI) and plays a central role in various industries such as healthcare, finance, retail, entertainment, and transportation.

• ML enables computers to learn from data and make predictions or decisions without being explicitly programmed.

• It uses advanced algorithms to handle large datasets and compute results efficiently.

2. Machine Learning Process (5 Steps)

1. Input Data – Collection of past data (e.g., text, images, audio).

2. Algorithms – Used to learn patterns; can be supervised, unsupervised, or reinforcement learning.

3. Trained Data – Feeding historical data to train the model.

4. Prediction – Making forecasts using new data.

5. Output – Producing the final result based on predictions.

3. Supervised Learning

• Involves training models on labeled data.

• The model learns the relationship between input and output to predict future outcomes.

Two Main Types:

1. Classification – Predicts categories (e.g., Spam or Not Spam).

   • Binary classification – Two classes.

   • Multiclass classification – More than two possible labels.

2. Regression – Predicts continuous values (e.g., company profits, weather conditions).

4. Support Vector Machine (SVM)

• A powerful supervised learning algorithm used for classification and regression.

• It creates a hyperplane that separates classes with maximum margin.

• Types:

  • Linear SVM – Works with linearly separable data.

  • Non-linear SVM – Uses kernel tricks to handle complex data.

5. K-Nearest Neighbor (KNN)

• A non-parametric, lazy learning algorithm mainly used for classification.

• It classifies data based on the closest K data points using distance metrics like:

  • Euclidean Distance

  • Manhattan Distance

  • Minkowski Distance

Example:

• Customer review prediction using KNN on labeled dataset.

• Based on the 3 nearest neighbors, KNN predicts that Sachin's review is "Excellent" using majority voting from closest labeled reviews.

Conclusion

In these model comparative performances of Supervised Learning different between Support Vector Machine and K-Nearest Neighbor Algorithms. An extensive or effective list of information and statistical and mathematical measures for a dataset. After a better understanding of the strengths and limitations of each methods, to solve the problem should be investigated. The KNNalgorithms that high level of correlation among to the output model more efficient. By comparing this two algorithms that the K-Nearest Neighbors(KNN) is simple and easier to understand than Support Vector Machine(SVM).

References

[1] Padraig Cuningham; Sarah Jane Delany (2007) k-Nearest Neighbour classifiers. [2] Tom Mitchell, McGrew Hill (2015) Machine Learning. [3] Sisodia D, Sisodia DS Prediction of Diabetes using classification algorithms. Procedia Comput Sci. 2018;132:1578-85. [4] Shai Shalev-Shwartz and Shai Ben-David (2014). Understanding Machine Learning From Theory To Algorithms. [5] Amanpreet Singh, Narina Thakur, Aakanksha Sharma (2016). Areview of supervised machine learning algorithms. In Computing for Sustainable Global Development (INDIACom), 2016 3rd International Conference on. [6] Boukerche A, Wang J. Machine learning-based traffic prediction models for intelligent transportation systems. ComputNetw. 2020;181. [7] Essien A, Ptrounias I, Sampaio P, Sampaio S. A deep-learning model for urban traffic flow prediction with traffic events mined from twitter. In: World Wide Web, 2020: 1-24.

Copyright

Copyright © 2025 G. Divya, Dr. V. Maniraj. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.