13

MarThe support vector machine is a machine learning algorithm that follows the supervised learning paradigm and can be used for both classifications as well as regression problems though this is primarily a classification algorithm. This algorithm primarily works by designing the hyperplanes and increasing the margin between them.

In the domain of machine learning, support vector machines belong to the associated learning under the broader domain of supervised learning. These models can be used for classification as well as regression. This algorithm particular belongs to the class of non-probabilistic binary classifiers though under few variations like Platt scaling and allied stuff that these can also be used probabilistic classifiers. Being a binary classifier, SVM owing to its inherent design can only assign the data points to two classes only. A model utilizing SVM maps the data points in space in such a fashion that two classes are separated by a clear gap and are concerned about increasing the gap.

In support vector machines that utilize the n-features for its input, the data items are plotted as a point in N-Dimension space with each feature corresponding to the value of a particular coordinate. Then, we check the class to which it belongs. By the basic definition of SVM, it’s a binary classifier and thus, utilizes the normal XY-coordinate geometry which is depicted in figure 1, below.

The data points that are close to the hyperplane and have an impact on the position and alignment of the hyperplane are called support vectors. These data points this algorithm to its name as well. The main aim of using a support vector is to increase the functional margin which is introduced with the hyperplane. Changing or removing these datapoints alter the position and orientation of hyperplane as well.

The decision boundaries that help classify the data are called the hyperplanes. Data points that fall on either side of the hyperplane belong to the same class. The number of dimensions to which a hyperplane exists also varies according to the number of features in the input space. If there are only two features in the input space, then the hyperplane is a straight line as shown in Fig. 1. Whereas, if there 3 features then hyperplane is a dimensional plane. For dimensional greater then 3D, it will also exit but the visualization is not possible. The hyperplane generated ios separated from the support vectors with some margin. This margin should be as large as possible. Figure 2 depicts the concept of margin which is technically known as the functional margin and it remains an important concept in support vector machines.

Read: Data Science vs Machine Learning - What you need to know?

The functioning of the support vector machines is all about increasing the margin between the data points and the hyperplane. A loss function named as hinge loss is one of the most commonly used loss function to perform this activity. Mathematically, Hinge loss is given by:

c(x,y,f(x)= f(x)={0,if y*f(x)≥1 1-y*f(x), else ……………………………………………………………….(1) Eq. (1) can also be represented as c(x,y,f(x))=(1-y*f(x))_+ ………………………………………………………………………………..(2)

If the predicted and the actual value happen to be of the same sign then the cost becomes 0 as evident from figure 1. Scenarios, where they are not zero the calculation of the loss value takes place. In Hinge loss, a regularization parameter is added to balance the margin at maximum separation and bring the loss at a minimum. Once, the regularization parameter is added equation (2), looks like:

〖〖min〗_w λ|(|w|)|^2+∑_(i=1)^n(1-y_i<x_i,w>〗_+…………………………………………………………………….(3)

The derivative of equation(2), can be used to update the weight with the help of the gradient descent algorithm.

A support vector machine happens to be the type of binary classifier. Thus, by definition, they can be used to classify only 2 classes. But, can also be used for a multi-class classifier. This can be done by manipulating the dataset. If we have 3 classes for classification. Native support vector machines are useless. But instead of saying 3 class, if we say, that every class is a negation of its own class then we can have 3 classifiers for saying are they that class or not. IN this way, support vector machines can also be used for multi-class classification. The number classifier trained in this case is

(n*(n-1))/2 ……………………………………………………………………………………………………………………..(4)

Thus, the number of classifier increase but the model is able to handle the multiclass data. This function comes in hand with sklearn as SVC.

In this section, python is utilized with its sklearn library. Different SVM based classifiers will be demonstrated over an iris dataset. Details of the iris dataset can be found here.

Read: How to Do Data Manipulation of Packages Using R?

Importing the libraries:

import numpy as np import matplotlib.pyplot as plt from sklearn import SVM, datasets

The next step is to create the mesh for plotting points:

def make_meshgrid(iris_in, target, h=.02): iris_in_min, iris_in_max = iris_in.min() - 1, iris_in.max() + 1 target_min, target_max = target.min() - 1, target.max() + 1 vector, label = np.meshgrid(np.arange(iris_in_min, iris_in_max, h), np.arange(target_min, target_max, h)) return vector, label

Once the mesh is created, the decision boundries for the classifier are created:

def plot_contours(ax, model, vector, label, **params): A = model.predict(np.c_[vector.ravel(), label.ravel()]) A = A.reshape(vector.shape) out = ax.contourf(vector, label, A, **params) return out

Now, the data set is being imported and 3 classifiers over a 2D space will be created and classifier are trained using a for and plotted:

iris = datasets.load_iris() #loading the dataset X = iris.data[:, :2] #taking only 2 features as for dimentions greater the 2, visualization won’t be possible y = iris.target C = 1.0 # SVM regularization parameter models = (svm.SVC(kernel='linear', C=C), #defining models svm.LinearSVC(C=C, max_iter=10000), svm.SVC(kernel='poly', degree=3, gamma='auto', C=C)) models = (model.fit(X, y) for model in models) titles = ('SVC with linear kernel', 'LinearSVC (linear kernel)', 'SVC with polynomial (degree 3) kernel') fig, sub = plt.subplots(2, 2) #defining the plots and plots 2*2 = 4 grid plt.subplots_adjust(wspace=0.4, hspace=0.4) X0, X1 = X[:, 0], X[:, 1] xx, yy = make_meshgrid(X0, X1) for models, title, ax in zip(models, titles, sub.flatten()): plot_contours(ax, models, xx, yy, cmap=plt.cm.coolwarm, alpha=0.8) ax.scatter(X0, X1, c=y, cmap=plt.cm.coolwarm, s=20, edgecolors='k') ax.set_xlim(xx.min(), xx.max()) ax.set_ylim(yy.min(), yy.max()) ax.set_xlabel('Sepal length') ax.set_ylabel('Sepal width') ax.set_xticks(()) ax.set_yticks(()) ax.set_title(title) x=plt.show()

This will produce a grid of 2*2 = 4 grids which will depict the decision boundaries of the classifier trained. One box remains empty as we are using plt.subplot which makes 4 grids as per specifications and there are only 3 classifiers.

**Advantages:**

Read: An Easy To Understand Approach For K-Nearest Neighbor Algorithm

- It is possible to introduce L2 regularization in the Support vector machines. Thus, helps in prevent over-fitting.
- kernel functions can be utilized in the support vector machines to handle non-linearity in the data.
- Support vector machines can be used for both classifications as well as regression.
- In the cases of small changes in the support vector machines. The hyper-plane is not impacted much. Resulting in a much stable model.

**Disadvantages:**

- While handling non-linear data in support vector machine, it becomes very tricky to select the appropriate kernel function.
- If the input space consists of N-dimension space with N being sufficiently high, the model will generate a high number of support vector which will impact the training speed.
- Scaling of features is required before training of the model is done.
- Space and time complexity for support vector machines is very high.
- In comparison to viewable models like decision trees, the model generated by support vector machines is difficult to interpret.

**End Notes**

Support vector machines can produce models that are robust as well as accurate even in scenarios where the input dataset is non-monotonous or is not linearly separable. Thus, they are convenient to use. Since the data is separated linearly, thus they don’t need human expertise for training. These days there are a number of tools available for implementing support vector machines and these have shown remarkable results in text classification and allied stuff.

Please leave the query and comments in the comment section.

A dynamic, highly professional, and a global online training course provider committed to propelling the next generation of technology learners with a whole new way of training experience.

AWS

- AWS & Fundamentals of Linux
- Amazon Simple Storage Service
- Elastic Compute Cloud
- Databases Overview & Amazon Route 53

DevOps

- Intro to DevOps
- GIT and Maven
- Jenkins & Ansible
- Docker and Cloud Computing

Data Science

- Data Science Introduction
- Hadoop and Spark Overview
- Python & Intro to R Programming
- Machine Learning

Hadoop

- Architecture, HDFS & MapReduce
- Unix Shell & Apache Pig Installation
- HIVE Installation & User-Defined Functions
- SQOOP & Hbase Installation

Salesforce

- Salesforce Configuration Introduction
- Security & Automation Process
- Sales & Service Cloud
- Apex Programming, SOQL & SOSL

QA

- Introduction and Software Testing
- Software Test Life Cycle
- Automation Testing and API Testing
- Selenium framework development using Testing

Business Analyst

- BA & Stakeholders Overview
- BPMN, Requirement Elicitation
- BA Tools & Design Documents
- Enterprise Analysis, Agile & Scrum

MS SQL Server

- Introduction & Database Query
- Programming, Indexes & System Functions
- SSIS Package Development Procedures
- SSRS Report Design

Python

- Features of Python
- Python Editors and IDEs
- Data types and Variables
- Python File Operation

Artificial Intelligence

- Components of AI
- Categories of Machine Learning
- Recurrent Neural Networks
- Recurrent Neural Networks

Machine Learning

- Introduction to Machine Learning & Python
- Machine Learning: Supervised Learning
- Machine Learning: Unsupervised Learning

Tableau

- Introduction to Tableau Desktop
- Data Transformation Methods
- Configuring tableau server
- Integration with R & Hadoop

- Salesforce
- Azure
- QA Testing
- SQL Server
- Business Analyst
- Hadoop
- AWS
- DevOps
- Data Science
- Java
- Digital Marketing
- Dotnet
- PMP
- Selenium
- Worth To Visit
- Machine Learning
- Python
- Oracle DBA
- Data Analyst
- Tableau
- Six Sigma
- Scrum Master
- Blockchain
- Artificial Intelligence
- Android
- Cyber Security
- VMware
- Online IT Training

Search Posts

Trending Posts

Related Posts

Receive Latest Materials and Offers on **Data Science Course**

**Interviews**

- Business Analyst Interview Questions
- DevOps Interview Questions
- AWS Interview Questions
- QA Testing Interview Questions
- Software Testing Interview Questions
- SQL Interview Questions
- Salesforce Interview Questions
- Java Interview Questions
- Hibernate Interview Questions
- Spark Interview Questions
- Vmware Interview Questions
- Data Science Interview Questions
- Digital Marketing Interview Questions
- API Testing Interview Questions
- SSAS Interview Questions
- Power BI Interview Questions
- Cloud Computing Interview Questions
- SSRS Interview Questions
- Manual Testing Interview Questions
- Social Media Interview Questions
- Performance Testing Interview Questions
- MSBI Interview Questions
- QTP Interview Questions
- Automation Testing Interview Questions
- SSIS Interview Questions
- GIT Interview Questions