Support Vector Machines Tutorial on SVM Tutorial

SVMs - An overview of Support Vector Machines

Sun, 26 Feb 2017 18:44:28 +0000

Today we are going to talk about SVMs in general.

I recently received an email from a reader of my serie of articles about the math behind SVM:

I felt I got deviated a lot on Math part and its derivations and assumptions and finally got confused what exactly SVM is ? And when to use it and how it helps ?

Here is my attempt to clarify things.

What exactly is SVM ?

SVM is a supervised learning model

It means you need a dataset which has been labeled.

Support Vector Machines Succinctly released

Tue, 24 Oct 2017 20:01:27 +0000

My ebook Support Vector Machines Succinctly is available for free.

About Support Vector Machines Succinctly

While I was working on my series of articles about the mathematics behind SVMs, I have been contacted by Syncfusion to write an ebook in their “Succinctly” e-book series. The goal is to cover a particular subject in about 100 pages. I gladly accepted the proposition and started working on the book.

SVM - Understanding the math - Part 1 - The margin

Sun, 02 Nov 2014 19:04:29 +0000

Introduction

This is the first article from a series of articles I will be writing about the math behind SVM. There is a lot to talk about and a lot of mathematical backgrounds is often necessary. However, I will try to keep a slow pace and to give in-depth explanations, so that everything is crystal clear, even for beginners.

If you are new and wish to know a little bit more about SVMs before diving into the math, you can read the article: an overview of Support Vector Machine.

How to prepare your data for text classification ?

Sun, 19 Oct 2014 19:30:17 +0000

Understand how data is represented

For text classification, you often begin with some text you want to classify. For instance, you have quotes and wants to find the quotes about love. Or you have emails and you want to separate spam from legitimate emails.

SVM is a supervised-learning algorithm. It means you will need to manually label some data with what you think is the correct choice. Then you train a SVM model with it. Eventually you can use it to predict unlabeled data.

A retrospective of SVM Tutorial and some new content

Mon, 28 Sep 2020 08:00:00 +0000

Almost six years ago, on October 11, 2014, I published my first article about Support Vector Machines. It was a short article describing how to classify text in C#. My original goal was to create a blog containing a series of tutorials about Support Vector Machines, hence the name of the site: svm-tutorial.

Early days 2014-2015

As I started using this machine learning algorithm, something bothered me. I felt like I could not find any “simple” explanation of the topic. Of course, there were already some resources online, but most of them were talking to a reader with deep mathematical knowledge. It became my goal to explain support vector machines in an approachable way so that anyone who takes the time to read about them can understand them.

SVM - Understanding the math - Part 2

Sun, 09 Nov 2014 19:00:17 +0000

This is Part 2 of my series of tutorial about the math behind Support Vector Machines.
If you did not read the previous article, you might want to start the serie at the beginning by reading this article: an overview of Support Vector Machine.

In the first part, we saw what is the aim of the SVM. Its goal is to find the hyperplane which maximizes the margin.

Linear Kernel: Why is it recommended for text classification ?

Sun, 19 Oct 2014 19:42:11 +0000

The Support Vector Machine can be viewed as a kernel machine. As a result, you can change its behavior by using a different kernel function.

The most popular kernel functions are :

the linear kernel
the polynomial kernel
the RBF (Gaussian) kernel
the string kernel

The linear kernel is often recommended for text classification

It is interesting to note that :

The original optimal hyperplane algorithm proposed by Vapnik in 1963 was a linear classifier [1]

SVM - Understanding the math - the optimal hyperplane

Mon, 08 Jun 2015 17:35:43 +0000

This is the Part 3 of my series of tutorials about the math behind Support Vector Machine.

If you did not read the previous articles, you might want to start the serie at the beginning by reading this article: an overview of Support Vector Machine.

What is this article about?

The main focus of this article is to show you the reasoning allowing us to select the optimal hyperplane.

Here is a quick summary of what we will see:

How to classify text using SVM in C#

Sun, 19 Oct 2014 19:34:07 +0000

SVM Tutorial : Classify text in C#

In this tutorial I will show you how to classify text with SVM in C#.

The main steps to classify text in C# are:

Create a new project
Install the SVM package with Nuget
Prepare the data
Read the data
Generate a problem
Train the model
Predict

Step 1: Create the Project

Create a new Console application.

SVM - Understanding the math - Unconstrained minimization

Sun, 11 Sep 2016 18:31:32 +0000

This is the Part 4 of my series of tutorials about the math behind Support Vector Machines.
Today we are going to learn how to solve an unconstrained minimization problem.

If you did not read the previous articles, you might want to start the serie at the beginning by reading this article: an overview of Support Vector Machine.

About this Part

It took me a while to write this article because the subject is vast and assume a lot of prior knowledge. What should I explain and what should I skip was kind of a hard line to trace. After a while, I ended up with a large Part 4 which was too long to read. So I decided to split it. Welcome, Part 4, Part 5 and Part 6!

SVM Tutorial: How to classify text in R

Sun, 23 Nov 2014 17:08:24 +0000

In this tutorial I will show you how to classify text with SVM in R.

The main steps to classify text in R are:

Create a new RStudio project
Install the required packages
Read the data
Prepare the data
Create and train the SVM model
Predict with new data

Step 1: Create a new RStudio Project

To begin with, you will need to download and install the RStudio development environment.

SVM - Understanding the math - Convex functions

Sun, 11 Sep 2016 18:42:52 +0000

This is the Part 5 of my series of tutorials about the math behind Support Vector Machines.
Today we are going to study convex functions.

If you did not read the previous articles, you might want to start the serie at the beginning by reading this article: an overview of Support Vector Machine.

How can we find a global minimum?

There is one simple way to find the global minimum:

Find all the local minima
Take the smallest one; it is the global minimum.

Another approach is to study the function we are trying to minimize. If this function is convex, then we are sure its local minimum is a global minimum.

Support Vector Regression with R

Thu, 23 Oct 2014 00:35:54 +0000

In this article I will show how to use R to perform a Support Vector Regression.
We will first do a simple linear regression, then move to the Support Vector Regression so that you can see how the two behave with the same data.

A simple data set

To begin with we will use this simple data set:

I just put some data in excel. I prefer that over using an existing well-known data-set because the purpose of the article is not about the data, but more about the models we will use.

SVM - Understanding the math - Duality and Lagrange multipliers

Sun, 11 Sep 2016 18:47:54 +0000

This is the Part 6 of my series of tutorials about the math behind Support Vector Machines.
Today we will learn about duality, optimization problems and Lagrange multipliers.

If you did not read the previous articles, you might want to start the serie at the beginning by reading this article: an overview of Support Vector Machine.

Duality

In mathematical optimization theory, duality means that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem (the duality principle). The solution to the dual problem provides a lower bound to the solution of the primal (minimization) problem. (Wikipedia)