Black Friday Deal : Up to 40% OFF! + 2 free self-paced courses + Free Ebook  - SCHEDULE CALL

- Python Blogs -

Naive Bayes: An Easy To Interpret Classifier

Naive Bayes is one of the simplest methods to design a classifier. It is a probabilistic algorithm used in machine learning for designing classification models that use Bayes Theorem as their core. Its use is quite widespread especially in the domain of Natural language processing, document classification and allied.

Why call these as Naive?

The term naïve in the name tells one of the basic assumptions in these classifiers that the input features are completely independent of each other. In other words, there will be no implicit change in the input features when one or more than one input parameter is changed, explicitly.

Naïve Bayes is an extremely popular algorithm owing to its probabilistic nature which provides it a significant advantage like fast-paced predictions, easy codes over other algorithms. This makes this model highly scalable.

Why call these as Naive?

Naïve Bayes classifier uses conditional probability and number probability distributions to train the machine. Thus, it becomes important to know the following points:

A). Conditional Probability:

In the domain of probability theory, conditional probability is the measure of an event A occurring when another event say B has taken place. This is represented by and is read as “the conditional probability of A given B”.

One of the classical example in the domain of conditional probability is flipping the coin. SO, while flipping a fair coin, the chances of having a head or tail are equal. IN other words, the probability of any of the events is 0.5. So, conditional probability talks about the chances of having a head once we already had a tail. In this case, theoretically, it remains at 0.5 as well. Bayes theorem provides a mathematical model for calculating these.

B). Bayes Theorem:

It is also sometimes called as the god’s theorem. It describes the probability of occurrence of an event, based upon the existing knowledge about the conditions that are related to that specific event. Say, diabetes happens at some particular age X, and then by using the Bayes theorem, the age of a person can be used to forecast the chances that they will have diabetes and the results will be much better as compared to a situation when we had no idea about their age.

Mathematically, Bayes’ theorem is given by the following equation:

Read: A Comprehsnive Python Tutorial for Beginners and Professionals

Bayes Theorem

The Naïve Bayes:

The Bayes theorem as depicted in figure (1), shows that it can reflect the reverse of an event. Just the event shown should be known. When the Bayes rule in applied to a set of events that are completely independent, the resulting model is called naïve Bayes.

A and B in the standard formula can be replaced with X and Y where X is the independent variable and Y is the dependent variable.

Now for non-linear cases of X where more than one instance of the same exists, the instance can be reduced to a linear instance by simply each instance to be a separate instance and applying the Bayes rule:

Naïve Bayes

As a comparison of equation (3) and (4) depicts, it can be inferred from these that the likelihoods of all the X’s can be multiplied and is called the probability of likelihood of evidence. This can be known from the training dataset by filtering records where .

The multiplicative term with the probability of likelihood of evidence is called the prior which depicts the overall probability of Y= q, where q is the class label of Y.

C). Types of Naïve Bayes:

As can be observed from the above equations, the Naïve Bayes theorem only depicts the probability. Hence, only in its nascent capacity, it cannot become a classifier. Thus, it is always used in combination with a probability distribution for designing the classifier. Most commonly used naïve Bayes based classifier is the following:

Types of Naïve Bayes

Read: Top 10 Python Libraries For Machine Learning

1). Gaussian naïve Bayes:

A Gaussian naïve Bayes uses Gaussian probability distribution for designing and is used to deal with continuous data. A Gaussian distribution is defined as:

Gaussian naïve Bayes

2). Multinomial  Naïve Bayes:

If the dataset set consists of probability in terms of frequency instead of continuous data Gaussian distribution cannot be applied. In this situation, Multinomial distribution is used.

3). Bernoulli Bayes:

In case the features are binary in nature, the Bernoulli naïve Bayes are utilized. It is very popular in case of documentation when the feature space is binary.

Designing a Naïve Bayes based classifier:

In this blog, for designing a naïve Bayes classifier, sklearn is utilized with python. A classical iris flower dataset is utilized. Details of the dataset can be found here.

First of all, libraries need to be imported. The commands for the same are:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB

Loading the dataset and splitting it into training and testing set

Read: Upgrade Your Programming Career by Learning Python & Getting Certified
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=0)

Training the model:

gnb = GaussianNB()

If one wishes to use any other model from the domain of Naïve Bayes based classifier. Then, that needs to be specified here.

Querying the query set and checking the results:

y_pred = gnb.fit(X_train, y_train).predict(X_test)
print("Number of mislabeled points out of a total %d points : %d" % (X_test.shape[0], (y_test != y_pred).sum()))

This query, in this case, will give the output as

Number of mislabeled points out of a total of 75 points: 4

Thus, out of all the test sets of 75, 4 were incorrect. Giving a final accuracy of almost 99%.

The pros and cons of Naïve Bayes based classifiers:

Pros:

  1. Naïve Bayes based classifiers are efficient in terms of time and space complexity.
  2. When the features follow independence, this classifier outperforms other types of the classifier.
  3. Naïve Bayes works well with continuous as well as categorical data.

Cons:

  1. If there happens to be a new class querying the test data set, that class will be given class probability of 0 and the classifier will be unable to make a classification. This is a typical case and is called “Zero Frequency”.
  2. The other problem is the assumption of independence in the features which give the classifier its name. If the slightest of dependency exists the classifier will not make the best fit. This challenges the concept of naïve as well as becomes interpersonal to the architect of the classifier to be certain about independence.

Uses of Naïve Bayes based classifier:

  1. Text Classification: Classifiers based on Naïve Bayes are mostly used in text classification as they outperform in case of multi-class classification.
  2. Spam filters: Checking for spam is a shows naïve bias as either it is spam or its not. Hence, is a suitable use case for naïve Bayes.
  3. Sentiment analysis: Sentiments like happy, sad, angry are also independent in nature. Thus, making an ideal use case.
  4. Recommendation systems: Naïve Bayes based classifiers are also used in recommendation systems where collaborative filtering or independence in feature space is observed.

Concluding on Naïve Bayes:

IN this blog, Naïve Bayes classifier was covered which belongs to the supervised learning domain of machine learning.  Naïve Bayes based classifier is extremely competent when the condition of independence in the feature space is met. It can be used as per the level of understanding and requirement of the project where the condition of independence is met.

Read: Python String Methods - What should you know?


fbicons FaceBook twitterTwitter lingedinLinkedIn pinterest Pinterest emailEmail

     Logo

    JanBask Training

    A dynamic, highly professional, and a global online training course provider committed to propelling the next generation of technology learners with a whole new way of training experience.


  • fb-15
  • twitter-15
  • linkedin-15

Comments

Trending Courses

Cyber Security Course

Cyber Security

  • Introduction to cybersecurity
  • Cryptography and Secure Communication 
  • Cloud Computing Architectural Framework
  • Security Architectures and Models
Cyber Security Course

Upcoming Class

12 days 14 Dec 2024

QA Course

QA

  • Introduction and Software Testing
  • Software Test Life Cycle
  • Automation Testing and API Testing
  • Selenium framework development using Testing
QA Course

Upcoming Class

-0 day 02 Dec 2024

Salesforce Course

Salesforce

  • Salesforce Configuration Introduction
  • Security & Automation Process
  • Sales & Service Cloud
  • Apex Programming, SOQL & SOSL
Salesforce Course

Upcoming Class

8 days 10 Dec 2024

Business Analyst Course

Business Analyst

  • BA & Stakeholders Overview
  • BPMN, Requirement Elicitation
  • BA Tools & Design Documents
  • Enterprise Analysis, Agile & Scrum
Business Analyst Course

Upcoming Class

11 days 13 Dec 2024

MS SQL Server Course

MS SQL Server

  • Introduction & Database Query
  • Programming, Indexes & System Functions
  • SSIS Package Development Procedures
  • SSRS Report Design
MS SQL Server Course

Upcoming Class

11 days 13 Dec 2024

Data Science Course

Data Science

  • Data Science Introduction
  • Hadoop and Spark Overview
  • Python & Intro to R Programming
  • Machine Learning
Data Science Course

Upcoming Class

4 days 06 Dec 2024

DevOps Course

DevOps

  • Intro to DevOps
  • GIT and Maven
  • Jenkins & Ansible
  • Docker and Cloud Computing
DevOps Course

Upcoming Class

4 days 06 Dec 2024

Hadoop Course

Hadoop

  • Architecture, HDFS & MapReduce
  • Unix Shell & Apache Pig Installation
  • HIVE Installation & User-Defined Functions
  • SQOOP & Hbase Installation
Hadoop Course

Upcoming Class

4 days 06 Dec 2024

Python Course

Python

  • Features of Python
  • Python Editors and IDEs
  • Data types and Variables
  • Python File Operation
Python Course

Upcoming Class

19 days 21 Dec 2024

Artificial Intelligence Course

Artificial Intelligence

  • Components of AI
  • Categories of Machine Learning
  • Recurrent Neural Networks
  • Recurrent Neural Networks
Artificial Intelligence Course

Upcoming Class

12 days 14 Dec 2024

Machine Learning Course

Machine Learning

  • Introduction to Machine Learning & Python
  • Machine Learning: Supervised Learning
  • Machine Learning: Unsupervised Learning
Machine Learning Course

Upcoming Class

25 days 27 Dec 2024

 Tableau Course

Tableau

  • Introduction to Tableau Desktop
  • Data Transformation Methods
  • Configuring tableau server
  • Integration with R & Hadoop
 Tableau Course

Upcoming Class

4 days 06 Dec 2024

Search Posts

Reset

Receive Latest Materials and Offers on Python Course

Interviews