What is Neural Network in Data Science?

Introduction

Deep learning is a hot technology nowadays. Why is this so special and what makes it unique from other aspects of machine learning? We need to learn basic concepts to get answers to these questions.

A neural network is an integration of basic elements neurons. Another term for neural network is artificial neural network, which is made of artificial neurons. The neural network can be connected to a biological neural network   made up of real biological neurons and is used for solving artificial intelligence use cases. The biological neurons are connected, and the connection is represented as weights.

Deep Learning and Neural Network's main use cases as self-driving cars, Video Analytics, image recognition in Medical fields, face recognition, Object detection, Voice recognition, etc. 

What is a Neural Network?

Neural Networks or Artificial Neural Network concept has been evolved from the human nervous system works.

Below is the basic structure of Neuron.

Below are major components of the neuron:

• Dendrites- It takes input from other neurons in the pattern of an electrical impulse

• Cell Body– It accepts the inputs and decide what action to take

• Axon terminals– this generates outputs in the form of electrical impulse

ANN also does the processing in the same way. Its general flow diagram can be as below.

This figure depicts a typical neural network with the working of a single neuron explained separately. Let’s understand this.

Read: Introduction of Decision Trees in Machine Learning

Dendrites are input for neurons. Neuron collates all the inputs and performs an operation on them. In the end, it transmits the output to all other neurons of further next layer. Neural Network structure can be divided into 3 layers.

• Input Layer: The Input observations are injected through these neurons
• Hidden Layers: These are the intermediate layers between the input and final output layers. These layers can be more than one. These hidden layers help to learn inherent relationships. Starting the hidden layer is more important and extracts more significant relationships.
• Output Layer: This is the final layer of the neural network which gives classification results.

Single Layer Perceptron (SLP)

A single layer perceptron has one layer of weights connecting the inputs and output. It is the simplest kind of feed-forward network. In this network, the information always flows in the forward direction.

Multi-Layer Perceptron Basics

A multilayer perceptron is a subclass of feedforward artificial neural network. Multilayer perceptrons are also referred to as "vanilla" neural networks if they have only one hidden layer.

A multilayer perceptron (MLP) has one input layer, one output payer and there can be one or more hidden layers. Each layer has many neurons that are interconnected with each other by some weights. On a broad level, it is good to have a count of neurons in the input layer as a number of features in the dataset, while neurons in the output layer will be the number of classes in the dependent feature.

Types of Activation functions

Activation functions introduce non-linearity into the output. In the Neural network, the sum of products of inputs(X) and their corresponding Weights(W) is calculated and the Activation function f(x) is applied over it to get the output of that layer and feed it as an input to the next layer.

A neural network without an activation function is simply just a linear regression model whose usage is very limited power and can solve only simple use cases. Neural Network not just learns and computes a linear function but expected to do something more complicated(images, videos, audio, speech data) analysis.

Some commonly used Activation functions

• Sigmoid or Logistic
• Tanh - Hyperbolic tangent
• Relu -Rectified linear units

Sigmoid Activation function: It is an activation function with formula f(x) = 1 / 1 + exp(-x) The sigmoid transformation generates a range between 0 and 1. It is a S-shaped curve. It is easy to understand and apply but it has below issues which made its usage a bit limited.

Vanishing gradient problem (Its gradient is low so when multiplied with weight, weights become low passing through different hidden layers)

Read: SQL- A Leading Language for Data Science Experts

its output isn’t zero centered. That’s why gradient updates go too far in different directions.

Vanishing gradient problem (Its gradient is low so when multiplied with weight, weights become low passing through different hidden layers)

its output isn’t zero centered. 

Hyperbolic Tangent function- Tanh: This function enables activation functions to range from -1 to +1. It is a shifted version of the sigmoid function. Tanh functions always perform better than sigmoid function.

f(x) = tanh(x) = 2/(1 + e-2x) - 1

ReLU- Rectified Linear units: The full form for this activation function is rectified linear unit. This is one of the most popular activation functions. One condition is that this can be implemented in only hidden layers of network. As per study using this function has given six times improvement in convergence compared to Tanh function. The major benefit of this is that it avoids vanishing gradient problems.

The output of the function is X for all values of  X>0 and 0 for all values of X<=0. The function looks like this:

SoftMax Function: SoftMax activation functions usually used in the last output layer for architecture which gives final classification output. It is similar to the sigmoid function, the only difference is being that the outputs are normalized, to sum up to 1. 

It is commonly used for multiple class classification problem statements. The SoftMax function compresses the outputs for each class and it gives probabilities for each class in the target field.

Cost Function – Cost function is used to increase the accuracy and performance of the neural network model. The cost function increases the performance of the model by penalizing the network when it makes errors. So the target should be to reduce the cost function which will reduce error value and hence prediction accuracy will be increased. The cost function is like a mean squared error.

Learning Rate – Learning rate basically regulates the speed of the training network and how soon loss can be optimized. We can say the learning rate is defined as the amount of reduction in the cost function in each iteration. This learning rate is like tuning hyperparameters for designing and an optimizing network. The learning rate is an important configuration hyperparameter that can be tuned for training neural network models. It has a positive value normally between 0 and 1. We should select the optimum learning rate value. Its value should not be very high so that the optimal solution is passed and neither value should be very low so that a lot of time is required to converge the network.

 

Read: Data Science Career Path: Your Roadmap to Become Data Scientist Pro

Epochs – As we know we don’t all input data in one single iteration. We pass in many batches for more stable training. One complete training iteration for all batches in both forward and backpropagation is called epochs. One epoch is a single forward and backwards pass of the entire dataset. Batch-size must be less than a number of samples in the training dataset.

Generally, networks with a higher number of epochs would give a more accurate model, but the high value of epoch may force networks to converge in a long time also the large value of epoch may cause overfitting also.

Dropout – Dropout layer is placed in the neural network to tackle overfitting. It is a type of regularization technique. In the Dropout layer, few neurons in the hidden layer are dropped in a random fashion. So, there will be different network combinations with a different random neuron connected. This can be thought of as different training networks as resembling. So there will be different network architectures where the output of multiple networks is then used to produce the final output. So the Dropout layer implements an ensembling algorithm.

Neural Networks and Deep Learning

If the Neural network has many hidden layers, dropout layers then training on input data goes deeper layers and this architecture is referred to as learning. If input datasets size is high for example in the case of image data, deep neural network is advantageous because it can process more complex information quickly and find complex relationships in data.

Applications of Neural Networks

Neural networks cannot learn anything an intelligent human could not theoretically learn given enough time from the complex data. For example LinkedIn, for instance, neural networks along with linear text classification models to detect spam toxic content in their live feed LinkedIn also uses neural nets to different content categories posted on LinkedIn — for example whether it’s news articles or jobs or online classes. This content categorization can be used for Building live recommender systems. Here are further current examples of NN business applications:

• Medical: Some of the NN use cases in the medical field are Lung cancer classification, Cancer cell analysis, Breasts cancer classification, ECG and EEG analysis, and building smart hospital systems
• Character Recognition –character recognition has become very important and it's used in many daily life applications. Neural networks can be used to recognize handwritten characters. Some use cases are Meter reading(reading readings using Mobile OCR), Voucher Scanning and Promotional Code Scanning, Mobile OCR used in Roads, Streets, Parking Validation, Ticket Id validation of Events, Security Access control, Passport Control in Airports.
•  Image Compression –Internet usage is increasing day by day and many sites are using images a lot which causes memory, storage issues. So neural networks for image compression are worth a look.
• Securities: Some of the use cases are Automatic bond rating, Stock value prediction, and stock trading advisory systems
• Financial: Corporate bond ratings, corporate financial analysis, credit line use analysis, currency price prediction, loan advising, mortgage screening, real estate appraisal, and portfolio trading
• Banking: Credit card attrition, Payment transaction Frauds prediction, credit and loan application evaluation, repayment prediction, fraud, risk evaluation and loan delinquencies

Advantages of Neural Networks

• Fault tolerance capability: Even if any cell/neuron of the network failed, still it gives output. So neural network comes with fault-tolerant architecture.
• Distributed Memory: Neural Network provides parallel processing capability due to it’s distributed memory capability. 
• Parallel processing ability: Neural Network compute more than one job simultaneously. Because of having parallel processing attribute.
• Missing values handling: Neural Network can produce good results to a certain limit even with missing data in records. But definitely with missing values treatment accuracy can be improved.

Disadvantages: Neural Networks

• Building Network Architecture: There is no theoretical or fixed rule for defining network architecture. Optimized network structure can be achieved through hit and trial, experience
• Data Type Constraints: ANN can take only numeric input. Non-numerical values have to be translated into numerical values before they are passed into ANN architecture.
• Hardware Requirements: Since Artificial neural networks provide parallel processing so there is a dependency on having processors with parallel processing power. Also, for big data of images normal CPU cannot work so GPU is required for processing very big data That’s why there is hardware dependency
• Blackbox Architecture: ANN is basically Black Box So nobody knows how ANN is predicting results and what is the logic behind this. This reduces trust in the network. 

Conclusion

With this blog, we have understood the concept of a series of networks, that is a neural network. Hope you will now be able to perform time-series predictions, anomaly detection, and understand natural language processing. Please let us know in the comment section below if you have more questions.

FaceBook Twitter Google+ LinkedIn Pinterest Email