Independence Day Offer - FLAT 30% OFF | JTINFLAT30

- Data Science Blogs -

What do you Understand by AutoEncoders (AE)?



Introduction

Autoencoders are a particular kind of feed-forward neural systems where the input is equivalent to the output. They pack the input to a lower-dimensional code and afterward reproduce the output from this portrayal. The code is a smaller “summary” of the input, likewise called the latent space representation. Autoencoders are trained similarly like ANNs.                                      

Autoencoder, by configuration, reduces information measurements by figuring out how to ignore the noise in the information. An encoder consist of three components and these components are as follows:

  • Encoder
  • Code
  • Decoder

 For the construction of an autoencoder, we require 3 things: an encoding strategy, decoding strategy, and a loss function which is used to compare the output and the objective.

Data Science Training - Using R and Python

  • Learn from the videos anytime anywhere
  • Pocket-friendly mode of learning
  • Complimentary eBook available
  • Discount Voucher on Live-class

Autoencoders are chiefly a dimensionality reduction calculation with two or three significant properties:

  1. Data specific: Autoencoders are just ready to genuinely compression data like what they have been prepared on. Since they learn highlights specific for the given training data, they are unique with a standard information compression algorithm like gzip. So we can't expect an autoencoder prepared on manually written digits to compress landscape photographs. 
  2. Lossy: The output of the autoencoder won't be equivalent to the data, it will be a nearby however corrupted representation. If you need lossless compression they are not the best approach. 
  3. 3Unsupervised: To prepare an autoencoder we don't have to do anything extravagant, simply throw the raw input data at it. Autoencoders are viewed as an unsupervised learning strategy since they don't require express labels to prepare on. But to be more precise, they are self-supervised because they create their label from the training data.

Architecture:

Both the encoder and decoder are completely associated with feed-forward neural systems, basically the ANNs. Code is a single layer of an ANN with our preferred dimensionality. The number of nodes in the code layer or can say code size is a hyperparameter that we set before preparing the autoencoder. This is a progressively visualized representation of an autoencoder. First, the data goes through the encoder, which is a completely associated ANN, to create the code. The decoder, which has the comparative ANN structure, at that point creates the input by just utilizing the code. The objective is to get an output indistinguishable with the data. Note that the decoder design is the identical representation of the encoder. This isn't a necessity however it's ordinarily the situation. The main prerequisite is the dimensionality of the data and output should be the equivalent. Anything in the center can be played with. 

Data Science Training - Using R and Python

  • Personalized Free Consultation
  • Access to Our Learning Management System
  • Access to Our Course Curriculum
  • Be a Part of Our Free Demo Class

There are four hyperparameters that we need to set before training an autoencoder:

  1. Code Size: It can be explained as the number of nodes in the center layer. Smaller code size results in more compression. 
  2. The number of layers: It can be as deep as we want.
  3. The number of nodes in each layer: The number of nodes per layer decreases with each layer of encoder and vice versa happens in the decoder.
  4. Loss Function: We can use any of the methods either mean squared error or binary cross-entropy.

Types of autoencoders.

There are mainly 6 types of autoencoders

  • Denoising autoencoder: Denoising autoencoders make a ruined duplicate of the input by presenting some noise. These autoencoders take a somewhat corrupted input while preparing to recover the first undistorted information.
  • Sparse Autoencoder: Sparse autoencoders have hidden nodes more noteworthy than input nodes. They can, in any case, find significant highlights from the data. Their limitations were shown on the hidden layer.
  • Deep Autoencoder: Deep Autoencoders comprise of two indistinguishable deep conviction networks, one system for encoding and another for decoding.
  • Contractive Autoencoder: The target of a contractive autoencoder is to have a strong learned representation which is less delicate to little variation in the data.
  • Under complete Autoencoder: The target of under complete autoencoder is to catch the most significant highlights present in the data. Under complete autoencoders have a littler measurement for the hidden layer contrasted with the input layer.
  • Variational Autoencoder: Variational autoencoder models make solid suspicions concerning the appropriation of latent factors. They implement a variational approach for latent representation learning, which brings about a loss component and a particular estimator for the algorithm training called the Stochastic Gradient Variational Bayes estimator.

Working of autoencoders

Autoencoders are found out naturally from information models. It particulates that there is anything but difficult to prepare particular examples of the calculation that will perform well on a particular sort of information and it doesn't require any new technology, just the preparing information which fits the autoencoder.

Data Science Training - Using R and Python

  • No cost for a Demo Class
  • Industry Expert as your Trainer
  • Available as per your schedule
  • Customer Support Available

Since autoencoders consist of encoder and decoder both. The former one is used to transform the input by generating codes which are crisp and short from the high dimensional image whereas the latter one is used to transform the generated code into the high dimensional image.

When one is working with autoencoders, one must have known the dimensionality of the image. For a p-dimensional code, the encoder will be:

autoencoder2.png

And the decoder will be:

autoencoder3.png

On combining both encoder and decoder, full autoencoder will be:

autoencoder4.png

Autoencoders are prepared to save however much data as could be expected when information is gone through the encoder and afterwards the decoder, but at the same time are prepared to cause the new portrayal to have different decent properties. Various types of autoencoders intend to accomplish various types of properties. 

Implementation

Importing libraries:

from keras.layers import Input, Dense

from keras.models import Model

from keras.datasets import mnist

import numpy as np

import matplotlib.pyplot as plt

  • Loading and partitioning of dataset into train and test set respectively.
(X_train, _), (X_test, _) = mnist.load_data()

X_train = X_train.astype('float32') / 255.

X_test = X_test.astype('float32') / 255.

X_train = X_train.reshape((X_train.shape[0], -1))
X_test = X_test.reshape((X_test.shape[0], -1))
  • Designing the encoder
INPUT_SIZE = 784 

 ENCODING_SIZE = 64 

 input_img = Input(shape=(INPUT_SIZE,)) 

 encoded = Dense(ENCODING_SIZE, activation='relu')(input_img) 

 decoded = Dense(INPUT_SIZE, activation='relu')(encoded) 

 autoencoder = Model(input_img, decoded) 
  • Training and building our model using ADAM optimizer and mean squared error loss.
autoencoder.compile(optimizer='adam', loss='mean_squared_error')
autoencoder.fit(X_train, X_train, epochs=50, batch_size=256, shuffle=True, validation_split=0.2)
  • Decoding the test set to look the compression clarity.
decoded_imgs = autoencoder.predict(X_test)
  • Visualizing values using Matplotlib
plt.figure(figsize=(20, 4)) 

 for i in range(10): 

     # original 

     plt.subplot(2, 10, i + 1) 

     plt.imshow(X_test[i].reshape(28, 28)) 

     plt.gray() 

     plt.axis('off') 

     # reconstruction 

     plt.subplot(2, 10, i + 1 + 10) 

    plt.imshow(decoded_imgs[i].reshape(28, 28)) 

     plt.gray()  

     plt.axis('off') 

 plt.tight_layout() 
 plt.show()

Usage of autoencoders.

There are several usages of autoencoders, some major usages are:

  • Colouring of images

Autoencoders is used for colouring of any black and white picture. Depending on the picture it can be easily identified that which colour should be used for colouring.

  • Dimensionality Reduction

Autoencoders are used to reconstruct a given image by reducing its dimensions. Dimensionality reduction using autoencoders results from a similar image with reduced pixel values. Loss of picture information is almost minimal.

  • Removing noises from images.

Autoencoders are used to remove noises from input images. In this process, the original input image was reconstructed from the noisy image.

  • Feature Variation

One can extract the required feature of an image using autoencoder and generates the desired output by eliminating the noise or unnecessary interruption from the image.

  • Removing watermarks from images.

Using autoencoders, one can also remove the watermarks present in an image.

Advantages of autoencoders:

  • In general, autoencoders provide you with multiple filters that can best fit your data.
  • Autoencoders also improves the performance of the data in some case.
  • Autoencoders provides you with a model based on your data rather than providing a predefined filter.

Data Science Training - Using R and Python

  • Detailed Coverage
  • Best-in-class Content
  • Prepared by Industry leaders
  • Latest Technology Covered

Disadvantages of autoencoders:

  • The only downside that is there with autoencoder is the additional computation time.

Conclusion:

Autoencoders are a valuable dimensionality reduction technique. They are well known as an encouraging material in early on deep learning courses, in all likelihood because of their straightforwardness. An autoencoder is a neural system design fit for finding structure inside information so as to build up a compressed representation of the input. A wide range of variations of the general autoencoder design exist with the objective of guaranteeing that the compressed representation speaks to important properties of the first data input; commonly the greatest test when working with autoencoders is getting your model to really gain proficiency with a significant and generalizable latent space representation. Since autoencoders figure out how to compress the data-dependent on traits (i.e. relationships between the info include vector) found from data during preparing, these models are commonly just fit for remaking information like the class of perceptions of which the model saw during preparing.


    Janbask Training

    A dynamic, highly professional, and a global online training course provider committed to propelling the next generation of technology learners with a whole new way of training experience.


Comments

Trending Courses

AWS

  • AWS & Fundamentals of Linux
  • Amazon Simple Storage Service
  • Elastic Compute Cloud
  • Databases Overview & Amazon Route 53

Upcoming Class

1 day 13 Aug 2020

DevOps

  • Intro to DevOps
  • GIT and Maven
  • Jenkins & Ansible
  • Docker and Cloud Computing

Upcoming Class

8 days 20 Aug 2020

Data Science

  • Data Science Introduction
  • Hadoop and Spark Overview
  • Python & Intro to R Programming
  • Machine Learning

Upcoming Class

5 days 17 Aug 2020

Hadoop

  • Architecture, HDFS & MapReduce
  • Unix Shell & Apache Pig Installation
  • HIVE Installation & User-Defined Functions
  • SQOOP & Hbase Installation

Upcoming Class

3 days 15 Aug 2020

Salesforce

  • Salesforce Configuration Introduction
  • Security & Automation Process
  • Sales & Service Cloud
  • Apex Programming, SOQL & SOSL

Upcoming Class

2 days 14 Aug 2020

QA

  • Introduction and Software Testing
  • Software Test Life Cycle
  • Automation Testing and API Testing
  • Selenium framework development using Testing

Upcoming Class

5 days 17 Aug 2020

Business Analyst

  • BA & Stakeholders Overview
  • BPMN, Requirement Elicitation
  • BA Tools & Design Documents
  • Enterprise Analysis, Agile & Scrum

Upcoming Class

2 days 14 Aug 2020

MS SQL Server

  • Introduction & Database Query
  • Programming, Indexes & System Functions
  • SSIS Package Development Procedures
  • SSRS Report Design

Upcoming Class

2 days 14 Aug 2020

Python

  • Features of Python
  • Python Editors and IDEs
  • Data types and Variables
  • Python File Operation

Upcoming Class

5 days 17 Aug 2020

Artificial Intelligence

  • Components of AI
  • Categories of Machine Learning
  • Recurrent Neural Networks
  • Recurrent Neural Networks

Upcoming Class

1 day 13 Aug 2020

Machine Learning

  • Introduction to Machine Learning & Python
  • Machine Learning: Supervised Learning
  • Machine Learning: Unsupervised Learning

Upcoming Class

4 days 16 Aug 2020

Tableau

  • Introduction to Tableau Desktop
  • Data Transformation Methods
  • Configuring tableau server
  • Integration with R & Hadoop

Upcoming Class

6 days 18 Aug 2020

Search Posts

Reset

Receive Latest Materials and Offers on Data Science Course

Interviews