Independence Day Offer - FLAT 30% OFF | JTINFLAT30

- AWS Blogs -

AWS Object Storage and CDN (S3, Glacier and CloudFront)



1. Introduction

Cloud computing needs a robust and reliable storage infrastructure to enable the applications to perform seamlessly. Amazon Cloud (AWS) provides numerous storage-related  services for this purpose. These include:

  • Amazon S3
  • Amazon Glacier
  • Amazon Backup
  • Amazon Elastic Block Store
  • Amazon Elastic File Store 

The stored content also needs to be made available to the end-users  at a fast rate to deliver a pleasant user experience. This is made possible by Amazon’s Content Delivery Network(CDN). It consists of a worldwide network of caching servers that  improve the access time for content download. Amazon’s Content Delivery Network service is known as AWS CloudFront.

AWS Solution Architect Training and Certification

  • Learn from the videos anytime anywhere
  • Pocket-friendly mode of learning
  • Complimentary eBook available
  • Discount Voucher on Live-class

2. Amazon S3

S3 stands for Simple Storage Service. This is the primary AWS object storage service,  It provides 99.999999999% (11 9's) durability and secured access to data. It also supports  data query, data selection, archival and disaster recovery. Data is stored in ‘buckets’. An S3 bucket can be visualized as a Hard drive for objects in the cloud. S3 service can be managed using administrative interfaces such as  Management console, CLI and SDK.

A  A bucket represents an entity of  object storage in AWS. Each bucket can store multiple objects/files.

An S3 bucket can be created from the S3 console.

Once the bucket has been created, it can be populated with content.

As we have seen, S3 can be used for storing content. It can also be used for storing static content for a website. The steps are as follows:

AWS Solution Architect Training and Certification

  • No cost for a Demo Class
  • Industry Expert as your Trainer
  • Available as per your schedule
  • Customer Support Available

  • Select S3 in the AWS Console.
  • Select your bucket.
  • Click the “Properties” section.
  • Click the “Static website hosting” option.
  • Select “Use this bucket to host a website”.
  • Clear ”Block all public access” on Bucket properties.
  • Add a bucket policy to make the content publicly accessible.
  • Enter “index.html” as the Index document for the website.
  • Test your site using URL of the form http://.s3-website-.amazonaws.com

An S3 bucket can be used for storing multiple variants of the object in the AWS object based storage. This is done using versioning. S3 adds a unique version ID to an object when it is added to a versioning-enabled bucket. In one bucket, for example, you can have two objects with the same key, but different version IDs.Different versions of the object can be listed individually using the management console for the object storage in AWS.

order to accomplish fault tolerance and disaster recovery, S3 makes available the mechanism of cross – region replication. Replication enables automatic, asynchronous copying of objects across Amazon S3 buckets. For this, we can add a replication configuration to the source bucket. This provides a number of advantages such as:

AWS Solution Architect Training and Certification

  • Detailed Coverage
  • Best-in-class Content
  • Prepared by Industry leaders
  • Latest Technology Covered

  • Control latency for users in different geographical regions
  • Maintaining object copies
  • Adhere to compliance requirements

The source bucket owner must have the source and destination AWS Regions enabled for their account. Also, both source and destination buckets need to have versioning enabled. 

An object stored in an S3 bucket undergoes various stages during its lifecycle. Lifecycle management defines actions you want S3 to take during these different stages in the object's lifetime.

Each object in Amazon S3 has a storage class associated with it. 

  • Standard – General storage of frequently accessed data
  • Intelligent Tiering – Changing access patterns
  • Standard IA – Infrequently accessed data
  • One Zone IA - Infrequently accessed data
  • Glacier – Data archival
  • Glacier Deep Archive – Deep data archival

Lifecycle policy enables automatic transfer to a different storage class without any changes to your application.

 

S3 is designed to provide 99.999999999% durability and 99.99% availability of objects over a given year. It has been designed to sustain the concurrent loss of data in two facilities

AWS Solution Architect Training and Certification

  • Personalized Free Consultation
  • Access to Our Learning Management System
  • Access to Our Course Curriculum
  • Be a Part of Our Free Demo Class

Objects are redundantly stored on multiple devices across multiple facilities in an Amazon S3 Region. S3 object lock prevents an object from being deleted or overwritten for a specified amount of time. Access Policies are used to control access to buckets by users. Cloudwatch, CloudTrail and Access Logs are used for logging and monitoring of the S3 resources.

3. AWS Glacier

AWS Glacier is a very low cost storage service on Amazon Cloud platform. It is primarily used for data archiving and backup. The archive is the base unit of storage in Glacier(e.g., file, photo, video…).Archive IDs are unique and 138 bytes long. Glacier vault is a container for storing archives. Glacier updates the vault inventory approximately once a day. The management console can be used to create and delete vaults. All other interactions take place using CLI or SDK.

AWS Command Line Interface can be used for managing Glacier vault. For example: 

$ aws glacier initiate-multipart-upload --account-id - --archive-description "multipart upload test" --part-size 1048576 --vault-name myvault

"uploadId": "19gaRezEXAMPLES6Ry5YYdqthHOC_kGRCT03L9yetr220UmPtBYKk-OssZtLqyFu7sY1_lR7vgFuJV6NtcV5zpsJ", x

"location": "/123456789012/vaults/myvault/multipart-uploads/19gaRezEXAMPLES6Ry5YYdqthHOC_kGRCT03L9yetr220UmPtBYKk-OssZtLqyFu7sY1_lR7vgFuJV6NtcV5zpsJ" 

}

$ aws glacier describe-vault --account-id - --vault-name myvault

{

    "SizeInBytes": 3178496,

    "VaultARN": "arn:aws:glacier:us-west-2:123456789012:vaults/myvault",

    "LastInventoryDate": "2018-12-07T00:26:19.028Z",

    "NumberOfArchives": 1,

    "CreationDate": "2018-12-06T21:23:45.708Z",

    "VaultName": "myvault"

}

4. AWS CloudFront

Amazon CloudFront is the content delivery network (CDN) service of Amazon.The CloudFront network has 197 points of presence (PoPs). CloudFront provides low latency and high data transfer speeds.Content to be distributed can be published in origin server ( S3 for static content, EC2 for dynamic content).Origin servers can be registered with Amazon CloudFront through a API call/console. This will return a CloudFront.net domain name ( e.g., test765.cloudfront.net) that can be uses to distribute content via the Amazon CloudFront service.

CloudFront CDN content is organized into distribution where content and delivery properties are configured. The steps to create a distribution are as follows:

  • Log in to the AWS CloudFront console
  • Choose ”Create Distribution”.
  • Specify the distribution properties:
  • Content origin— S3 bucket/MediaPackage channel/HTTP server from which CloudFront gets the files for distribution.
  • Access— Which users/groups have access to the content.
  • Security—e.g., Users must use HTTPS to access your content.
  • Cookie or query-string forwarding—whether you want CloudFront to forward cookies or query strings to your origin.
  • Geo-restrictions—Restrict access in selected geographies.
  • Access logs—Create access logs for analysis.

AWS Object Storage Gateway represents hybrid cloud storage between on-premises environments and the AWS Cloud. AWS object based storageGateway sits between your applications and Amazon storage services.

The gateway connects your applications to AWS storage over standard storage interfaces.

Supports three storage interfaces: file, tape, and volume. 

It provides caching efficient data transfer and integration with AWS monitoring and security services.

If we need to transfer large amounts of data between Amazon S3 and your data center we can consider using AWS Snowball which is an actual physical AWS cloud object  storage device for data transfer.

50 TB Snowball device has 42 TB of usable space. 80 TB Snowball device has 72 TB of usable space.

All jobs for AWS Snowball are created and managed using the AWS Management console or API.

Snowball client is one of the tools that you can use to transfer from your on-premises data source to the Snowball.

Transfer Acceleration enables fast, easy, and secure transfers of files over long distances between your client and an S3 bucket. It makes use of optimized network protocols and the AWS edge infrastructure. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path. Transfer acceleration for S3 buckets can be enabled from the Management console. Transfer speed improvements from 50% - 500% are observed.

5. Conclusion

S3 or Simple Storage Service is the aws cloud object object storage service from Amazon. Buckets are the containers for stored objects like documents, images, video etc.S3 buckets can be created from the management console, CLI or API. Multiple versions of the objects can be stored in the S3 bucket. Buckets can be replicated across different regions for higher resilience and disaster recovery. Data can be automatically moved from one storage class to another as per the defined LifeCycle management policy. A glacier is the data archival service from AWS.S3 supports both Server level and Client level encryption for stored data. Storage gateway acts as the intermediary between your local applications and AWS storage. Snowball is used to transfer huge data volumes from the data center to the S3 cloud and vice versa using the Snowball device. Data transfer to/from S3 buckets can be accelerated using the Transfer Acceleration feature.


    Janbask Training

    A dynamic, highly professional, and a global online training course provider committed to propelling the next generation of technology learners with a whole new way of training experience.


Comments

Trending Courses

AWS

  • AWS & Fundamentals of Linux
  • Amazon Simple Storage Service
  • Elastic Compute Cloud
  • Databases Overview & Amazon Route 53

Upcoming Class

1 day 13 Aug 2020

DevOps

  • Intro to DevOps
  • GIT and Maven
  • Jenkins & Ansible
  • Docker and Cloud Computing

Upcoming Class

8 days 20 Aug 2020

Data Science

  • Data Science Introduction
  • Hadoop and Spark Overview
  • Python & Intro to R Programming
  • Machine Learning

Upcoming Class

5 days 17 Aug 2020

Hadoop

  • Architecture, HDFS & MapReduce
  • Unix Shell & Apache Pig Installation
  • HIVE Installation & User-Defined Functions
  • SQOOP & Hbase Installation

Upcoming Class

3 days 15 Aug 2020

Salesforce

  • Salesforce Configuration Introduction
  • Security & Automation Process
  • Sales & Service Cloud
  • Apex Programming, SOQL & SOSL

Upcoming Class

2 days 14 Aug 2020

QA

  • Introduction and Software Testing
  • Software Test Life Cycle
  • Automation Testing and API Testing
  • Selenium framework development using Testing

Upcoming Class

5 days 17 Aug 2020

Business Analyst

  • BA & Stakeholders Overview
  • BPMN, Requirement Elicitation
  • BA Tools & Design Documents
  • Enterprise Analysis, Agile & Scrum

Upcoming Class

2 days 14 Aug 2020

MS SQL Server

  • Introduction & Database Query
  • Programming, Indexes & System Functions
  • SSIS Package Development Procedures
  • SSRS Report Design

Upcoming Class

2 days 14 Aug 2020

Python

  • Features of Python
  • Python Editors and IDEs
  • Data types and Variables
  • Python File Operation

Upcoming Class

5 days 17 Aug 2020

Artificial Intelligence

  • Components of AI
  • Categories of Machine Learning
  • Recurrent Neural Networks
  • Recurrent Neural Networks

Upcoming Class

1 day 13 Aug 2020

Machine Learning

  • Introduction to Machine Learning & Python
  • Machine Learning: Supervised Learning
  • Machine Learning: Unsupervised Learning

Upcoming Class

4 days 16 Aug 2020

Tableau

  • Introduction to Tableau Desktop
  • Data Transformation Methods
  • Configuring tableau server
  • Integration with R & Hadoop

Upcoming Class

6 days 18 Aug 2020

Search Posts

Reset

Receive Latest Materials and Offers on AWS Course

Interviews