fbpx

get started on cloud now!

AWS Basics

by Mohdqaz

AWS S3

What is AWS S3

What does AWS S3 stands for?

AWS S3 stands for Simple Storage Service. As the name tells us, S3 provides storage to store files in the Cloud (Amazon Cloud).

And it is not only to store normal files, it can also be used as archival and even to host a static website. 

It have unlimited storage and file size can be from 0 byte to 5TB.  

Files are stored in bucket  and we always hear people mention S3 bucket. But what is actually an Amazon S3 bucket? It is just AWS term for folders. Meaning, we are storing our files or S3 objects in folders. 

S3 Objects

S3 is object based where we can imagine that each of the file that we can upload is an object

S3 Objects consists of the following:

  • Key (name of the object)
  • Value (data is made up of a sequence of bytes)
  • Version ID (for versioning)
  • Metadata (data about the data that you are storing)
  • Subresources

S3 Features

S3 has following features (you can click on each of the features to know more details) :

S3 Data Consistency

How does data consistency work for S3? 

  • Read after write Consistency for PUTS of new Objects
    • If the user write a new file & read it immediately, user will be able to read the data  
  • Eventual Consistency for overwrite PUTS and DELETES (will take a bit of time to propagate)
    • If the user update EXISTING file or DELETE a file and read it immediately, user may get either
      • Older version 
      • New version 
      • And will get the new version after few seconds

S3 Guarantees (based on S3 standard)

AWS S3 is built for 99.99% availability for AWS S3 platform

Amazon guarantee 99.9% availability (SLA) for their S3 service

  • What S3 availability means is probability that you will be able to get the content back the moment you try to access it
  • So, 0.1% of the time user cannot retrieve the file is due to few possible reason such as file corruption

Amazon Guarantee durability of 99.999999999% (11 9s if we want to remember this)

  • As for S3 Durability, it means that probability  the user is going to get the file back
  • How does S3 durability being achieved? By storing those objects redundantly across multiple facilities within an S3 region

S3 Storage Classes (S3 Storage Types)

What is S3 Storage classes? It’s type of S3 storage that user can go for according to their requirements. To understand more about the options, below are S3 storage classes explanation:

  • S3 Standard
    • 99.99% availability
    • 99.999999999% durability
    • Stored redundantly across multiple devices in multiple facilities (multiple host in multiple datacenter)
    • Thus can sustain loss of 2 facilities (datacenter) concurrently
  • S3 Standard – IA (infrequently access)
    • Data accesses less frequently (eg: access only once a month)
    • but requires rapid access when needed
    • Lower fee than S3 standard
    • But will be charged a retrieval fee
  • S3 One zone – IA (One Zone Infrequently Access)
    • Lower-cost option for infrequently access data
    • But do not require multiple AZ (Availability zone)
  • S3 – Intelligent tiering
    • Optimized cost by automatically moving data to the most cost efficient access tier
    • Without performance impact or operational overhead
    • Using machine learning
    • Eg: S3 standard can be moved to S3-IA
  • S3 Glacier
    • Secure, durable & low cost storage class for data archiving
    • Reliably store any amount of data at costs that are competitive or cheaper than on-premise solution
    • Retrieval time configurable from minutes to Hours
  • S3 Glacier deep archive
    • Lowest cost storage class
    • REtrieval time of 12 hours is accepted
    • Need to put a request

S3 Storage Classes Comparison

S3 StandardS3 Intelligent TieringS3 Standard-IAS3 One Zone-IAS3 GlacierS3 Glacier Deep Archive
Designed for Durability99.999999999% (11 9's)99.999999999% (11 9's)99.999999999% (11 9's)99.999999999% (11 9's)99.999999999% (11 9's)99.999999999% (11 9's)
Designed for Availability99.99%99.9%99.9%99.5%N/AN/A
Availability SLA99.9%99%99%99%N/AN/A
Availability Zones>= 3>= 3>= 31>= 3>= 3
Minimum capacity charge per objectN/AN/A128KB128KB40KB40KB
Minimum storage duration chargeN/A30 days30 days30 days90 days180 days
Retrieval FeeN/AN/Aper GB retrievedper GB retrievedper GB retrievedper GB retrieved
First byte latencyMillisecondsMillisecondsMillisecondsMillisecondsSelect minutes or hoursSelect hours

AWS S3 Object Lifecycle (Lifecycle Management)

As we have gone through type of storages that AWS is offering, it is now easier to understand what is S3 Object Lifecycle or Lifecycle management means to us. 

S3 Object Lifecycle is set of rules that automate the migration of the Object Storage Class to a different storage class based on specified time intervals. 

For example on the scenario: 

  1. I have a work file that I’m going to access daily for 30 days
    • Usage: Frequent
    • Best storage: S3 standard
  2. After 30 days, I may only need to access the file once a week for the next 60 days
    • Usage: Infrequent access
    • Best storage: S3-IA (Infrequent access)
  3. After 90 days, I probably will never access that file but still want to keep the file just in case somebody ask for it.
    • Usage: Most likely will never need the file
    • Best storage: Glacier 

Although there is a cost being charged if we use lifecycle management, there is also a saving that we can enjoy when implementing it. This is due to the more efficient usage of the storage based on what we required. Keep in mind that S3 standard is the most expensive  and it will be cheaper towards Glacier Deep archive. 

So, why not move the files to the cheaper storage if we can save some money from there. 

Difference between S3 Object Lifecycle and S3 Intelligent tiering

After understanding what is S3 Object Lifecycle and storage class particularly S3 Intelligent tiering, one may ask “what is the difference between those two feature then?”

Actually what makes those two differs from each other are:

  • S3 Intelligent tiering only support moving object from S3 Standard to S3 Standard – IA
  • Whereas Lifecycle Management support all S3 storage class

This might change in the future, so stay tune for future updates from AWS. 

S3 Version Control 

S3 versioning with S3 

  • Stores all versions of an object (writes / delete)  
  • Can be used as Backup tool as we can easily revert back to the previous version of the object
  • Once enabled, Versioning cannot be disabled!! >> only suspended
    • You can only delete the bucket and create a new one if you don’t want versioning anymore 
  • Integrates with Life cycle rules 
  • Versioning have MFA Delete capability
    • Use MFA to prevent accidentally deletion. This will become another layer of security 
  • When we enable versioning, please remember that it will incur some cost as each of the version will add additional size to the object

S3 Encryption

It is really important to secure data that we transfer to and from AWS S3. 

This is to prevent any data that we stored to fall into wrong hands such as hackers. 

There are two scenarios that we need to take into consideration when protecting our data: 

  1. Data while in-transit (Encryption in transit)
    • Data as it travels to and from S3 (AWS Datacenter)
  2. Data at rest (Encryption at Rest)
    • While data is stored on disks in AWS datacenter
    • Encryption at rest can then be split into another two types:
      • Server-Side Encryption
        • S3 Managed Keys (SSE-S3)
        • AWS Key Management Service (SSE KMS)
        • Server-side encryption with customer-provided key (SSE-C)
      • Client-side encryption
Below is an example of how encryption is being done for server-side encryption

Amazon S3 Pricing

How much does Amazon S3 cost (S3 Pricing)? 

Amazon S3 charges will be a bit different for each of the region. But we’ll get S3 pricing example based on North Virginia this time

Below are examples of S3 charges or S3 pricing: 

S3 Standard S3 Standard Infrequent Access S3 One Zone - IA S3 Glacier

STORAGE

First 50 TB / mo

$0.023 / GB
$0.0125 / GB
$0.01 / GB
$0.004 / GB

Next 450 TB / mo

$0.022 / GB
$0.0125 / GB
$0.01 / GB
$0.004 / GB

Over 500 TB / mo

$0.021 / GB
$0.0125 / GB
$0.01 / GB
$0.004 / GB

REQUEST & RETRIEVAL

Put, Copy, Post, List

$0.005 / 1000 request
$0.01 / 1000 request
$0.01 / 1000 request
$0.05 / 1000 request

Data retrievals

$0.000 / GB
$0.01 / GB
$0.01 / GB
* Refer below

*For S3 Glacier:

  • Object archived have a minimum of 90 days storage. Object deleted before 90 days will be charged pro-rated charge which is equal to the remaining storage charge.
  • Object deleted, overwritten and transferred to a different storage before minimum period (90 days), will be charged normal storage usage + pro-rated request charge for the remaining of minimum storage duration
  • Object stored longer than 90 days WILL NOT be charged minimum request charge

Above is just portion of the S3 charges as an example on how S3 pricing works. To get more information for different S3 storage classes cost and to check out on S3 pricing based on your own region, head on to https://aws.amazon.com/s3/pricing/ 

S3 Permission

When we touch on S3 permission, we cannot run from talking about Bucket policies and access control list. 

Anyway, all buckets and objects are PRIVATE by default. Only resource owner have access at first. Owner will then grant access to other users through: 

  • S3 resource based policy
  • IAM User policy

Let’s dive deep into what is S3 resource based policy and IAM User Policy? 

S3 Bucket Policy

  • S3 Bucket Policy is a policy attached to S3 bucket.
  • Permission in the policy applied to all objects in the bucket.
  • Policies determine what actions are allowed or denied for particular user

S3 Access Control List

  • Grant access to users in other Account or to Public
  • Both bucket or object has ACL (Access Control List)
  • Object Access Control list allow us to share S3 object via URL with the Public

AWS S3 FAQ

  • A: No, we explained a little bit on S3 pricing at S3 pricing section above. But if you register for AWS Free tier account, you will get 5GB of S3 standard storage for 12 months. Not really free I would say, it’s more to the customer to try it out
  • A: S3 is part of AWS Free Tier offering and during the 12 months free tier period, you will able to enjoy 5GB of S3 standard storage for free. For more information, go to: https://aws.amazon.com/free

Tips for AWS Solution Architect Associate Exam (related to S3 topic)

For those who is planning to sit for AWS Solution Architect Associate exam or currently doing your revision, below are some tips to prepare for the exam especially if you are currently preparing on AWS S3 topic. Take note that there a lot of questions on S3 in this exam so, S3 topic is definitely one of the topic that you need to master on. 

  1. Go through all the S3 F.A.Q in AWS website: https://aws.amazon.com/s3/faqs/
  2. Find a good AWS Online training. One of the best I went through so far is from Ryan Kroonenburg on Udemy 
  3. Make sure you do a lot of lab exercise (practical knowledge) and practice questions. I find Whizlabs having a good resources for this (practice labs and practice questions) with an affordable price too. (you can try the free test first before purchasing the practice question). What I like about the practice question is they have around 6 set of full questions (all topics) and also question on specific topic for us to focus on the topic that we are weak on. 
Good luck for your exam

Share this: