↖↑↓ Developer Associate

6/2017 - 8/2017

↖↑↓ AWS Fundamentals

↖↑↓ Global infrastructure

Region - grouping of data centers
AZ - indidvidual data center in a region. Redundancy throughout AZs in one region
Edge Location - location to deliver cached data fast -> Use Cloudfront CDN to cache data close to where it's being used

↖↑↓ Storage overview

Instance store volumes

Temporary block storage
Physically attached to the host computer of the instance
Useful for often-changing data like caches & buffers
Data is lost when EC2 instance stops or terminates (ephemeral data)

Elastic Block Storage (EBS)

Permanent block storage, independent to instance
Attachable to running EC2 instances (same AZ)
Only accessible by a single instance
Can take snapshots from
Can be encrypted
Stores redundantly in single AZ
Different volume options:
- General purpose SSD
- Provisioned IOPS
- Magnetic volumes

Elastic File System (EFS)

Scalable file storage for use with Amazon EC2 instances
Elastic storage capacity, growing and shrinking as files are added or removed
Multiple EC2 instances from multiple AZs can access an EFS file system at the same time
Stores redundantly in multiple AZs

Amazon Glacier

Low cost, very slow retrieval
Can be intergrated with S3 lifecycle policy

Database Storage

DynamoDB
RDS
DBs on EC2 instances
AWS Redshift (data warehouse service)

In-memory caching

ElastiCache (Memcached and Redis)
Software on EC2 instances

Storage gateway

Integrate existing on-premises storage infrastructure and data with the AWS Cloud

↖↑↓ Security Concepts

Shared responsibility environment
AWS is responsible for:
- Server / Host level and below
- Physical environment security
- Hardware decommissioning
- Traffic security (Networks, ACLs, SSL, DDOS-protection)
- EC2 hypervisor isolation
User is responsible for:
- IAM
- MFA
- Password/key-rotation
- Access advisor (shows used permissions)
- Trusted advisor (validates best practices)
- Security groups
- ACL (resource based policy)
- VPC

↖↑↓ Services

↖↑↓ IAM

IAM is a global service that helps to securely control access to AWS resources.

Users hold credentials
Groups hold users, typically only provides permission to assume a role
Roles hold policies.
- Can have trust relationships with trusted entities that can assume this role
Policies can be attached to users, groups or roles (preferred)
An instance profile is a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts.
Users and / or services assume roles

Policies

Any actions on resources that are not explicitly allowed are denied by default
Structure
- E - effect (allow / deny)
  - What the effect will be when the user requests the specific action
- P - prinicpal (ARN)
  - The account or user who is allowed access to the actions and resources in the statement
  - IAM policies do not have a principal (because they are attached to users, groups or roles)
- A - action or notaction
  - Describes the specific action or actions that will be allowed or denied
- R - resource or notresource
  - Specifies the object or objects that the statement covers
- C - condition
  - Specifies conditions for when a policy is in effect
Can use policy variables
- aws:currentTime, aws:userid, ...

	{
		"Version": "2012-10-17",
		"Statement": [
			{
				"Effect": "Allow",
				"Action": "s3:ListAllMyBuckets",
				"Resource": "arn:aws:s3:::*"
			},
			{
				"Effect": "Allow",
				"Action": [
						"s3:ListBucket",
						"s3:GetBucketLocation"
				],
				"Resource": "arn:aws:s3:::productionapp"
			},
			{
				"Effect": "Allow",
				"Action": [
					"s3:GetObject",
					"s3:PutObject",
					"s3:DeleteObject"
				],
				"Resource": "arn:aws:s3:::productionapp/*"
			}
		]
	}

IAM Policies

Managed policies (the new way)
- Can be attached to multiple users, groups and roles
- AWS managed policies
  - Updated by AWS if new API come out
Inline policies (the old way)

Limits

.	.
Groups per account	100
Instance profiles	100
Roles	500
Server certificates	20
Users	5000

↖↑↓ Secure Token Service (STS)

Allows to grant temporary access to authenticated users
- IAM users
- Web-based identity providers (google, facebook, ...)
- Organization's existing identity system
Returns temporary credentials that expire after some time:
- Access key
- Session token

Terms

Federation
- Trust relationship between identity provider and AWS
Identity broker
- Broker in charge of mapping user to the right set of credentials
Identity store
- Eg Google or Facebook
Identities
- Users

Scenarios

Temporary credentials with EC2
- Assign IAM role to instance
- Get temp credentials from instance metadata
Temporary credentials with SDK
- Call assumeRole, extract temp credentials
Options for temporary credentials with API calls
- Sign request with temp credentials
- Add AC / SK to request (header or query string)

Amazon Simple Storage Service (S3) is object storage with a simple web service interface to store and retrieve any amount of data from anywhere on the web. It is designed to deliver 11x9 durability and scale past trillions of objects worldwide.

Key-value storage (folder-like structure is only a UI representation)
Bucket size is unlimited. Objects from 0B to 5TB.
HA and scalable, transparent data partitioning
Bucket lifecycle events can trigger SNS, SQS or AWS Lambda
- New object created events
- Object removal events
- Reduced Redundancy Storage (RRS) object lost event
Bucket names have to be globally unique, should comply with DNS naming conventions.
- http://bucket.s3.amazonaws.com
- http://bucket.s3-aws-region.amazonaws.com
- http://s3.amazonaws.com/bucket
- http://s3-aws-region.amazonaws.com/bucket

Perfomance & Consistency

Bucket operations get - list - put - delete - head
- Implemented through http operations: GET - PUT - DELETE - HEAD
- Read-after-write consistency for PUT of new objects.
- Eventual consistency for overwrite PUT and DELETE (stale reads but low latency).
Can only delete a bucket that is empty.
Scales automatically, up to a certain limit:
- Consistent:
  - >100 PUT/LIST/DELETE/s
  - >300 GET/s
- Bursts:
  - >300 PUT/LIST/DELETE/s
  - >800 GET/s
Key names are used to determine which partition to store the object in.
- Make sure keys are spread out (not sequential)
  - E.g. by adding a random prefix to the key name
For GET requests put AWS CloudFront in front of S3 bucket
- Internal caching
- Reduced latency - objects are physically closer to the consumer.
Multipart upload
- Recommended for objects >=100MB, mandatory for >=5GB
- Supports parallel uploads
- Can pause & resume
- Can upload file while it's being created
- 3 step process:
  - Initiate multipart upload
    - POST /ObjectName?uploads HTTP/1.1
  - Upload of all parts
    - PUT /ObjectName?partNumber=PartNumber&uploadId=UploadId HTTP/1.1
  - Complete Multipart upload
    - ```
      POST /ObjectName?uploadId=UploadId HTTP/1.1`
      <CompleteMultipartUpload>...
```

Hosting Static Websites

<bucket-name>.s3-website-<AWS-Region>.amazonaws.com

Bucket name must match domain name. Every hosted bucket recieves its own URL
Use AWS Route 53 to integrate custom domains (also to automatically fail-over from dynamic website)
Specify index & error documents
In AWS Route 53: create hosted zone & record set
Might need to add CORS configuration to bucket (cross origin resource sharing)

Access Control

Effect – This can be either allow or deny
Principal – Account or user who is allowed access to the actions and resources in the statement
Actions – For each resource, S3 supports a set of operations
Resources – Buckets and objects are the resources
Authorization works as a union of IAM & bucket policies and bucket ACLs

Defaults

Bucket is owned by the AWS account that created it
- Bucket ownership is not transferable
Bucket owner gets full permission (ACL)
The person paying the bills always has full control.
A person uploading an object into a bucket owns it by default.

IAM

IAM policies (in general) specify what actions are allowed or denied on what AWS resources
Defined as JSON
Attached to IAM users, groups, or roles (so they cannot grant access to anonymous users)
Use if you’re more interested in “What can this user do in AWS?”

Bucket policies

Specify what actions are allowed or denied for which principals on the bucket that the policy is attached to
Defined as JSON
Attached only to S3 buckets. Can however effect object in buckets.
Contain principal element (unnecessary for IAM)
Use if you’re more interested in “Who can access this S3 bucket?”
Easiest way to grant cross-account permissions for all s3:* permission. (Cannot do this with ACLs.)

ACLs

Defined as XML. Legacy, not recomended any more.
Can
- be attached to individual objects (bucket policies only bucket level)
- control access to object uploaded into a bucket from a different account.
Cannot..
- have conditions
- cannot explicitely deny actions
- grant permission to bucket sub-resources (eg. lifecycle or static website configurations)
Other than object ACLs there are bucket ACLs as well - only for writing access log objects to a bucket.

How to specify resources in a policy:

.	.
`arn:partition:service:region:namespace:relative-id`	`arn:aws:s3:::mybucket`
`arn:aws:s3:::*`	All buckets and objects in account
`arn:aws:s3:::mybucket`	`mybucket`
`arn:aws:s3:::mybucket/*`	All objects in `mybucket`
`arn:aws:s3:::mybucket/mykey`	`mykey` in `mybucket`
`arn:aws:s3:::mybucket/developers/($aws:username)/`	folder matching the accessing user's name

Pre-signed URLs

All objects are private by default. Only the object owner has permission to access these objects. However, the object owner can optionally share objects with others by creating a pre-signed URL, using their own security credentials, to grant time-limited permission to download the objects.

Logging

AWS CloudTrail logs S3-API calls for bucket-level operations (and many other information) and stores them in an S3 bucket. Could also send email notifications or trigger SNS notifications for specific events.
S3 Server Access Logs log on object level.

Versioning

Works on bucket level (for all objects)
Versioning can either be unversioned (default), enabled or suspended
Version ids are automatically assigned to objects
- Ids cannot changed.
- As long as versioning is disabled, id is set to null
- Once enabled, versioning can only be suspended (but not disabled)
- PUT creates a new version, GET returns the latest version. Specific versions can be requested.
- DELETE (without version) marks latest version as deleted and returns a 404 for subsequent GETs.
  - Older versions (pre-delete) can still be requested.
  - Restore old version by deleting the new version or by copying the old version on top of the bucket.
- DELETE (with a version) permanently deletes that version.
- If versioning is suspendend, S3 automatically adds a null version ID to every subsequent object stored thereafter
Lifecycle Management policies can automatically handle old versions, e.g. permanently delete or move to AWS Glacier.
Different versions of the same object can have different permissions.

Encryption

Protecting data in transit

Using an AWS KMS–Managed Customer Master Key (CMK)
- Before uploading to S3, Client makes request to KMS, receives plain text encryption key and cypher blob, to upload to S3 as object metadata. Decrypt by sending cypher blob to KMS, retrieving plain text back, use for decryption.
- Before downloading from S3, The client first downloads the encrypted object from Amazon S3 along with the cipher blob version of the data encryption key stored as object metadata. The client then sends the cipher blob to AWS KMS to get the plain text version of the same, so that it can decrypt the object data.
Using a Client-Side Master Key
- Clients provides a master key, S3 client generates random data key and encrypts with client's master key.
- Uploads material description as part of the object metadata.
- On download S3 client uses metadata to determine the right master key to use for decryption.
Use SSL encryption

Protecting data at rest

Uses AES-256 (or others)
Encryption can be enforced via bucket policy.
Enable server-side encryption by adding specific header to request (x-amz-server-side-encryption).
Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3)
- Each object is encrypted with a unique key employing strong multi-factor encryption
- Furthermore it encrypts the key itself with a master key that is rotated regularly
Server-Side Encryption with AWS KMS-Managed Keys (SSE-KMS)
- Similar to SSE-S3, with extra benefits
- Separate permissions for the use of an envelope key
- Has audit trail
Server-Side Encryption with Customer-Provided Keys (SSE-C)
- Key is not stored with AWS (stores salted HMAC valued instead)

Storage classes

.	.
S3 Standard	Durability 11x9
Availability 4x9
S3 IA (infrequent access)	Durability 11x9
Availability 3x9
S3 RRS (reduced redundancy storage)	Durability 4x9
Availability 4x9

Request/response headers

Request	Response
`x-amz-content-sha256`	`x-amz-delete-marker`
`x-amz-date`	`x-amz-id-2`
`x-amz-security-token`	`x-amz-request-id`
`x-amz-version-id`

Error codes

.	.
400 Bad Request	`ExpiredToken`
400 Bad Request	`InvalidToken`
400 Bad Request	`InvalidArgument`
400 Bad Request	`InvalidRequest`
400 Bad Request	`IncompleteBody`
400 Bad Request	`IncompleteDigest`
400 Bad Request	`InvalidBucketName`
403 Forbidden	`AccessDenied`
403 Forbidden	`InvalidAccessKeyId`
404 Not Found	`NoSuchBucket`
404 Not Found	`NoSuchKey`
409 Conflict	`BucketAlreadyExists`
409 Conflict	`BucketNotEmpty`

Limits

.	.
Buckets per account	100
Bucket policy max size	20KB
Object size	0B to 5TB
Object size in a single `PUT`	5GB

↖↑↓ Dynamo DB

Overview

Fully managed NoSQL database
HA through different AZs, automatically spreads data and traffic accross servers
- 3 geographically distributed regions per table
Can scale up and down depending on demand (no downtime, no performance degradation)
Built-in monitoring
User controlled read/write capacity (recently added: auto-scaling)
Big data: Integrates with AWS Elastic MapReduce and Redshift
No joins - create references to other tables manually (table1#something)
Option between eventual consistency or strongly consistency
Conditional updates and concurrency control (atomic counters)

Core components

A table is a collection of items.
- Can be updated through a single UpdateTable command at a time (ACTIVE -> UPDATING)
An item is a group of one or more attributes that is uniquely identifiable among all of the other items. (row in a traditional db)
An attribute is a fundamental data element, something that does not need to be broken down any further. Can be nested up to 32 levels. (column in a traditional db)
Primary keys are used to uniquely identify each item in a table. Apart from that DynamoDB is schemaless, which means that neither the attributes nor their data types need to be defined beforehand
Secondary indexes are used to provide more querying flexibility
Control plane operations create and manage DynamoDB table
Data plane operations perform CRUD actions on data in a table
DynamoDB streams operations capture data modification events in DynamoDB tables

Keys and indexes

Partion key (PK)

Partition key is also called hash attribute or primary key
Must be unique, used for internal hash function (unordered)
Used to retrieve data

PK & Sort key

Composite PK: index composed of hashed PK (unordered) and SK (ordered)
Sort key is also called range attribute or range key
Different items can have the same PK, must have different SK

Secondary indexes

Associated with exactly one table, from which it obtains its data
Allows to query or scan data by an alternate key (other than PK/SK)
All secondary indexes are automatically maintained by DynamoDB as sparse objects
- Items will only appear in an index if they exist in the base table
- Makes querying very efficient
Only for read operations, write is not supported.
Tables with secondary indexes need to be created sequentially (LimitExceededException)

Projected attributes

Attributes copied from the base table into an index
Makes them queryable
Different projection types
- KEYS_ONLY - Only the index and primary keys are projected into the index
- INCLUDE - Only the specified table attributes are projected into the index
- ALL - All of the table attributes are projected into the index

Local secondary index

Uses the same PK, but offers different SK
Every partition of a local secondary index is scoped to a base table partition that has the same partition key value
Local secondary indexes are extra tables that dynamo keeps in the background
Cannot be created after the base table has already been created.
Can choose eventual consistency or strong consistency at creation time
Local as in "co-located on the same partition"
Can request not-projected attributes for query or scan operation
Consumes read/write throughput from the original table.

Global secondary index

Uses different PK and offers additional SK (or none).
PK does not have to be unique (unlike base table)
Queries on the global index can span all of the data in the base table, across all partitions
Can be created after the base table has already been created.
Only support eventual consistency
Have their own provisioned read/write throughput
Global secondary keys are distributed transactions across multiple partitions
Global as in "over many partitions"
Cannot request not-projected attributes for query or scan operation

Capacity provisioning

Unit for operations:
- 1 strongly consistent read per second (up to 4KB/s)
- 2 eventual consistent read per second (up to 8KB/s)
- 1 write per second (up to 1KB)
Algorithm

.	.
.	300 strongly consistent reads of 11KB per minute
Calculate read / writes per second	`300r/60s = 5r/s`
Multiply with payload factor	`5r/s * (11KB/4KB) = 15cu`
If eventual consistent, devide by 2	`15cu / 2 = 8cu`

More throughput -> more reads / writes per second
Exceeding allocated throughput may result in throttling of the operation. Check return code.
Failing to distribute data accross partions can result in ProvisionedThroughputExceededException
Local secondary index
- Read
  - If read only index keys and projected attributes use same calculation
  - If more than index keys and projected attributes add extra latency and read capacity cost
    - Use read capacity from the index and for every item from the table
- Write (to items in the base table that are indexed)
  - 1 for adding an item
  - 2 for changing the value of an item
  - 1 for deleting and item
Global secondary index
- Read
  - Only supports eventual consistency, so 8KB/s base unit
  - Calculated the same as in tables, except that the size of the index entries is used instead of the size of the entire item
- Write (to items in the base table that are indexed)
  - Putting, Updating, or Deleting items in a table consumes the index' write capacity units

Query and scan operation

Query

Finds items based on PK values
Can only query any table or secondary index that have a composite primary key
Has to use PK, can specify SK
Very efficient, only searches index
Result is orderd by SK
Returns all attributes or only specified subset
Eventually consistent per default, can request consistent read
Can use conditional attributes

Scan

Reads every item in table (much worse performance than queries)
Can filter result (slows down performance)
The larger the data set in the table the slower the scan
Eventual consistent reads by default, can specify strongly consistent
Try to avoid scans
Use Page Size to limit how much data is retrieved at the same time

Atomic and conditional updates

Atomic Counters

Increment or decrement the value of an existing attribute without interfering with other writes
Request are applied in the order they are received
Not idempotent

Conditional updates

Only proceed if condition is met
Idempotent

How to grant temporary access

Web Identity Federation - use existing OpenId provider, eg. Amazon, Google, Facebook
Amazon Cognito does Web Identity Federation, also synchronizes app data
IAM - contains role for users to assume

API

Control Plane

Create and manage tables	.
`CreateTable`	Creates a table and specifies the primary index used for data access
`DescribeTable`	Returns information such as primary key schema, throughput settings, index information
`ListTables`	Returns the names of all of your tables in a list
`UpdateTable`	Modifies the settings of a table or its indexes
`DeleteTable`	emoves a table and all of its dependent objects

Data Plane

Creating data	.	conditional?
`PutItem`	Creates a new item, or replaces an old item with a new item	yes
`BatchWriteItem`	Puts or deletes multiple items in one or more tables	no
Called in a loop it typically checks for unprocesses items and submits a new `BWI` request for those

Reading data	.	conditional?
`GetItem`	Returns a set of Attributes for an item that matches the PK	no
`BatchGetItem`	Returns the attributes for multiple items from multiple tables using their PKs	no
`Query`	Gets one or more items using the table PK, or from a secondary index using the index key	no
`Scan`	Gets all items and attributes by performing a full scan across the table or a secondary index	no

Updating data	.	conditional?
`UpdateItem`	Modifies one or more attributes in an item	yes

Deleting data	.	conditional?
`DeleteItem`	Deletes a single item in a table by primary key	yes
`BatchWriteItem`	Puts or deletes multiple items in one or more tables	no
Called in a loop it typically checks for unprocesses items and submits a new `BWI` request for those

Limits

.	.
Tables per account/region	256
Max read / write per table partition	3000 reads / 1000 writes
Partition key	min 1B, max 2048B
Sort key	min 1B, max 1024B
Local secondary index per table	5
Global secondary index per table	5
Item size	1B to 400KB, including name & value
Simultaneous `CreateTable`, `UpdateTable`, `DeleteTable`	up to 10
Single `BatchGetItem`	Max 100 items, must be <16MB
Single `BatchWriteItem`	Up to 25 PutItem or DeleteItem, must be <16MB
Query and Scan result set limit	1MB data per call

↖↑↓ Elastic Compute Cloud (EC2)

Resizable compute capacity in the cloud
Amazon Machine Image (AMI)
- Unit of deployment
- Packaged-up environment that includes all the necessary bits to set up and boot an instance
- Can create AMI from configured EC2 instance

Different options

Payment models
- On-demand instances
  - Pay for compute capacity by the hour, can be terminated by Amazon
- Reserved instances
  - Provide a significant discount compared to On-Demand pricing and provide a capacity reservation when used in a specific Availability Zone
  - Can transfer between AZs
- Spot instances
  - Bid on spare Amazon EC2 computing capacity, not available for all instance types
- Dedicated hosts
  - A physical server with EC2 instance capacity fully dedicated to your use
Instance sizes & types
- Sizes: nano / micro / small / medium / large
- Types: general purpose / computer optimized / memory optimized / gpu / storage optimized
Pricing by
- Compute time
- Data transfer
- Storage
- Elastic IP address
- Monitoring
- Elastic load balancer

Instance metadata & userdata

Data about an instance that can be used to configure or manage the running instance
Available from running instance under http://169.254.169.254/latest/meta-data/
Contains various data about the current instance (static & dynamic)
Can specify user-data
- Allows to launch individual instances from same AMI

API

.	.
`DescribeImages`	Describe an Amazon Machine Image
`RegisterImage`	Final process of creating an AMI

Limits:

.	.
Elastic IP addresses for EC2-Classic	5

↖↑↓ Elastic Load Balancer (ELB)

Distributes traffic between instances that belong to the ELB group
Stops sending requests to unhealthy instances
Can store SSL certificates (offloads encryption to load balancer level)
Can configure session stickyness:
- LB issued cookie
  - Easy to implement, not best balancing
- Application issued cookie
  - Cookies based on application session, marginally better
- ElastiCache
  - Better distribution, requires state to be stored in DB or in EC memory.
  - EC memory is the much better option
Relies on DNS / Route53
Can route traffic into instances running in private subnets
- Needs to be configured with (empty) public subnets though.

Limits:

.	.
Total load balancers per region (ALB & ELB)	20

↖↑↓ SNS

Publishes messages to subscribers via topic
Pub-Sub-Service for messaging
- Scenarios:
  - Fanout: Many subsribers process event parallel and asyncronously
  - Push to SQS: Services pull from SQS, when they become available
  - Alert: Notification triggered by event or threshold
Mobile Notifications to mobile devices
- Sends push notifications to iOS, Android, Fire OS, Windows and Baidu-based devices

Components

Publisher (producer)
- Communicates asynchronously with subscribers
- Policies determine which topic(s) publishers can write to
Topics
- Unique name up to 256 characters
- Stored redundantly on multitple servers and datacenters
Subscribers (consumer)
- Subscribes to topic
- Endpoints like mobile app, web server, email, AWS SQS, AWS Lambda
  - Email subscriptions need to be confirmed
Messages
- Json-formatted key-value pairs
  - Fixed set + additional attributes if required
    - POSTs to https endpoints with specific headers
      - Contains topics- and subscription-ARN
      - To identify messages without parsing the body
  - Up to 10 for SQS.
  - Provider-specific for mobile push notifications
- Messages can be signed and verified
- Message data:
  - Message, MessageId, Signature, SignatureVersion, SigningCertURL, Subject, Timestamp, TopicArn, Type, UnsubscribeURL

Managing access

Owner creates topic and controls access to it
Can use own API (Access Control) and / or IAM, similar to S3
- Access control policies
  - Default denies, needs explicit allow
  - Can grant access across account (API call: AddPermission)
- IAM
  - More fine grained or very coarse, can include conditions
  - Can grant temporary security credentials

Mobile push notifications

Does not push to endpoint, but to PN-service (platform/provider specific)
1. Request credentials from mobile platforms (ADM, APNS, etc...)
2. Request a token from mobile platforms (registrations id for some platforms)
3. Create a platform application object
4. Create a platform endpoint object
5. Publish a message to the mobile endpoint
A single message can contain different data for different platforms

API

.	.
`CreateTopic`	Create a new topic.
`DeleteTopic`	Delete a topic and all its subscriptions.
`Publish`	Publish a new message to the topic.
`ListTopics`	List of topics owned by a particular user (AWS ID).
`ListSubscriptions`	List subscriptions owned by a particular user (AWS ID)
`ListSubscriptionsByTopic`	List of subscriptions for a particular topic
`Subscribe`	Register a new subscription on a topic, will generate a confirmation message from Amazon SNS
`ConfirmSubscription`	Respond to a confirmation message, confirming to receive notifications from the topic
`UnSubscribe`	Cancel a previously registered subscription

Limits:

.	.
Topics per account	100,000

↖↑↓ SQS

Scalable message queue service
Allows loose coupling and asynchronous processing
Pull from SQS (Push to SNS)
PCI compliant
Allows for asynchronous processing
Protection against data loss on application failure

Core features

Redundant infrastructure
Multiple readers / writers at the same time
Access control via SQS policies (similar to IAM)
Standard queue
- Guarantees message delivery at least once
- No guarantee on message order
- No guarantee on not receiving duplicates (app has to deal with it)
FIFO queue
- Guaranteed order
- Exactly-once processing
- Message groups - multiple ordered message groups within a single queue
- Name ends in .fifo
Delay queues
- Controls when a message becomes available
- Between 0s and 15min, default 0s
Visibility timeout
- Controls when a polled message becomes visible again
- Configurable and extendable for individual messages
- Between 0s and 12h, default 30s
Message retention period
- Amount of time a message will live in the queue if it's not deleted
- Between 1min and 14d, default 4d
In flight message
- Sent to a client but have not yet been deleted or have not yet reached the end of their visibility window
Deadletter queue
- Queue that other queues can send messages to when these were not successfully processed.
Receive message wait time
- Value >0 enables long polling
- Between 0s and 20s, default 0s (short polling)

Message lifecycle

Component 1 sends message A to queue
- SendMessage/SendMessageBatch
Component 2 retrieves A from queue.
- A remains in queue while it's being processed, but is not returned to any other components
- Message is now considered to be in flight.
- ReceiveMessage
Component 2 deletes A from queue during visibility timeout
- Otherwise it will get processed again
- SQS will never delete messages
- DeleteMessage/DeleteMessageBatch

Long polling vs short polling

Short polling returns immediately, could be false empty (e.g. message not fully propagated yet)
Long polling won't return unless there's a message in the queue or receive message wait time is exceeded. Also checks every server to avoid false empty responses

API

.	.
`SendMessage`/`SendMessageBatch`	Delivers a message to the specified queue (up to 20, <= 256KB)
`ReceiveMessage`	Retrieves one or more messages (up to 10), `WaitTimeSeconds` for long poll
`ChangeMessageVisibility`/`ChangeMessageVisibilityBatch`	Changes the visibility timeout of a message
`DeleteMessage`/`DeleteMessageBatch`	Deletes the specified message from the specified queue
`SetQueueAttribute`	e.g `DelaySeconds`, `MessageRetentionPeriod`
`GetQueueURL`
`CreateQueue`
`DeleteQueue`
`ListQueues`

Limits:

.	.
Max message size	256KB
Max inflight messages	120,000

↖↑↓ Cloudformation

Allows to create and provision resources in a reusable template fashion
- A CloudFormation template is a JSON or YAML formatted text file
Related resources are managed in a single unit called a stack
- All the resources in a stack are defined by the stack's CloudFormation template
Two ways to update a stack
- Direct update
- Create change set
  - Summary of proposed changes
Will rollback stack if it fails to create (can be disabled via API / console)

Anatomy of template

AWSTemplateFormatVersion
Description
Metadata
- Details about the template
Parameters
- Values to pass in right before template creation
- Allows validation per regular expression
Mappings
- Maps keys to values (eg different values for different regions)
Conditions
- Check values before deciding what to do
Resources
- Creates resources
Outputs
- Values to be exposed from the console or from API calls.
- Can be used in a different stack (cross stack references)

Intrinsic Functions

Used to pass in values that are not available until runtime
Usable in resource properties, metadata attributes, and update policy attributes (auto-scaling)
Fn::GetAtt
- Returns the value of an attribute from a resource in the template
Fn::FindInMap
- Returns the value corresponding to keys in a two-level map that is declared in the Mappings section
Fn::Join
- Appends a set of values into a single value, separated by the specified delimiter
Fn::GetAZs
- Returns an array that lists Availability Zones for a specified region
Fn::Select
- Returns a single object from a list of objects by index
Fn::ImportValue
- Returns the value of an Output exported by another stack
Fn::Split
- Split a string into a list of string values so that you can select an element from the resulting string list
Fn::Sub
- Substitutes variables in an input string with values that you specify
Ref
- Returns the value of the specified parameter or resource

Limits:

.	.
Max stacks per region	200
Max templates per region	unlimited
Parameters	60
Mappings	100
Resources	200
Outputs	60

↖↑↓ Elastic Beanstalk (EB)

Full stack that provisions capacity, sets up load balancing and auto-scaling and configures monitoring
No need to create / manage infrastructure
Not good if full control of resource configuration is needed
Not everything fits into the EB model

AWS-Stack

EC2 instance
Instance Security Group
Elastic Load Balancer
Load Balancer Security Group
Auto Scaling Group
- Automatically replaces instances if they become unavailable
S3 Bucket
- Source code, logs & othe artifacts
CloudWatch Alarm
- 2 alarms that monitor load on instances & Auto Scaling group scaling up / down
Cloudformation stack
Domain name
- subdomain.region.elasticbeanstalk.com

Supports

Platform-specific application source bundle (e.g. Java war for Tomcat)
- Go
- Java with Tomcat
- Java SE
- .NET on Windows Server with IIS
- Node.js
- PHP
- Python
- Ruby (Passenger Standalone)
- Ruby (Puma)
- Single Container Docker
- Multicontainer Docker
- Preconfigured Docker (Glassfish)
- Preconfigured Docker (Python 3.x)
- Preconfigured Docker (Go)

Core components

Application
- Logical collection of Elastic Beanstalk components, including environments, versions, and environment configurations. In Elastic Beanstalk an application is conceptually similar to a folder.
Application version
- An application version refers to a specific, labeled iteration of deployable code for a web application
Environment
- An environment is a version that is deployed onto AWS resources
- Runs only a single application version at a time
- Can run the same version or different versions in many environments at the same time
Environment Configuratoin
- Collection of parameters and settings that define how an environment and its associated resources behave
- Updating a configuration will cause AWS to automatically apply the changes
Configuration template
- Starting point for creating unique environment configurations

Limits:

.	.
Applications	75
Application Versions	1000
Environments	200

↖↑↓ Simple Workflow Service (SWF)

Task coordination and state management service
Distributed, scales up and down depending on task
Works with on-premise and cloud apps
Allows for synchronous or asynchronous processing
Can contain human events
Guaranteed order of execution
Tasks can live up to one year (31,536,000 seconds)

Core components

Workflow
- A workflow is a set of activities that carry out some objective, together with logic that coordinates the activities.
Domain
- Scope of a workflow
- An account can have multiple domains, each of which can contain multiple workflows
- Workflows in different domains cannot interact
Workflow Starter
- Any application that can initiate workflow executions
Activity
- Things carried out by a workflow
Activity Task
- Represents one invocation of an activity
- Can run synchronously or asynchronously
- Gets assigned to worker
- Decision task tells decider that state of workflow has changed
Activity Worker
- Is a program that receives activity tasks, performs them, and provides results back
- Might be used by a person
- Can live on EC2 or on-premise
Decider
- Coordination logic in a workflow
- Schedules activity tasks, provides input data to the activity workers, processes events that arrive while the workflow is in progress, and ultimately ends (or closes) the workflow when the objective has been completed.

Limits:

.	.
Maximum registered domains	100

↖↑↓ Virtual Private Cloud (VPC)

Provisions a logically isolated section of the AWS cloud
Spans over all AZs in a region
Allows to create layered architecture
Shared or dedicated tenancy (exclusive hardware or not)
Security groups and subnet network ACLs
Ability to extend on-premise network to cloud

Overview

Default VPC (Amazon specific)

Gives easy access to a VPC without having to configure it from scratch
Has different subnets in different AZs and an internet gateway per AZ
Each instance launched automatically receives a public IP (very different to non-default VPC)
Cannot be restored if deleted

Non-default VPC (regular VPC)

Only has private IP addresses
Resources only accessible through Elastic IP, VPN or internet gateways

VPC Peering

Connect VPCs through direct network routing
Can occur between different accounts and VPCs, but must be in the same region
Allows instances to communicate with each other as if they were in the same network

VPC Scenarios

VPC with private subnet only -> single tier apps
VPC with public and private subnets -> layered apps
VPC with public, private subnets and hardware connected VPN -> extending apps to on-premise
VPC with private subnets and hardware connected VPN -> extended VPN

Components

Subnet
- In exactly one AZ
- If traffic is routed to an Internet gateway, the subnet is known as a public subnet
- If a subnet doesn't have a route to the Internet gateway, it's known as a private subnet
- EC2 instances are launched into subnets
- Use ssh-agent forwarding to connect from public to private instances
- Sometimes grouped into Subnet Groups, e.g. for caching or DB. Typically across AZs
Route Table
- Contains a set of rules, called routes that determine where network traffic is directed to
- Each VPC automatically comes with a main route table that can be configured
- Each subnet in a VPC must be associated with a route table; the table controls the routing for the subnet. A subnet can only be associated with one route table at a time, but multiple subnets can be associated with the same route table
- Each route in a table specifies a destination CIDR and a target
- Every route table contains a local route for communication within the VPC
- Can have a default route 0.0.0.0/0 to route everything that doesn't have a specific rule
Elastic IP
- Static IPv4 address mapped to an instance or network interface
- If attached to network interface it's decoupled from the instance's lifecycle
- Routes to private IP address of instance
- Can be remapped in case of failure.
- For use in a specific region only
- Can only map to instances in public subnets
Gateways
- Internet Gateway
  - Horizontally scaled, redundant, and highly available VPC component that allows communication between instances in a VPC and the internet
  - Provides a target in VPC route tables for internet-routable traffic
  - Performs network address translation (NAT) for instances that have been assigned public IPv4 addresses
- Virtual Private Gateway
  - Has VPN connection to customer gateway attached
  - Serves as VPN concentrator on the Amazon side of the VPN connection
- Customer Gateway
  - A physical device or software application on your side of the VPN connection
NAT
- NAT Instances
  - Manually configured instance from an NAT AMI
- NAT Gateway
  - AWS-mananged service

Structure & package flow

VPC (has CIDR)
- Gateway (Internet or VPN)
- Routes (one per subnet, can be shared)
- Network ACL (one per subnet, can be shared)
- Subnets (CIDRs match VPC's CIDR)
- Security Group (on VPC level)
- Instance (needs public IP for internet communication, either ELB or Elastic IP)

Security

Network ACL

Subnet level, acting as firewall
Rules for inbound and outbound traffic
Rules have numbers and are evaluated from low to high
Stateless

Security Groups

Acts as a virtual firewall to control inbound and outbound traffic to instances
Acts on instance level, not subnet level
Rules for inbound and outbound traffic
Stateful - will always allow response to (allowed) outbound traffic
Can refer to other security group, e.g. allow traffic from there

Limits:

.	.
VPCs per region	5
Subnets per VPC	200
Customer gateways per region	50
Gateway per region	5 Internet
Elastic IPs per account per region	5
VPN connections per region	50
Route tables per region	200
Security groups per region	500

↖↑↓ Relational Database Service (RDS)

Set up, operate, and scale a relational database in the cloud
Supports
- Amazon Aurora
- MySQL
- MariaDB
- Oracle
- SQL Server
- PostgreSQL
Automates common administrative tasks such as performing backups and software patching
- Automated backups
  - Performs a full daily snapshot
  - Enables point-in-time recovery
- DB Snapshots
  - User-initiated
  - As frequent as desired
Supports encryption at rest for all database engines
DB instance
- Database environment in the cloud with specified compute and storage resources
Multi-AZ deployments
- Provide enhanced availability and durability for DB Instances, making them a natural fit for production database workloads
DB subnet group
- Collection of subnets that you are designated for the RDS DB Instances in a VPC
Maintenance window
- Needs to be specified (or defaults to weekly) for maintenance events like scaling and patching
DB Parameter group
- Acts as a “container” for engine configuration values that can be applied to one or more DB Instances

↖↑↓ Etc

us-east-1 is the default region for all SDKs
Penetration tests need to be anounced

↖↑↓ Developer Associate

↖↑↓ AWS Fundamentals

↖↑↓ Global infrastructure

↖↑↓ Storage overview

Instance store volumes

Elastic Block Storage (EBS)

Elastic File System (EFS)

Amazon Glacier

Database Storage

In-memory caching

Storage gateway

↖↑↓ Security Concepts

↖↑↓ Services

↖↑↓ IAM

Policies

IAM Policies

Limits

↖↑↓ Secure Token Service (STS)

Terms

Scenarios

↖↑↓ S3

Perfomance & Consistency

Hosting Static Websites

Access Control

Defaults

IAM

Bucket policies

ACLs

How to specify resources in a policy:

Pre-signed URLs

Logging

Versioning

Encryption

Protecting data in transit

Protecting data at rest

Storage classes

Request/response headers

Error codes

Limits

↖↑↓ Dynamo DB

Overview

Core components

Keys and indexes

Partion key (PK)

PK & Sort key

Secondary indexes

Projected attributes

Local secondary index

Global secondary index

Capacity provisioning

Query and scan operation

Query

Scan

Atomic and conditional updates

Atomic Counters

Conditional updates

How to grant temporary access

API

Limits

↖↑↓ Elastic Compute Cloud (EC2)

Different options

Instance metadata & userdata

API

Limits:

↖↑↓ Elastic Load Balancer (ELB)

Limits:

↖↑↓ SNS

Components

Managing access

Mobile push notifications

API

Limits:

↖↑↓ SQS

Core features

Message lifecycle

Long polling vs short polling

API

Limits:

↖↑↓ Cloudformation

Anatomy of template