AWS - Compute Fundamentals

Elastic Cloud Compute (EC2)

This is the platform that lets you deploy virtual servers within the AWS environment. The following subsections discuss each one of the elements that make EC2.

Amazon Machine Images (AMI's)

AMI's are templates of pre-configured EC2 instances. An AMI comprises an operating system, applications and custom configuration. When configuring an EC2 instance selecting your AMI is the first choice you need to make, Amazon offers a lot of AMI configurations but you can also create your own AMIs and reuse them to start you EC2 instances.

To create an AMI instance you start with one of the existing Amazon AMIs (you can select one of the AMIs offered directly by Amazon or pick one from the AMI Marketplace) and create an EC2 instance from it, then you proceed to install your custom applications and configuration to end with a customized EC2 instance which now you can save as an AMI template.

You can also create your own AMI from scratch, instead of starting with an AMI you start with your own VM image, but there are some prerequisites and limitations to this approach, for more info see https://www.elastic.co/blog/create-an-ami-from-your-own-vm-image .

EC2 Instance Types

After selecting your AMI the next step is to select an EC2 instance type. An instance type is a definition of a virtual machine that comprises the following dimensions:




There are 5 families of EC2 instances:

  • General Purpose: Balanced CPU, memory and storage. Ideal for small and medium applications (databases, development servers and back-end servers).
  • Compute Optimized: These instances have high performing processors and are ideal for high performance front-ends, web servers, science and engineering applications.
  • GPU: Optimized for graphic intensive applications or other applications that could take advantage of GPU architectures.
  • Memory Optimized: The focus in on memory and is the type of instance you'll use for large-scale, enterprise-class, in-memory applications.
  • Storage Optimized: SSD backed instance storage for low-latency and high I/O performance. This is recommended for analytic workloads and NoSQL stores, data file systems and lock processing applications.

Instance Purchasing Options

  • On-Demand instances:
    • Can be launched at any time.
    • Can be used for as long as needed.
    • The have a flat rate which is typically by the hour.
    • Used for irregular uninterruptible workloads.
    • Best fit for testing development environments.
  • Reserved instances:
    • Purchased for a set period of time for a reduced cost compared to on-demand (this reduction can be up to 75%). These reservation are purchased in one to three year time-frames. You can pay All Upfront (largest discount), Partial Upfront (smaller discount), no upfront (smallest discount).
    • This instance type is recommended for long term, predictable workloads.
  • Spot Instances:
    • Allows you to bid for unused EC2 Compute resources.
    • No guarantee of having the resource for a fixed period of time.
    • You must bid higher than the than the current spot price set by AWS.
    • Fluctuation of prices based on supply and demand.
    • As soon as your bid prices becomes lower than the fluctuating spot price you get a two minutes warning before the instance is terminated.
    • Possible to purchase large EC2 instances at a very low price.
    • Useful for processing data that can be suddenly interrupted (for example, batch jobs, background processing of data).
  • Dedicated Instance:
    • Instances hosted in hardware no other customer can access.
    • May be required to meet compliance.
    • Dedicated instances incur additional charges.
  • Dedicated host:
    • Similar to a dedicated instance but offer additional visibility and control over the physical host.
    • You can use the same host for a number of instances.
    • May be required to meet compliance.

User Data

During the configuration of your EC2 instance there is a section called "User Data" which allows you to enter commands that will run during the first boot cycle of that instance.

Storage Options

Storage can be classified as persistent or ephemeral.
  • Persistent
    • EBS volumes are physically separated from the EC2 instance.
    • EBS volumes are logically attached via the AWS network.
    • The data in these volumes is automatically replicated to other volumes within the same availability zone.
    • You can disconnect an EBS volume from the EC2 instance maintaining data and attach it to another EC2 instance if needed.
    • You can implement encryption and take backup snapshot of all the data.
  • Ephemeral
    • Instance store volumes are physically attached to the underlying host.
    • When the instance is stopped or terminated, all saved data on disk is lost.
    • If you reboot your data will remain intact.
    • Unable to detach instance volumes from the instance.

EBS Volume Types

  • General Purpose SSD (GP2):
    • General purpose, balances price and performance.
    • Ratio of 3 IOPS per GB with up to 10,000 IOPS and the ability to burst up to 3,000 IOPS for extended periods of time for volumes at 3334 GiB and above.
  • Provisioned IOPS SSD (IO1):
    • Designed for I/O intensive applications such as large relation database and NoSQL databases.
    • Use if you need more than 10,000 IOPS.
    • Can provision up to 20,000 IOPS per volume.
  • Throughput Optimized HDD (ST1):
    • Big Data
    • Data Warehousing
    • Log Processing.
    • Cannot be a boot volume.
  • Cold HDD (SC1):
    • Lowest cost storage for infrequently accessed workloads.
    • File server
    • Cannot be a boot volume.
  • Magnetic (Standard):
    • Lowest cost per gigabyte of all EBS volume types that is bootable. Magnetic volumes are idea for workloads where data is accessed infrequently, and applications where the lowest storage cost is important.

Security Groups

Security groups are virtual firewalls for instances.
  • Source and Destination restrictions.
  • Inbound and outbound restrictions.
  • Ports and protocols usage.

Key Pairs

When creating an instance, a key pair must be selected or downloaded. The function of the key pairs is to encrypt the login information for Linux and Windows EC2 instances and the decrypt the same information allowing you to authenticate onto the instance.
  • The public key is kept by AWS.
  • The private key is your responsability.
  • You can use the same key pair for multiple instances.

 Elastic Load Balancing (ELB)

ELB is used to direct and route traffic to multiple instances. There are two ELB types: Classic Load Balancer and Application Load Balancer.

Traffic is sent to the ELB which distributes the requests across the servers. If one of the instances fails the ELB will continue to send traffic to the remaining instances. ELB can also balance berween different availability zones just in case something goes wrong at that level.

Defining your Load Balancer

You must choise either and internal balancer or an external balancer.
  • External Load Balancer:
    • Public IP Address
    • Receives traffic from the internet and distributes to instances.
  • Internal Load Balancer:
    • Private IP Address
    • Can only be accessed from your private AWS network.
Once the ELB is setup you can select the availability zones, it is a good choice to pick at least two. You must ensure instances exist in all the selected availability zones.

For ELB to know that an instance has failed you need ton configure the health checks, the ELB will ping the instances and if one o them fails it will be removed from the ELB configuration.

Auto Scaling

Auto Scaling is a mechanism that automatically allows you to increase or decrease resources. Through defined metrics you can increase and decrease the size of your EC2 fleet. Auto Scaling will give you these benefits:
  • Automatic Infrastructure Management
  • Better user experience.
  • Cost reduction.
There are two steps to configure auto scaling:
  •  Launch Configuration: 
    • It is a template used by the Auto Scaling Group to launch new instances.
    • Select an AMI, instance type, IP address, storage volumes, user data requirements, storage volume configuration, security groups, etc.
    • Similar step to when creating an EC2 instance.
  • Auto Scaling Group.
    • Defines the desired capacity of the group using scaling policies.
    • Defines where the group should place resources.


Web Server experience:
  • CORS issue with 3 files (these three files had been cached after a failed CORS request), I don't have idea how this happened.
  • After disabling the load balancer messing around with the instance configuration I discovered the web server with the elastic IP was no longer accessible from the internet... in the end I discovered that when playing the LB configuration I had changed the security group config that was also used by that single instance and that I had limited inbound traffic to connections from the load balancer.
  • I stopped the instances in the auto-scaling group and called it a day, a couple of days later i got a bill notification from my bank. Well, the auto-scaling group put the instances back in operation when it detected that I was below my number of minimum instances defined in the autoscaling configuration.

Comentarios

Publicar un comentario