Amazon held it’s first ever global customer and partner
conference called re:Invent for it’s AWS (Amazon Web Services) business. Six-thousand people from 60 countries were in
attendance.
Most people have heard of
Amazon’s cloud service and the more highly technical amongst you are familiar
with their compute and storage offerings.
AWS has come a long way, a very long way and they’re not stopping
anytime soon.
At the core, AWS is a set of building blocks so
organizations can put their infrastructure “In the Cloud” and deploy their
applications into virtual environments.
AWS is now a BIG set of building blocks. It would behoove any infrastructure engineer,
software developer or product manager to become familiar with their services. The fundamental principles of deploying in
the cloud are similar to a private virtualized environment, but there’s a lot more choice (which means
although there is much more flexibility, only knowledge of all the services and
design patterns will lead to the best solution) plus it’s a different language. Knowledge of architecting in AWS will provide the
appropriately robust and cost-effective solution. AWS most certainly facilitates deploying for
developers.
Here’s a very valid solution in AWS: Launch EC2 with EBS behind ELB with your
domain on Route 53 and your videos on Cloudfront, back up to S3 and your db on
RDS with Multi-AZ….. makes perfect
sense… right? Different language?
For those interested, I’ve listed the core AWS building
block services (think Lego’s for data architects) at the end, in the areas of
compute, storage, database, deployment & management, CDN, application
services and network. That will help
unlock the key to the AWS language example given above.
The keynotes and sessions we attended were fascinating. The hour plus keynotes can be seen here
from Jeff Bezos, Werner Vogels (CTO), and Andy Jassy (SVP).
Powerpoint presentations from the various sessions can be
found here.
Here’s an interesting statistic. In 2003 when amazon.com was a $5B retail business
the total amount of servers to support the back-end of that business is the same as what they
deploy every single day now for AWS! In S3 they have about 1 Trillion objects
stored and they handle 800K requests per second. Talk about scale !
Here’s another tidbit related to new products and feature
releases. In 2007, the first full year
of AWS operations, there were 9 releases.
In 2011, there were 82 releases.
For 2012 there will be 150 new releases. In terms of software releases into AWS that happens every 12 seconds.. yes, 12 seconds!
One thing that struck me was the amount of enterprise and
government agencies that are using AWS. AWS
now has hundred’s of thousands of customers from 190 countries. They have 300 government agency customers
and 1500 educational institutions in addition to thousands of enterprise
customers. Their vast partner ecosystem
has helped fuel their growth. The keynotes and sessions included presentations from customers including Netflix,
Nasdaq, SAP, Pinterest and more.
Major announcements on day 1 included a 25% cost reduction
on S3 and two new services. The first called Redshift, is their first data
warehouse solution that scales to Petabytes in the cloud. Compared to conventional data warehouse
solutions Redshift should be ten times lower in cost with high performance. The second is a new workflow service called
Data Pipeline which is more focused on data-related workflows like backups,
running analytics, generating reports, etc.
The link to the Jeff Bezos keynote above is actually a one hour
fireside chat between Bezos and Werner Vogels.
The main theme from Jeff was focus on your customers and think
long-term. Towards the end he spoke
about innovation and how the ability to spin up instances on demand allows
entrepreneurs to increase their experimentation rate. He also offered the following advice to
entrepreneurs and those within existing companies:
- - Don’t chase today’s fads
- - Be passionate about what you do
- - Be a missionary not a mercenary
- - Start with your customer and work backwards.
Werner Vogel, Amazon’s CTO, took the stage to Nirvana’s
music playing on day 2.
He said that AWS was first developed because Amazon needed
to scale and they had difficulty dealing with peaks and valleys (think Black
Friday). Werner showed a chart which
illustrated that to provision their “old” infrastructure for their peaks, on a
yearly basis they were wasting about 40% of their server capacity. If they considered 15% above peak for
November they were wasting almost 76% of their capacity. This is the primary benefit of the
cloud. You provision what you need on
demand and scale up and down accordingly.
Other advice from Werner included:
- - Don’t think about resources in the “old” way; an
EC2 instance is not a server
- - Decompose into small loosely coupled stateless
building blocks
- - Automate your application and processes
- - Let business levers control the system
- - Architect with cost in mind
- - Protect your customer as a first priority
- - In production, deploy to at least two availability
zones
- - Integrate security into your application from
the ground up
- - Build, test, integrate and deploy continuously
- - Don’t treat failure (equip, system,etc) as an
exception, treat it as a normal possible state
So all in all, it was a fascinating few days attending
Amazon’s AWS re:Invent
Amazon’s core building
block services include:
Compute:
EC2 (Elastic
Compute Cloud) for pay as you go compute capacity
EMR (Elastic
MapReduce) to cost effectively process vast amounts of data
Auto-Scaling to
automatically scale your EC2 capacity up or down (this is huge!)
ELB (Elastic Load
Balancing) automatically distributes traffic across EC2 instances
Storage:
S3 (Simple Storage Service) fully
redundant data storage
Glacier is a very
low cost storage for archive because retrieval is 3 to 5 hours
EBS (Elastic Block
Storage) for block level storage volumes with EC2
Import/Export for
getting data in/out of AWS the ‘old way’ on disk drives, etc.
Storage Gateway for
connecting on-premise appliances to cloud storage
Database:
RDS (Relational
Database Service) for MySQL, SQL server or Oracle set-up
DynamoDB is a fully
managed NoSQL db service
SimpleDB managed
NoSQL db service for smaller datasets
ElasticCache to
deploy, operate and scale an in-memory cache
Redshift is a data
warehouse solution
Deployment &
Management
IAM (Identity and
Access Management)
Cloudwatch for
monitoring AWS services starting with EC2
Elastic Beanstalk
simplifies deployment and management through automation
CloudFormation
simplifies provisioning through templates and version control
Data Pipeline is a
workflow solution for data related jobs (backup, analytics, etc.)
CDN (Content Delivery
Network)
Cloudfront is the
AWS CDN similar to Akamai or Limelight
Application Services
CloudSearch is a
managed search service
SWF (Simple
Workflow Service) is a workflow engine for your execution state
SQS (Simple Queue
Service) provides a hosted queue for storing messages
SNS (Simple
Notification Service) to manage notifications from the cloud
SES (Simple Email
Service) for scalable, bulk & transactional email sending
AWS Marketplace to
buy software that runs in the cloud
Networking
Amazon Route 53 is a robust DNS (Domain
Name Service)
VPC (Virtual
Private Cloud) lets you provision a private, isolated section of AWS
Direct Connect
simplifies establishing a dedicated network connection to AWS
And more………
No comments:
Post a Comment