2. 503
Service Temporarily Unavailable
The server is temporarily unable
to service your request due to
maintenance downtime or capacity
problems. Please try again later.
11. Database Options
Self-Managed
Database Server
on Amazon EC2
Your choice of
database running on
Amazon EC2
Bring Your Own
License (BYOL)
Fully-Managed
Amazon RDS
Relational Database
as a managed
service
Flexible licensing:
BYOL or License
Included
Amazon
DynamoDB
Managed NoSQL
database service
using SSD storage
Seamless scalability
Zero administration
12. But how do I choose what
DB technology I need?
SQL? NoSQL?
18. Why SQL?
Established and well worn technology
Lots of existing code, communities, books, tools, etc
Clear patterns to scalability
You aren’t going to break SQL DBs in your first 10
million users. No really, you won’t
19. Amazon Relational Database Service (RDS)
Feature
Platform support
Preconfigured
Automated patching
Details
Create MySQL, SQL Server and Oracle
Get started instantly with sensible default
settings
Keep your database platform up to date
automatically
Backups
• Database-as-a-Service
• No need to install or manage database
instances
• Scalable and fault tolerant configurations
Automatic backups and point in time
recovery using snapshots
Manual DB snapshots
Failover
Automated failover to slave hosts in event of
a failure
Replication
Easily create read-replicas of your data and
seamlessly replicate data across availability
zones
20. Auto-Scaling
Automatic resizing of
compute clusters based on
demand
Feature
Amazon
CloudWatch
Trigger auto-scaling policy
Details
Control
Define minimum and maximum instance pool
sizes and when scaling and cool down occurs.
Integrated to Amazon
CloudWatch
Use metrics gathered by CloudWatch to drive
scaling.
Instance types
Run Auto Scaling for On-Demand and Spot
Instances. Compatible with VPC.
as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones us-east-1a
--min-size 4
--max-size 200
30. Production 1.0 Architecture
Well-designed, 2 Tier architecture
Highly Available due to Multiple Availability Zone
Load Balancing & Auto-Scaling for full scalability
Fully managed Database included
Capable of serving >10K-100Ks users
32. Production 1.0 Architecture
Wasted server capacity for static content
Reliability and durability are not yet optimal
End-user experience could be improved thru
offloading & caching
35. Simple Storage Service (S3)
Feature
Flexible object store
Access control
Server-side encryption
Multi-part uploads
Object versioning
Object expiry
Durable storage, any object
99.999999999% durability of objects
Unlimited storage of objects of any type
Up to 5TB size per object
Access logging
Web content hosting
Notifications
Import/Export
Details
Buckets act like drives, folder structures within
Granular control over object permissions
256bit AES encryption of objects
Improved throughput & control
Archive old objects and version new ones
Automatically remove old objects
Full audit log of bucket/object actions
Serve content as web site with built in page handling
Receive notifications on key events
Physical device import/export service
36. CloudFront
• World-wide content distribution
network
• Easily distribute content to end
users with low latency, high data
transfer speeds, and no
commitments
Feature
Fast
Integrated with other services
Dynamic content
Streaming
Details
Multiple world-wide edge locations to serve content as close to your users as possible
Works seamlessly with S3 and EC2 origin servers
Supports static and dynamic content from origin servers
Supports rtmp from S3 and includes support for live streaming from Adobe FMS and Microsoft
Media Server
38. Production 1.2 Architecture
Well-designed, 2 Tier architecture
Highly Available due to Multiple Availability Zone
Load Balancing & Auto-Scaling for full scalability
Fully managed Database included
Static content stored in durable, consistent way
Improved end-user experience through CDN
Capable of serving >100K-1M+ users
44. Elastic MapReduce (EMR)
• Managed, elastic Hadoop cluster
• Integrates with S3 & DynamoDB
• Leverage Hive & Pig analytics scripts
Feature
Scalable
Integrated with other
services
Comprehensive
Cost effective
Monitoring
Details
Use as many or as few compute instances running Hadoop as you want. Modify the number of
instances while your job flow is running
Works seamlessly with S3 as origin and output. Integrates with DynamoDB
Supports languages such as Hive and Pig for defining analytics, and allows complex definitions in
Cascading, Java, Ruby, Perl, Python, PHP, R, or C++
Works with Spot instance types
Monitor job flows from with the management console
45. Foursquare…
…generates a lot of Data
Founded in 2009
112M in Venture Capital
33 million users
1.3 million businesses using the service
3.5 billion check-ins
15M+ venues,
Terabytes of log data
46. Uses EMR for
Evaluation of new features
Machine learning
Exploratory analysis
Daily customer usage reporting
Long-term trend analysis
47. Benefits of EMR
Ease-of-Use
“We have decreased the processing time for urgent data-analysis”
Flexibility
To deal with changing requirements & dynamically expand reporting clusters
Costs
“We have reduced our analytics costs by over 50%”
49. Production 1.3 Architecture
Well-designed, 2 Tier architecture
Highly Available due to Multiple Availability Zone
Load Balancing & Auto-Scaling for full scalability
Static content stored in durable, consistent way
Improved end-user experience through CDN
Big Data analytics built in for continuous optimization
Capable of serving >1m-10M+ users
Amazon Web Services allows you to scale from one EC2 Instance to [Click]
to many thousands. Just dial up and down as required. The Power of Elasticity…[Click] And all this is fully automated without you loosing sleep [Click]
TIME TO MARKETNeed to launch the business quicklyLong development cycles and high costsInability to experiment and test the hypotheses that underpin the businessSCALABILITYUnpredictable demandNeed to deal with spiky traffic or sudden increase in usersNeed to scale out to cover new markets / regionsCOST & REVENUENo CAPEX budget Inability to forecast demand & commit long term contractsNeed to run a lean business & focus on generating revenue
Let us see how AWS helps to scale your Web Application to support 10’s of Millions of users. Start Small and grow big, build an architecture that scales at each progressive stage.
After a few feedbacks and tinkering for a better customer experience we have finally gone live and this is our Production 1.0 Architecture. If you notice we have now enabled the Multi AZ feature in our Database. All it takes is a single click or an API call to make your Database highly available Over the course of the last fee slides we covered how you can scale progressively through various stages of your application development and deployment. This again underlines the ability to scale seamlessly and pay for only what you use and provision when you need to.
Amazon EC2 enables our partners and customers to build and customize Amazon Machine Images (AMIs) with software based on your needs. These are the database servers available for use today within Amazon EC2: Oracle Database 11g,Microsoft SQL Server Standard,MySQL Enterprise,IBM DB2,IBM Informix Dynamic Server. http://aws.amazon.com/ec2/Amazon Relational Database Service (Amazon RDS) is a web service that makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while managing time-consuming database administration tasks, freeing you up to focus on your applications and business. Amazon RDS gives you access to the capabilities of a familiar MySQL or Oracle database. This means that the code, applications, and tools you already use today with your existing databases can be used with Amazon RDS. Amazon RDS automatically patches the database software and backs up your database, storing the backups for a user-defined retention period and enabling point-in-time recovery. You benefit from the flexibility of being able to scale the compute resources or storage capacity associated with your relational database instance via a single API call. In addition, Amazon RDS for MySQL makes it easy to use replication to enhance availability and reliability for production databases and to scale out beyond the capacity of a single database deployment for read-heavy database workloads. As with all Amazon Web Services, there are no up-front investments required, and you pay only for the resources you use. http://aws.amazon.com/rds/Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. With a few clicks in the AWS Management Console, customers can launch a new Amazon DynamoDB database table, scale up or down their request capacity for the table without downtime or performance degradation, and gain visibility into resource utilization and performance metrics. Amazon DynamoDB enables customers to offload the administrative burdens of operating and scaling distributed databases to AWS, so they don’t have to worry about hardware provisioning, setup and configuration, replication, software patching, or cluster scaling. http://aws.amazon.com/dynamodb/Amazon Redshift is a managed data warehouse service in the Amazon cloud. Redshift is optimized for data sets ranging from 100’s of GB to peta-byte scale. It uses columnar storage to compress and accelerate scan operations against large data sets, while providing a SQL interface for easy integration with reporting and query tools. All Redshift operations occur as massively parallel processes, including data loading, query, resizing, backup and restore. Redshift users can provision a cluster and load data directly from S3 in a few minutes, and be assured that their data is protected by VPC and encryption, both at rest and in-flight (via SSL).
Founded in 2004, raised 56M in Venture Capital, went IPO