SlideShare ist ein Scribd-Unternehmen logo
1 von 55
What’s new in AWS
Philip Fitzsimons, Manager, Solutions Architecture
Amazon Web Services UK
What’s new in AWS
Amazon Kinesis

Managed Service for Real-time Big Data Processing

Amazon WorkSpaces

Fully Managed Desktops in the Cloud

AWS CloudTrail (beta)

Get log files of API call made on your AWS account

Amazon AppStream

Stream resource-intensive applications from the Cloud

Amazon RDS for PostgreSQL (beta)

Amazon RDS makes it easy to set up, operate, and scale
PostgreSQL deployments in the cloud
Amazon Kinesis
Managed Service for
Real-time Big Data Processing
Introducing Amazon Kinesis
• Managed service that can scale without any down time
– Create a streaming data intake pipeline and do analysis in near real time
– Scale to hundreds of thousands of sensors or producers
– Scale to gigabytes to terabytes of throughput per hour

• Understand and take action based on data in seconds
– Using Amazon Kinesis Client Library on Amazon EC2 instances to process data
– Leverage pre-built connectors to store the data in S3, Redshift or DynamoDB

• High availability of data streams at low cost
– Data ingested in is stored in 3 different physical locations
– Pay by the hour with no commitments or up front fees
Why Amazon Kinesis?
Internal AWS experiences provided inspiration

Big Data

Real-time Big Data
•

CloudWatch metrics: what just went wrong
now

Weekly / Monthly Bill: What you spent this
past billing cycle?

•

Real-time spending alerts/caps:
guaranteeing you can’t overspend

•

Daily customer-preferences report from your
website’s click stream: tells you what deal or
ad to try next time

•

Real-time analysis: tells you what to offer
the current customer now

•

Daily fraud reports: tells you if there was
fraud yesterday

•

Real-time detection: blocks fraudulent use
now

•

Daily business reports: tells me how
customers used AWS services yesterday

•

Fast ETL into Amazon Redshift: how are
customers using AWS services now

•

Hourly server logs: how your systems were
misbehaving an hour ago

•
Sample Use Cases of Amazon Kinesis
Financial Services Leader

Digital Advertising Tech. Pioneer

Maintain real-time audit trail of every single market/
exchange order

Generate real-time metrics, KPIs for online ads
performance for advertisers

Custom-built solutions operationally complex to
manage, & not scalable

End-of-day Hadoop based processing pipeline slow, &
cumbersome

Kinesis enables customer to ingest all market order
data reliably, and build real-time auditing applications

Kinesis enables customers to move from periodic batch
processing to continual, real-time metrics and reports
generation

Accelerates time to market of elastic, real-time
applications – while minimizing operational overhead

Generates freshest analytics on advertiser performance to
optimize marketing spend, and increases responsive to
clients
Clickstream Analytics with Amazon Kinesis

Clickstream Archive
Aggregate
Clickstream
Statistics

Clickstream Trend Analysis

Clickstream Processing App
Simple Metering & Billing with Amazon Kinesis

Metering Record Archive
Incremental Bill
Computation

Billing Management Service

Billing Auditors
Amazon Kinesis: Key Developer Benefits
Easy Administration
Managed service for real-time streaming
data collection, processing and analysis.
Simply create a new stream, set the
desired level of capacity, and let the
service handle the rest.

S3, Redshift, & DynamoDB Integration
Reliably collect, process, and transform all
of your data in real-time & deliver to AWS
data stores of choice, with Connectors for
S3, Redshift, and DynamoDB.

Real-time Performance

High Throughput. Elastic

Perform continual processing on streaming
big data. Processing latencies fall to a few
seconds, compared with the minutes or
hours associated with batch processing.

Seamlessly scale to match your data
throughput rate and volume. You can
easily scale up to gigabytes per second.
The service will scale up or down based on
your operational or business needs.

Build Real-time Applications

Low Cost

Client libraries that enable developers to
design and operate real-time streaming
data processing applications.

Cost-efficient for workloads of any scale.
You can get started by provisioning a small
stream, and pay low hourly rates only for
what you use.

9
Amazon Kinesis
Managed Service for Real-Time Processing of Big Data
App.1

Data
Sources
Availability
Zone

Data
Sources

Data
Sources

Availability
Zone

S3
App.2

AWS Endpoint

Data
Sources

Availability
Zone

[Aggregate &
De-Duplicate]

Shard 1
Shard 2
Shard N

[Metric
Extraction]
DynamoDB
App.3
[Sliding
Window
Analysis]
Redshift

Data
Sources

App.4
[Machine
Learning]
Putting data into Kinesis
Managed Service for Ingesting Fast Moving Data
•

Streams are made of Shards
•

•

Each shard ingests up to 1MB/sec of data and up to 1000
TPS

•

All data is stored for 24 hours

•
•

A Kinesis stream is composed of multiple Shards

You scale Kinesis streams by adding or removing Shards

Simple PUT interface to store data in Kinesis
•

Producers use a PUT call to store data in a stream

•

A Partition Key is used to distribute the PUTs across
Shards

•

A unique Sequence # is returned to the Producer upon a
successful PUT call
Getting data out of Kinesis
Client library for fault-tolerant, at least-once, real-time processing
•

In order to keep up with the stream, your application must:
•
•

Be fault tolerant, to handle failures in hardware or software

•
•

Be distributed, to handle multiple shards
Scale up and down as the number of shards increase or decrease

Kinesis Client Library (KCL) helps with distributed processing:
•
•

Automatically starts a Kinesis Worker for each shard

•

Changes number of Kinesis Workers as number of shards changes

•
•

Simplifies reading by abstracting your code from individual shards

Uses checkpoints to keep track of a Worker’s location in the stream

Use the KCL with Auto Scaling Groups
•

Auto Scaling policies will restart EC2 instances if they fail

•

Automatically add EC2 instances when load increases
Amazon Kinesis Resources
• Sign up for Limited Preview
– http://aws.amazon.com/kinesis/limited-preview/
– Get SDK and Endpoint information after getting into Limited Preview

• Getting Started Guide
– http://docs.aws.amazon.com/kinesis/latest/dev/getting-started.html

• Developer Guide
– http://docs.aws.amazon.com/kinesis/latest/dev/introduction.html
Amazon WorkSpaces
Fully Managed Desktops in the Cloud
Sample Use Cases (there are many more)

• Mobile Device Access
• Secure WorkSpaces
• Remote Employees

• Seasonal Workers
• Student WorkSpaces
• Developer WorkSpaces
Key Benefits
• Fully Managed
• Support Multiple
Devices
• Keep Data Secure
and Available

• Choose Software &
Hardware
• Pay as You Go
• Corporate Directory
Integration
Fully Managed

WorkSpaces

• Launch the number of WorkSpaces needed
• All heavy lifting taken care of by AWS
• Users receive email to install clients and
connect
Support Multiple Devices
•
•
•
•
•

iPad
Kindle Fire HDX (Keyboard & Mouse)
Android Tablet
Microsoft Windows
Mac
Keep Data Secure and Available

• No data stored on end-user device
• Only Pixels delivered to users (PCoIP)
• User volume backed by Amazon S3
Choose Software and Hardware
WorkSpaces Bundle

Hardware Resources

Applications

Standard

1 vCPU, 3.75 GiB Memory,
50 GB User Storage

Utilities (Adobe Reader, Internet Explorer 9,
Firefox, 7-Zip, Adobe Flash, JRE)

Standard Plus

1 vCPU, 3.75 GiB Memory,
50 GB User Storage

Microsoft Office Professional 2010, Trend
Micro Worry-Free Business Security, Utilities
(Adobe Reader, Internet Explorer 9, Firefox,
7-Zip, Adobe Flash, JRE)

Performance

2 vCPU, 7.5 GiB Memory,
100 GB User Storage

Utilities (Adobe Reader, Internet Explorer 9,
Firefox, 7-Zip, Adobe Flash, JRE)

Performance Plus

2 vCPU, 7.5 GiB Memory,
100 GB User Storage

Microsoft Office Professional 2010, Trend
Micro Worry-Free Business Security, Utilities
(Adobe Reader, Internet Explorer 9, Firefox,
7-Zip, Adobe Flash, JRE)

All WorkSpaces Bundles provide the Windows 7 Experience to users (provided by Windows Server 2008 R2 with RDS).
Pay as You Go
WorkSpaces Bundle
Standard

$35

Standard Plus

$50

Performance

$60

Performance Plus
•
•
•

Monthly Price

$75

No up-front commitment
Delete WorkSpaces at any time
Price includes infrastructure (compute, storage, bandwidth) and bundle’s software
Corporate Directory Integration

• Users: Get to use existing Enterprise Credentials
• IT: WorkSpaces control like regular desktops
Getting Started – What steps do customers take?

•
•
•
•
•
•

Integrate with Corporate Active Directory
Choose WorkSpaces Bundle
Select Users to receive WorkSpaces
Launch WorkSpaces
Users receive email when provisioned
Users connect to WorkSpaces
Availability
• Now: Limited Preview
• Q1 2014: Public Beta (Initially US-West & US-East)
• Q1/Q2 2014: Region Expansion
Resources
aws.amazon.com/WorkSpaces
•
•
•
•
•

Detail Page
Pricing
Limited Preview Sign-Up
Documentation will follow at Public Beta
Contact aws-bdms-workspaces@amazon.com
– Prioritizing customers in Limited Preview, specific briefings
AWS CloudTrail (beta)

AWS CloudTrail is a web service that records AWS API
calls for your account and delivers log files to you.
The recorded information includes:
•
•
•
•
•

The identity of the API caller
The time of the API call
The source IP address of the API caller
The request parameters
The response elements returned by the AWS service
AWS CloudTrail Use Cases
• Security Analysis
–

You can use the AWS API call history produced by CloudTrail as an input into log
management and analysis solutions to perform security analysis and to detect user
behaviour patterns.

• Track Changes to AWS Resources
–

You can use the AWS API call history produced by CloudTrail to track changes to AWS
resources, including creation, modification, and deletion of AWS resources such as Amazon
EC2 instances, Amazon VPC security groups and Amazon EBS

• Troubleshoot Operational Issues
–

You can use the AWS API call history produced by CloudTrail to troubleshoot operational
issues. For example, you can quickly identify the most recent changes made to resources in
your environment.

• Compliance Aid
–

CloudTrail makes it easier to ensure compliance with internal policies and regulatory
standards by providing AWS API call history. Integrates with AWS Partner solutions like Alert
Logic
Features and Benefits
• Increased Visibility
– CloudTrail provides increased visibility into your user activity by
recording AWS API calls.
– You can answer questions such as, what actions did a given
user take over a given time period? For a given resource, which
user has taken actions on it over a given time period? What is
the source IP address of a given activity? Which activities failed
due to inadequate permissions?
Features and Benefits
• Durable and Inexpensive Log File Storage
– CloudTrail uses Amazon S3 for log file storage and delivery, so
log files are stored durably and inexpensively.
– You can use Amazon S3 lifecycle configuration rules to further
reduce storage costs. For example, you can define rules to
automatically delete old log files or archive them to Amazon
Glacier for additional savings.
Features and Benefits
• Easy Administration
– CloudTrail is a fully managed service.
– You simply turn on CloudTrail for your account using the AWS
Management Console, the Command Line Interface, or the
CloudTrail SDK and start receiving CloudTrail log files in the
Amazon Simple Storage Service (Amazon S3) bucket that you
specify.
Features and Benefits
• Reliability
– CloudTrail continuously transports events from AWS services
using a highly available and fault tolerant processing pipeline.
– Turning on CloudTrail has no impact on performance of your
AWS resources or API call latency.
Features and Benefits
• Timely Delivery & Notification
– CloudTrail typically delivers events within 15 minutes of the API
call and can be configured to publish a notification for each log
file delivered.
– This feature enables you to automatically take action upon log
file delivery. CloudTrail uses the Amazon Simple Notification
Service (SNS) for notifications.
Features and Benefits
• Log File Aggregation
– CloudTrail can be configured to aggregate log files across
multiple accounts and regions.
– If you use multiple AWS regions, you can choose where log files
are delivered for each region. For example, you can have a
separate Amazon S3 bucket for each region, or you can
aggregate log files from all regions in a single S3 bucket.
Features and Benefits
• Choice of Partner Solutions
– Multiple partners are available including AlertLogic, Boundary,
Loggly, Splunk and Sumologic.
– These partners offer integrated solutions to analyze CloudTrail
log files. These solutions include features like change tracking,
troubleshooting, and security analysis.
Getting Started
CloudTrail can be turned on in as few as two clicks from the AWS
Management Console. CloudTrail generates log files containing
detailed information about API calls made, and periodically saves these
files into an Amazon S3 bucket of your choosing. You can also choose
to create an SNS topic to receive a notification every time a new log file
is delivered.
•
•

•

To turn on CloudTrail, just provide a name for an Amazon S3 bucket where you want your
log files delivered.
If you use multiple AWS regions, you can choose where log files are delivered for each
region. For example, you can have a separate Amazon S3 bucket for each region, or you
can aggregate log files from all regions in a single S3 bucket.
There is no additional charge for CloudTrail, but standard rates for Amazon S3 and
Amazon SNS usage apply.
Amazon AppStream
Stream resource-intensive applications from the Cloud
Sample Use Cases
•
•
•
•

Games
Media and Entertainment
Simulation Software
3D Graphics Development
Key Benefits
• Remove Device
• Instant On
Constraints
• Improved Security
• Multi Platform Support • Automatic Scaling
• Easy Updates
Remove Device Constraints

• No longer a need to snap to the device capabilities
• AppStream helps reach broadest audience
• Users get rich experience across devices
Multi Platform Support
•
•
•
•
•

Kindle Fire
Android
iOS
Windows 7
Mac OS (2014)
Easy Updates
• Provide a new application version to AppStream
• Immediately upgrade all users
• No action on users’ part
Instant On
• Users can start using application immediately
• No large file downloads
• No time consuming installations
Improved Security
• Create Entitlement Service to authorize connections
• Clients connect to Entitlement Service for access
• Your application is secured in AWS Cloud
Automatic Scaling
• Set the limits for scaling of your application
• The AppStream service takes care of scaling
• AppStream uses the G2.2XLarge Instance
– Support for more instance types coming in 2014

• Focus on your application, not infrastructure
Getting Started – What steps do customers take?

•
•
•
•

Integrate AppStream SDK with application
Deploy Application to AppStream
Create Entitlement Service
Create Client Apps
Availability
• Now: Limited Preview
• Public Beta Date: TBD
Amazon RDS for PostgreSQL

Amazon RDS makes it easy to set up, operate,
and scale PostgreSQL deployments in the cloud.
With Amazon RDS, you can deploy scalable
PostgreSQL deployments in minutes with costefficient and resizable hardware capacity.
What does RDS for PostgreSQL enable?
•
•
•
•

Large-scale web applications
Internal and departmental applications
Excellent vehicle for ETL into analytics engines
Geospatial and mobile applications
Benefits of RDS for PostgreSQL
• Easy, managed deployments
– Free up time from undifferentiated admin tasks
– Database instances with pre-configured parameters, automated patching,
monitoring, and notifications

• Familiar environment for developers
– All your existing PostgreSQL apps work

• Fast, predictable performance
– Use Provisioned IOPS to tune on the fly

• Backup and recovery
– Automated backups and database snapshots

• High availability
– Deploy in multiple Availability Zones

Available in all regions
Benefits of PostgreSQL for AWS Customers
• The preferred open-source database for many
enterprise developers and startups
• Support for geospatial queries using the
PostGIS extensions
• Support for full-text search
• Support for advanced data types: JSON and
key/value stores (“hstore”)
Getting started with Amazon RDS for PostgreSQL

• Review the Getting Started Guide for RDS
http://docs.aws.amazon.com/gettingstarted/lates
t/awsgsg-intro/gsg-aws-intro.html
• Review the RDS Free Tier
http://aws.amazon.com/free
• Create a PostgreSQL DB instance from the RDS
console
https://console.aws.amazon.com/rds/
Resources for Amazon RDS for PostgreSQL
• User Guide
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/
• Data Import Guide
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/P
ostgreSQL.Procedural.Importing.html
• Common tasks, including setting up PostGIS geospatial
extensions
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/A
ppendix.PostgreSQL.CommonDBATasks.html
• RDS forum
https://forums.aws.amazon.com/forum.jspa?forumID=60
Details for Developers
• Launching with PostgreSQL 9.3.1
• Wide selection of available instances
– Including the new high-memory db.cr1.8xlarge

• Choose Multi-AZ deployment for high availability
– Synchronous replication to a secondary in a different AZ

• Use Provisioned IOPS for predictable performance
– Convert to IOPS with a brief availability impact
– Then add IOPS or storage on the fly
What’s new in AWS
Amazon Kinesis

Managed Service for Real-time Big Data Processing

Amazon WorkSpaces

Fully Managed Desktops in the Cloud

AWS CloudTrail (beta)

Get log files of API call made on your AWS account

Amazon AppStream

Stream resource-intensive applications from the Cloud

Amazon RDS for PostgreSQL (beta)

Amazon RDS makes it easy to set up, operate, and scale
PostgreSQL deployments in the cloud

Talk to someone from AWS or our partners at Stand 1070

Weitere ähnliche Inhalte

Was ist angesagt?

Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSBig Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSAmazon Web Services
 
New Database Migration Services & RDS Updates
New Database Migration Services & RDS UpdatesNew Database Migration Services & RDS Updates
New Database Migration Services & RDS UpdatesAmazon Web Services
 
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMR
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMRBDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMR
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMRAmazon Web Services
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Optimizing Storage for Big Data Analytics Workloads
Optimizing Storage for Big Data Analytics WorkloadsOptimizing Storage for Big Data Analytics Workloads
Optimizing Storage for Big Data Analytics WorkloadsAmazon Web Services
 
SRV420 Analyzing Streaming Data in Real-time with Amazon Kinesis
SRV420 Analyzing Streaming Data in Real-time with Amazon KinesisSRV420 Analyzing Streaming Data in Real-time with Amazon Kinesis
SRV420 Analyzing Streaming Data in Real-time with Amazon KinesisAmazon Web Services
 
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...Amazon Web Services
 
AWS Webcast - AWS Kinesis Webinar
AWS Webcast - AWS Kinesis WebinarAWS Webcast - AWS Kinesis Webinar
AWS Webcast - AWS Kinesis WebinarAmazon Web Services
 
Best Practices for Migrating your Data Warehouse to Amazon Redshift
Best Practices for Migrating your Data Warehouse to Amazon Redshift Best Practices for Migrating your Data Warehouse to Amazon Redshift
Best Practices for Migrating your Data Warehouse to Amazon Redshift Amazon Web Services
 
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Amazon Web Services
 
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of ThingsDay 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of ThingsAmazon Web Services
 
Getting Started with Amazon QuickSight
Getting Started with Amazon QuickSightGetting Started with Amazon QuickSight
Getting Started with Amazon QuickSightAmazon Web Services
 
Building Big Data Applications with Serverless Architectures - June 2017 AWS...
Building Big Data Applications with Serverless Architectures -  June 2017 AWS...Building Big Data Applications with Serverless Architectures -  June 2017 AWS...
Building Big Data Applications with Serverless Architectures - June 2017 AWS...Amazon Web Services
 
AWS May Webinar Series - Streaming Data Processing with Amazon Kinesis and AW...
AWS May Webinar Series - Streaming Data Processing with Amazon Kinesis and AW...AWS May Webinar Series - Streaming Data Processing with Amazon Kinesis and AW...
AWS May Webinar Series - Streaming Data Processing with Amazon Kinesis and AW...Amazon Web Services
 
Introduction to Real-time, Streaming Data and Amazon Kinesis. Streaming Data ...
Introduction to Real-time, Streaming Data and Amazon Kinesis. Streaming Data ...Introduction to Real-time, Streaming Data and Amazon Kinesis. Streaming Data ...
Introduction to Real-time, Streaming Data and Amazon Kinesis. Streaming Data ...Amazon Web Services
 
(ARC202) Real-World Real-Time Analytics | AWS re:Invent 2014
(ARC202) Real-World Real-Time Analytics | AWS re:Invent 2014(ARC202) Real-World Real-Time Analytics | AWS re:Invent 2014
(ARC202) Real-World Real-Time Analytics | AWS re:Invent 2014Amazon Web Services
 
Spark and the Hadoop Ecosystem: Best Practices for Amazon EMR
Spark and the Hadoop Ecosystem: Best Practices for Amazon EMRSpark and the Hadoop Ecosystem: Best Practices for Amazon EMR
Spark and the Hadoop Ecosystem: Best Practices for Amazon EMRAmazon Web Services
 
AWS Data migration services
AWS Data migration servicesAWS Data migration services
AWS Data migration servicesArun Sirimalla
 

Was ist angesagt? (20)

Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWSBig Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWS
 
New Database Migration Services & RDS Updates
New Database Migration Services & RDS UpdatesNew Database Migration Services & RDS Updates
New Database Migration Services & RDS Updates
 
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMR
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMRBDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMR
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMR
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Optimizing Storage for Big Data Analytics Workloads
Optimizing Storage for Big Data Analytics WorkloadsOptimizing Storage for Big Data Analytics Workloads
Optimizing Storage for Big Data Analytics Workloads
 
SRV420 Analyzing Streaming Data in Real-time with Amazon Kinesis
SRV420 Analyzing Streaming Data in Real-time with Amazon KinesisSRV420 Analyzing Streaming Data in Real-time with Amazon Kinesis
SRV420 Analyzing Streaming Data in Real-time with Amazon Kinesis
 
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
 
AWS Webcast - AWS Kinesis Webinar
AWS Webcast - AWS Kinesis WebinarAWS Webcast - AWS Kinesis Webinar
AWS Webcast - AWS Kinesis Webinar
 
Best Practices for Migrating your Data Warehouse to Amazon Redshift
Best Practices for Migrating your Data Warehouse to Amazon Redshift Best Practices for Migrating your Data Warehouse to Amazon Redshift
Best Practices for Migrating your Data Warehouse to Amazon Redshift
 
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
 
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of ThingsDay 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
 
Getting Started with Amazon QuickSight
Getting Started with Amazon QuickSightGetting Started with Amazon QuickSight
Getting Started with Amazon QuickSight
 
Building Big Data Applications with Serverless Architectures - June 2017 AWS...
Building Big Data Applications with Serverless Architectures -  June 2017 AWS...Building Big Data Applications with Serverless Architectures -  June 2017 AWS...
Building Big Data Applications with Serverless Architectures - June 2017 AWS...
 
AWS May Webinar Series - Streaming Data Processing with Amazon Kinesis and AW...
AWS May Webinar Series - Streaming Data Processing with Amazon Kinesis and AW...AWS May Webinar Series - Streaming Data Processing with Amazon Kinesis and AW...
AWS May Webinar Series - Streaming Data Processing with Amazon Kinesis and AW...
 
Introduction to Real-time, Streaming Data and Amazon Kinesis. Streaming Data ...
Introduction to Real-time, Streaming Data and Amazon Kinesis. Streaming Data ...Introduction to Real-time, Streaming Data and Amazon Kinesis. Streaming Data ...
Introduction to Real-time, Streaming Data and Amazon Kinesis. Streaming Data ...
 
AWS Analytics
AWS AnalyticsAWS Analytics
AWS Analytics
 
(ARC202) Real-World Real-Time Analytics | AWS re:Invent 2014
(ARC202) Real-World Real-Time Analytics | AWS re:Invent 2014(ARC202) Real-World Real-Time Analytics | AWS re:Invent 2014
(ARC202) Real-World Real-Time Analytics | AWS re:Invent 2014
 
Spark and the Hadoop Ecosystem: Best Practices for Amazon EMR
Spark and the Hadoop Ecosystem: Best Practices for Amazon EMRSpark and the Hadoop Ecosystem: Best Practices for Amazon EMR
Spark and the Hadoop Ecosystem: Best Practices for Amazon EMR
 
Introduction to Amazon Athena
Introduction to Amazon AthenaIntroduction to Amazon Athena
Introduction to Amazon Athena
 
AWS Data migration services
AWS Data migration servicesAWS Data migration services
AWS Data migration services
 

Ähnlich wie What's new in AWS?

Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...
Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...
Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...Amazon Web Services
 
Deep dive and best practices on real time streaming applications nyc-loft_oct...
Deep dive and best practices on real time streaming applications nyc-loft_oct...Deep dive and best practices on real time streaming applications nyc-loft_oct...
Deep dive and best practices on real time streaming applications nyc-loft_oct...Amazon Web Services
 
AWS re:Invent 2016: The State of Serverless Computing (SVR311)
AWS re:Invent 2016: The State of Serverless Computing (SVR311)AWS re:Invent 2016: The State of Serverless Computing (SVR311)
AWS re:Invent 2016: The State of Serverless Computing (SVR311)Amazon Web Services
 
Accelerate your Cloud Success with Platform Services
Accelerate your Cloud Success with Platform ServicesAccelerate your Cloud Success with Platform Services
Accelerate your Cloud Success with Platform ServicesAmazon Web Services
 
Easy Analytics with AWS - AWS Summit Bahrain 2017
Easy Analytics with AWS - AWS Summit Bahrain 2017Easy Analytics with AWS - AWS Summit Bahrain 2017
Easy Analytics with AWS - AWS Summit Bahrain 2017Amazon Web Services
 
Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017
Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017
Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017Amazon Web Services
 
Wicked rugby
Wicked rugbyWicked rugby
Wicked rugbyDklumb4
 
AWS Chicago user group - October 2015 "reInvent Replay"
AWS Chicago user group - October 2015 "reInvent Replay"AWS Chicago user group - October 2015 "reInvent Replay"
AWS Chicago user group - October 2015 "reInvent Replay"Cohesive Networks
 
Uses, considerations, and recommendations for AWS
Uses, considerations, and recommendations for AWSUses, considerations, and recommendations for AWS
Uses, considerations, and recommendations for AWSScalar Decisions
 
Build your Cloud Solution for Success - Tel Aviv Summit 2018
Build your Cloud Solution for Success - Tel Aviv Summit 2018Build your Cloud Solution for Success - Tel Aviv Summit 2018
Build your Cloud Solution for Success - Tel Aviv Summit 2018Amazon Web Services
 
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...Amazon Web Services
 
Cloud First: New Architecture for New Infrastructure
Cloud First: New Architecture for New InfrastructureCloud First: New Architecture for New Infrastructure
Cloud First: New Architecture for New InfrastructureAmazon Web Services
 
Getting started with Amazon Kinesis
Getting started with Amazon KinesisGetting started with Amazon Kinesis
Getting started with Amazon KinesisAmazon Web Services
 
Getting started with amazon kinesis
Getting started with amazon kinesisGetting started with amazon kinesis
Getting started with amazon kinesisJampp
 
Aws re invent 2018 recap
Aws re invent 2018 recapAws re invent 2018 recap
Aws re invent 2018 recapCloudHesive
 
From Batch to Streaming - How Amazon Flex Uses Real-time Analytics
From Batch to Streaming - How Amazon Flex Uses Real-time AnalyticsFrom Batch to Streaming - How Amazon Flex Uses Real-time Analytics
From Batch to Streaming - How Amazon Flex Uses Real-time AnalyticsAmazon Web Services
 

Ähnlich wie What's new in AWS? (20)

Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...
Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...
Introducing Amazon Kinesis: Real-time Processing of Streaming Big Data (BDT10...
 
Deep dive and best practices on real time streaming applications nyc-loft_oct...
Deep dive and best practices on real time streaming applications nyc-loft_oct...Deep dive and best practices on real time streaming applications nyc-loft_oct...
Deep dive and best practices on real time streaming applications nyc-loft_oct...
 
AWS re:Invent 2016: The State of Serverless Computing (SVR311)
AWS re:Invent 2016: The State of Serverless Computing (SVR311)AWS re:Invent 2016: The State of Serverless Computing (SVR311)
AWS re:Invent 2016: The State of Serverless Computing (SVR311)
 
Accelerate your Cloud Success with Platform Services
Accelerate your Cloud Success with Platform ServicesAccelerate your Cloud Success with Platform Services
Accelerate your Cloud Success with Platform Services
 
Easy Analytics with AWS - AWS Summit Bahrain 2017
Easy Analytics with AWS - AWS Summit Bahrain 2017Easy Analytics with AWS - AWS Summit Bahrain 2017
Easy Analytics with AWS - AWS Summit Bahrain 2017
 
Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017
Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017
Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017
 
Agile BI - Pop-up Loft Tel Aviv
Agile BI - Pop-up Loft Tel AvivAgile BI - Pop-up Loft Tel Aviv
Agile BI - Pop-up Loft Tel Aviv
 
Wicked rugby
Wicked rugbyWicked rugby
Wicked rugby
 
AWS Big Data Solution Days
AWS Big Data Solution DaysAWS Big Data Solution Days
AWS Big Data Solution Days
 
AWS Chicago user group - October 2015 "reInvent Replay"
AWS Chicago user group - October 2015 "reInvent Replay"AWS Chicago user group - October 2015 "reInvent Replay"
AWS Chicago user group - October 2015 "reInvent Replay"
 
Uses, considerations, and recommendations for AWS
Uses, considerations, and recommendations for AWSUses, considerations, and recommendations for AWS
Uses, considerations, and recommendations for AWS
 
Build your Cloud Solution for Success - Tel Aviv Summit 2018
Build your Cloud Solution for Success - Tel Aviv Summit 2018Build your Cloud Solution for Success - Tel Aviv Summit 2018
Build your Cloud Solution for Success - Tel Aviv Summit 2018
 
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
Real-time Streaming and Querying with Amazon Kinesis and Amazon Elastic MapRe...
 
Cloud First: New Architecture for New Infrastructure
Cloud First: New Architecture for New InfrastructureCloud First: New Architecture for New Infrastructure
Cloud First: New Architecture for New Infrastructure
 
Windows on AWS
Windows on AWSWindows on AWS
Windows on AWS
 
Getting started with Amazon Kinesis
Getting started with Amazon KinesisGetting started with Amazon Kinesis
Getting started with Amazon Kinesis
 
Getting started with amazon kinesis
Getting started with amazon kinesisGetting started with amazon kinesis
Getting started with amazon kinesis
 
Aws re invent 2018 recap
Aws re invent 2018 recapAws re invent 2018 recap
Aws re invent 2018 recap
 
Real-Time Streaming Data on AWS
Real-Time Streaming Data on AWSReal-Time Streaming Data on AWS
Real-Time Streaming Data on AWS
 
From Batch to Streaming - How Amazon Flex Uses Real-time Analytics
From Batch to Streaming - How Amazon Flex Uses Real-time AnalyticsFrom Batch to Streaming - How Amazon Flex Uses Real-time Analytics
From Batch to Streaming - How Amazon Flex Uses Real-time Analytics
 

Mehr von Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

Mehr von Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

What's new in AWS?

  • 1. What’s new in AWS Philip Fitzsimons, Manager, Solutions Architecture Amazon Web Services UK
  • 2. What’s new in AWS Amazon Kinesis Managed Service for Real-time Big Data Processing Amazon WorkSpaces Fully Managed Desktops in the Cloud AWS CloudTrail (beta) Get log files of API call made on your AWS account Amazon AppStream Stream resource-intensive applications from the Cloud Amazon RDS for PostgreSQL (beta) Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the cloud
  • 3. Amazon Kinesis Managed Service for Real-time Big Data Processing
  • 4. Introducing Amazon Kinesis • Managed service that can scale without any down time – Create a streaming data intake pipeline and do analysis in near real time – Scale to hundreds of thousands of sensors or producers – Scale to gigabytes to terabytes of throughput per hour • Understand and take action based on data in seconds – Using Amazon Kinesis Client Library on Amazon EC2 instances to process data – Leverage pre-built connectors to store the data in S3, Redshift or DynamoDB • High availability of data streams at low cost – Data ingested in is stored in 3 different physical locations – Pay by the hour with no commitments or up front fees
  • 5. Why Amazon Kinesis? Internal AWS experiences provided inspiration Big Data Real-time Big Data • CloudWatch metrics: what just went wrong now Weekly / Monthly Bill: What you spent this past billing cycle? • Real-time spending alerts/caps: guaranteeing you can’t overspend • Daily customer-preferences report from your website’s click stream: tells you what deal or ad to try next time • Real-time analysis: tells you what to offer the current customer now • Daily fraud reports: tells you if there was fraud yesterday • Real-time detection: blocks fraudulent use now • Daily business reports: tells me how customers used AWS services yesterday • Fast ETL into Amazon Redshift: how are customers using AWS services now • Hourly server logs: how your systems were misbehaving an hour ago •
  • 6. Sample Use Cases of Amazon Kinesis Financial Services Leader Digital Advertising Tech. Pioneer Maintain real-time audit trail of every single market/ exchange order Generate real-time metrics, KPIs for online ads performance for advertisers Custom-built solutions operationally complex to manage, & not scalable End-of-day Hadoop based processing pipeline slow, & cumbersome Kinesis enables customer to ingest all market order data reliably, and build real-time auditing applications Kinesis enables customers to move from periodic batch processing to continual, real-time metrics and reports generation Accelerates time to market of elastic, real-time applications – while minimizing operational overhead Generates freshest analytics on advertiser performance to optimize marketing spend, and increases responsive to clients
  • 7. Clickstream Analytics with Amazon Kinesis Clickstream Archive Aggregate Clickstream Statistics Clickstream Trend Analysis Clickstream Processing App
  • 8. Simple Metering & Billing with Amazon Kinesis Metering Record Archive Incremental Bill Computation Billing Management Service Billing Auditors
  • 9. Amazon Kinesis: Key Developer Benefits Easy Administration Managed service for real-time streaming data collection, processing and analysis. Simply create a new stream, set the desired level of capacity, and let the service handle the rest. S3, Redshift, & DynamoDB Integration Reliably collect, process, and transform all of your data in real-time & deliver to AWS data stores of choice, with Connectors for S3, Redshift, and DynamoDB. Real-time Performance High Throughput. Elastic Perform continual processing on streaming big data. Processing latencies fall to a few seconds, compared with the minutes or hours associated with batch processing. Seamlessly scale to match your data throughput rate and volume. You can easily scale up to gigabytes per second. The service will scale up or down based on your operational or business needs. Build Real-time Applications Low Cost Client libraries that enable developers to design and operate real-time streaming data processing applications. Cost-efficient for workloads of any scale. You can get started by provisioning a small stream, and pay low hourly rates only for what you use. 9
  • 10. Amazon Kinesis Managed Service for Real-Time Processing of Big Data App.1 Data Sources Availability Zone Data Sources Data Sources Availability Zone S3 App.2 AWS Endpoint Data Sources Availability Zone [Aggregate & De-Duplicate] Shard 1 Shard 2 Shard N [Metric Extraction] DynamoDB App.3 [Sliding Window Analysis] Redshift Data Sources App.4 [Machine Learning]
  • 11. Putting data into Kinesis Managed Service for Ingesting Fast Moving Data • Streams are made of Shards • • Each shard ingests up to 1MB/sec of data and up to 1000 TPS • All data is stored for 24 hours • • A Kinesis stream is composed of multiple Shards You scale Kinesis streams by adding or removing Shards Simple PUT interface to store data in Kinesis • Producers use a PUT call to store data in a stream • A Partition Key is used to distribute the PUTs across Shards • A unique Sequence # is returned to the Producer upon a successful PUT call
  • 12. Getting data out of Kinesis Client library for fault-tolerant, at least-once, real-time processing • In order to keep up with the stream, your application must: • • Be fault tolerant, to handle failures in hardware or software • • Be distributed, to handle multiple shards Scale up and down as the number of shards increase or decrease Kinesis Client Library (KCL) helps with distributed processing: • • Automatically starts a Kinesis Worker for each shard • Changes number of Kinesis Workers as number of shards changes • • Simplifies reading by abstracting your code from individual shards Uses checkpoints to keep track of a Worker’s location in the stream Use the KCL with Auto Scaling Groups • Auto Scaling policies will restart EC2 instances if they fail • Automatically add EC2 instances when load increases
  • 13. Amazon Kinesis Resources • Sign up for Limited Preview – http://aws.amazon.com/kinesis/limited-preview/ – Get SDK and Endpoint information after getting into Limited Preview • Getting Started Guide – http://docs.aws.amazon.com/kinesis/latest/dev/getting-started.html • Developer Guide – http://docs.aws.amazon.com/kinesis/latest/dev/introduction.html
  • 14. Amazon WorkSpaces Fully Managed Desktops in the Cloud
  • 15. Sample Use Cases (there are many more) • Mobile Device Access • Secure WorkSpaces • Remote Employees • Seasonal Workers • Student WorkSpaces • Developer WorkSpaces
  • 16. Key Benefits • Fully Managed • Support Multiple Devices • Keep Data Secure and Available • Choose Software & Hardware • Pay as You Go • Corporate Directory Integration
  • 17. Fully Managed WorkSpaces • Launch the number of WorkSpaces needed • All heavy lifting taken care of by AWS • Users receive email to install clients and connect
  • 18. Support Multiple Devices • • • • • iPad Kindle Fire HDX (Keyboard & Mouse) Android Tablet Microsoft Windows Mac
  • 19. Keep Data Secure and Available • No data stored on end-user device • Only Pixels delivered to users (PCoIP) • User volume backed by Amazon S3
  • 20. Choose Software and Hardware WorkSpaces Bundle Hardware Resources Applications Standard 1 vCPU, 3.75 GiB Memory, 50 GB User Storage Utilities (Adobe Reader, Internet Explorer 9, Firefox, 7-Zip, Adobe Flash, JRE) Standard Plus 1 vCPU, 3.75 GiB Memory, 50 GB User Storage Microsoft Office Professional 2010, Trend Micro Worry-Free Business Security, Utilities (Adobe Reader, Internet Explorer 9, Firefox, 7-Zip, Adobe Flash, JRE) Performance 2 vCPU, 7.5 GiB Memory, 100 GB User Storage Utilities (Adobe Reader, Internet Explorer 9, Firefox, 7-Zip, Adobe Flash, JRE) Performance Plus 2 vCPU, 7.5 GiB Memory, 100 GB User Storage Microsoft Office Professional 2010, Trend Micro Worry-Free Business Security, Utilities (Adobe Reader, Internet Explorer 9, Firefox, 7-Zip, Adobe Flash, JRE) All WorkSpaces Bundles provide the Windows 7 Experience to users (provided by Windows Server 2008 R2 with RDS).
  • 21. Pay as You Go WorkSpaces Bundle Standard $35 Standard Plus $50 Performance $60 Performance Plus • • • Monthly Price $75 No up-front commitment Delete WorkSpaces at any time Price includes infrastructure (compute, storage, bandwidth) and bundle’s software
  • 22. Corporate Directory Integration • Users: Get to use existing Enterprise Credentials • IT: WorkSpaces control like regular desktops
  • 23. Getting Started – What steps do customers take? • • • • • • Integrate with Corporate Active Directory Choose WorkSpaces Bundle Select Users to receive WorkSpaces Launch WorkSpaces Users receive email when provisioned Users connect to WorkSpaces
  • 24. Availability • Now: Limited Preview • Q1 2014: Public Beta (Initially US-West & US-East) • Q1/Q2 2014: Region Expansion
  • 25. Resources aws.amazon.com/WorkSpaces • • • • • Detail Page Pricing Limited Preview Sign-Up Documentation will follow at Public Beta Contact aws-bdms-workspaces@amazon.com – Prioritizing customers in Limited Preview, specific briefings
  • 26.
  • 27. AWS CloudTrail (beta) AWS CloudTrail is a web service that records AWS API calls for your account and delivers log files to you. The recorded information includes: • • • • • The identity of the API caller The time of the API call The source IP address of the API caller The request parameters The response elements returned by the AWS service
  • 28. AWS CloudTrail Use Cases • Security Analysis – You can use the AWS API call history produced by CloudTrail as an input into log management and analysis solutions to perform security analysis and to detect user behaviour patterns. • Track Changes to AWS Resources – You can use the AWS API call history produced by CloudTrail to track changes to AWS resources, including creation, modification, and deletion of AWS resources such as Amazon EC2 instances, Amazon VPC security groups and Amazon EBS • Troubleshoot Operational Issues – You can use the AWS API call history produced by CloudTrail to troubleshoot operational issues. For example, you can quickly identify the most recent changes made to resources in your environment. • Compliance Aid – CloudTrail makes it easier to ensure compliance with internal policies and regulatory standards by providing AWS API call history. Integrates with AWS Partner solutions like Alert Logic
  • 29. Features and Benefits • Increased Visibility – CloudTrail provides increased visibility into your user activity by recording AWS API calls. – You can answer questions such as, what actions did a given user take over a given time period? For a given resource, which user has taken actions on it over a given time period? What is the source IP address of a given activity? Which activities failed due to inadequate permissions?
  • 30. Features and Benefits • Durable and Inexpensive Log File Storage – CloudTrail uses Amazon S3 for log file storage and delivery, so log files are stored durably and inexpensively. – You can use Amazon S3 lifecycle configuration rules to further reduce storage costs. For example, you can define rules to automatically delete old log files or archive them to Amazon Glacier for additional savings.
  • 31. Features and Benefits • Easy Administration – CloudTrail is a fully managed service. – You simply turn on CloudTrail for your account using the AWS Management Console, the Command Line Interface, or the CloudTrail SDK and start receiving CloudTrail log files in the Amazon Simple Storage Service (Amazon S3) bucket that you specify.
  • 32. Features and Benefits • Reliability – CloudTrail continuously transports events from AWS services using a highly available and fault tolerant processing pipeline. – Turning on CloudTrail has no impact on performance of your AWS resources or API call latency.
  • 33. Features and Benefits • Timely Delivery & Notification – CloudTrail typically delivers events within 15 minutes of the API call and can be configured to publish a notification for each log file delivered. – This feature enables you to automatically take action upon log file delivery. CloudTrail uses the Amazon Simple Notification Service (SNS) for notifications.
  • 34. Features and Benefits • Log File Aggregation – CloudTrail can be configured to aggregate log files across multiple accounts and regions. – If you use multiple AWS regions, you can choose where log files are delivered for each region. For example, you can have a separate Amazon S3 bucket for each region, or you can aggregate log files from all regions in a single S3 bucket.
  • 35. Features and Benefits • Choice of Partner Solutions – Multiple partners are available including AlertLogic, Boundary, Loggly, Splunk and Sumologic. – These partners offer integrated solutions to analyze CloudTrail log files. These solutions include features like change tracking, troubleshooting, and security analysis.
  • 36. Getting Started CloudTrail can be turned on in as few as two clicks from the AWS Management Console. CloudTrail generates log files containing detailed information about API calls made, and periodically saves these files into an Amazon S3 bucket of your choosing. You can also choose to create an SNS topic to receive a notification every time a new log file is delivered. • • • To turn on CloudTrail, just provide a name for an Amazon S3 bucket where you want your log files delivered. If you use multiple AWS regions, you can choose where log files are delivered for each region. For example, you can have a separate Amazon S3 bucket for each region, or you can aggregate log files from all regions in a single S3 bucket. There is no additional charge for CloudTrail, but standard rates for Amazon S3 and Amazon SNS usage apply.
  • 37. Amazon AppStream Stream resource-intensive applications from the Cloud
  • 38. Sample Use Cases • • • • Games Media and Entertainment Simulation Software 3D Graphics Development
  • 39. Key Benefits • Remove Device • Instant On Constraints • Improved Security • Multi Platform Support • Automatic Scaling • Easy Updates
  • 40. Remove Device Constraints • No longer a need to snap to the device capabilities • AppStream helps reach broadest audience • Users get rich experience across devices
  • 41. Multi Platform Support • • • • • Kindle Fire Android iOS Windows 7 Mac OS (2014)
  • 42. Easy Updates • Provide a new application version to AppStream • Immediately upgrade all users • No action on users’ part
  • 43. Instant On • Users can start using application immediately • No large file downloads • No time consuming installations
  • 44. Improved Security • Create Entitlement Service to authorize connections • Clients connect to Entitlement Service for access • Your application is secured in AWS Cloud
  • 45. Automatic Scaling • Set the limits for scaling of your application • The AppStream service takes care of scaling • AppStream uses the G2.2XLarge Instance – Support for more instance types coming in 2014 • Focus on your application, not infrastructure
  • 46. Getting Started – What steps do customers take? • • • • Integrate AppStream SDK with application Deploy Application to AppStream Create Entitlement Service Create Client Apps
  • 47. Availability • Now: Limited Preview • Public Beta Date: TBD
  • 48. Amazon RDS for PostgreSQL Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the cloud. With Amazon RDS, you can deploy scalable PostgreSQL deployments in minutes with costefficient and resizable hardware capacity.
  • 49. What does RDS for PostgreSQL enable? • • • • Large-scale web applications Internal and departmental applications Excellent vehicle for ETL into analytics engines Geospatial and mobile applications
  • 50. Benefits of RDS for PostgreSQL • Easy, managed deployments – Free up time from undifferentiated admin tasks – Database instances with pre-configured parameters, automated patching, monitoring, and notifications • Familiar environment for developers – All your existing PostgreSQL apps work • Fast, predictable performance – Use Provisioned IOPS to tune on the fly • Backup and recovery – Automated backups and database snapshots • High availability – Deploy in multiple Availability Zones Available in all regions
  • 51. Benefits of PostgreSQL for AWS Customers • The preferred open-source database for many enterprise developers and startups • Support for geospatial queries using the PostGIS extensions • Support for full-text search • Support for advanced data types: JSON and key/value stores (“hstore”)
  • 52. Getting started with Amazon RDS for PostgreSQL • Review the Getting Started Guide for RDS http://docs.aws.amazon.com/gettingstarted/lates t/awsgsg-intro/gsg-aws-intro.html • Review the RDS Free Tier http://aws.amazon.com/free • Create a PostgreSQL DB instance from the RDS console https://console.aws.amazon.com/rds/
  • 53. Resources for Amazon RDS for PostgreSQL • User Guide http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/ • Data Import Guide http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/P ostgreSQL.Procedural.Importing.html • Common tasks, including setting up PostGIS geospatial extensions http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/A ppendix.PostgreSQL.CommonDBATasks.html • RDS forum https://forums.aws.amazon.com/forum.jspa?forumID=60
  • 54. Details for Developers • Launching with PostgreSQL 9.3.1 • Wide selection of available instances – Including the new high-memory db.cr1.8xlarge • Choose Multi-AZ deployment for high availability – Synchronous replication to a secondary in a different AZ • Use Provisioned IOPS for predictable performance – Convert to IOPS with a brief availability impact – Then add IOPS or storage on the fly
  • 55. What’s new in AWS Amazon Kinesis Managed Service for Real-time Big Data Processing Amazon WorkSpaces Fully Managed Desktops in the Cloud AWS CloudTrail (beta) Get log files of API call made on your AWS account Amazon AppStream Stream resource-intensive applications from the Cloud Amazon RDS for PostgreSQL (beta) Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the cloud Talk to someone from AWS or our partners at Stand 1070

Hinweis der Redaktion

  1. Nasdaq sending in all market order data across exchanges in real-time into Kinesis , so that their real-time processing workers (built with Kinesis libraries) would maintain a consolidated audit trail that collects and accurately identifies every order, cancellation, modification and trade execution for all exchange-listed equities and options across all U.S. markets. Bizo, an AdTech company – ingesting customers online advertising performance data in real-time to generating real-time metrics, KPIs about Ad performance, conversion, yield, and more. They would previously do this end of day with a Hadoop job.
  2. Stream of metering records.Incremental bill computation, uploaded every few minute to Redshift so that customers can query their estimated bill.Archive metering records, aggregated to 1 hour buckets, in S3Billing alerts in real-time.
  3. [2 minutes]KINESIS is a new service that scales elastically for near realtime processing of streaming big data. The service will store large streams of data in durable, consistent storage, reliably, for near realtime processing of data by an elastically scalable fleet of data processing servers. Large streams means millions of records per second, GBs of data per second and near real-time means order of a few secondsStreaming data processing has two layers: a storage layer and a processing layer. The storage layer needs to support specialized ordering and consistency semantics that enable fast, inexpensive, and replayable reads and writes of large streams of data. Kinesis is the storage layer in Kinesis / Kinesis. The processing layer is responsible for reading data from the storage layer, processing that data, and notifying the storage layer to delete data that is no longer needed. Kinesis supports the processing layer. Customers compile the Kinesis library into their data processing application. Kinesis notifies the application (the Kinesis Worker) when there is new data to process. The Kinesis / Kinesis control plane works with Kinesis Workers to solve scalability and fault tolerance problems in the processing layer.
  4. Key Questions: Is 24 hour retention OK?At its core, Kinesis’s storage system is a high-performance, strongly-consistent, replicated log system. Kinesis employs chain replication, which maintains “high throughput and availability without sacrificing strong consistency guarantees.” Chain replication, running on commodity hardware, enables us to drive to a low entitlement cost for our append only workload. Elastic scale-out is achieved by employing many independent AlfBus logs, called Chain Pools, while high availability and durability is achieved by spreading each Chain Pool across three different availability zones. Finally, Kinesis’s high availability model, which is designed to withstand AZ failures, and solves backpressure issues that result from lost hosts such or prevents overprovisioning as the issues described by Flipboard.Finally, testing suggests that a TP99.9 latency of less than 4 seconds can be achieved for data flowing through Kinesis. Kinesis’s high availability and strong consistency is central to near real-time processing. Partition KeysWhen records are added to a Kinesis Stream a Partition Key must be provided to Kinesis so that the record can be directed at the appropriate shard within the stream Kinesis currently will generate an MD5 hash of the provided partition key to determine the proper shard to place the recordIt is important to not exceed the per shard data rate of any one unique partition key Kinesis guarantees that records will be placed in order with in a data stream for each unique partition key
  5. Kinesis Client library simplifies parallel processing of streaming big data by allowing customers to write simple applications for processing records. Kinesis is a Java library that customers compile into their data processing application. The application with the Kinesis library is called a Kinesis Worker. Kinesis’s library notifies the customer’s processing code when there is new data to process. Kinesis Workers will often process data and write output to an external data store such as DynamoDB, EMR, Redshift, or S3. Kinesis guarantees that every record within a Kinesis Data stream is processed at least onceThe grey box on the top right – will be EC2 instances in your space. Within there is the Kinesis worker process in green. That is your specific application and business logic. That logic leverages our libraries that connect back to Kinesis (in pink on the left) . There is also another VIP on the GET side.
  6. Amazon WorkSpaces is a fully managed desktop computing service in the cloud. Amazon WorkSpaces allows customers to easily provision cloud-based desktops that allow end-users to access the documents, applications and resources they need with the device of their choice, including laptops, iPad, Kindle Fire, or Android tablets.
  7. Amazon WorkSpaces can provide value to customers in numerous scenarios where they might use desktops. The list of use cases here is not exhaustive and is just a sample to set context with customers. There are many other scenarios, too numerous to list, where customers might find WorkSpaces would provide value to them.
  8. This slide lists the key benefits of WorkSpaces. The following slides in the presentation go into more detail on each benefit.
  9. Amazon WorkSpaces is a fully managed service. All the heavy lifting and provisioning of hardware and software is done for customers by AWS. All they need to do is launch the number of WorkSpaces they need. The service takes care of all underlying provisioning processes – once the WorkSpaces are ready, users will receive an email providing them with instructions on how to obtain a client and connect to their WorkSpaces.
  10. Amazon WorkSpaces provides customers with a choice of devices they can use to connect to their desktop. They can use an iPad, a Kindle Fire HDK (including the ability to use a keyboard and mouse), a Windows or Mac desktop. The iPad and Android clients have numerous optimizations to make a desktop experience on the device intuitive, such as a slide out radial control to access commonly used functions and a choice of mouse modes.
  11. Amazon WorkSpaces delivers only pixels to users, using the Teradici PCoIP protocol, and customer data does not stay on the end user’s device. The user volume provided for a user’s WorkSpace is regularly backed up to Amazon S3 as a snapshot, helping ensure data durability even in the case of hardware failure.
  12. With Amazon WorkSpaces, customers have a choice of WorkSpaces Bundles. There are two hardware choices (Standard and Performance) and a “Plus” option for each hardware choice allowing customers to purchase software via AWS if they would like to do so. Customers always have the opportunity to install any software they like onto their WorkSpace at any time, subject to the conditions of any applicable licensing agreement.
  13. Amazon WorkSpaces features Pay as You Go pricing. There is no up-front commitment required from customers; WorkSpaces can be deleted at any time. The monthly WorkSpaces fee includes all necessary infrastructure (compute, storage and bandwidth) and the appropriate software for the bundle selected by the customer.
  14. Amazon WorkSpaces integrates with customers’ Corporate Directories. This means that all WorkSpaces provisioned by a customer will join the customer’s Active Directory domain. This means that users can continue to use their existing corporate credentials to get seamless access to corporate resources (eg. Exchange, Sharepoint, other internal applications). This also means that for administrators, as the WorkSpaces join the customer’s Active Directory domain, they can be managed just like any other desktops with management tools or processes that customers are already using.
  15. Once customers have access to the WorkSpaces service, getting started is simple. If customers want integration with their corporate Active Directory, they will need to have a VPC configured with a hardware VPN connection back to their corporate network. Once they’ve configured their directory, they just need to select the WorkSpaces Bundle they require, choose the users who will receive WorkSpaces and launch those WorkSpaces. Once the WorkSpaces are provisioned (which will include them joining the customer’s Active Directory domain if they are integrating their directory), users get an email telling them how they can install the client and connect to their WorkSpaces
  16. WorkSpaces is available now as a Limited Preview and customers can join the waitlist on the details page. Public Beta is planned for Q1 2014 (Active Directory integration will follow the initial Public Beta which will offer “Cloud Only” mode, where WorkSpaces will join a standalone directory) and in Q1/Q2 the service will be rolled out to further AWS regions.
  17. The WorkSpaces detail page provides information about the service benefits, pricing and allows customers to sign up. For customers to be prioritized in the Limited Preview or for specific briefing requests, please contact the WorkSpaces BDM team using the email address above.
  18. Amazon AppStream is a flexible, low-latency service that lets customers stream resource intensive applications and games from the cloud. It deploys and renders applications on AWS infrastructure and streams the output to mass-market devices, such as personal computers, tablets, and mobile phones.
  19. AppStream does not have a fixed set of use cases; it’s a service that allows customers to build many different types of applications. The use cases listed on this slide are samples only.
  20. Here are the key benefits of AppStream, which will be covered in more detail later on in the deck.
  21. Traditionally, developers had to make tough trade-offs for applications. If they wanted to provide the richest possible experience for their users, they might have had to require high end PC hardware and thus limited the reach of their application. If they opted for lower specifications then they had to potentially provide a lower quality experience for users, or engineer different versions of their application for multiple different devices. With AppStream, developers no longer have to make these trade-offs. They can deliver a rich and immersive experience to users across a range of different devices and leave the heavy lifting of the complex graphical work to AWS.
  22. AppStream supports multiple different devices as per the detail on the slide, with support for Mac OS coming in 2014. This enables developers to have a broad reach with their applications.
  23. When developers update their applications, all they need to do is provide a new version of the application to the AppStream service. All users will immediately get the updated version with no action or lengthy updates or downloads for the user to wait for.
  24. When users start to use an application that is running from AppStream, they get a great “Instant On” experience. They don’t have to wait for lengthy downloads or installation procedures- they just start using the application straight away.
  25. AppStream provides developers with increased security. Their application remains in the AWS cloud, not on end user devices, protecting it against analyses such as reverse engineering. Developers create an entitlement service (a sample is provided with the SDK) which clients connect to ensure entitlement to the application. Once entitled, the application will be streamed.
  26. Developers set limits for scaling the application and the service takes care of the scaling within those constraints. AppStream uses G2.2XLarge instances today; support for other instance types will come in 2014. AppStream lets developers focus on developing amazing applications, not undifferentiated heavy lifting.
  27. To get started with AppStream, once customers have access to the service, they simply integrate their application with the AppStream SDK. Then, they deploy their application to AppStream. They create an entitlement service (sample provided in SDK), create client applications for whichever platform they choose…and then end users can enjoy those applications.
  28. AppStream is available as a Limited Preview now. Customers can sign up to be added to the waitlist. We will communicate details of timing for Public Beta soon.