Matt Wood of AWS "Cloud Research" Europe April 2011 @ the Eagle Genomics Symposium

1. Cloud Research
Matt Wood
T E C H N O L O G Y E VA N G E L I S T

2. Hello.

3. Text

4. Thank you.

5. The Cloud by Example

6. The Cloud by Example

7. Infrastructure
services

8. ?

9. On demand

10. Pay as you go

11. Pay for what you use

12. Elastic capacity

13. Capacity
Estimated
demand
Time

14. Capacity
Infrastructure
Investment Estimated
demand
Time

15. Capacity
Infrastructure
Real
demand
Time

16. Capacity
Elastic
capacity
Real
demand
Time

17. Agility

18. Faster to
prototype

19. Faster to
production

20. Undifferentiated
heavy lifting

21. Tools for
accelerating
research

22. 300
225
150
75
Q4 2006
Q4 2007
Q4 2008
0
Q4 2009
Q4 2010

23. The Cloud by Example

25. Data management

26. Biomarker Warehouse
pre-clinical, clinical, 3rd party data and publications
;<./5'=>?6@' !)*(%"&&'
23,3415'61789:1' !#%&$(%&&&'
+,'-./01' !"#$%"&&'
6178170' 6A.7341' B817-135'
Estimated cost: 10 TB warehouse over 3 years

27. Data processing

29. http://cyclecomputing.com

30. http://web.mit.edu/stardev/cluster/

31. sudo gem install cloud-crowd
http://cyclecomputing.com
http://wiki.github.com/documentcloud/cloud-crowd

32. http://www.rightscale.com

33. Amazon Elastic MapReduce
Amazon EC2 Instances
End
Deploy Application
Hadoop Hadoop Hadoop
Elastic Elastic
MapReduce MapReduce
Hadoop Hadoop Hadoop Notify
Web Console, Command
line tools Input output
dataset results
Input
S3
Output
S3
Get Results
Input Data
bucket bucket
Amazon S3

34. Crossbow: Rapid whole genome SNP analysis
Preprocessed reads
Map: Bowtie
Sort: Bin and partition
Reduce: SoapSNP
Langmead B, Schatz MC, Lin, J, Pop M, Salzberg SL. Genome Biol 10(11): R134.

35. CloudBurst
Catalog k-mers Collect seeds End-to-end alignment
http://cloudburst-bio.sourceforge.net; Bioinformatics 2009 25: 1363-1369

36. ASSEMBLING GENOMES
140
million
454
reads
Image:
Ma)
Wood

37. BLAT @ U. PENN
Map 100 million, 100 base paired end reads
Quad core with 5 GB of RAM would take 16 days
30 high-memory instances; 32 hours; $195

38. HEAVY-ION COLLISIONS @
RHIC
Problem: Quark physics conference imminent
but no compute resources handy
Solution: NIMBUS context broker allowed
researchers to provision 300 nodes and get the
simulations done

39. Collaboration

40. http://aws.amazon.com/publicdatasets/

42. http://www.cloudbiolinux.com/

43. http://usegalaxy.org/cloud

45. Applications and
platforms

46. http://heroku.com

47. http://chempedia.com/

48. Security

49. Shared
responsibility

50. Requirement
based access

51. Certification

52. ISO 27001
+
SAS 70 Type II

53. PCI DSS
Level 1

54. Control objectives
Security organisation Employee lifecycle
Logical security Secure data handling
Physical security Environmental safeguards
Change management Incident handling
Availability and
Data integrity
redundancy

55. Data access
control

56. Identity and
access

57. Geographically
Independent buildings Separate flood zones
separated
Redundant Redundant
Highly monitored
power connectivity

58. Default deny
firewall

59. Security groups

60. DDOS
Man in the Middle
IP spoofing

61. Resource
isolation
Virtual Private Cloud

62. Customer’s isolated AWS resources
Subnet 1 Subnet 2
VPN Gateway Router
Secure VPN connection
over the internet
Subnet 3 Subnet 4
Customer’s network Amazon Web Services infrastructure

63. Dedicated
instances
Virtual Private Cloud

64. aws.amazon.com/security

65. Data stays local

66. aws.amazon.com

67. Thank you!

68. Q U E S T I O N S + C O M M E N T S
matthew@amazon.com
@mza
O N T W I T T E R