My Presentation to Sydney JUG in Aug 2010

1. Playing God in the clouds...
SJUG Aug 2010 @Atlassian
Michael Neale
michael.neale@gmail.com
twitter.com/michaelneale
www.michaelneale.net
Copyright © 2010 Opscode, Inc - All Rights Reserved 1

2. Tiny bit of background...
Copyright © 2010 Opscode, Inc - All Rights Reserved 2

3. #define cloud
Copyright © 2010 Opscode, Inc - All Rights Reserved 3

4. SaaS
PaaS
IaaS <--
(NIST)
Copyright © 2010 Opscode, Inc - All Rights Reserved 4

5. IaaS:
Illusion of infinite
servers
Low level resources
Requires Sysadmin/
Devops
Copyright © 2010 Opscode, Inc - All Rights Reserved 5

6. IaaS - fire data centre
Copyright © 2010 Opscode, Inc - All Rights Reserved 6

7. PaaS - fire the
sysadmins
Copyright © 2010 Opscode, Inc - All Rights Reserved 7

8. SaaS - fire everone else
Copyright © 2010 Opscode, Inc - All Rights Reserved 8

9. #define devops
Copyright © 2010 Opscode, Inc - All Rights Reserved 9

10. Infrastructure as code
Useful tools:
jclouds
FOG
deltacloud
puppet/chef
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

11. What Does Cloud
Provisioning Look Like?
Copyright © 2010 Opscode, Inc - All Rights Reserved 11

12. Two Answers
Copyright © 2010 Opscode, Inc - All Rights Reserved 12

13. You building a platform
(building an image?)
Copyright © 2010 Opscode, Inc - All Rights Reserved 13

14. You deploying it
Copyright © 2010 Opscode, Inc - All Rights Reserved 14

15. Build your platform

16. Chef, puppet, by hand...
snapshotting
Copyright © 2010 Opscode, Inc - All Rights Reserved 16

17. Deploy to a (IaaS)
platform
http://www.byteonic.com/2009/overview-of-java-support-in-google-app-engine/

18. Command line, web UI,
or API
Copyright © 2010 Opscode, Inc - All Rights Reserved 18

19. cloud + devops =
better, faster, more
Copyright © 2010 Opscode, Inc - All Rights Reserved 19

20. “Enable the reconstruction of the
business from nothing but a source
code repository, an application data
backup, and “... a credit card
Copyright © 2010 Opscode, Inc - All Rights Reserved 20

21. Cloud Provisioning is
Step 1
Copyright © 2010 Opscode, Inc - All Rights Reserved 21

22. Cloud Provisioning is
Step 1
Provisioning
Copyright © 2010 Opscode, Inc - All Rights Reserved 21

23. Cloud Provisioning is
Step 1
Provisioning
Configuration
Copyright © 2010 Opscode, Inc - All Rights Reserved 21

24. Cloud Provisioning is
Step 1
Provisioning
Configuration
Systems Integration
Copyright © 2010 Opscode, Inc - All Rights Reserved 21

26. Cloudy
environments

27. Cloudy
environments
• automate EVERYTHING

28. Cloudy
environments
• automate EVERYTHING
• seed/snapshotting

29. Cloudy
environments
• automate EVERYTHING
• seed/snapshotting
• safely update configuration

30. github jclouds/
runs on the JVM (including )
(with-compute-service [compute]
(run-node "webserver"))
pis pis
d a ud a
lou clo
o c t of
k- in t -ou
loc lock
zero zero
nodes = compute.runNodesWithTag(“webserver”, 1);

31. jclouds
open source
feels like java (and clojure)
portability between clouds
deal with web complexity
unit testability
thread-safe and scalable

32. Cross cloud API
abstractions:
Copyright © 2010 Opscode, Inc - All Rights Reserved 26

33. Cross cloud API
abstractions:
Generic binary storage
BlobStore
Copyright © 2010 Opscode, Inc - All Rights Reserved 26

34. Cross cloud API
abstractions:
Generic binary storage
BlobStore
Generic service provisioning
ComputeService
Copyright © 2010 Opscode, Inc - All Rights Reserved 26

35. Cross cloud API
abstractions:
Generic binary storage
BlobStore
Generic service provisioning
ComputeService
but you have low level access when
need it
Copyright © 2010 Opscode, Inc - All Rights Reserved 26

36. jclouds-blobstore github jclouds/jclouds
blobstore = new BlobStoreContextFactory().createContext(
“s3”, identity, credential).getAsyncBlobStore();
blob = blobstore.newBlob(key);
blob.getMetadata().getUserMetadata().put("date", now());
blob.setPayload(snapshot());
blob.getPayload().setContentMD5(snapshotMD5());
blobstore.putBlob(containerName, blob)
.addListener(publishAMQP(key),sameThreadExecutor());

37. jclouds-compute github jclouds/jclouds
service = new ComputeServiceContextFactory().createContext(
“ec2”, identity, credential).getComputeService();
template = service.templateBuilder().fastest().build();
template.getOptions().runScript(installGemsAndRunChef);
nodes = service.runNodesWithTag(“webserver”, 5, template);

38. jclouds github jclouds/jclouds
service = new ComputeServiceContextFactory().createContext(
“rimuhosting”, user, password ).getComputeService();
template = service.templateBuilder().any().biggest().build();
template.getOptions().installPrivateKey(privateRSA)
.authorizePublicKey(publicRSA)
.runScript(installGemsAndRunChef);
nodes = service.runNodesWithTag(“webserver”, 5, template);

39. Common demoniator
Copyright © 2010 Opscode, Inc - All Rights Reserved 30

40. Common demoniator
BlobStore: OK
Copyright © 2010 Opscode, Inc - All Rights Reserved 30

41. Common demoniator
BlobStore: OK
ComputeService:
Not all clouds created equal, you may need to dive down
and “lock in” just a tiny bit.. I have !
Copyright © 2010 Opscode, Inc - All Rights Reserved 30

42. Common demoniator
BlobStore: OK
ComputeService:
Not all clouds created equal, you may need to dive down
and “lock in” just a tiny bit.. I have !
Still early days:
Copyright © 2010 Opscode, Inc - All Rights Reserved 30

43. Common demoniator
BlobStore: OK
ComputeService:
Not all clouds created equal, you may need to dive down
and “lock in” just a tiny bit.. I have !
Still early days:
No Volume abstractions, yes, no LBs, firewalls etc
Copyright © 2010 Opscode, Inc - All Rights Reserved 30

44. To avoid lockin:
Copyright © 2010 Opscode, Inc - All Rights Reserved 31

45. To avoid lockin:
IaaS Cloud primitives (my opinion on low level)
Volume management
Server provisioning **
Blob storage / Bulk data**
Firewall/access control
Image construction ***
Copyright © 2010 Opscode, Inc - All Rights Reserved 31

46. To avoid lockin:
IaaS Cloud primitives (my opinion on low level)
Volume management
Server provisioning **
Blob storage / Bulk data**
Firewall/access control
Image construction ***
** This is the currently successfully abstracted
ones - generic enough via various apis
Copyright © 2010 Opscode, Inc - All Rights Reserved 31

47. To avoid lockin:
IaaS Cloud primitives (my opinion on low level)
Volume management
Server provisioning **
Blob storage / Bulk data**
Firewall/access control
Image construction ***
** This is the currently successfully abstracted
ones - generic enough via various apis
*** plenty of “traditional” tools help here - chef !
Copyright © 2010 Opscode, Inc - All Rights Reserved 31

48. Bright future
Copyright © 2010 Opscode, Inc - All Rights Reserved 32

49. Bright future
Eucalyptus
In house “private clouds”
Copyright © 2010 Opscode, Inc - All Rights Reserved 32

50. Bright future
Eucalyptus
In house “private clouds”
OpenStack
Rackspace/NASA - comparable to EC2 in
scalecapability
Copyright © 2010 Opscode, Inc - All Rights Reserved 32

51. Bright future
Eucalyptus
In house “private clouds”
OpenStack
Rackspace/NASA - comparable to EC2 in
scalecapability
Copyright © 2010 Opscode, Inc - All Rights Reserved 32

52. Copyright © 2010 Opscode, Inc - All Rights Reserved 33

53. Thanks !
twitter.com/michaelneale
http://www.jclouds.org/
http://deltacloud.org/
http://www.openstack.org/
Copyright © 2010 Opscode, Inc - All Rights Reserved 33

54. Thanks !
twitter.com/michaelneale
http://www.jclouds.org/
http://deltacloud.org/
http://www.openstack.org/
Copyright © 2010 Opscode, Inc - All Rights Reserved 33