A preliminary version of my OSCON 2011 talk, given for the Raleigh Ruby Brigade on June 21, 2011.

1. OpenStack APIs: Present and
Future
OSCON Beta Talk
H. Wade Minter
Chief Architect, TeamSnap Inc.
@minter

2. What Is OpenStack?
OpenStack is a global collaboration of developers and cloud
computing technologists producing the ubiquitous open source
cloud computing platform for public and private clouds. The
project aims to deliver solutions for all types of clouds by being
simple to implement, massively scalable, and feature rich. The
technology consists of a series of interrelated
projects delivering various components for a cloud
infrastructure solution.

3. Community Members And Sponsors
• Rackspace Hosting (Storage)
• NASA (Compute)
• Citrix
• Dell
• Peer1
• Over 70 others

4. Why Does OpenStack Matter?
"All of the code for OpenStack is freely available under the
Apache 2.0 license. Anyone can run it, build on it, or submit
changes back to the project. We strongly believe that an open
development model is the only way to foster badly-needed
cloud standards, remove the fear of proprietary lock-in for cloud
customers, and create a large ecosystem that spans cloud
providers."
Run your own API-driven internal cloud and get access to
a wealth of community resources already built on top of it.

5. OpenStack Technologies

6. The OpenStack Universe
• Storage (Swift)
• Compute (Nebula)
• Image Service (Glance)
• Authentication (Keystone)

7. OpenStack Storage
"Swift"

8. OpenStack Storage
• Terminology: "Containers" and "Storage Objects"
• Scalable back-end storage for files
• Replicates objects in software to multiple drives
• Multi-tenant with integrated reseller support
• Scales to multi-petabytes and billions of files
• Stores objects of unlimited size (in 5GB chunks)
• Software written in Python
• API-driven access: Native API or S3-Compatible (under
development)

9. OpenStack Storage Native API
• REST API
• Accepts data in either XML or JSON
• Returns data in either XML or JSON
• Authenticates with Username and API Key, returns access
token.
• Well-documented.

10. CURL Example - Authentication
$ curl
-H 'X-Auth-User: minter'
-H 'X-Auth-Key: XXXX'
-I
-X GET
https://auth.api.rackspacecloud.com/v1.0
HTTP/1.1 204 No Content
Server: Apache/2.2.13 (Red Hat)
X-Storage-Url: https://storage101.dfw1.clouddrive.com/v1/
MossoCloudFS_012559c7-030d-4bbd-8538-7468896b0e7f
Content-Type: application/octet-stream
Date: Sat, 18 Jun 2011 18:10:07 GMT
X-Auth-Token: 6312661f-1a18-41c8-a823-d28e44b6c2f9
X-Storage-Token: 6312661f-1a18-41c8-a823-d28e44b6c2f9
X-Server-Management-Url: https://servers.api.rackspacecloud.com/
v1.0/361291
Connection: Keep-Alive
X-CDN-Management-Url: https://cdn.clouddrive.com/v1/
MossoCloudFS_012559c7-030d-4bbd-8538-7468896b0e7f
Content-Length: 0

11. CURL Example - List Containers
$ curl
-H 'X-Auth-Token: 6312661f-1a18-41c8-a823-
d28e44b6c2f9'
-X GET https://
storage101.dfw1.clouddrive.com/v1/
MossoCloudFS_012559c7-030d-4bbd-8538-7468896b0
e7f
.CDN_ACCESS_LOGS
Books
CWX
Disk Images
Raw Video
[...]

12. CURL Example - List Containers (JSON)
$ curl
-H 'X-Auth-Token: 6312661f-1a18-41c8-a823-
d28e44b6c2f9'
-X GET
https://storage101.dfw1.clouddrive.com/v1/
MossoCloudFS_012559c7-030d-4bbd-8538-7468896b0
e7f?format=json
[{"name":".CDN_ACCESS_LOGS","count":
49,"bytes":31871},{"name":"Books","count":
9,"bytes":48055492},{"name":"CWX","count":
14,"bytes":37933163},{"name":"Disk
Images","count":4,"bytes":6062775740},
{"name":"Raw Video","count":399,"bytes":
29976077454},[....]]

13. CURL - Create Container
$ curl
-I
-H 'X-Auth-Token: 6312661f-1a18-41c8-a823-
d28e44b6c2f9'
-X PUT
https://storage101.dfw1.clouddrive.com/v1/
MossoCloudFS_012559c7-030d-4bbd-8538-7468896b0
e7f/a_test_container
HTTP/1.1 201 Created
Content-Length: 18
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txaa7b23c0fb53437cb29b3720102c0528
Date: Sat, 18 Jun 2011 18:23:47 GMT

14. CURL - Create Object With Data
$ curl
-T -
-I
-H 'Content-Length: 28'
-H 'Content-Type: text/plain'
-H 'X-Auth-Token: d17f67b1-13da-4352-b769-b9dfe9f06458'
-X PUT
https://storage101.dfw1.clouddrive.com/v1/
MossoCloudFS_012559c7-030d-4bbd-8538-7468896b0e7f/
a_test_container/first_object.txt
HTTP/1.1 100 Continue
This is the body of my file
HTTP/1.1 201 Created
Content-Length: 118
Content-Type: text/html; charset=UTF-8
Etag: 4ca503d05d7e613d963fac927ac4ce3e
Last-Modified: Sat, 18 Jun 2011 20:02:59 GMT
X-Trans-Id: tx55427a97e6264ecc9c51edfa7bcf096c
Date: Sat, 18 Jun 2011 20:03:21 GMT

15. But There Has To Be An Easier Way
Language bindings for Rackspace Cloud Files are available.
Since OpenStack Storage is Cloud Files minus the CDN, you
can use those.
Official bindings (github.com/rackspace):
• Ruby
• Python
• PHP
• C#
• Java

16. Ruby Cloud Files Gem
https://github.com/rackspace/ruby-cloudfiles
gem install cloudfiles
require 'cloudfiles'
Patches welcome!

17. Ruby - OpenStack Authentication
irb(main):002:0> require 'cloudfiles'
=> true
irb(main):003:0> cf = CloudFiles::Connection.new(:auth_url
=> 'http://swift.chmouel.com:8080/auth/v1.0', :username =>
'test:tester', :api_key => 'XXXX')
=> #<CloudFiles::Connection:0x10194d700
@storagescheme="http", @authok=true, @retry_auth=true,
@authuser="test:tester", @storageport=8080, @proxy_host=nil,
@authtoken="AUTH_tkcb32965fb22c4108b0d8510bcbb86854",
@storagepath="/v1/AUTH_01835552-7bf4-4b00-9b90-
ab9b4e82e359", @proxy_port=nil, @auth_url="http://
swift.chmouel.com:8080/auth/v1.0", @http={},
@storagehost="127.0.0.1", @snet=nil, @authkey="XXX">

18. Ruby - Cloud Files Authentication
irb(main):001:0> require 'cloudfiles'
=> true
irb(main):002:0> cf = CloudFiles::Connection.new(:username
=> 'minter', :api_key => 'XXXX')
=> #<CloudFiles::Connection:0x10196d9b0 @http={},
@storagehost="storage101.dfw1.clouddrive.com",
@cdnmgmtscheme="https", @snet=nil, @cdn_available=true,
@proxy_port=nil, @authkey="XXXX", @storagescheme="https",
@authok=true, @cdnmgmtport=443, @retry_auth=true,
@authuser="minter", @storageport=443, @proxy_host=nil,
@cdnmgmtpath="/v1/
MossoCloudFS_012559c7-030d-4bbd-8538-7468896b0e7f",
@authtoken="3f9a3610-beaf-488f-8270-5f54f42619af",
@storagepath="/v1/
MossoCloudFS_012559c7-030d-4bbd-8538-7468896b0e7f",
@cdnmgmthost="cdn.clouddrive.com", @auth_url="https://
auth.api.rackspacecloud.com/v1.0">

19. Ruby - Containers
irb(main):003:0> cf.containers
=> [".CDN_ACCESS_LOGS", "Books", "CWX", "Disk
Images", "Raw Video", "a_test_container",...]
irb(main):004:0> c =
cf.create_container('another_container')
=> #<CloudFiles::Container:0x101924120
@name="another_container"....>
irb(main):006:0> cf.containers
=> [".CDN_ACCESS_LOGS", "Books", "CWX", "Disk
Images", "Raw Video", "a_test_container",
"another_container",...]

20. Ruby - Objects
irb(main):007:0> c =
cf.container('another_container')
=> #<CloudFiles::Container:0x1018be668
@name="another_container", ...>
irb(main):008:0> o = c.create_object('myfile.txt')
=> #<CloudFiles::StorageObject:0x1018ac170
@make_path=false, @name="myfile.txt",...>
irb(main):009:0> o.write("O HAI WORLD")
=> true
irb(main):010:0> o.data
=> "O HAI WORLD"

21. Ruby - Convenience Methods
irb(main):012:0> o = c.create_object("Code
Monkey.mp3")
=> #<CloudFiles::StorageObject:0x101849570
@make_path=false, @name="Code Monkey.mp3",...>
irb(main):014:0> o.load_from_filename('/Users/
minter/Music/iTunes
/iTunes Music/Jonathan Coulton/JoCo Looks
Back/01 Code Monkey.mp3')
=> true
irb(main):015:0> o.bytes
=> "7636175"

22. Other Notable API Operations
• Set metadata for containers and objects
• Server-side copy of objects
• Create pseudo-hierarchical directory support
• Offset/marker support for large collections
• Fetching object data via byte offset and range
• Chunked encoding for uploads
• Manifest support for objects >5G

23. Changing The Content-Type
$ curl -I -H "X-Auth-Token: XXXXX" http://
swift.chmouel.com:8080/v1/AUTH_test/krack/H.
%20Wade%20Minter%20%2D%20RADIO%20H%20%2D
%20KRACK%20Live%20%281%2D15%2D2010%29.mp3
HTTP/1.1 200 OK
Last-Modified: Tue, 21 Jun 2011 14:56:05 GMT
Etag: 608cc90b0a516fed3e10b869e5c2cb6a
X-Object-Meta-Foo: bar
Accept-Ranges: bytes
Content-Length: 88434590
Content-Type: application/octet-stream
Date: Tue, 21 Jun 2011 15:02:04 GMT

24. Do A Copy To Itself
$ curl -I -X PUT -H "X-Auth-Token: XXX"
-H "X-Copy-From: /krack/H.%20Wade%20Minter%20%2D%20RADIO
%20H%20%2D%20KRACK%20Live%20%281%2D15%2D2010%29.mp3"
-H "Content-Type: audio/mpeg"
-H "Content-Length: 0"
http://swift.chmouel.com:8080/v1/AUTH_test/krack/H.%20Wade
%20Minter%20%2D%20RADIO%20H%20%2D%20KRACK%20Live
%20%281%2D15%2D2010%29.mp3
HTTP/1.1 201 Created
Content-Length: 118
Content-Type: text/html; charset=UTF-8
Etag: 608cc90b0a516fed3e10b869e5c2cb6a
X-Copied-From: krack/H.%20Wade%20Minter%20-%20RADIO%20H%20-
%20KRACK%20Live%20%281-15-2010%29.mp3
X-Copied-From-Last-Modified: Tue, 21 Jun 2011 14:56:05 GMT
X-Object-Meta-Foo: bar
Last-Modified: Tue, 21 Jun 2011 15:05:40 GMT
Date: Tue, 21 Jun 2011 15:05:42 GMT

25. Check The Results
$ curl -I
-H "X-Auth-Token: XXXX"
http://swift.chmouel.com:8080/v1/AUTH_test/
krack/H.%20Wade%20Minter%20%2D%20RADIO%20H
%20%2D%20KRACK%20Live20%281%2D15%2D2010%29.mp3
HTTP/1.1 200 OK
Last-Modified: Tue, 21 Jun 2011 15:05:40 GMT
Etag: 608cc90b0a516fed3e10b869e5c2cb6a
X-Object-Meta-Foo: bar
Accept-Ranges: bytes
Content-Length: 88434590
Content-Type: audio/mpeg
Date: Tue, 21 Jun 2011 15:05:53 GMT

26. Or Do It The Easy Way (git HEAD)
irb(main):012:0> o = c.object(c.objects.first)
=> #<CloudFiles::StorageObject:0x10174f4a8
@name="H. Wade Minter - RADIO H - KRACK Live
(1-15-2010).mp3", ...>
irb(main):013:0> o.content_type
=> "application/octet-stream"
irb(main):014:0> o.content_type = "audio/mpeg"
=> "audio/mpeg"
irb(main):015:0> o.refresh
=> true
irb(main):016:0> o.content_type
=> "audio/mpeg"
irb(main):017:0>

27. Second-Level Tools
• Fog - Cloud abstraction library that supports both Cloud
Files and Cloud Servers
• paperclip-cloudfiles - Fork of Thoughtbot's paperclip with
support for Cloud Files and OpenStack
• Carrierwave - Rack/Rails asset storage
• Wordpress CDN Tools - Store assets on Cloud Files
• Safe - backup tool for local or cloud storage

28. Example Time
Simple Sinatra app interfacing with
Cloud Files

29. OpenStack Compute
Under Construction

30. OpenStack Compute API 1.0
(Cloud Servers)
• In production now on Rackspace Cloud Servers
• Based on Slicehost technology and API
• Integrated server image support
• Hooks into Cloud Files for backup storage
• IPV4 Only

31. OpenStack Compute API 2.0
(NASA Nebula)
Enhancements:
• IPv6 Support
• Returns resources in full URLS (for accessing resources
provided elsewhere)
Differences:
• No support for shared IP groups
• No support for backup schedules
• Image service soon to move to its own project (Glance)
• Adjustable limits (rates, personality files, etc)

32. References
OpenStack: http://openstack.org/
Rackspace Cloud APIs: http://docs.rackspace.com/api/
Rackspace Github: https://github.com/rackspace/
Me:
• Twitter: @minter
• Github: https://github.com/minter/
• Email: minter@lunenburg.org
• My Startup: http://teamsnap.com/