Big Data Camp LA 2014, Computable Object Store with Openstack swift and ZeroVM By Adrian Otto of Rackspace

1. Computable Object Store
With OpenStack Swift and ZeroVM
Date: June 14th, 2014
Adrian Otto, Principal Architect

2. Adrian Otto
Principal Architect
OASIS CAMP TC Co-Chair
Racker Since 2007
Solum PTL

3. 3
The Future

4. 4
The Future

5. 5
The Future

6. 6
Virtualization

7. 7
Cloud

8. 8
Containers

9. 9
Speed

10. 10
Pick Two
Cheap
Fast
Good

11. 11
Today
Fast
Fast
Fast

12. 12
Start

13. 13
~2-20 min

14. 14
< 5 sec

15. 15
< 5 ms

16. 16
ZeroVM

17. 17
ZeroVM
Threads
Ethernet Interfaces
Concurrency
1
0
0
Filesystems 0

18. 18
Principles
Small, Light, Fast Secure
Functional Hyper-Scalable
Open Source Embeddable

19. 19
C99, POSIX
API

20. 20
Single Thread
Deterministic

21. 21
IO
STDIN
STDOUT
STDERR
Input
Output
Log

22. 22
Restrictions
Memory
Timeout
Read/Write
Access Restricted
Advance Set Life
Advance Set Limit

23. 23
Architecture

24. 24
Restrictions
C1 Once loaded into memory, the binary is not writable, enforced by OS-level protection during execution.
C2 The binary is statically linked at a start address of zero, with the first byte of text at 64K.
C3 All indirect control transfers use a nacljmp pseudo-instruction (defined below).
C4 The binary is padded up to the nearest page with at least one hlt instruction (0xf4).
C5 The binary contains no instructions or pseudo-instructions overlapping a 32-byte boundary.
C6 All valid instruction addresses are reachable by a fall-through disassembly starting at the base address.
C7 All direct control transfers target valid instructions.

25. 25
Memory

26. 26
Why?
Secure

27. 27

28. 28
Inertia
F = ma

29. 29
Data Inertia
t = Bytes/Rate

30. 30
Network

31. 31
Speed

32. 32
Data Inertia
t = Bytes/Rate

33. 33
Why Not?

34. 34
ZeroCloud

35. 35
ZeroCloud Arch
Storage Node Storage Node Storage Node Storage Node Storage Node Storage Node Storage N…
Proxy Node Proxy Node Proxy N…
ZVMZVMZVM
ZVM ZVMZVM ZVM ZVMZVM ZVM

36. 36
ZeroCloud Arch

37. 37
Use Cases

38. 38
Transcoding
video.ogg video.mp4 video.flv

39. q75SaChR2rgbq8Cx76ZOq5vsmx3sNiWCJYbloXGxSlODhu4XXPVp50bss3uzyaeAvKCOcY114R2nfROukqVnuIGPPBMxi5Eknq1eSDT2ZEjG31C7afIcoKp6bf
yBSi805jgqzpCue32TwJ35aHOrGmp20WjuoBeKZvDvXT8jkcIMYuXzqU6IuXXV7AQmLRzrQFgpxTA7Hpgkxmd2TPAotAEa8sCLEI0nrmQAoFq4SUymOSGHsV
CyOtekNdnCuZGpwhWeyDsMyJt2o0enmDvXtzW6pc3SFrDMAqDyLwdg82AJDxr64kPW5U3eRvtnTolnLraBCa6Ii0M11q1gOZeKEWtCLEgbn3vrL2vXYFRIOk8
LNoxiPpXP4vjKxO5osapnvedDAqPSx3duQkNG2a7vgVU455K0fmqTV4Z2awji5AsHQWNNJB0zHQEZido4N17kzR8KlwMCiEYvAmkeu2JsGOSJtRIymn7Phznd
qFPr42t5nqOuqpfqLyMaPpNLaMiSCt4jOtCpX4i4C5Y4P7mxafJTQeQ335837zKSr6v8oedKnPKKHBzQnyEtqzzesJWkKnlaTLUF4TS8nOkWBSU0hUSItodiCdR
JUUOyK4j3TLo3tIMeX6RxTqLSCi65q0qwNE6O2NAYaFpuSsFqXImx5bAr6Vy6rE36aOgfR4r4CRBR75X8NP2e386Bd1pd6zIbi42jrbr5xB2kwOX1t5f8vRSIYid7C
hyMEfLmVhsjbGWDs86dNLN45nbW7VpRQUrRbdGISULhnFzFkblffBGlZWbNzGVkKIorJDGQgex0dXrapv3AVXbnyaQRckKCrwLVvZ20ftL8p2CJbmilHkbjN5d
m5BgaRNsoEljz8RMvVtuSFwvvj7sKUvxaH0NrUqqiH6ZaKuXubVmNMZRhSCfgg0BGPcYfrHNGVNChMG2Q1nrfNVqM040DLrqg63NazhmT2SKBCwr7Lqws74
jYtTYF1pWAelmZXatxdpIOgHWO7ZIhF6qXOnJX4NeFqLwvwRtf3TkPB0yr5pR3bIEcvv2ZmG0L3CsecOBUCYHiB14AdjNacpPsW4iWFnsQxPpFspeuIbEXjL6a
DBXygaT7gJNAllbAELjiS64YEZSSf850HcStG7FxL8DPkf5J5XKOwMqsn5GxeB0rtH2eJfNHMt7pMRchwJbgiInZut846Fl5yEeJLjPgx0NgWeTEZehhjCTA2Muvvr
FQCDgiceuVl7e3k8yTYrForpPTO3aa6CuUQFw1EvXfqCOSMvSTIS37EYq5MiWX17hJ5sg0DN23DHSTPsP2GkLulOZoptQfnWEbmPIqT2QvLS225WhEloJj7D
y1fIcrGhPRvXdkr1rfPQxdsT3Fy88td1U6wc8zu5Ylk7A805pgF1B10EMiWUWJTG7mXC6YY7WI6SacK11bykb7pPOLGXOgC8FOT7vn17d6pKHKftValwIHRVQg
fsPMJCkJQzjSOmassuNMP4Vyr0viDyAdSdUC7eC28WknHJxdtQVRsBY3oaTQcoXwhN8HpAXg4LnbvI23eOHXNZODCII3sdrIFkseiLiskyhC3IvcvoDzd5uyIhR
Vv3tTeYOndYqHQu1SZz3wF1lTa78nLvQKHhEJLElDjbHTXlV3ALBC1NaEnxBUqMzqBQUROxYgGjHIQdpPatUdcPjWFV2NFsYHWC4tQIuigKZknYKx3NMDkG
TIWFgFnSbGIIWgE0Pcy7WJT3jN5ZCInMjdaenvt6fcaxQyOzylB25mpvwlY3uOb8I1ykmjMEMRDY7Pk2jUuUXQj30vFcVGxMb1H8XOINvRs4nzobj5ZNpuwKwk
WJHdnMImMAOwzDybz8N4wmXgBTpK1nkPeD3cupNBqeuGKDV3yUpxd0t0weF3utNu1xFeMd1X8hnG8HIOEjPZS1AxaLAmkqtPBrkwdzHAlwbnECYzdr4kAa
sLHrYIY0VffkfnIs1JU6iYD2q2enxeJSxDM5JN5fnWlX3CLiA5PsxB3OSL2m5gF86QkrBeceuW4TIGDFqj1yafcUlq3p2baqyl7kGPfHsJkzY3Tl3qOnt1XBLBO2pFo
FLPbfVHvJqMqRvJ2VEyr47yNpsgRsrBfR4nNnZ868ZUDHdMEK38J77C66rTXN5DWkJtrGsgj0ieVBp07cbplBALhcA2KJ6kUd8yn7wF8RURjdbn6aNjUFYmEKB
06GXuiXhefMpLJfx4uPPzQZn7d1a8v6EAUlMjAogTFu5UqPstLXs0vphv6WyOvsBlk6ZohZIkYjaZsX5QqBzf4S3MGWmpltNKmUkImZa4h2th64MviyZjsbqKbmxf
gb1MZdrEC8KrGVjWg70biL8TCqDTLF6SglchIo3DZ3AhYajyNAnCc6DJy5YqBn42N2vtySnSyRFlVrLaOBd5nHsYsK8AeAhYZq46ktn6BeO0DY57fGYJa1mNjzZ
wmDKkei5zf0N2xGVHGx3vi0bQQnTbUMb2WaXNkucE7lfqGrYPo3TQrLsguo1xzL388ai4Xa2yshFLJ4pCA6hMH6S5dNF0BO2TdRiB44lyYVODPStDMSRR6Hd
RottcAm8l5pLx6r2YuG4IxLag6AyMgoscwYi8g6xHwj3wRblFSsWo5sm4tz2OdxEdwcOyX1mxIhSeiuuTsMCSMuOnGCRR1WJxWe60JBLvxFsEP3zhtzkHAutfDY
OVhLoh04VP1ZrtvkulAY2dlXFOKv8M7OiiMsh0Olq8GuJyBW0KCed0ILIfIYogThYkuuiFl7b31aaMyNtcJZaLbJT5Pm3u4H4leOgV3tXdASDZnBBcnrUFF2gAw8nu
MyeneP8tCkq7ZIxOSBkcvO0n5ausFPBBw25etFm5NQ4Zo7mQpDRITM7UgQfwOommLSEDpf7fzezDiDmN2aGAh7szHE2OInwRyJfIUH3iYn6bDuat8u68P0Jo
IfPR38j1ESqtyvgJG1QDF4dF0g0WWnlca0bsqnCYxwG0oOsAXTaorSpN5jjneMWO4tye7y8avizpL7I6OgPCVeIzDM5E8rcQ7jgpE4GsiIOR820uYAkoJzGuBVR3I
BzuKP2uPgmmXZ7yVfbmOIfgd0HRXY0Vtt8g6Z1dCUQcfaLRm4uGtCBBpmQ44zPWvNhFgCpnLRqqcW7Z2KImC128txlDUys6lcg4klenmWWOtZCdPjMe6HG
vbe67nZmYJ4mLcgNESy4Xi3qQdSNhSOcfOqd0yupGjB8d8Re476v48S2pe2AHEQpVrr5KFCcMHzFNxmrxrSU4CeynBLX0UCv1uMo6YKjyLYH3bnJ7lefoN4Vv
6HLsM83nVAAcseHNx6hDs3THBZxMF5DGMyu2QyYs8rv3dC2YRNAFBTPH7GoMNHPvIqMuVtXapAEUm1twFSdxyMFoUzTPEyZsRwzBhYkFyr6mE82NJrz2
MBoa8JEeeepO6Ilrjmm4Zrr2nSHDPIh0eULpr0C1AZzk7ZnOkodFJc3keIGrwB3M7YL7LELchLMpSdunKUmnBb3U42oF0QP3nEUaoRAkWjGdyRKqvmM4oFV
VmJ6nci0LGMlUvUTDJKTj7sAIWTyjMyAfk8wPFsvefcd18j03MN4ulyJ0lzdK8khPLrsoh3NzkoGgNu6kHiQVpt2RGSJbpN1x0qZ2AlIfyUyg8Q1lTnRsHS1jNzXJNg
JdT6pBfqHweEr538bULY6pgPxGF4jZOOdPEfxKXKjRK7CunkPk4eF3K8epAFVvYhVSeeinU2SP1uOqGUAQLaNKmIuGIpj18ji6VViSeMLhqNlDSZpsVu71ziajl6
beibJinZMT8TaZHKWIOtrMdYVG1FcKxKLk3MlqNw8zdrwA2spygUCCIAgI5yBUJF2fzYEYZleRRUe0aO3pNGEAmWtfMPniGcBGkiDunfmYiqC2zX3PHJiMS5g
Xxnfyk51QzS2U2ei730ocjCb0UsiROZAjyblWo1xNUPxIOEAb6Uy5hLtfM1EMAGYoscwYi8g6xHwj3wRblFSsWo5sm4tz2OiS64YEZSSf850HcStG7FxL8DPkf5Jd
xE
39
Encryption

40. Map Reduce
40

41. 41
cat petabyte | grep

42. 42
Imagine

43. 43
ZeroCloud Usage
execution:
groups:
- name: "sysinfo"
path: file://python:python
args: "sysinfo.py"
devices:
- name: python
- name: stdout
…
bundling:
- "sysinfo.py"
$ zpm bundle
$ zpm deploy dir sysinfo.zapp
$ zpm execute dir sysinfo.zapp
sysinfo.zapp

44. 44
zebra.zerovm.org

45. 45
@adrian_otto
http://adrianotto.com