The OSI Superboard II was the computer on which I first learned to program back in 1979. Python is why programming remains fun today. In this tale of old meets new, I describe how I have used Python 3 to create a cloud computing service for my still-working Superboard--a problem complicated by it only having 8Kb of RAM and 300-baud cassette tape audio ports for I/O.

1. Using Python 3 to Build a
Cloud Computing Service for
my Superboard II
David Beazley
(http://www.dabeaz.com)
Presented at PyCon 2011
Atlanta

2. Note: This talk involves a number of live
demonstrations. You will probably enjoy
it more by watching the PyCon 2011
video presentation
http://pycon.blip.tv/file/4878868/

3. It Begins
(Drumroll...)

5. Byte Magazine, January, 1979

6. Personal History
• Superboard II was my family's first computer
• What I first learned to program (~1979)
• It had everything that a kid could want

7. Easy Setup!

8. Peripherals!

9. Potentially Lethal Suggestions!

10. Schematics!

12. You could get
an electric
shock
You could
look inside!

13. Pure Awesome!
You could program

14. Pure Awesome!
You could hack!
Note the
encouragement

15. Backstory
• 1982-2010. Superboard sits in mom's basement
• July 2010. Eric Floehr mentions SB at Scipy
• August 2010. Brother brings SB to Chicago
• It still works! (to my amazement)

16. Question
• What do you do with an old Superboard II
• 1 Mhz 6502 CPU
• 8K RAM
• 8K Microsoft Basic (vers. 1.0)
• 300 Baud Cassette Audio Interface
• 1K Video Ram (24x24 visible characters)
• Not a modern powerhouse

17. TO THE CLOUD!

19. HOW?
(with Python 3 awesomeness of course)
(plus some ØMQ and Redis)
(and some 6502 assembler)

20. Getting to the Cloud
This is the only I/O
A pair of RCA audio jacks
Used for cassette tape storage
Not too promising

21. A Possible Solution

22. Cassette Interface
• Behind the RCA jacks,
sits a real serial "port"
• Motorola 6850 ACIA
• Ports are a 300 baud
audio stream encoded
via Kansas City Standard
• Might be able to hack it

23. Byte, Feb. 1976

26. Python Audio Processing
• pyaudio
• http://http://people.csail.mit.edu/hubert/pyaudio/
• A Python interface to the portaudio C library
• Allows real-time access to audio line in/out
• I ported it to Python 3

27. Reading Audio Input
import pyaudio
p = pyaudio.PyAudio()
stream = p.open(format=pyaudio.paInt8,
channels=1,
rate=9600,
chunksize=1024,
input=True)
while True:
sample = stream.read(1024)
process_sample(sample)

28. Building a Python Modem
writer thread
pyaudio byte
writer encoder
TCP
client
socket
pyaudio byte
reader decoder
reader thread
• A network socket bridged to two real-time
audio encoding/decoding threads

29. KCS Audio Encoding
0= 1=
(4 cycles @ 1200 Hz) (8 cycles @ 2400 Hz)
'A' = 0x41 (01000001)
0 0 1 0 0 0 0 0 1 1 1
start data stop

30. KCS Audio Decoding
samples (8-bit)
89, 103, 117, 151, 194, 141, 99, 64, 89, 112, 141, 203, 152, 107, 88, ...
sign bits (0 = neg, 1= pos)
000111000111000111000111000111111000000111111000000111111000000111111...
sign changes (XOR adjacent bits)
0001001001001001001001001001000001000001000001000001000001000001000001...
Idea: Each 1 bit represents a 0-crossing

31. Example Code
def generate_sign_change_bits(stream):
previous = 0
while True:
samples = stream.read(CHUNKSIZE)
# Emit a stream of sign-change bits
for byte in samples:
signbit = byte & 0x80
yield 1 if (signbit^previous) else 0
previous = signbit

32. KCS Bit Decoding
idle (constant 1) 1 0 11 0 0 0 0
start data byte stop
Sign Change Bits push
1001001001001001001001001001 1001001001001001 discard
Sample buffer (size = audio samples for 1 bit)
~ 8 sign changes ~ 16 sign changes
0 1

33. Example Code
from collections import deque
# Sample buffer
sample = deque(maxlen=SAMPLES_PER_BIT)
for b in generate_sign_change_bits(stream):
sample.append(b)
nchanges = sum(sample)
if nchanges < 10:
# Detected a start bit
# Sample next 8 data bits
...
(The actual code is more optimized)

34. Demo
My Mac (linked via audio)

35. Interlude
• It's Alive!
• Basic communication is just the start!

36. This is uploading machine code
~500 lines of 6502 assembler
...
getvar_copy_string:
! ;; Move the variable address to INDIRECT where we c
! LDA![0x95, Y]! ; Length
! STA!%MEM_LENGTH
! INY
! LDA![0x95, Y]! ; address (low)
! STA!%INDIRECT
! INY
! LDA![0x95, Y]! ; address (high)
! STA!%INDIRECT+1
! LDY!#0x00
...
Wrote a 6502 assembler
~500 lines of Python 3

37. Under the covers
This is uploading machine code
msgdrv.asm
~500 lines of 6502 .1C00/20
assembler
asm6502.py 05
... AE
getvar_copy_string: EE
! ;; Move the variable address to INDIRECT where we c
1E
! msgdrv.hex Y]! ; Length
LDA![0x95, 1E
! STA!%MEM_LENGTH AD
! INY 1E
! LDA![0x95, Y]! ; address (low)
1E
! STA!%INDIRECT 8D
! INY
! pymodem Y]! ; address (high)
LDA![0x95,
! STA!%INDIRECT+1
! LDY!#0x00
...
Wrote a 6502 assembler
~500 lines of Python 3

38. Messaging Driver code
Under the covers
This is uploading machine
request
msgdrv.asm
~500 lines of 6502 .1C00/20 Client
Superboard II control assembler
asm6502.py 05
... AE
• Superboard issues requests
getvar_copy_string:
!
! msgdrv.hex Y]! ; Length
LDA![0x95,
EE
;; Move the variable address to INDIRECT where we c
1E
1E
• Client responds and gets control
!
!
!
STA!%MEM_LENGTH
INY
AD
1E
LDA![0x95, Y]! ; address (low)
1E
• Driver coexists with BASIC
! STA!%INDIRECT 8D
! INY
! pymodem Y]! ; address (high)
LDA![0x95,
! STA!%INDIRECT+1
1024LDY!#0x00 message driver
! bytes
...
Wrote a 6502 Workspace
7168 bytes BASIC
assembler
~500 lines of Python 3

39. Example of makingcovers
Messaging Driver code
Under the a request
This is uploading machine
request
10 A = 96
msgdrv.asm
~500 lines
Superboard II of 6502 assembler
control Client
asm6502.py
20 B = 42 .1C00/20
05
... AE
• SuperboardY]!issues1Erequests
getvar_copy_string: = "FOO"
!
!
30 the variable address to INDIRECT where we c
;; Move
msgdrv.hex
C$
40 S =; Length
LDA![0x95,
EE
USR(37)
1E
• Client responds and gets control
!
Client
!
!
STA!%MEM_LENGTH
INY
control
1E
AD
LDA![0x95, Y]! ; address (low)
1E
• Driver coexists- Get a BASIC region
! STA!%INDIRECT 8D
! INY
PEEK with memory
! pymodem Y]! ; address (high)
LDA![0x95,
!
! POKE - Set a memory region
STA!%INDIRECT+1
1024LDY!#0x00 message driver
bytes
...
GET - Get a BASIC variable
SET - Set a BASIC variable
Wrote a 6502 -Workspace to BASIC
RETURN assembler
7168 bytes BASIC Return
~500 lines of shared memory!
Distributed Python 3

41. Messaging Architecture
• There are two parts (driver and a client)
superboard message client
message
driver audio socket
pymodem
BASIC Python 3
• Uses a binary message protocol
USR(37) : x20 x06 x01 x25 x00 x02
Command Size Seq Data LRC
• Protocol details not so interesting

42. Message Driver
• Interacts directly with Microsoft BASIC
• Uses "Undocumented" ROM routines
• Accesses BASIC interpreter memory
• For this, some resources online
• http://osiweb.org

46. Client Architecture
• Client bridges Superboard II to the outside world
• Uses ØMQ (http://www.zeromq.com)
• And pyzmq (http://github.com/zeromq/pyzmq)
Message client Services
audio socket
pymodem ØMQ
Python 3

47. Request Publishing
• Requests are broadcast on a ØMQ PUB socket
• Retransmitted every few seconds if no response
Message client
USR(37) ØMQ PUB "37 1" "37 1" "37 1" ... To the
"Cloud"
Python 3
• Requests include the USR code and a sequence #

48. Service Subscription
• Services simply subscribe to the request feed
import zmq
context = zmq.Context()
requests = context.socket(zmq.SUB)
requests.connect("tcp://msgclient:21001")
requests.setsockopt(zmq.SUBSCRIBE,b"37 ")
• Now wait for the requests to arrive
while True:
request = requests.recv()
• Clients are separate programs--live anywhere

49. Request Response
• Message client has a separate ØMQ REP socket
• Used by services to respond
Message client
USR(37) ØMQ PUB "37 1"
driver ØMQ REP Service
commands (subscribed to 37)
Python 3
• Service initially acks by echoing request back
• On success, can issue more commands

50. Command Connection
• Setting up the command socket
commands = context.socket(zmq.REQ)
commands.connect("tcp://msgclient:21002")
• Complete request/response cycle
while True:
request = requests.recv()
commands.send(request) # Echo back
resp = commands.recv()
if resp[:2] == b'OK':
# In control of Superboard
# Do evil stuff
...

51. Commands
• Commands are just simple byte strings
b"RETURN VALUE"
b"PEEK ADDR SIZE"
b"POKE ADDR DATA"
b"GET VARNAME"
b"SET VARNAME VALUE"
• Response codes
b"OK DATA"
b"FAIL MSG"
b"IGNORE"
b"BUSY"

52. Interactive Demo
>>> request_sock.recv()
b'37 1'
>>> command_sock.send(b'37 1')
>>> command_sock.recv()
b'OK'
>>> command_sock.send(b'GET A')
>>> command_sock.recv()
b'OK 96.0'
>>> command_sock.send(b'GET B')
>>> command_sock.recv()
b'OK 42.0'
>>> command_sock.send(b'SET Q 2')
>>> command_sock.recv()
b'OK'
>>> command_sock.send(b'SET R 12')
>>> command_sock.recv()
b'OK'
>>> command_sock.send(b'RET 0')
>>> command_sock.recv()
b'OK'
>>>

54. Big Picture
Message client
Up to 65536 Service IDs (N)
ØMQ PUB
USR(N)
ØMQ REP
Big Iron
• Services can live anywhere
• Written in any language
• No real limit except those imposed by ØMQ

55. Superboard Emulation
• I dumped the BASIC and system ROMS
• Loaded them into Py65
• Py65 : A 6502 Emulator Written in Python
https://github.com/mnaberez/py65
• Work of Mike Naberezny
• I ported it to Python 3

56. Emulation in 60 Seconds
You start with the
Superboard II memory
map (provided)

57. Emulation in 60 Seconds
You identify
hardware devices

58. Emulation in 60 Seconds
You read
(about keyboards)

59. Emulation in 60 Seconds
You read
(about video ram)

60. Emulation in 60 Seconds
You read
(about ACIA chips)

61. Emulation in 60 Seconds
Then you just plug it into py65 (sic)
def map_hardware(self,m):
# Video RAM at 0xd000-xd400
m.subscribe_to_write(range(0xd000,0xd400),
self.video_output)
# Monitor the polled keyboard port
m.subscribe_to_read([0xdf00], self.keyboard_read)
m.subscribe_to_write([0xdf00], self.keyboard_write)
# ACIA Interface
m.subscribe_to_read([0xf000], self.acia_status)
m.subscribe_to_read([0xf001], self.acia_read)
m.subscribe_to_write([0xf001], self.acia_write)
# Bad memory address to force end to memory check
m.subscribe_to_read([0x2000], lambda x: 0)

62. Interactive Demo

63. Question
• What do you do with an emulated Superboard?

64. A Superboard Cloud
10 PRINT "I WILL THINK OF A"
15 PRINT "NUMBER BETWEEN 1 AND
100"
20 PRINT "TRY TO GUESS WHAT IT IS"
25 N = 0
30 X = INT(RND(56)*99+1)
35 PRINT
40 PRINT "WHATS YOUR GUESS ";
Program
Storage
50 INPUT G
10 PRINT "HELLO WORLD"
20 END
10 FOR I = 1 TO 1000
20 PRINT I
30 NEXT I
20 END
Cloud
Datastore
Service
Stored
Instances
(images of
running
Virtualized machines)
Superboard CPUs

65. Building The Cloud
• I built it using Redis (http://redis.io)
• Ported py-redis to Python 3
• Redis is cool
• Can use it as a key-value store
• Has other data structures (sets, hashes, etc.)
• Queuing
• Atomic operations

66. Redis Example
import redis
db = redis.Redis()
# Key-value store
db.set('foo',data)
data = db.get('foo')
# Queuing
db.lpush('queue',work)
work = db.brpop('queue')

67. Superboard Cloud Features
• Remote program store
• Load/save programs
• Instance creation
• Creation
• Run with input
• Distributed shared memory
• It must be usable from the Superboard II

68. Program Load/Store
BASIC
strings
msg cloud set
driver service get
program redis
settings
• BASIC program & workspace memory
directly manipulated by the message driver
• Stored in Python object and pickled to Redis

69. Instances
• Instances are a running Superboard
• 8K Program Memory
• 1K Video RAM
• Stored CPU context (registers, etc.)
• Stored in a Python object
• Pickled to Redis when inactive

70. Instance Execution
• "Runner" programs watch a Redis queue
import redis
r = redis.Redis()
...
while True:
work = r.brpop("queue") # Wait for work
...
inst = load_instance() # Get instance
run_emulation(work) # Run emulation
save_instance(inst) # Save instance
• Based on supplying keyboard input to SB
• Instance runs until no more input available

71. Instance Concurrency
• Can have arbitrary number of runners
Redis
Runners
• Asynchronous execution (w/ Superboard)
• Uses Redis setnx() for locking

73. import superboard as skynet
pymodem
Up to 65536 Service IDs (N)
ØMQ PUB
ØMQ REP
Big Iron
Cloud
Service
programs
10 PRINT "I WILL THINK OF A"
15 PRINT "NUMBER BETWEEN 1 AND
Virtualized 100"
20 PRINT "TRY TO GUESS WHAT IT IS"
25 N = 0
Superboard CPUs
30 X = INT(RND(56)*99+1)
35 PRINT
40 PRINT "WHATS YOUR GUESS ";
redis
50 INPUT G
10 PRINT "HELLO WORLD"
20 END
Stored 10
20
30
FOR I = 1 TO 1000
PRINT I
NEXT I
Instances
20 END

74. WHY?!

75. Non-Answer
• I don't actually want to use my Superboard II
• It's awful!
• It was painful even in 1980

76. A Better Answer
• For fun
• Also to create a glorious mess!
• Everything a systems hacker could want!
• Hardware, device drivers, signal
processing, protocols, networks, message
passing, threads, synchronization,
debugging, software design, testing,
deployment, etc.
• Awesome!

77. Real Answer : Python 3
• Can Python 3 be used for anything real?
• I needed to check it out myself
• On a non-trivial project with many moving parts
• And with a variety of library dependencies

78. Lessons Learned
• You can use Python 3 to do real work
• However, it’s bleeding edge
• You really have to know what you’re doing
• Especially with respect to bytes/unicode
• Must be prepared to port and/or patch

79. Porting Experience
• py65 (easy, some bytes issues)
• pyredis (easy, bytes/unicode issues)
• pypng (hard, really messy I/O)
• pyaudio (hard, C extensions)
• pyzmq (worked, Cython patch for Py3.2)

80. Finding Python 3 Code
• Every single package I used was downloaded
from development repositories (github,
subversion, etc, etc)
• You can often find Python 3 compatible
libraries in project forks or issue trackers if
you look for it

81. My Thoughts
• Python 3 is cool
• It keeps getting better
>>> import numpy
>>>
• It’s different and fun
• It might be a great language for distributed
computing, messaging, and other applications

82. That’s All Folks!
• Thanks for listening!
• Look for the “Python
Cookbook, 3rd” edition
in late 2011!
• More info on my blog
• http://www.dabeaz.com