CocoaHeads Ann Arbor presentation on handling MIDI events in Mac OS X and iOS with Core MIDI and Core Audio

1. CoreMIDI and Friends
Chris Adamson, CocoaHeads Ann Arbor
Sept. 8, 2011

2. Road Map
• MIDI basics
• MIDI APIs on OSX
• MIDI APIs on iOS

3. MIDI Basics
• MIDI = Musical Instrument Digital Interface
• Circa 1982!
• Sends messages between devices, not sounds
• Notes to play (pitch, impact, vibrato), patch
selection, timing synchronization, etc.

4. MIDI Terms
• Device — participant in a MIDI network
• Endpoint — one connection into or out of a
device
• Source — endpoint that sends data
• Destination — endpoint that receives data

5. MIDI Messages
STATUS DATA 1 DATA 2

6. MIDI Messages
• Channel Voice Messages — Note On, Note
Off, After-touch, Pitch wheel
• High nybble of status is command, low
nybble is channel number
• Channel Mode, System Messages
• Various extensions to the spec over the years
http://www.midi.org/techspecs/midimessages.php

7. MIDI APIs on OSX
• CoreMIDI — Implementation of MIDI
messaging
• Instrument Units — Audio Units that generate
sound in response to MIDI events
• Music Device — Sends commands to
Instrument Units
• Music Sequence / Music Player — Plays MIDI
(.mid) files

8. Core MIDI
• C-based API in Audio Toolbox
• Largely similar to other Core Audio APIs
• Uses Core Foundation opaque types,
memory management, etc.

9. Core MIDI Types
• MIDIClientRef — Holds state for your MIDI
session
• MIDIDeviceRef — A device (real or virtual)
with “entities” (MIDIEntityRef), which are
logical sub-systems
• MIDIEndPointRef — Source or destination
• MIDIPortRef — An input or output port

10. Creating client & input port
!MIDIClientRef client = NULL;
!MIDIClientCreate(CFSTR("Core MIDI to System Sounds Demo"),
MyMIDINotifyProc, self, &client);
!
!MIDIPortRef inPort = NULL;
!MIDIInputPortCreate(client, CFSTR("Input port"),
MyMIDIReadProc, self, &inPort);

11. Discovering and connecting
sources
!unsigned long sourceCount = MIDIGetNumberOfSources();
!for (int i = 0; i < sourceCount; ++i) {
!! MIDIEndpointRef src = MIDIGetSource(i);
!! CFStringRef endpointName = NULL;
!! OSStatus nameErr = MIDIObjectGetStringProperty(src,
kMIDIPropertyName, &endpointName);
!! if (noErr == nameErr) {
!! ! NSLog (@" source %d: %@n", i, endpointName);
!! }
!! MIDIPortConnectSource(inPort, src, NULL);
!}

12. Read Proc
• Callback function indicated by
MIDIInputPortCreate()
• Receives MIDIPacketList
• numPackets and MIDIPacket[]
• Receives user-info/context pointers that you
set up in MIDIInputPortCreate() and
MIDIPortConnectSource()

13. MIDIPacket
• Contains timeStamp, length, and Byte[]
• Byte[] data is the MIDI message
• data[0]: status
• data[1]: MIDI Data 1
• data[2] (if present): MIDI Data 2

14. Parsing a NOTE ON message
!MIDIPacket *packet = (MIDIPacket *)pktlist->packet;!
!Byte midiCommand = packet->data[0] >> 4;
!// is it a note-on?
!if (midiCommand == 0x09) {
!! Byte note = packet->data[1] & 0x7F;
!! Byte velocity = packet->data[2] & 0x7F;

15. Now What?

16. Instrument Units
• Audio Units that generate sound in response to
MIDI events
• kAudioUnitType_MusicDevice
• Do not currently exist on iOS

17. Instrument to audio H/W
I/O Unit
Unit
AU Graph

18. Instrument to audio H/W
Effect Unit I/O Unit
Unit
AU Graph

19. Instrument to audio H/W
I/O Unit
Unit
AU Graph

20. Mu
si
cD
ev
ic
eM
ID
IE
ve
nt
()
Instrument to audio H/W
I/O Unit
Unit
AU Graph

21. MusicDevice.h
• Small API to deliver MIDI events to
instrument units
• Not in Xcode documentation. Check out the
header file
• Only 4 functions
• MusicDeviceMIDIEvent() sends status,
data1, data2 to a MusicDeviceComponent
(i.e., an instrument Audio Unit)

22. Demo

23. MIDI on iOS
• Core MIDI added in iOS 4.2
• Device connectivity is via dock port
• Custom hardware
• iPad Camera Connection Kit

24. MIDI via the CCK
• MIDI-to-USB adapters semi-officially blessed
by Apple
• Adapter must be USB MIDI Class-
compliant (i.e., doesn’t need drivers)
• Bus-powered devices may not work, due to
low power supplied by iPad. Powered USB
devices generally work.
• http://iosmidi.com/devices/

25. Playing sounds on iOS
• No instrument units on iOS. Options:
• Synthesize your own with a render callback
• Play sampled sounds with Audio Queue, AV
Player, System Sounds, etc.

26. Demo

27. Demo

28. AUSampler
• New instrument audio unit in Lion… and… …
• Takes a sampled waveform and pitch-shifts it
to make it into an instrument
• Call with MusicDeviceMIDIEvent(), just
like other instrument units

29. AUSampler
• Configuration is a huge hassle
• Huge and tota!y undocumented hassle
• Build an .aupreset with AU Lab utility, or
load DLS bank or SoundFont 2 files, or
provide your own files
• See WWDC Session 411 (“Music in iOS
and MacOSX”), then file documentation
bugs against absence of sample code

30. In Summary
• Musicians love MIDI. Devices are cheap and
plentiful
• OSX and iOS love media, MIDI included
• By comparison, Android has crap MIDI
support (no javax.sound.midi, just .mid file
support in android.media.JetPlayer)
• If you’re doing Mac or iOS media apps, you
should consider supporting MIDI device I/O

31. Also, you should buy my book
http://www.mypearsonstore.com/bookstore/product.asp?isbn=9780321636843
http://my.safaribooksonline.com/9780321636973
http://www.informit.com/promotions/promotion.aspx?promo=137039