A brief about Web Audio and Web MIDI APIs that were added to the open web platform recently

1. Web & Sound
Playing with Sound APIs recently introduced to the Web

2. Overview + Demo

3. New Sound APIs
◇ Web Audio (http://webaudio.github.io/web-audio-api/)
￭ Real-time audio synthesis
￮ cf. SuperCollider, Pure Data, Max/MSP
￭ SFX (Pan, Delay, Filter, etc)
◇ Web MIDI (http://webaudio.github.io/web-midi-api/)
￭ Real-time communication
￮ Web ⇄ Musical Instruments
● Synths, DJ and lighting controllers
◇ WebRTC (http://w3c.github.io/webrtc-pc/)
￭ Camera & Microphone

4. Web Audio Demo
◇ Web Audio Playground (http://webaudioplayground.appspot.com/)
￭ Straight-forward GUI wrapper
◇ WebPd (https://github.com/sebpiq/WebPd)
￭ Pure Data clone
◇ CoffeeCollider (https://github.com/mohayonao/CoffeeCollider)
￭ Coffee Script + SuperCollider
◇ T’SS JS edition (https://github.com/toyoshim/tss)
￭ Chiptune sound emulation (demo, tsdsp)

5. MIDI is old, and new
◇ 30th anniversary
◇ MIDI is not only SMF but a protocol
￭ SMF: Standard MIDI File
￮ Protocol dump file with timestamps
￭ Real-time communication - lighting and more
￮ 冨田勲×初音ミク「イーハトーヴ」@ YouTube
◇ De facto for music production tools
￭ Software Synth in DAW: Audio Unit, VSTi, DXi
￭ Hardware: DIN, USB, FireWire, RTP, Bluetooth(*)
￮ (*): supported by Yosemite, Windows 10, Android M

6. Web MIDI Demo
◇ Physical controller for online sound systems
￭ Web Audio Synth v2 (http://aikelab.net/websynthv2/)
￭ Web FM Synthesizer (http://www.taktech.org/takm/WebFMSynth/)
￭ Online DAW: audiotool (http://www.audiotool.com/)
￭ Online VJ: MajVj (GGT 2014 demo)
◇ Demo and Productions by YAMAHA
￭ Pocket Miku (Product page & Web App)
￭ Web Music Platform (toyoshim’s fork)
￭ Reface + Soundmondo (Product page)
￮ Social to share your sound and music via Web MIDI

7. WebRTC
◇ microphone
￭ getUserMedia({audio:true}, …)
￮ Can connect to Web Audio via
createMediaStreamSource()
◇ camera
￭ getUserMedia({video: true}... )
￮ Can use it as a texture for WebGL

8. Web Apps be Secure!
◇ Hardware is hard to use securely
￭ What happens on WebGL?
￮ Need GPU process and GPU command validator
￭ Recent APIs to use hardware
￮ Permission prompt (infobar/bubble UI)
￮ Secure origin, e.g. https or localhost, is mandatory
￭ Real P0 issue happened on Web MIDI
￮ Microsoft GS Wavetable SW Synth crashes
● on receiving a special valid sequence
● outside Chrome sandbox, inside the driver

9. How to learn
◇ Web Audio
￭ Getting Started with Web Audio API html5rocks
￭ Web Audio API Ryoya Kawai
￭ Web Audio API 解説 g200kg
◇ Web MIDI
￭ Web MIDI API (x-webmidi編) Ryoya Kawai
￭ ブラウザで電子楽器を作ってみよう HTML5 Experts
￭ 最高に乱暴なWeb MIDI API利用方法 tadfmac

10. Lesson: Web Audio 1
◇ Simple sine wave
var audio = new AudioContext();
var osc = audio.createOscillator();
osc.frequency.value = 440; // in Hz
osc.connect(audio.destination);
osc.start(0);

11. Lesson: Web Audio 2
◇ Other nodes
￭ Audio source
￮ AudioBufferSourceNode (from Audio resources, e.g., mp3, m4a)
￮ MediaElementAudioSourceNode (from DOM media object)
￮ MediaStreamSource (from microphone via getUserMedia)
￮ ScriptProcessorNode (deprecated) => AudioWorkerNodeProcessor (from JS)
￮ OscillatorNode (from various kinds of oscillators)
￭ Effect
￮ GainNode
￮ DelayNode
￮ (Stereo)PannerNode / AudioListener
￮ ConvolverNode (e.g., Vocoder, Reverb, and so on)
￮ DynamicsCompressorNode / WaveShaperNode
￮ BuquadFilterNode (e.g., LPF, HPF, BPF, and so on)
￭ Misc
￮ AnalyserNode (FFT)
￮ ChannelSplitterNode / ChannelMergerNode (Mixer)

12. Lesson: Web MIDI 1
◇ Device Query
var promise = navigator.requestMIDIAccess();
promise.then(function(midi) {
for (var input of midi.inputs.values())
console.log(input); // MIDIInput
for (var output of midi.outputs.values())
console.log(output); // MIDIOutput
});

13. Lesson: Web MIDI 2
◇ MIDIInput
for (var input of midi.inputs.values())
input.onmidimessage = handler;
function handler (e) {
// Single MIDI message in Array.
console.log(e.data);
}

14. Lesson: Web MIDI 3
◇ MIDIOutput
for (var output of midi.outputs.values()) {
// Send single MIDI message or multiple
// MIDI messages.
output.send([0x90, 0x40, 0x7f]); // Note ON
}

15. Lesson: Promise
◇ New replacement for callbacks
foo(successCallback, errorCallback);
↓
foo().then(successCallback, errorCallback);
foo().then(bar)
.then(baz)
.catch(qux);
◇ Chaining
Promise.all([foo(),
bar()])
.then(baz, qux);
◇ Wait multiple async functions

16. Lesson: Maplike
◇ readonly maplike<DOMString, MIDIInput> inputs
◇ readonly maplike<DOMString, MIDIOutput> outputs
◇ keys() … MIDIPort.id is used as an unique key
◇ values() … MIDIInput / MIDIOutput
for (var input of inputs.values()) {
// New syntax for ES6 iterable
}

17. Lesson: Web MIDI 4
◇ Hotplug
midi.onstatechange = function (e) {
var newport = e.port;
if (newport.state == “connected”)
console.log(newport.name + “ is attached”;
else if (newport.state == “disconnected”)
console.log(newport.name + “ is detached”;
};

18. Fine ||

19. References
◇ Slide template
￭ from Slides Carnival
￮ http://www.slidescarnival.com/imogen-free-
presentation-template/399