Pd (Pure Data) , MobMuPlat, libpd, pd-for-ios, Xcode, iOS SDK

1. Experimental Mobile Music
Pd/iOS Workshop
Open Media Lab
School of Intermedia Art
China Academy of Art
May 23-31, 2013
Masayuki Akamatsu / IAMAS

2. Introduction

3. akaMasayuki Akamatsu, Ph.D
● Media Artist
● Professor at IAMAS
● Fellow at some comapnies
Gocco., Soneru, Ma-tilde…
● Sound, Visual, Network, Mobile…
Real-time Performance
Interactive Installation
Mobile/Network Experience
● http://akamatsu.org/
赤松 正行

4. akaIAMAS
● Institute of Advanced Media and Sciences
● Graduate school for new media arts
20 students x 2 years = 40 students
19 professors
24 hours / 364 days facilities
Personal desks / Shared studios
● http://www.iamas.ac.jp/E/

5. akaincubator, 1999
● Autonomous Distributed Cooperative System

6. akaTime Machine!, 2002
• Image analysis / Image Processing

7. akaEarthen Bodied Augur, 2007
● Image/Audio Analysis and Processing

8. akaATOMxGALAXIES, 2007-
● Image Processing / Reversed Projection Mapping

9. akaGame Changer in 2007

10. akaComputer or Mobile ?

11. akaDinosaurs vs. Rats (Mammals)

12. akaMobile Information Environment
● Bodily Information Environment
● Sensory Information Environment
● Universal Information Environment
● World-wide Information Environment

13. akaMobile Apps

14. akaSekai Camera, 2008

15. akaSnowflakes, 2008

16. akaOkeanos Buoys, 2010

17. akaSync for Japan, 2011

18. akaUroboros Torch, 2012

19. akaMIDIWriter, 2012

20. ● Using App Store (on iOS) or iTunes (on OS X)
● Search Keywords
“Masayuki Akamatsu” for my apps
“ARART” for classic paintings
“Bloom Card” for flowers
akaTry Apps

21. akaWhat’s your favorite app ?

22. akaWhy create an app?
● Many excellent music apps
● Create your own music
with your own idea
with your own app
Animoog by Moog Music Inc.

23. MobMuPlat:Basics

24. akaCreate a Music app with Xcode
● Xcode
● Objective-C
● Core Audio
● DSP algorithm
● Difficult!!
Okeanos by Masayuki Akamatsu
mini-composer by Karl Bartos and Masayuki Akamatsu

25. akaCreate a Music app with Pd
● MobMuPlat
Pd + Editor
No programming code
Easy to develop
Limited UI/Graphics
Limited OS Access
For Personal Use
● libpd / Pd for iOS
Pd + Xcode
Objective-C code
Hard to develop
Full UI/Graphics
Full OS Access
For App Store Distribution

26. akaQuestions: Do you have a skill of…
● Max
● Pd
● C/C++
● iOS SDK

27. akaInstall MobMuPlat
● http://mobmuplat.com/
● Install MobMuPlat on iOS device
● Install MobMuPlat distribution on Mac

28. akaMobMuPlat (iOS app)

29. akaMobMuPlat Workflow
1. Create an audio patch with Pd
2. Create a GUI with MobMuPlatEditor
3. Test them with PdWrapper.pd
4. Transfer them with iTunes
5. Play it with MobMuPlat (iOS)
Mac iPhone

30. akaPd tips for Max users
Run/Edit mode
Audio On/Off
Audio On/Off
option+click
Create an object

31. akaApp 1 - Drums
Drums set by Pearl

32. akaStep 1: Read an Audio File
● Open Pd (on Mac)
● Open a new patch
● Create and connect objects
● Save the patch as “Drums.pd”
into the same folder of the audio file
● Click the button
You’ll see the waveform
Object
Message
Button
Array
Object
Patch Cord

33. akaStep 2: Play the Audio Buffer
● Create and connect objects
● Start DSP
● Click the button
You’ll hear the sound

34. akaStep 3: Create a Communication
● Create and Connect Objects
● What’s “fromGUI” & “/claps”?

35. ● Open MobMuPlatEditor
You’ll see a new document
● Open “Doc” tab
● Choose Screen Size
● Choose your Pd file
● Save the document
as “Drums.mmp”
akaStep 4: Create a GUI document
1
2
2

36. akaStep 5: Create a Button
● Open “Add” tab
● Click “button”
● Adjust the position
Do not cover “i” button
2
3 Drag
1

37. akaStep 6: Set the Properties
● Open “Prop” tab
● Set the Address
as “/claps”
1
2

38. akaStep 7: Test the GUI
Pd
● Open “PdWrapper.pd”
MobMuPlatEditor
● Open “Lock” tab
● Click the button
You’ll hear the sound
You’ll see the messages
1
2

39. akaStep 8: Transfer Files
● Connect iPhone to Mac via USB cable
● Open iTunes
● Choose your iPhone, Apps tab and MobMuPlat
● Put files into MobMuPlat Documents
1
2
3
4
Drag&Drop

40. akaStep 9: Hit your Drum
● Open MobMuPlat (on iPhone)
● Click “i” button
● Choose “Drums.mmp”
● Play!

41. akaExercises: Build your own Drums
● Add more buttons
● Use your own audio files
● Use a image
Add a panel and choose a image file
Make a button transparent
● Control the volume
Play softly or loudly

42. ● Use “Tidy Up” in Edit menu in Pd
● .mmp & .pd are text files you can edit
akaPd Editing Tips

43. MobMuPlat:Advanced

44. akaApp 2 - Theremin
Léon Theremin and his invention “Theremin”

45. ● Receive a system message
“fromSystem” and “/tilts”
● Convert values to pitch and volume
Tilt X “-1.0~1.0” -> MIDI Note Number -> Frequency
Tilt Y “-1.0~1.0” -> Volume
● Smoothing
line
● Generate a sine waveform
osc~
*~
dac~
akaTheremin Patch

46. akaTheremin GUI
Tilt emulation
Panel with a image

47. akaExercises: Build your own Synth
● Subtractive synthesis
● Additive synthesis
● Wavetable synthesis
● Frequency Modulation synthesis
● Phase Distortion synthesis
● Physical Modeling synthesis
● Sample-based synthesis
MOOG Synthesizer

48. akaTutorial & Reference
● Programming Electronic Music in Pd
http://www.pd-tutorial.com/english/index.html
● Pure Data Reference Card
http://puredata.info/docs/manuals/pdrefcards

49. akaApp 3 - Delay
Echoplex, a tape delay machine by Mike Battle

50. akaDelay Patch (incomplete)
Input Volume Control
Delay Feedback
Input Volume
Audio Input
Audio Output

51. akaDelay GUI

52. ● Delay
● Reverb
● Distortion
● Overdrive
● Flanger
● Chorus
● Phaser
● Tremolo
● EQ
● Wah
● Pitch Shift
akaExercises: Build your own Effector
BOSS Compact Effectors

53. akaAudio I/O
● Headset Mic Earphone
● Breakout adapter
● Audio Interface
● Feedback is also fun!

54. akaApp 4 - Chorus
The China National Chorus

55. akaChorus Patch (Standalone)
Calculate Frequency Calculate Duration
Produce Cycle
Produce Envelop
Generate Sine wave
Modify Amplitude

56. akaChorus GUI

57. akaChorus Patch (Network)
Send to Network Receive from Network
Set GUI Values
set value without output

58. akaStandalone vs. Network
Standalone
Network
Your iPhone
GUI Audio
toNetwork fromNetwork
Network
Another iPhone
GUI Audio
toNetwork fromNetwork
Your iPhone
GUI Audio
Another iPhone
GUI Audio

59. ● Multicast (Broadcast)
● Port Number
akaOSC (Open Sound Control)

60. ● Unison
● Harmonic
● Inharmonic
● Beating/Chorus
● Overtone
● Texture
● Additive Synthesis
● Granular Synthesis
akaExercises: Build your own Harmony
Harmonic (from Wikipedia)

61. akaNetwork(s)
● Natural Network
Through the Air
Physical Phenomenon
● Human Network
Play Session
Feelings / Rules
● Electronic Network
LAN / The Internet
Protocols

62. akaLimitations of MobMuPlat
● Pd-vanilla (not Pd-extended)
● Poor Audio functions
● Poor GUI elements
● Poor visual expressions
● Poor iOS accessibilities
● Can’t create a TRUE app
● But Fun and Easy to use!
SpaceLab by Iglesia Intermedia

63. libpd/Pd for iOS

64. akalibpd
● http://libpd.cc
● Pd as Audio Engine + iOS SDK (Xcode)

65. akaInstall Xcode
● Open App Store (on Mac)
● Search “Xcode”
● Click “Free”

66. akaInstall pd-for-ios
● https://github.com/libpd/pd-for-ios
● Clone Repository
Click “Clone in Mac”
● Update Submodules
Follow “README.txt”

67. ● Open PdTest01.xcodeproj
● Choose Scheme
● Click “Run”
akaRun Sample Projects
1
2

68. akaTrue App ;-) Test

69. akaTest Patch
Receive a message
from app (Objective-C code)
Send a message
to app (Objective-C code)

70. akaCreate a new Project
● Open Xcode
● Choose File > New > Project…

71. akaChoose a template
1
2

72. akaChoose options
1
2

73. akaSave the project
1
2

74. akaProject Window

75. akaCreate a User Interface
1
2
3
Interface Builder
Drag&Drop
Drag&Drop

76. akaTest the User Interface
1
2

77. akaAction and Outlet

78. akaCreate an Action
control-drag
12
3
4
5
6
Code Editor

79. akaCreate an Outlet
control-drag1 2
3

80. akaGenerated Codes
#import <UIKit/UIKit.h>
@interface TestViewController : UIViewController
- (IBAction)changeFreq:(UISlider *)sender;
@property (retain, nonatomic) IBOutlet UILabel *freqLabel;
@end
#import "TestViewController.h"
@interface TestViewController ()
@end
@implementation TestViewController
- (void)viewDidLoad
{
[super viewDidLoad];
! // Do any additional setup after loading the view,
typically from a nib.
}
- (void)didReceiveMemoryWarning
{
[super didReceiveMemoryWarning];
// Dispose of any resources that can be recreated.
}
- (IBAction)changeFreq:(UISlider *)sender {
}
- (void)dealloc {
[_freqLabel release];
[super dealloc];
}
@end
TestViewController.h TestViewController.m

81. akaAdd Code and Run
#import "TestViewController.h"
@interface TestViewController ()
@end
@implementation TestViewController
- (void)viewDidLoad
{
[super viewDidLoad];
! // Do any additional setup after loading the view, typically from a nib.
}
- (void)didReceiveMemoryWarning
{
[super didReceiveMemoryWarning];
// Dispose of any resources that can be recreated.
}
- (IBAction)changeFreq:(UISlider *)sender {
! self.freqLabel.text = [NSString stringWithFormat:@"%f", sender.value];
}
- (void)dealloc {
[_freqLabel release];
[super dealloc];
}
@end
TestViewController.m

82. akaXcode Tips
● Snapshot

83. akaAdd libpd.xcodeproj
Drag&Drop
Close any other projects first because of workspaces problem

84. akaAdd Target Dependencies
1
2
3
5
4
7
6

85. akaAdd Libraries/Frameworks
2
3
4
Add libpd-ios.a
AudioToolbox.framework
and AVFoundation.framework
1

86. akaAdd Header Search Paths
1
4
Drag&Drop
Double Click
3
Add libpd/objc
and libpd/pure-data/src
2

87. akaAdd the Pd Patch
1Drag&Drop
2
3

88. akaAdd Codes to TestViewController.h
#import <UIKit/UIKit.h>
#import "PdBase.h"
#import "PdAudioController.h"
@interface TestViewController : UIViewController <PdReceiverDelegate>
- (IBAction)changeFreq:(UISlider *)sender;
@property (retain, nonatomic) IBOutlet UILabel *freqLabel;
@property (strong, nonatomic) PdAudioController *audioController;
@end

89. - (void)viewDidLoad
{
[super viewDidLoad];
! // Do any additional setup after loading the view, typically from a nib.
! // Create Audio Controller
! self.audioController = [[[PdAudioController alloc] init] autorelease];
! [self.audioController configureAmbientWithSampleRate:44100 numberChannels:2 mixingEnabled:YES];
! self.audioController.active = YES;
!
! // Open Pd Patch
! [PdBase openFile:@"Test.pd" path:[[NSBundle mainBundle] bundlePath]];
!
! // Set PdBase Delegate
! PdBase.delegate = self;!
! // Subscribe messages from Pd
! [PdBase subscribe:@"freqInHz"];
}
- (IBAction)changeFreq:(UISlider *)sender {
! // Send a message to Pd
! [PdBase sendFloat:sender.value toReceiver: @"freq" ];
}
// Handle float messages from Pd
- (void)receiveFloat:(float)received fromSource:(NSString *)source {
! if ([source isEqualToString:@"freqInHz"]) {
! ! self.freqLabel.text = [NSString stringWithFormat:@"%f Hz", received];
! }
}
akaAdd Codes to TestViewController.m

90. akaRun the True App !

91. akaUser Interface - Code - Patch
- (IBAction)changeFreq:(UISlider *)sender {
! [PdBase sendFloat:sender.value toReceiver: @"freq" ];
}
- (void)receiveFloat:(float)received fromSource:(NSString *)source {
! if ([source isEqualToString:@"freqInHz"]) {
! ! self.freqLabel.text = [NSString stringWithFormat:@"%f", received];
! }
}
Action
Outlet
Receive method
(Delegate)
Send method

92. akalibpd Project Summary
● Add libpd.xcodeproj
● Add Target Dependencies
libpd-ios
● Add Libraries/Frameworks
libpd-ios.a, AudioToolbox.framework, AVFoundation.framework
● Add Header Search Paths
libpd/objc, libpd/pure-data/src
● Add Pd Patch
● Add Codes
Create PdAudioController, Set it up and Make it active
Open Pd patch
Set PdBase delegate and Subscribe Pd messages
Send messages to Pd patch and Receive messages from Pd patch

93. akaFurther References
● Pd for iOS wiki
https://github.com/libpd/pd-for-ios/wiki/ios
● libpd Objective-C API
https://github.com/libpd/libpd/wiki/objc
● iOSの教科書
http://iosbook.net/

94. Students Project

95. ● Solo or Group
● 3 minutes
● Theme/Motif/Feeling
● Audio Processing
● User Interface
● Composition
● Performance
akaCreate your own music
Kraftwerk / Pocket Calculator

96. akaWhy? (again)
● Many excellent music apps
● Create your own music
with your own idea
with your own app
Animoog by Moog Music Inc.

97. akaMake a Documentation
● Format
PDF
Single Page
Landscape
● Contents
Title
Name
Description
Images (Screen, Photo, etc.)
Title: China My Love
Name: Masayuki Akamatsu
Description: “China My Love” is a simple app that transmits MIDI note
messages by hitting keys on a keyboard. Playing an input phrase as a
sequence is also possible. The tempo and the loop can be set. You can
use either the on-screen software keyboard or a Bluetooth/Dock-connected
hardware keyboard for iOS.

98. One more thing…

99. akaiDonation
Unneeded iPhone, iPad, iPod touch,
I am asking for a kind donation
http://akamatsu.org/aka/ios/idonation/

100. 謝謝
Thanks to Prof. Dajuin Yao and students