code coverage measurement tools (Flexcover has gotten a lot of attention this year)
visual editor frameworks (I'm just about to release a new open source project called Moccasin that distills a lot of Noteflight's editing infrastructure into a general framework)
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
Joe Berkovitz - Dynamic Audio Synthesis on the Flash Flex Platform
1. Dynamic Audio
Synthesis
on the Flash/Flex Platform
Joe Berkovitz
2. Overview
• Bit o’ background
• Flash 9 sound capabilities
• The Flash 10 Sound API
• StandingWave audio synthesis library
• New avenues for Flash audio and music
5. Player 9 Sound API
• Sound objects can be loaded from URLs...
• ...or embedded as assets
• Load ‘em, play ‘em, change volume or stereo
pan – that’s about it!
• Unpredictable latency, synchronization
6. Player 9 Audio
// loading a sound over the network
var s:Sound = new Sound();
s.addEventListener(Event.COMPLETE, handleComplete);
s.load(new URLRequest(“TheBomb.mp3”);
function handleComplete(e:Event) {
Sound(event.target).play();
}
7. Player 9 Audio
// loading a sound from a library asset
[Embed(source=”TheBomb.mp3”)]
private static var MySoundAsset:Class;
function playSound():void
{
var s:Sound = new MySoundAsset();
var sc:SoundChannel = s.play();
sc.soundTransform =
new SoundTransform(0.5 /*volume*/);
// set other transform properties...
// examine playback position
trace(sc.position);
}
8. What we want to do...
• Dynamically compute sounds from scratch
• Process samples on the fly to shift pitch,
add effects, equalization...
• Play more than 32 sounds at the same time
• Compute continuous sound output
9. FP10 to the rescue
• Synchronization and continuity:
SAMPLES_CALLBACK event
• Access to codecs: Sound.extract()
• The Vector data type
• Adobe Pixel Bender runtime
• Alchemy
10. Dynamic Audio Output
var s:Sound = new Sound();
s.addEventListener(SampleDataEvent.SAMPLE_DATA,
handleSampleData);
var sc:SoundChannel = s.play();
function handleSampleData(e:SampleDataEvent) {
for (i = 0; i < /*number of frames*/ ; i++)
{
e.data.writeFloat(/* left channel */);
e.data.writeFloat(/* right channel */);
}
}
11. Codec Access
[Embed(source=”TheBomb.mp3”)]
private static var MySoundAsset:Class;
function getSoundSamples(numFrames:uint):void
{
var s:Sound = new MySoundAsset();
var bytes:ByteArray = new ByteArray();
s.extract(bytes, numFrames, _position);
bytes.position = 0;
for (var i:Number = 0; i < numFrames; i++)
{
... = bytes.readFloat(); // Left channel
... = bytes.readFloat(); // Right channel
}
}
12. Some things to know
• Between 2048 and 8192 samples per event
• The fewer samples you provide, the lower
the latency (time before a sample is heard)
• Initial latency > steady-state latency
• More samples/callback = more stability
• Don’t take too long to generate samples!
14. Going beyond the
FP10 sound API
• Goal: to work at a higher level than samples
• High level concepts: Sources, Filters
• Play back timed, overlapping sequences of
sonic or musical events: Performances
• Create a modular, extensible system
15. The StandingWave Project
• Open source AS3 library
• Outgrowth of Noteflight
• Requires Player 10
• Enables new class of audio applications
19. Audio Processing Concepts
timed sequence
starts at 0.0 sec of sounds
Source Filter
starts at 0.2 sec
Audio
Performance
Source Filter
Player
starts at 0.7 sec
Source Filter
(...continues...)
20. Audio Processing Concepts
Samples
Sample
Sample
starts at 0.0 sec
Source Filter
Samples
Sample
Sample
starts at 0.2 sec
Audio Audio
Performance
Source Filter
Performer Player
starts at 0.7 sec
Source Filter
...
21. Audio Processing Concepts
starts at 0.0 sec
Audio
Source Filter
Player
starts at 0.2 sec
Audio
Performance
Source Filter Filter
Performer
starts at 0.7 sec
Source Filter
(...continues...)
22. IAudioSource
public interface IAudioSource
{
function get descriptor():AudioDescriptor;
function get frameCount():Number;
function get position():Number;
function resetPosition():void;
function getSample(numFrames:Number):Sample;
function clone():IAudioSource;
}
implementations:
Sample, SineSource, SoundSource...
23. IAudioFilter
public interface IAudioFilter extends IAudioSource
{
function get source():IAudioSource;
function set source(s:IAudioSource):void;
}
implementations:
EnvelopeFilter, EchoFilter, BandpassFilter...