Wouter Verweirder (@wouter) en Koen De Weggheleire (@newmovieclip) presented this session at FITC Amsterdam in februari 2011. You can find the demos of this presentation at http://factory.happy-banana.be/interact_fitc_ams_2011/

1. INTERACT! with
the Flash Platform

2. SESSION overview
 our DO’S and DONT’S
 interaction with
 Sound
 Camera
 Mobile /Other devices
 Q and hopefully A
 DEMO’s

3. our DO’S and DONT’S
Wouter Verweirder
Belgian Citizen, Lord of Sealand
@wouter

4. our DO’S and DONT’S
 I DO:
 Spend a yearly budget of € 200
to lightsaber apps, noble titles, …
 Teach hardcore AS3 at HOWEST
 Am a Happy Banana
 Cheat on Flash Games
 I DON’T
 Use the lightsaber apps or noble
titles.

5. our DO’S and DONT’S
@newmovieclip

6. our DO’S and DONT’S
 I DO:
 Lecture
 Write
 Organise
 Like Happy-Banana!
 I DON’T
 Do sports, eat healthy and have a
girlfriend 

7. MAKE SOME NOISE !!

8. INTERACT! with sound

9. INTERACT! with sound
 flash.media.sound
 More control about input:
 flash.media.Microphone
 flash.media.SoundChannel
 More control about output:
 flash.media.SoundTransform

10. INTERACT! with sound
 Microphone
 Static getMicrophone() method
 setLoopback() + setUseEchoSurpression()
 Microphone.activityLevel (0-100)
 Capturing Microphone
 SampleDataEvent dispatched
 Write to ByteArray

11. INTERACT! with sound
 SampleDataEvent
 Contains sample data: event.data
 = ByteArray instance
 readFloat() method avances through
available bytes 4 at a time.
soundBytes
9
10
11
12
13
14
15
16
17
18
readFloat() next starting point

12. INTERACT! with sound
 Microphone.rate determines how many samples
are used every time the SampleDataHandler is
called.
 rate: 44  44,100 khz  8192 samples
 rate: 22  22,050 kHz  4096 samples
 rate: 11  11,025 kHz  2048 samples

13. INTERACT! with sound
Capture the samples and save in ByteArray:
Create new Sound instance

14. INTERACT! with sound
Fill the Sound with the captured soundBytes

15. DEMO

16. INTERACT! with Camera

17. INTERACT! with Camera
 flash.media.Camera
 Camera.getCamera();
 Video.attachCamera(camera)
 Interactions:
 activityLevel + motionLevel  motion
detected
 Pixel changes

18. INTERACT! with Camera
 Track pixel changes
 Draw video into BitmapData instance
 Use Difference blendmode
 Set threshold and give pixels color
 Check for color change

19. INTERACT! with Camera
 flash.media.CameraRoll
 CameraRoll.browseForImage()
 CameraRoll.addBitmapData()
 Check CameraRoll.supportsAddBitmapData
property
 flash.media.CameraUI
 CameraUI.launch(MediaType.IMAGE);
 CameraUI.launch(MediaType.VIDEO);
 Listen for Event.SELECT, Event.CANCEL
 Event.data = MediaPromise instance
 Use Loader.loadFilePromise() to load media
promise

20. INTERACT! with Camera
 flash.media.MediaPromise
 = promise to deliver a media object (image or
video)
 Access to the actual file: use the File Property
 Eg: promise.file.url,
promise.file.nativePath,...

21. DEMO

22. INTERACT! with Touch

23. INTERACT! with Touch
 Native MouseEvents also supported.
 flash.ui.Multitouch
 TouchEvent
 GestureEvent
 Plan:
 Step 1: Set the inputMode
 Step 2: Check if MultiTouch is supported
 Step 3: Add listeners for user interaction
 Step 4: Write handler methods to react on UI

24. INTERACT! with Touch
 TouchEvent
 COOL: pressure property: between 0 and 1
 TOUCH_BEGIN, TOUCH_END,
TOUCH_TAP,...
 GestureEvent / TransformGestureEvent
 More complex user contact
 GESTURE_TWO_FINGER_TAP
 ROTATE, SWIPE, ZOOM,...

25. INTERACT! with Touch
 Multitouch.supportsTouchEvents
 Multitouch.maxTouchPoints
 TouchEvent has property touchPointID
 Dragging : startTouchDrag, stopTouchDrag

26. DEMO

27. INTERACT! with Devices

28. INTERACT! with Devices
 Use devices capabilities als input
 Adobe AIR for Android
 Adobe AIR for IOS
 flash.sensors.Accelerometer
 Using the motion sensor of Device

29. INTERACT! with Devices
 AccelerometerEvent contains data
 AccelerationEvent.accelerationX
 AccelerationEvent.accelerationY
 AccelerationEvent.accelerationZ
 Send the data to a Server
 UDP would be nice…
 Flash.net.Socket (TCP/IP)
 Create Socket Server with Adobe AIR 

30. INTERACT! with Devices
 CLIENT applications: Socket class
 CLIENT sents commands to the SocketServer

31. INTERACT! with Devices
 ServerSocket: listen for connections on port
 When client is connected: SOCKET_DATA

32. INTERACT! with Devices
 Handle the incoming data

33. DEMO

34. INTERACT! with Kinect
 I can see, in 3D!
 Kinect data to flash: Sockets 

35. INTERACT! with Kinect
 AS3Kinect – Camera Images
 http://www.as3kinect.org/
 TUIOKinect – Blob cursor tracking
 https://code.google.com/p/tuiokinect/
 http://bubblebird.at/tuioflash/tuio-as3-library/
 http://silviopaganini.posterous.com/
openkinect-flash-tuio-udp-flash-bridge
 OSCeleton – Skeleton Detection
 https://github.com/Sensebloom/OSCeleton

36. INTERACT! with Kinect
 as3osceleton
 Open-sourced today
 Runs on top of OSCeleton & TUIOKinect
 Translates joints into cursors using
TUIOKinect bridge
 initial setup

37. INTERACT! with Kinect
 Event Driven:
 UserEvent.USER_JOINED
 UserEvent.USER_LEFT
 UserEvent.SKELETON_CREATED
 JointChangedEvent.JOINT_CHANGED
 Translation of hands to multitouch cursors

38. LETS PLAY SOME MUSIC !

39. Questions and hopefully
Answers

40. THANKS !!

41. Presenta3on
/
THANKS !!
Source
Code
demos:
http://
factory.happy-
banana.be/
FITC2011