Slides from the talk about augmented reality given in the mDevcon tour (9 Apr 2014)

1. Augmented Reality
Joan Puig Sanz
9 Apr 2014

2. Agenda
1. What is Augmented Reality
2. Existing Platforms
3. BeyondAR Framework
1. What can we do with it?
4. What is the next step?

3. Augmented Reality
• According to WikiPedia:
“Augmented reality (AR) is a live direct or indirect view of a physical, real-
world environment whose elements are augmented (or supplemented)
by computer-generated sensory input such as sound, video, graphics or
GPS data”

4. Augmented Reality
• According to WikiPedia:
“Augmented reality (AR) is a live direct or indirect view of a physical, real-
world environment whose elements are augmented (or supplemented)
by computer-generated sensory input such as sound, video, graphics or
GPS data”

5. Make Augmented Reality
• Using geo localization
• Image recognition
• Sensors
– Accelerometer
– Magnetic field
– Compass
– Motion tracking camera
– …

6. EXISTING TOOLS

7. Existing Tools
• Vuforia
• Layar
• Wikitude
• Droidar
• Mixare
• Google Tango
• BeyondAR
• …

8. Vuforia
• Proprietary
• Available for Android and iOS
• Unity support
• Big community
• Collect some data form the user: related to the
scanned images
• Target recognition
– Device database: free < 100 images
– Cloud Database: not free >100

9. Vuforia
• https://www.youtube.com/watch?v=UOfN1pl
W_Hw

10. Layar
• Proprietary
• SDK Available for Android and iOS
–Geo localization
–Image recognition

11. Layar
• App browser (Android and iOS)
– Geo Layers and image recognition
Demo time!

12. Wikitude
• Proprietary
• SDK Available for Android, iOS, Epson
Moverio and Vuzix.
–Geo localization
–Image recognition

13. Wikitude
• Proprietary
• SDK Available for Android, iOS, Epson
Moverio and Vuzix.
–Geo localization
–Image recognition

14. Wikitude
• Proprietary
• SDK Available for Android, iOS, Epson
Moverio and Vuzix.
–Geo localization
–Image recognition

15. Wikitude
• Proprietary
• SDK Available for Android, iOS, Epson
Moverio and Vuzix.
–Geo localization
–Image recognition

16. Wikitude
• Extensions for PhoneGap and Titanium
• App browser (Android, iOS and BB10)
– Geo Layers and image recognition

17. Droidar
• GPL3
• SDK for Android
• Location based
• Supports 3D modeling
/bitstars/droidar

18. Mixare
• GPL3
• SDK for Android and iOS
• Location based
• Canvas
– Slow performance
• No code changes since 2 years ago
/mixare

19. Google Tango
https://www.youtube.com/watch?v=Qe10ExwzCqk

20. BeyondAR
• Apache 2
• SDK for Android
• Location based
• 3D
• Active development
/BeyondAR

21. LET’S CODE!
/BeyondAR
Example App available on
Google play

22. BeyondAR
Getting started
• Import the library in your project
• Update Manifest
<!-- Minimum permissions for BeyondAR -->
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<!-- For BeyondAR this is not mandatory unless you want to load something from the network -->
<uses-permission android:name="android.permission.INTERNET" />
<!-- BeyondAR needs the following features-->
<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.sensor.accelerometer" />
<uses-feature android:name="android.hardware.sensor.compass" />
/BeyondAR

23. BeyondAR
Getting started
Create the UI
<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:id="@+id/parentFrameLayout" >
<fragment
android:id="@+id/beyondarFragment"
android:name="com.beyondar.android.fragment.BeyondarFragmentSupport"
android:layout_width="match_parent"
android:layout_height="match_parent" />
</FrameLayout>
/BeyondAR

24. BeyondAR
Getting started
Create the UI
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.example);
// ...
mBeyondarFragment = (BeyondarFragmentSupport)
getSupportFragmentManager().findFragmentById (R.id.beyondarFragment);
// ...
}
BeyondarFragment: Class that manages the camera and the OpenGL surface
/BeyondAR

25. BeyondAR
Getting started
Create the AR world and add an AR object
// We create the world object
World myWorld = new World(this);
myWorld.setGeoPosition(41.90533734214473d, 2.565848038959814d);
World: Container for all the BeyondarObjects
• Try to manage all BeyondarObjects using this class
• Responsible for the user location
• You can add plugins
/BeyondAR

26. BeyondAR
Getting started
Create the AR world and add an AR object
// We create the world object
World myWorld = new World(this);
myWorld.setGeoPosition(41.90533734214473d, 2.565848038959814d);
// Create an object
GeoObject go1 = new GeoObject();
go1.setGeoPosition(41.90523339794433d, 2.565036406654116d);
go1.setImageResource(R.drawable.my_image);
go1.setName(”Hello World");
// Add the object
myWorld.addBeyondarObject(go1);
// give the world to the fragment
mBeyondarFragment.setWorld(myWorld);
/BeyondAR

27. BeyondAR
Getting started
Create the AR world and add an AR object
// We create the world object
World myWorld = new World(this);
myWorld.setGeoPosition(41.90533734214473d, 2.565848038959814d);
// Create an object
GeoObject go1 = new GeoObject();
go1.setGeoPosition(41.90523339794433d, 2.565036406654116d);
go1.setImageResource(R.drawable.my_image);
go1.setName(”Hello World");
// Add the object
myWorld.addBeyondarObject(go1);
// give the world to the fragment
mBeyondarFragment.setWorld(myWorld);
/BeyondAR

28. BeyondAR
Getting started
Interaction with the AR Objects
/BeyondAR

29. BeyondAR
Getting started
Interaction with the AR Objects
OnTouchBeyondarViewListener OnClickBeyondarObjectListener
mBeyondarFragment.setOnTouchBeyondarViewListener(this);
mBeyondarFragment.setOnClickBeyondarObjectListener(this);
@Override
public void onClickBeyondarObject(ArrayList<BeyondarObject> beyondarObjects) {
if (beyondarObjects.size() > 0) {
Toast.makeText(this, "Clicked on: " +
beyondarObjects.get(0).getName(), Toast.LENGTH_LONG).show();
}
}
/BeyondAR

30. BeyondAR
Getting started
Interaction with the AR Objects
OnTouchBeyondarViewListener OnClickBeyondarObjectListener
mBeyondarFragment.setOnTouchBeyondarViewListener(this);
mBeyondarFragment.setOnClickBeyondarObjectListener(this);
@Override
public void onClickBeyondarObject(ArrayList<BeyondarObject> beyondarObjects) {
if (beyondarObjects.size() > 0) {
Toast.makeText(this, "Clicked on: " +
beyondarObjects.get(0).getName(), Toast.LENGTH_LONG).show();
}
}
/BeyondAR

31. OTHER FEATURES
1. Location utils
2. Views in the AR world
3. Take screenshots
4. Plugins
5. …
/BeyondAR

32. BeyondAR
Using location utils
• Easy way to use the location services
LocationManager locationManager = (LocationManager)
getSystemService(Context.LOCATION_SERVICE);
// We need to set the LocationManager to the BeyondarLocationManager.
BeyondarLocationManager.setLocationManager(locationManager);
BeyondarLocationManager: Helper to use the location services.
• Retrieves the best location using GPS and the network providers
• Can be used with LocationListener, World or GeoObject
/BeyondAR

33. BeyondAR
Using location utils
Create the AR world and add an AR object
@Override
protected void onResume() {
super.onResume();
// When the activity is resumed it is time to enable the
// BeyondarLocationManager
BeyondarLocationManager.enable();
}
@Override
protected void onPause() {
super.onPause();
// To avoid unnecessary battery usage disable BeyondarLocationManager
// when the activity goes on pause.
BeyondarLocationManager.disable();
}
BeyondarLocationManager.addWorldLocationUpdate(myWorld);
BeyondarLocationManager.addGeoObjectLocationUpdate(user);
/BeyondAR

34. OTHER FEATURES
1. Location utils
2. Views in the AR world
3. Take screenshots
4. Plugins
5. …
/BeyondAR

35. BeyondAR
Working with Views
• Attach a View to a BeyondarObject
Extend BeyondarViewAdapter  It follows the same pattern than the
ListAdapter
- Remember to recycle your Views!!
@Override
public View getView(BeyondarObject beyondarObject, View recycledView, ViewGroup parent) {
if (recycledView == null)
recycledView = inflater.inflate(R.layout.beyondar_object_view, null);
TextView textView = (TextView) recycledView.findViewById(R.id.titleTextView);
textView.setText(beyondarObject.getName());
Button button = (Button) recycledView.findViewById(R.id.button);
button.setOnClickListener(myClickListener);
// Once the view is ready we specify the position
setPosition(beyondarObject.getScreenPositionTopRight());
return recycledView;
}
Createtheview
/BeyondAR

36. BeyondAR
Working with Views
• Make a BeyondarObject from a View
ImageUtils.storeView(view, path, imageName);
// If there are no errors we can tell the object to use the
// view that we just stored
beyondarObject.setImageUri(path + imageName);
/BeyondAR

37. BeyondAR
Working with Views
• Make a BeyondarObject from a View
ImageUtils.storeView(view, path, imageName);
// If there are no errors we can tell the object to use the
// view that we just stored
beyondarObject.setImageUri(path + imageName);
/BeyondAR

38. OTHER FEATURES
1. Location utils
2. Views in the AR world
3. Take screenshots
4. Plugins
5. …
/BeyondAR

39. BeyondAR
Screenshoots
mBeyondarFragment.takeScreenshot(myOnScreenshotListener);
@Override
public void onScreenshot(Bitmap screenshot) {
ImageDialog dialog = new ImageDialog();
dialog.setImage(screenshot);
dialog.show(getSupportFragmentManager(), "ImageDialog");
}
When you are done with the image, remember to recycle it
/BeyondAR

40. BeyondAR
Screenshoots
mBeyondarFragment.takeScreenshot(myOnScreenshotListener);
@Override
public void onScreenshot(Bitmap screenshot) {
ImageDialog dialog = new ImageDialog();
dialog.setImage(screenshot);
dialog.show(getSupportFragmentManager(), "ImageDialog");
}
When you are done with the image, remember to recycle it
/BeyondAR

41. OTHER FEATURES
1. Location utils
2. Views in the AR world
3. Take screenshots
4. Plugins
5. …
/BeyondAR

42. BeyondAR
Plugins
• Main goal:
– Easy to use
– Make your own features
/BeyondAR

43. BeyondAR
Plugins
• Example: Google Maps Plugin
// As we want to use GoogleMaps, we are going to create the plugin and
// attach it to the World
mGoogleMapPlugin = new GoogleMapWorldPlugin(this);
// Then we need to set the map in to the GoogleMapPlugin
mGoogleMapPlugin.setGoogleMap(mMap);
// Now that we have the plugin created let's add it to our world.
// NOTE: It is better to load the plugins before start adding object in to the world.
mWorld.addPlugin(mGoogleMapPlugin);
/BeyondAR

44. OTHER FEATURES
1. Location utils
2. Views in the AR world
3. Take screenshots
4. Plugins
5. …
/BeyondAR

45. What is the next step?
• Get over the WOW effect
• Choose your platform
• Make apps
– Education
– Traveling
– Gaming

46. beyondar.com
@joanpuigsanz
/Beyondar
46
@beyondar