Nghiên cứu kỹ thuật Q1-2018 của công ty GMO-Z.com VietnamLab

1. Surveillance on
SLAM Technology

2. Outline
● Introduction
● Surveillance
● Implementation
● Conclusion

4. SLAM
● Simultaneous localization and mapping
● Given a series of sensor observations ot over
discrete time steps t, the SLAM problem is to
compute an estimate of the agent's location xt
and a map of the environment mt

5. Applications of SLAM
● Exploration
● Self-driving cars
● Augmented reality

6. Outline
● Introduction
● Surveillance
● Implementation
● Conclusion

7. Surveillance
● Wikitude
● Kudan
● Tango, ARCore
● ORB-SLAM2

8. Wikitude
● Advantage
○ SLAM is runnable in Unity 2017
○ Support many features
● Disadvantage
○ Fee require

9. Demo Wikitude

10. Kudan
● Advantage
○ Demo video is really good
○ Free for Indie
● Disadvantage
○ The example is not runnable in Unity 2017

12. Tango/ARCore
● Advantage
○ Develop by Google
○ Free
● Disadvantage
○ Support only limited devices :(

14. ORB-SLAM2
● Advantage
○ Open source in C++
○ Strong MonoSLAM
○ Nice demo video
● Disadvantage
○ Cost time to understand details/algorithm
○ MonoSLAM is good only with 2D movement
https://github.com/raulmur/ORB_SLAM2

17. Outline
● Introduction
● Surveillance
● Implementation
● Conclusion

18. Implementation device
● Kinect
○ Mono camera + Depth sensor
● Mobile
○ Mono camera + gyro sensor + accelerometer
● Stereo camera
○ Able to estimate depth map
=> Mobile

19. Implementation technology
● Extendable with new technology / open source
● Save money
● Able to implement new algorithm
=> OpenCV (C++) + Unity + Android

20. Derivation
● Tutorial Unity + Android + OpenCV C++
● Face Recognition
● Ship Shooter

21. Tutorial Unity + Android + OpenCV C++
● C++ with Unity
● OpenCV C++ with Unity Android
● Object Recognition with Unity Android

22. C++ with Unity
● Write code in C++
● Build as Shared Object (in Visual Studio 2015)
● Add builded file to Unity as library and call in
Unity
● Can Marshall simple object from C# to C++ and
from C++ to C#

23. C++ with Unity
● Is it able to pass by reference from C# to C++ ?
○ Yes, use pointer in C++ and ref/out keyword in C#
● Can I pass an array from C# to C++ ?
○ Yes, you can pass array of primitive type from C# to
C++ (I did not try if array struct type)
● Can I receive an array from C++ to C# ?
○ Yes, you can. There are 2 ways to archive
○ Void function with pass by reference
○ Pointer-returned function with marshall in C#

24. OpenCV with Unity Android
● Download opencv4android from OpenCV
homepage
● Create Shared Object project with OpenCV config
● Write your image processing code
● Build your source code in ARM -> .so file
● Add .so file to Unity as library

25. OpenCV with Unity Android
● How to pass image ?
○ Unity image: RGBA type (not RGB)
○ Can pass as unsigned char* array or Color32 array
○ Receive raw data and re-create image with OpenCV
Mat
● How to process image ?
○ Cannot use opencv.hpp
○ Use core.hpp, imgproc.hpp, highgui.hpp instead
○ Be careful about memory management !!!

26. Example code

27. Example code

28. OpenCV Object Recognition With Unity
● Use objdetect.hpp
● Using Cascade to detect object
○ How to load Cascade data ???

29. OpenCV Object Recognition With Unity
● Can I read files in Unity/apk from C++ ?
○ No, you can’t
● How can I load cascade file to use in OpenCV ?
○ Read cascade xml file by using unity to string
○ Pass string to C++
○ Use cv::FileStorage to load it in memory
○ Init cascade classifier by read the FileStorage created
above

30. Example code

31. Example code

32. OpenCV Object Recognition With Unity
● Why face detection cascade is too slow ?
○ Smartphone camera has high resolution (720p);
therefore, process on this image is really slow
● How can I speed up the detection process ?
○ Detect on low resolution image, then convert the result
and display on high resolution image
○ Use ORB/FAST feature to detect (really hard)

33. Demo Face Recognition

34. Ship Shooter
● Spawning Ship by Face Recognition
● Ship Distance Change by face size (Simple depth
estimate)
● Fire the ship !!!

35. Demo Ship Shooter

36. Outline
● Introduction
● Surveillance
● Implementation
● Conclusion

37. Conclusion
● SLAM
● Some technologies
○ ORB-SLAM2, Wikitude, Kudan
● Implementation
○ Tutorial OpenCV with OpenCV Android
○ Face Detection, Ship Shooter

38. Q/A

39. Thanks for listening