作者：劉凌偉 【視覺進化論】AI智慧視覺運算技術論壇 ►活動日期：2018/9/26(三) 13:30-16:30 ►活動網址：https://makerpro.cc/events/180926_smart_vision/ ►聯繫Sertek：www.sertek.com.tw / 886-2-2696-0055

1. 3D深度影像之AI視覺技術與應用契機
LukeLiu

2. 2
2
◆ Company Overview
◆ Product Portfolio
◆ AI base on 3D Data with
OpenVINO
◆ Demo
AGENDA

4. Founded in 2013 by a team consisting of MIT research scholars from CSAIL.
History & Business Model
• 3D depth cameras based on ToF、Active Stereo technologies and Structured Light
• Standard recognition middleware
• Standard solutions
• Customized camera design, middleware, and total solution projects for developers and marketers worldwide
Positioned as 3D Sensing solution provider:
4

5. LIPS’ Capabilities
LIPS is able to provide tire 1~3 levels of solution from hardware
only, SDK and MW integration, all the way up to application
products
5

6. 6
The art of seamless integration between OP,ME,EE, and Software

7. 77
ToF DL
FoV 75.4ox60.2o
640x480@60fps
113x37x26mm
w/o IMU
ToF AT
(Cyclone V based)
FoV 74ox58o
640x480@30fps
190x50x56mm
w/o IMU
• Depth cameras models are categorized by technologies it’s based on
• ToF stands for Time of Flight. Active Stereo is a hybrid design of structured light and stereo technologies.
• AT is a standalone model that doesn’t require a host.
ToF M3
FoV 74.1ox57.5o
80x60@120fps
37x28x15mm
w/ IMU
Active Stereo M5
FoV 60 o x47 o
640x480@60fps
70x15x5mm
w/ IMU
LIPSedge Series Depth Cameras

8. 8
LIPS 3D Vision Middleware 3/9
3D Ruler
3D Metrology middleware currently offers automatic
measurement over L-W-H sizes of cuboid-shape
object typically cardboard boxes.
• Automatically measures the sizes of box in rectangular
cuboid under the following conditions:
• Max Measuring Size (L x W x H): 1500 x 1500 x 1500 (mm)
• Min Measuring Size (L x W x H): 200 x 200 x 200 (mm)
• Shape: Cuboid
• Distance Tolerance: Optimal measuring distance ±20%
• Measuring Accuracy: ±5%
• Measuring Latency: <0.5 Sec
• Box tagging
• Online demo: box measurement & bounding box
Windows, Linux, Android People Counting
People counting can differ dramatically in the way
how people is to be counted as to from what angle
the depth camera is mounted.
• Functions:
• Can detect multiple people (i.e. overlapping at the
detection section) at the same time walking at both
direction at normal walking speed across vertical
detection cross section.
• Can detect adults or kids (i.e. of different heights)
• Can count the amount of leaving event and entering
event.
• Count accuracy (recall rate): 98%
• Online demo: more data
• Online demo: field try
• Online demo: elevator density meter
Windows, Linux Driver Fatigue Detection
It analyzes facial features including closing both eye
lids and/or head’s forward pitch angle greater than
pre-defined degree for a pre-defined length of
seconds, that signifies fatigue.
• Functions:
• Can detect the fatigue related feature include: eyes
closing, nodding and yawning.
• Can calculate the level of fatigue and sends
notification event.
• Online demo: field try
Windows, Linux

9. 9
Hand Gesture
This middleware captures detected hand’s various
gestures moving in a relatively natural motion.
• Motions:
• Swipe left
• Swipe right
• Swipe up
• Swipe down
• Push forward
• Finger count
• Detection range: 0.5~2 meters (initial detection
range, once detected can be tracked up to 4 meters)
• Online demo: hand posture
• Online demo: hand gesture
Windows, Linux User Tracking
It can be invoked directly in the same fashion you
would to any regular API.
• Functions:
• Full-body user tracking with 25 specific skeleton joints.
• 32 users skeletons tracking simultaneously
• Online demo: skeleton tracking @ real time
Windows, Linux, Android Facial Detection
Facial recognition includes identity, gender, age,
liveness, and face’s pitch/yaw/roll angles.
• Functions:
• Can extract 47 facial landmarks.
• Can detect identity, gender, and age group (kids,
young adult, adult, senior) Face properties include:
liveness, face’s pitch/yaw/roll angles, face ROI.
• Liveness check from depth map
• Pure 3D facial recognition
• Online demo: 3D recognition on Android
• Online demo: liveness check
LIPS 3D Vision Middleware 6/9

10. 1
0
VSLAM & Object Avoidance
Realtime map creation, navigation and obstacle
avoidance
• VSLAM: environment reconstruction
• Obstacle avoidance
• Plan route
• Online demo: VSLAM and object avoidance
Factory Automation: Robot ARM
Base on LIPS 3D camera, and SDK to find the target
object and plan a route, so that plasma fit on the
target.
• Functions:
• Object finding
• Route planning
• Online demo: automatic plan path for plasma
Factory Automation: Object Sorting
Through LIPS’ active-stereo depth camera, the 3D
models of the various objects moving on the
conveyor pre-scanned in the database are matched in
a real-time and non-stop fashion and sorted by
robotic arm to their designated container. The system
takes advantage of LIPS’ 3D scanning system (a.k.a.
LIPScan 3D) to create object’s 3D model for arbitrary
360-degree matching and sorting that requires no
learning process beforehand.
• Online demo : object sorting
LIPS 3D Vision Middleware 9/9

11. 1
1
LIPS ADAS Solution Online demo: VD, PD and LD
Online demo: with 3D camera

12. 1
2
LIPS 3D Scan Solution Online demo: on Android

13. 1
3
What’s 3D Information(Depth), Point Cloud

14. 1
4
The Benefit of Depth for Deep Learning
• Easy to segmentation, and feature extraction
• Get the real dimension of the object, shape with different view
• Get the relative distance and angle
• Not a projected image plane, but multi view object
• And the most important, not effective by light or others

15. 1
5
Big Data + Deep Representation Learning

16. 1
6
Neuro Network

17. 1
7
Results on Object Part Segmentation

18. 1
8
Results on Object Part Segmentation

19. 1
9
Visualizing Global Point Cloud Features

20. 2
0
Some Tips for OpenVINO
• Support float type input
• Support float accuracy inference
• GPU: fp16
• CPU: fp32
• The performance up to 10 times, after optimize the dataset
• LIPS is software alliance of Intel IoT, RealSense and Altera
• Demo

21. THANK YOU!
LIPS Corporation
www.lips-hci.com