At our fourth edition of Behavior Design AMS meetup series April 24, 2014, we invited Dariu Gavrila to tell us how cars are recognising behavior now and in the future. Dariu works as research at the University of Amsterdam and at Daimler Benz. http://www.meetup.com/Behavior-Design-AMS/events/173715882/

1. Smart Cars that See (and Act)
Dariu M. Gavrila
Environment Perception
Research and Development
Behavior Design Meetup, 24-04-2014, Amsterdam

2. D. M. Gavrila
2
We originally thought Machine Intelligence would look like
1956 "Forbidden Planet"
Robby the Robot (Flickr)

3. D. M. Gavrila
3
Then more recently, some suggested it would be more like
2004 iRobot 1983-2009 Terminator

4. D. M. Gavrila
when in fact, Machine Intelligence is already with us,
and has a familiar embodiement …
The new Mercedes-Benz S Class (2013)
4

5. D. M. Gavrila
Sensors and Coverage (Mercedes-Benz E- and S-Class)
5
360°CAMERA
4 m range

6. D. M. Gavrila
6
Driver Assistance at Mercedes Benz (E- and S-Class, 2013)
Adaptive High Beam
Pre-Crash Braking (longitudinal & lateral traffic)
(Active)
Lane Keeping
Nightview AttentionTraffic Signs
Parking Adaptive Cruise Control
with Steering Assist
(Active) Body Control
S-Class only

7. D. M. Gavrila
7
Challenge: Active Pedestrian Safety

8. D. M. Gavrila
8
Worldwide Traffic Fatalities 2010 –
Share of Vulnerable Road Users
Illustration: Bosch

9. D. M. Gavrila
Typical Scenario
9

10. D. M. Gavrila
10
Why is it difficult?
Large variation in pedestrian appearance (viewpoint, pose, clothes).
Dynamic and cluttered backgrounds.
Pedestrians can exhibit highly irregular motion.
Real-time processing required.
Stringent performance requirements (especially for emergency maneuvres).

11. D. M. Gavrila
11
Better Algorithms: System Architecture
Pedestrian
Classification
Tracking / Fusion
Driver Warning /
Vehicle Control
Path Prediction &
Risk Assessment
Regions of Interest
(Stereo, Motion, Geometry)
Gavrila and Munder, „Multi-Cue Pedestrian Detection and Tracking from a Moving Vehicle”. Int. J. Computer Vision, 2007
S. Munder, C. Schnörr and D.M. Gavrila. “Pedestrian Detection and Tracking Using a Mixture of View-Based Shape-
Texture Models.” IEEE Transactions on Intelligent Transportation Systems, 2008

12. D. M. Gavrila
12
Dense Stereo: Better ROIs, Classification and Localization
C. Keller, M. Enzweiler, M. Rohrbach, D.-F. Llorca, C. Schnörr, and D.M. Gavrila. „The Benefits of Dense Stereo
for Pedestrian Detection.“ IEEE Trans. on Intelligent Transportation Systems, 2011.

13. D. M. Gavrila
13
Lots of Real Data …
Ulm
Stuttgart
München
Aachen
Amsterdam
Parma
Brussels
Tokyo
Paris
Barcelona
San Francisco
…
O(107) images
O(106) labels

14. D. M. Gavrila
14
EU WATCH-OVER (2008)
85%
50 km/h
10
Pedestrian Recognition Performance Last Decade (Downtown)
Correctly
recognized
pedestrians
Number of falsely recognized pedestrian trajectories per hour
100%
50%
10 10000 100100
EU SAVE-U (2005)
65%
40 km/h
EU PROTECTOR (2003)
40%
600
30 km/h
N.B. # False alarms per hour << # Falsely recognized trajectories per hour
DE AKTIV-SFR (2010)
90%

15. D. M. Gavrila
Driver Assistance at Mercedes Benz (E- and S-Class, 2013)
Adaptive High Beam
Pre-Crash Braking (longitudinal & lateral traffic)
Nightview AttentionTraffic Signs
Parking
with Pedestrian Recognition (Active) Body Control
S-Class only
15
(Active)
Lane Keeping
Adaptive Cruise Control
with Steering Assist

16. D. M. Gavrila
16

17. D. M. Gavrila
Will the Pedestrian Cross? (Through the Eyes of the Sytem)
17

18. D. M. Gavrila
Will the Pedestrian Cross? (Through the Eyes of the Driver)
18

19. D. M. Gavrila
(Future) Context-based Pedestrian Path Prediction
19
Preliminary experiments suggest that emergency braking could be applied 0.5 s earlier, without
increasing false alarms, reducing chances for hospital stay by 30% (TTC = 0.7s, vehicle speed = 50 km/h)
19

20. D. M. Gavrila
21
(Future) Automatic Evasion
C. Keller, T. Dang, A. Joos, C. Rabe, H. Fritz, and D.M. Gavrila. Active Pedestrian Safety by Automatic Braking and Evasive Steering,
IEEE Trans. on Intelligent Transportation Systems, 2011
300 ms from first sight of pedestrian to initiation of vehicle maneuver (braking or evasion)

21. D. M. Gavrila
(Future) Autonomous Driving
22
Mercedes-Benz S500 Intelligent Drive Research Vehicle
(Full videoclip available on YouTube)

22. D. M. Gavrila
Final Remarks
Dramatic progress on vision-based pedestrian sensing, mirroring the success of
computer vision technology in driver assistance over the past decade.
Perseverance pays off.
Challenges still remain for pedestrian sensing.
Additional focus: data fusion, situation understanding, new actuation concepts.
Accident analysis at Mercedes-Benz indicates that the currently deployed
pedestrian recognition technology could avoid 6 percent of pedestrian accidents
and reduce the severity of a further 41 percent.
23
1999 2013

23. D. M. Gavrila
24
Better Algorithms + Market-Ready Hardware + More Data + Extensive Testing =
More Lives Saved.

24. D. M. Gavrila
25
Thank you
Credits
Markus Enzweiler, Christoph Keller, Stefan Munder, Jan Giebel
and others …