1. Introduction to RoboticsPerception II CSCI 4830/7000 February 15, 2010 NikolausCorrell

2. Review: Sensing Important: sensors report data in their own coordinate frame Examples from the exercise Accelerometer of Nao Laser scanner Treat like forward kinematics

3. Today Perception using vision Practical angle: Why is vision hard Basic image processing How to combine image processing primitives into object recognition OpenCV / SwisTrack

4. Why is Vision Hard?The difference between seeing and perception. Gary Bradski, 2009 4 What to do? Maybe we should try to find edges …. Gary Bradski, 2005

6. Surface orientation discontinuity

7. Reflectance discontinuity (i.e., change in surface material properties)

8. Illumination discontinuity (e.g., shadow)Slide credit: Christopher Rasmussen 5 But, What’s an Edge?

9. To Deal With the Confusion, Your Brain has Rules...That can be wrong

10. We even see invisible edges…

11. And surfaces …

12. We need to deal with 3D Geometry 9 Perception is ambiguous … depending on your point of view! Graphic by Gary Bradski

13. And Lighting in 3D Which square is darker?

14. Lighting is Ill-posed … Perception of surfaces depends on lighting assumptions 11 Gary Bradski (c) 2008 11

15. Contrast 12 Which one is male and which one is female? Illusion by: Richard Russell,Harvard University Russell, R. (2009) A sex difference in facial pigmentation and its exaggeration by cosmetics. Perception, (38)1211-1219

16. Frequency

17. Color http://briantobin.info/2009/06/lost-and-found-visual-illusion.html

18. Pin-hole Model

19. Pin-Hole Camera A. Efros

20. Aperture

21. Increasing Aperture: Lens

22. Thin Lens Objects need to have the right distance to be in focus -> Depth-from-Focus method

23. Thresholds 20 20 http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm Screen shots by Gary Bradski, 2005

24. Canny Edge Detector 21 Gary Bradski (c) 2008 21

25. Morphological Operations Examples Morphology - applying Min-Max. Filters and its combinations Dilatation IB Opening IoB= (IB)B Erosion IB Image I Closing I•B= (IB)B TopHat(I)= I - (IB) BlackHat(I)= (IB) - I Grad(I)= (IB)-(IB)

26. Stereo Calibration Screen shots and charts by Gary Bradski, 2005 Gary Bradski (c) 2008 23 23

27. 3D Stereo Vision Find Epipolar lines: Align images: Triangulate points: Depth:

28. Example: Tomato-Picking Robot Challenges Foliage Reflections Varying size and shape Varying color Partly covered fruits http://swistrack.sourceforge.net N. Correll, N. Arechiga, A. Bolger, M. Bollini, B. Charrow, A. Clayton, F. Dominguez, K. Donahue, S. Dyar, L. Johnson, H. Liu, A. Patrikalakis, T. Robertson, J. Smith, D. Soltero, M. Tanner, L. White, D. Rus. Building a Distributed Robot Garden. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1509-1516, St. Louis, MO.

29. Filter-based object recognition Filter image Sobel Hough transform Color Spectral highlights Size and shape Weighted sum of filters highlights object location Color Sobel Hough Spectral Highlights

30. Group exercise Object recognition Goal Players Ball Field

31. Homework Read sections 4.2-5 (pages 145-180) Questionnaire on CU Learn