knowdiff.net Design of versatile vision-based robotic systems demands a solution with little or no dependence on system parameters. The problem of real-time vision-based control of robots has been long studied as robotic visual servoing. Most provably stable solutions to this problem require calibrated kinematic and camera models, because in a precisely calibrated system one can model the visual-motor function analytically. The uncalibrated approach has received limited attention mainly because the stability analysis is not as straightforward as that of calibrated image-based architecture. In an uncalibrated system the visual-motor function is not known, but partial derivative information (Jacobian) can be learned by tracking visual measurements during motion. In this talk, we study the uncalibrated image-based visual servoing and present different Jacobian learning methods.

1. Uncalibrated Image-based Control of Robots Azad Shademan PhD Candidate Computing Science, University of Alberta Edmonton, Alberta, CANADA [email_address]

2. Vision-Based Control A A A B B B current desired Left Image Right Image

3. Vision-Based Control Left Image Right Image B B B

8. Position-Based Desired pose Estimated pose

9. Position-Based Measurement noise State variable Process noise yaw pitch roll Measurement equation (projection) is nonlinear and must be linearized.

10. K x k-1,k-1 z k R k P k,k P k,k-1 C k Q k-1 x k,k x k,k-1 P k-1,k-1 Kalman Gain Measurement noise covariance A priori error cov. @ k-1 Process noise covariance Initial/previous state Linearization Measurement State update State prediction Error cov. prediction Error cov. update

11. Image-Based Desired Image feature Extracted image feature

12. Visual-motor Equation x 1 x 2 x 3 x 4 q=[q 1 … q 6 ] Visual-Motor Equation This Jacobian is important for motion control.

13. Visual-motor Jacobian Image space velocity Joint space velocity A A B B

15. Image-Based Control Law Desired Image feature Extracted image feature

25. K-NN Regression-based Method q 1 q 2 3 NN q 1 q 2 x 1 ?

26. (X,q) L ocally L east S quares Method q 1 q 2 x 1 ? K-neighbour(q)

28. Measuring the Estimation Error

29. Global Estimation Error

31. Eye-in-hand

32. Eye-in-Hand Experiments

33. Eye-in-Hand Experiments

34. Eye-in-Hand Experiments

35. Eye-in-Hand Experiments

36. Mean-Squared-Error Task 1 Task 2

37. Task Errors

40. Thank you!

41. Visual Ambiguity: Single Camera

42. Visual Ambiguity: Stereo