A 2D-3D Hybrid Vision System for Robotic Manipulation of Randomly Oriented Objects

This paper presents an new vision technique for robotic manipulation of randomly oriented objects in industrial applications. The proposed approach uses 2D and 3D vision for efficiently extracting the 3D pose of an object in the presence of multiple randomly positioned objects. 2D vision permits to quickly select the objects of interest for 3D processing with a new modified ICP algorithm (FaR-ICP), thus reducing significantly the processing time. The extracted 3D pose is then sent to the robot manipulator for picking. The tests show that the proposed system achieves high performances

Authors:

• Moulay A. Akhloufi

Keywords:

• 3D vision
• Hand-Eye calibration
• robot visual servoing
• random bin picking.

References:

[1] Wilson, W.J., Hulls, C.C.W., Bell, G.S.: Relative End-Effector Control
Using Cartesian Position Based Visual Servoing. IEEE Trans. Robotics
and Automation. Vol. 12, No. 5, pp. 684ÔÇö696 (1996)
[2] Grosso, E., Metta, G., Oddera, A., Sandini, G.: Robust Visual Servoing
in 3-D Reaching Tasks. IEEE Trans. Robotics and Automation RA. Vol.
12, No. 5, pp. 732ÔÇö742 (1996)
[3] Wijesoma, S.W., Wolfe, D.F.H., Richards, R.J.: Eye-to-Hand
Coordination for Vision-Guided Robot Control Applications.
International Journal of Robotics Research. Vol. 12, pp. 65--78 (1993)
[4] Sanchez, A.J., Martinez, J.M.: Robot-arm Pick and Place Behavior
Programming System Using Visual Perception. In: 15th International
Conference on Pattern Recognition, Vol. IV, pp. 507ÔÇö510, Barcelona,
Spain (2000)
[5] Hartley, R., Zisserman, A.: Multiple View Geometry in Computer
Vision. Cambridge University Press (2000)
[6] Lowe, David G. (1999). "Object recognition from local scale-invariant
features". Proceedings of the International Conference on Computer
Vision. 2. pp. 1150-1157.
[7] Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool "SURF:
Speeded Up Robust Features", Computer Vision and Image
Understanding (CVIU), Vol. 110, No. 3, pp. 346--359, 2008
[8] Krystian Mikolajczyk and Cordelia Schmid "A performance evaluation
of local descriptors", IEEE Transactions on Pattern Analysis and
Machine Intelligence, 10, 27, pp 1615--1630, 2005.
[9] Sarfraz, S., Hellwich, O.:"Head Pose Estimation in Face Recognition
across Pose Scenarios", Proceedings of VISAPP 2008, Int. conference
on Computer Vision Theory and Applications, Madeira, Portugal, pp.
235-242, January 2008
[10] Tarsha-Kurdi, F., Landes, T., Grussenmeyer, P., 2007a. Houghtransform
and extended RANSAC algorithms for automatic detection of
3d building roof planes from Lidar data. ISPRS Proceedings. Workshop
Laser scanning. Espoo, Finland, September 12-14, 2007
[11] D.H. Ballard, "Generalizing the Hough Transform to Detect Arbitrary
Shapes", Pattern Recognition, Vol.13, No.2, p.111-122, 1981
[12] P. Besl and N. McKay. A method for Registration of 3-D Shapes.
IEEE Transactions on Pattern Analysis and Machine Intelligence
(PAMI), 14(2):239 - 256, February 1992.
[13] Andreas N├╝chter, Kai Lingemann, and Joachim Hertzberg. Cached k-d
tree search for ICP algorithms. In Proceedings of the 6th IEEE
International Conference on Recent Advances in 3D Digital Imaging and
Modeling (3DIM '07), IEEE Computer Society Press, pp. 419-426,
Montreal, Canada, August 2007,
[14] K. S. Arun, T. S. Huang, and S. D. Blostein.
Least square fitting of two 3-d point sets.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
9(5):698 - 700, 1987.
[15] B. K. P. Horn. Closed-form solution of absolute orientation using unit
quaternions. Journal of the Optical Society of America A, 4(4):629 -
642, April 1987.
[16] B. K. P. Horn, H. M. Hilden, and Sh. Negahdaripour.
Closed-form solution of absolute orientation using orthonormal
matrices.
Journal of the Optical Society of America A, 5(7):1127 - 1135, July
1988.
[17] M. W. Walker, L. Shao, and R. A. Volz.
Estimating 3-d location parameters using dual number quaternions.
CVGIP: Image Understanding, 54:358 - 367, November 1991.
[18] A. Lorusso, D. Eggert, and R. Fisher. A Comparison of Four
Algorithms for Estimating 3-D Rigid Transformations. In Proceedings
of the 4th British Machine Vision Conference (BMVC '95), pages 237 -
246, Birmingham, England, September 1995.
[19] Y. Chen, G. G. Medioni. Object modelling by registration of multiple
range images. Image and Vision Comp. 10(3), pp 145-155, 1992.
[20] Z. Y. Zhang. Iterative point matching for registration of free-form curves
and surfaces. Int. J. of Computer Vision, 13(2), pp 119-15, Oct. 1994.
[21] T. Masuda, N. Yokoya. A robust method for registration and
segmentation of multiple range images. Comp. Vision and Image Under.
61(3), pp 295-307, May 1995.
[22] C. Dorai, J. Weng, A. K. Jain. Optimal registration of object views using
range data. IEEE Trans. Pat. Anal. and Mach. Intel. 19(10), pp 1131-
1138, Oct 1997.
[23] G. C. Sharp, S. W. Lee, D. K. Wehe. Invariant features and the
registration of rigid bodies. Proc. IEEE Int. Conf. on Robotics and
Autom., pp 932-937, 1999.
[24] Martin A. Fischler and Robert C. Bolles (June 1981). "Random Sample
Consensus: A Paradigm for Model Fitting with Applications to Image
Analysis and Automated Cartography". Comm. of the ACM 24: 381-
395
[25] Daniel B. Russakoff, Torsten Rohlfing, Calvin R. Maurer, Jr., "Fast
Intensity-based 2D-3D Image Registration of Clinical Data Using Light
Fields," iccv, vol. 1, pp.416, Ninth IEEE International Conference on
Computer Vision (ICCV'03) - Volume 1, 2003
[26] A. Kubias, F. Deinzer, T. Feldmann, D. Paulus, B. Schreiber, Th
Brunner. 2D/3D image registration on the GPU. Pattern Recognition and
Image Analysis, Vol. 18, No. 3. (2008), pp. 381-389
[27] M.A. Akhloufi,. Real-time 3D visual servoing of an industrial cutting
robot manipulator. Proc. International Conference on Control,
Automation, Robotics and Vision (ICCARV 2009), WASET Vol. 60,
pp. 432-436, ISSN: 2070-3724, Bangkok, Thailand, December 2009.
[28] Fanuc Robotics, http://www.fanucrobotics.com/
[29] KUKA Robotics, http://www.kuka-robotics.com/
[30] ABB Robotics, http://abb.com