Enhancement of Stereo Video Pairs Using SDNs To Aid In 3D Reconstruction

This paper presents the results of enhancing images from a left and right stereo pair in order to increase the resolution of a 3D representation of a scene generated from that same pair. A new neural network structure known as a Self Delaying Dynamic Network (SDN) has been used to perform the enhancement. The advantage of SDNs over existing techniques such as bicubic interpolation is their ability to cope with motion and noise effects. SDNs are used to generate two high resolution images, one based on frames taken from the left view of the subject, and one based on the frames from the right. This new high resolution stereo pair is then processed by a disparity map generator. The disparity map generated is compared to two other disparity maps generated from the same scene. The first is a map generated from an original high resolution stereo pair and the second is a map generated using a stereo pair which has been enhanced using bicubic interpolation. The maps generated using the SDN enhanced pairs match more closely the target maps. The addition of extra noise into the input images is less problematic for the SDN system which is still able to out perform bicubic interpolation.

• Lewis E. Hibell
• Honghai Liu
• David J. Brown

• Genetic Evolution
• Image Enhancement
• Neuron Networks
• Stereo Vision

[1] L. E. Hibell, H. Liu, and D. J. Brown, "Combining multi-frame images
for enhancement using self-delaying dynamic networks," in IEEE World
Congress on Computational Intelligence, Canada, 2006.
[2] "Eye movements," 2004. (Online). Available:
www.cis.rit.edu/vpl/eye movements.html
[3] J. Cooper, "All about strabismus," 2006. [Online]. Available:
http://www.strabismus.org/ all about strabismus.html
[4] J. P. C. Southall, Physiological Optics. New York, NY: Dover, 1937.
[5] R. D. Henkel, "Fast stereovision by coherence detection," in Computer
Analysis of Images and Patterns, 1997, pp. 297-304.
[6] ÔÇöÔÇö, "A simple and fast neural network approach to stereo vision," in
NIPS-97 in Denver. Cambridge: MIT Press, 1997, pp. 808-814.
[7] R. Tsai and T. Huang, Multiframe Image Restoration and Registration,
R. Tsai and T. Huang, Eds. JAI Press, 1984, vol. 1.
[8] M. Irani and S. Peleg, "Super resolution from image sequences," 10th
ICPR, vol. 2, pp. 115-120, 1990.
[9] ÔÇöÔÇö, "Improving resolution by image registration," CVGIP: Graphical
Models and Image Processing, vol. 53, no. 3, pp. 231-239, May 1991.
[10] R. Schultz and R. Stevenson, "Extraction of high resolution frames from
video sequences," IEEE Transactions on Image Processing, vol. 5, no. 6,
pp. 996-1011, 1996.
[11] S. Kim, N. Bose, and H. Valenzuela, "Recursive reconstruction of
high resolution image from noisy undersampled multiframes," IEEE
Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 6,
1990.
[12] S. Borman and R. Stevenson, "Spatial resolution enhancement of low
resolution image sequences. A comprehensive review with directions for
future research," University of Notre Dame, Tech. Rep., 1998.
[13] E. Salari and S. Zhang, "Integrated recurrent neural network for image
resolution enhancement from multiple image frames," in IEE Proceedings
on Vision, Image and Signal Processing, vol. 150, no. 5, 2003.
[14] C. Miravet and F. B. Rodr'─▒guez, "A hybrid MLP-PNN architecture for
fast image superresolution." in ICANN, 2003, pp. 417-424.
[15] ÔÇöÔÇö, "Accurate and robust image superresolution by neural processing
of local image representations." in ICANN (1), 2005, pp. 499-505.
[16] M. Negnevitsky, Artificial Intelligence, 2nd ed. Harlow, UK: Pearson,
2005.
[17] A. S. Ogale and Y. Aloimonos, "A roadmap to the integration of early
visual modules," International Journal of Computer Vision: Special Issue
Of Early Cognitive Vision, In Press.
[18] "Microsoft Research, Cambridge. i2i: 3D visual
communication," March 2006. (Online). Available:
http://research.microsoft.com/vision/cambridge/i2i/default.htm