BACK

Recovery of 3D structure and camera motion in videos

from a parametric estimation point of view

My PhD work has mainly adressed the problem of Structure and Motion recovery. My applicative goal was to obtain 3D models and a path for the camera accurate enough to add virtual 3D elements (in the following examples, some penguins) in a realistic way.

To attain such accuracy over long images sequences (30 to 100 frames), I have built complex models for both the structure and the motion that capture most of the available information issued from the tracking of geometric features like points, line segments or rectangles.

The reconstruction is computed in an optimal way, in the sense of Bayesian estimation, and the introduced models reduce the number of parameters to estimate to match the degrees of freedom of the system.

So finally, I can proudly show here the augmentation results at the end of the complete procedure, for various video scenes:

The augmented reality sequences above have been produced using Chebyshev polynomials as the motion model. More recently, I have investigated more sophisticated models to cope with noisy and composite motions. Here is an example with the best model so far, a piecewise Chebyshev polynomial with HMM-like state transitions:

Acknowledgements
This work has been initiated during my PhD in the MIRAGES Team at Inria Rocquencourt, and completed in the SHAPE Lab at Brown University.

I would like to thank the VISICS lab for the Begijnhof sequence, Alias|Wavefront and SGI for the stainX sequence, the MOVI team at Inria Rhône-Alpes for the House sequence, the ISA team at Inria Lorraine for the Loria sequence, Science - D - Vision for the Smint sequence and Mikros Image for the Panola sequence.

Further details
Modélisation et estimation du mouvement et de la structure dans une séquence d'images à partir d'indices géométriques épars, avec application à la post-production audio-visuelle, P. L. Bazin, PhD dissertation, 2001 (in French). Table of Contents and Chapters.

Integration of Geometric Elements, Euclidean Relations and Motion Curves for Parametric Shape and Motion Estimation - P.L. Bazin and J.M. Vezien - submited to IEEE Transactions on Pattern Analysis and Machine Intelligence. (postscript).

Motion curves for parametric shape and motion estimation, P. L. Bazin and J. M. Vezien, Proceedings of ECCV 2002. (postscript).

Shape and Motion Estimation from Geometry and Motion Modeling, P. L. Bazin and J. M. Vezien, IEICE Transactions, special issue on Machine Vision Applications, 2001. (postscript).

BACK