Layered Representation for Motion Analysis

John Y. A. Wang and Edward H. Adelson

Abstract

Standard approaches to motion analysis assume that the optic flow is smooth; such techniques have trouble dealing with occlusion boundaries. The most popular solution is to allow discontinuities in the flow field, imposing the smoothness constraint in a piecewise fashion. But there is a sense in which the discontinuities in flow are artifactual, resulting from the attempt to capture the motion of multiple overlapping objects in a single flow field. Instead we can decompose the image sequence into a set of overlapping layers, where each layer's motion is described by a smooth flow field. The discontinuities in the description are then attributed to object opacities rather than to the flow itself, mirroring the structure of the scene. We have devised a set of techniques for segmenting images into coherently moving regions using affine motion analysis and clustering techniques. We are able to decompose an image into a set of layers along with information about occlusion and depth ordering. We have applied the techniques to the ``flower garden'' sequence. We can analyze the scene into four layers, and then represent the entire 30-frame sequence with a single image of each layer, along with associated motion parameters.

J. Y. A. Wang and E. H. Adelson. Layered Representation for Motion Analysis. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1993, pp. 361-366, New York, June 1993.

J. Y. A. Wang and E. H. Adelson. Layered Representation for Image Sequence Coding. Proceedings of the 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 5, pp. 221-224, Minneapolis, April 1993.

Postscript version of paper [ps(2.7M), ps.gz(900K)]. bibtex.

Last modified: Tue Mar 19 10:20:18 1996