L. Zelnik-Manor and M. Irani
Abstract
Subspace based factorization methods are commonly used for a variety of applications, such as 3D reconstruction, segmentation and optical flow estimation. These are usually applied to a single video sequence. In this paper we present an analysis of the multi-sequence case and place it under a single framework with the single sequence case. In particular, we start by analyzing the characteristics of subspace based spatial and temporal segmentation. We show that in many cases objects moving with different 3D motions will be captured as a single object using multi-body (spatial) factorization approaches. Similarly, frames viewing different shapes might be grouped as displaying the same shape in the temporal factorization framework (Temporal factorization provides temporal grouping of frames by employing a subspace based approach to capture non-rigid shape changes). We analyze what causes these degeneracies and show that in the case of multiple sequences these can be made useful and provide information for both temporal synchronization of sequences and spatial matching of points across sequences.
CVPR’03 paper in pdf
Video sequences used for some of the results in the paper:
| Sequence showing two objects moving with the same rotation but different translation (0.2M avi) |
|
| Sequence showing a smiley face in two different expressions, smiling and sad, while moving with random rigid motions. (0.01M avi) |
|
| Sequence showing a person steping forward viewed by first camera. (2M avi) |
|
| Sequence showing a person steping forward viewed by second camera. (2M avi) |
|
| Synchronization result of the two sequences above. (2.7M avi) |
|
| A tiling of two video sequence each viewing a different person “dancing”. (3.4M avi) |
|
| Synchronization result of the dance sequences above. Now the people are dancing together. (2.2M avi) |