Squaring the Circles in Panoramas

L. Zelnik-Manor, G. Peters, P. Perona
ICCV’05

Abstract

Pictures taken by a rotating camera cover the viewing sphere surrounding the center of rotation. Having a set of images registered and blended on the sphere what is left to be done, in order to obtain a flat panorama, is projecting the spherical image onto a picture plane. This step is unfortunately not obvious — the surface of the sphere may not be flattened onto a page without some form of distortion. The objective of this paper is discussing the difficulties and opportunities that are connected to the projection from viewing sphere to image plane. We first explore a number of alternatives to the commonly used linear perspective projection. These are `global’ projections and do not depend on image content. We then show that multiple projections may coexist successfully in the same mosaic: these projections are chosen locally and depend on what is present in the pictures. We show that such multi-view projections can produce more compelling results than the global projections.


ICCV’2005 paper in pdf

Some more results (in pdf)

Summary

Images taken by a rotating camera can be registered and blended on the sphere using available software tools (e.g. Autostitch). To obtain a single flat image one has to project the sphere onto the plane. This projection is bound to introduce some perceptual distortions in the resulting panorama.
The two most common projections used for panoramas are perspective and geographic. Many other projections have been suggested by cartographers but all introduce some perceptual distortions.

Perspective:

perspective
  • Cannot handle wide fields of view as the projection plane size runs to infinity.
  • Distorts objects at large viewing angles.
Cubes under perspective projection at wide field of view An overlay of rectified face images taken by a rotating camera
Cubes at large viewing angle Perspective distortions

Geographic:

geographic projection
  • This is a cylindrical projection where the horizontal axis is linearly proportional to the horizontal angle and the vertical axis is linearly proportional to the vertical angle (same proportion for both axes).
  • It bends horizontal scene lines.

Mercator:

A conformal cylindrical projection. The horizontal axis is linearly proportional to the horizontal angle. The vertical axis is set to maintain conformality, i.e., local aspect proportions are maintained and small circles in the scene are projected onto circles in the panorama. This is different from the geographic projection which turns circles into ellipses.

Geographic: The shaft of the Pantheon ceiling is squished into an ellipse Mercator: The shaft circularity is maintained

Comparison of the three approaches:

Perspective: stretches too much Geographic: bends horizontal scene lines Mercator: bends horizontal scene lines

Suggested Alternative:  Multi-Plane Perspective Projection

Project the sphere onto multiple planes which are then unfolded into a single flat image. The projection onto each plane is perspective. The large discontinuities between the planes can be hidden where the world already has a discontinuity.

Geographic Our result:    Multi-Plane Perspective

Preserving Foreground Objects

Foreground (nearby, viewed at large angle) objects look distorted under perspective projection. To correct for that we allow multiple view points to reside in the same panorama. This is done by using a different projection for different objects in the scene.

Standard Perspective: Heads at wide angles look distorted Our Results:               Multi-View Perspective

The heads look good all over the panorama

For details on how we do that please refer to our paper.