Research Projects NTT-MIT Research Collaboration: a partnership in the future of communication and computation

Variable Viewpoint Reality


Start date: 07/98

Paul Viola and Eric Grimson

Ken'ichiro Ishii

Project summary

This work synthesizes information from many different cameras to produce arbitrary viewpoints of sporting or entertainment events.

Project description


In the foreseeable future, sporting events will be recorded in super high fidelity from hundreds or even thousands of cameras. Currently the nature of television broadcasting demands that only a single viewpoint be shown, at any particular time. This viewpoint is necessarily a compromise and is typically designed to displease the fewest number of viewers.

In this project we are creating a new viewing paradigm which will take advantage of recent and emerging methods in computer vision, virtual reality and computer graphics technology, together with the computational capabilities likely to be available on next generation machines and networks. This new paradigm will allow each viewer the ability to view the field from any arbitrary viewpoint -- from the point of view of the ball headed toward the soccer goal; or from that of the goalie defending the goal; as the quarterback dropping back to pass; or as a hitter waiting for a pitch.

In this way, the viewer can observe exactly those portions of the game which most interest him, and from the viewpoint that most interests him (e.g. some fans may want to have the best view of Michael Jordan as he sails toward the basket; others may want to see the world from his point of view).

Demos, movies and other examples


Our current system consists of a set of cameras, distributed around an open working area. These cameras can automatically self calibrate, so that their input can be coordinated.


The views from the different cameras capture a range of information about the person(s) moving in the site.


This information can be combined together to create a rough 3D model of the moving objects.


Models of humans can be fit to the reconstructed data, providing an articulated, moving model that represents the motion observed in the site.

The principal investigators

Presentations and posters

"Variable Viewpoint Reality", Paul Viola, Eric Grimson and Kenichiro Ishii, NTT, Musashino, Japan, January 2000.


Poster - page 1:
Poster - page 2:

January 1999 Meeting, MIT:

VVR Overview: 

VVR Activity Monitoring: 

March 1999 Meeting, NTT:

Progress Report: 

August 1999 Meeting, MIT:

Progress Report: 



"Exact Voxel Occupancy with Graph Cuts", Dan Snow, Paul Viola and Ramin Zabih, Submitted to CVPR 2000.
Not for distribution, but may be viewed by members of the NTT-MIT Collaboration.

"Roxels: Responsibility Weighted 3D Volume Reconstruction", J. S. De Bonet and P. Viola, Proceedings of ICCV, September, 1999

Proposals and progress reports


NTT Bi-Annual Progress Report, July to December 1998:

NTT Bi-Annual Progress Report, January to June 1999:

NTT Bi-Annual Progress Report, July to December 1999:

NTT Bi-Annual Progress Report, January to June 2000:

NTT Bi-Annual Progress Report, July to December 2000:

NTT Bi-Annual Progress Report, January to June 2001:

For more information