Research Projects NTT-MIT Research Collaboration

Variable Viewpoint Reality

9807-28

Start date: 07/98

Paul Viola and Eric Grimson
MIT AI Lab

Ken'ichiro Ishii
NTT

Project summary


This work synthesizes information from many different cameras to produce arbitrary viewpoints of sporting or entertainment events.

Project description


 

In the foreseeable future, sporting events will be recorded in super high fidelity from hundreds or even thousands of cameras. Currently the nature of television broadcasting demands that only a single viewpoint be shown, at any particular time. This viewpoint is necessarily a compromise and is typically designed to displease the fewest number of viewers.

In this project we are creating a new viewing paradigm which will take advantage of recent and emerging methods in computer vision, virtual reality and computer graphics technology, together with the computational capabilities likely to be available on next generation machines and networks. This new paradigm will allow each viewer the ability to view the field from any arbitrary viewpoint -- from the point of view of the ball headed toward the soccer goal; or from that of the goalie defending the goal; as the quarterback dropping back to pass; or as a hitter waiting for a pitch.

In this way, the viewer can observe exactly those portions of the game which most interest him, and from the viewpoint that most interests him (e.g. some fans may want to have the best view of Michael Jordan as he sails toward the basket; others may want to see the world from his point of view).


Demos, movies and other examples


 

Our current system consists of a set of cameras, distributed around an open working area. These cameras can automatically self calibrate, so that their input can be coordinated.

 

The views from the different cameras capture a range of information about the person(s) moving in the site.

 

This information can be combined together to create a rough 3D model of the moving objects.

 

Models of humans can be fit to the reconstructed data, providing an articulated, moving model that represents the motion observed in the site.


The principal investigators


Presentations and posters


January 1999 Meeting, MIT:

March 1999 Meeting, NTT:

August 1999 Meeting, MIT:

Posters:

Publications


"Exact Voxel Occupancy with Graph Cuts", Dan Snow, Paul Viola and Ramin Zabih,
Submitted to CVPR 2000.
Not for distribution, but may be viewed by members of the NTT-MIT Collaboration.

"Roxels: Responsibility Weighted 3D Volume Reconstruction", J. S. De Bonet and P. Viola,
Proceedings of ICCV, September, 1999

Proposals and progress reports


Proposals:

NTT Bi-Annual Progress Report, July to December 1998:

NTT Bi-Annual Progress Report, January to June 1999:

NTT Bi-Annual Progress Report, July to December 1999:

For more information