Interactive Sculpting of Virtual 3D Materials


Proposal for 1999-2000 Funding

Julie Dorsey and Leonard McMillan



Project Overview

We are developing a new 3D modeling system based on the metaphor of sculpting real materials. We believe that by combining haptic output devices, stereoscopic displays, physically-based surface models, and newly developed surface representations it will be possible to approach the feel, naturalness, and flexibility of interacting with materials such as stone, wood, metal, and plastic. This research also serves as a platform for studying the next generation of user interfaces, sensory fusion, and material representations. This work should find application in a variety of fields ranging from computer-aided design to entertainment. During fiscal year 1999-2000, we plan to address the following problems:

First, we will integrate a head-tracked stereoscopic display and a PHANToM (force feedback device) to provide registered visual and tactile feedback. We believe that this is essential to develop detailed models. The combination of tactile and stereo displays will aid in the comprehension of the surfaces being modeled, allow for direct manipulation using natural paradigms, and allow the user to interact with real-time simulations.


We plan to explore a new technology for stereo display, in particular, plasma displays. CRT's are too small too provide a compelling stereoscopic display. This is particularly true when we wish to image the three-dimensional object in front of the display surface (Most CRT-based stereo displays form the image behind the face of the CRT, as if the front of the CRT was a window). It is best to form the image in front of display if we want to allow the user to interact with it directly. Plasma displays offer many attractive properties with respect to this project. First, they are as bright as a CRT screen normally lit room. They are both large and flat, currently 45" displays that are less then 5" thick are commercially available, and 60" displays are in development. They also have perfectly registered pixels, that do not shift or jitter like CRT and projection based systems. A given pixel location turns on a specific area of the screen. Despite all these advantages, no one (to our knowledge) has built a stereo display based on plasma displays.

Layered and Composite Materials

We also plan to extend our basic volumetric data structure to support layered and composite materials, for example layers of brick and mortar with coatings of paint and stucco. This advance will allow the user to carve into models comprised of a mixture of materials. We are also interested in developing representations for materials that change shape and appearance through manipulation. Rather than simply carving, we will explore materials that deform, e.g metal foils and soft clays. We could also imagine materials whose appearance changes through manipulation, (i.e. polishing and buffing a surface).

Virtual User-Guided Simulation Environments

In addition, we plan to develop techniques to put the user in the simulation loop. This would allow a user to control parameters interactively to achieve the desired effect. For example, imagine waving a blowtorch over a piece of wood to burn away the soft areas and expose the grain.

We hope that our research will lead to the next generation 3D modeling system.

End of 1999 Goals

By the end of 1999, we hope to have a working prototype of the integrated head-tracked stereoscopic display and PHANToM. We also plan to have a new version of our basic volumetric data structure in place that will support the sculpting of assemblies of materials.