Research Projects NTT-MIT Research Collaboration: a partnership in the future of communication and computation

Interactive Sculpting of Virtual 3D Materials


Start date: 09/98

Julie Dorsey and Leonard McMillan

Tsutomu Horikoshi

Project summary

In this project, we are developing a new 3D modeling system based on the metaphor of interacting with real materials.

Project description

Our efforts are motiviated by images like the one shown here of a Venetian facade with complex weathered surfaces

The past thirty years have seen significant progress in the field of computer graphics, particularly in the area of rendering. However, the creation of realistic models is nearly as tedious today as it was 30 years ago, and many types of complex materials simply cannot be represented with today's graphics systems. To address these problems, we are developing a new 3D modeling system based on the metaphor of sculpting real materials. We believe that by combining haptic output devices, stereoscopic displays, physically-based surface models, user-guided simulations, and newly developed surface representations it will be possible to approach the feel, naturalness, and flexibility of interacting with materials such as stone, wood, metal, and plastic. This research also serves as a platform for studying the next generation of user interfaces, sensory fusion, and material representations. This work should find application in a variety of fields ranging from computer-aided design to entertainment.

System overview

The following diagram depicts a simplified view of the sculpting pipeline. The system takes as input scanned meshes or CAD models.

An initialization routine parses the input file and builds a volumetric surface representation of the object. This new data structure combines benefits of traditional surface- and volume-based object representations and provides a basis for for realistic physical simulations. Only the visible surfaces are sent to the graphic and haptic rendering routines to eliminate unnecessary overhead.

The haptic interface not only allows a user to "feel" the model surfaces, but also allows volume interaction. Sculpting commands are sent to an array of physical simulation routines, which modify the volumetric representation as necessary. Fracture simulation is performed by a series of routines that can remove volumetric regions to simulate the crumbling of rigid materials like brick or plaster. Finite element modeling can be used to simulate interaction with soft or maleable materials such as clay or wax.

Each material in the digital model can be associated with a unique set of physical properties. The graphic rendering process can simulate material color, texture, and specularity. The haptic rendering process can simulate material properties such as softness, roughness, and static and dynamic friction.

Demos, movies and other examples

The images shown here are screen shots of the realtime graphic interface. The final sculpted sphere model contains 45,500 tetrahedra and 13,910 visible polygons.

Click for a larger view

In Modeling and Rendering of Weathered Stone, we used volumetric surfaces to facilitate erosion simulation. Future work will unify this type of physical simulation with an intuitive realtime user interface.

Fresh Marble
Click for a larger view
Erosion and Iron Impurities
Click for a larger view

The principal investigators

Presentations and posters


Agarwala, Aseem. Volumetric Surface Sculpting. Master of Engineering Thesis, Massachusetts Institute of Technology, 1999.

Dorsey, J., Edelman, A., Jensen, H., Legakis, J., and Pedersen. Modeling and Rendering of Weathered Stone. In Computer Graphics Proceedings, Annual Conference Series, pages 225-234. ACM SIGGRAPH, August 1999.

Dorsey, J. and Hanrahan, P. Digital Materials and Virtual Weathering. To appear in Scientific American, February 2000.

Proposals and progress reports


NTT Bi-Annual Progress Report, July to December 1998:

NTT Bi-Annual Progress Report, January to June 1999:

NTT Bi-Annual Progress Report, July to December 1999:

NTT Bi-Annual Progress Report, January to June 2000:

For more information