Research Projects | |
Multilingual Conversational System Research9807-11 Start date: 07/98 |
James Glass and Stephanie Seneff MIT LCS Kiyoaki Aikawa NTT |
Project summary |
A Japanese conversational system is being built that allows users to interact with weather services over the telephone.
Project description |
The Spoken Language Systems group has been developing computer conversational systems for information access for over ten years. The Jupiter system for weather information is our most mature system, and has been available to the public via a toll-free telephone number for the past two years. The MIT/NTT collaboration project concerns the development of a version of the Jupiter system that communicates with the user in Japanese. It utilizes the same infrastructure as the English-based system, but recognizer vocabularies and grammar rules are provided for Japanese. The weather reports are obtained from multiple sites on the Web, and are represented in a generic "semantic frame" format. Generation rules map these meaning representations into Japanese. Ultimately, we expect to incorporate weather information available on the Web in Japanese for Japanese cities, which will augment our information sources for both the English-based and Japanese-based systems. We expect this research to help us understand how to build systems that can be ported to multiple languages efficiently and effectively. |
Demos, movies and other examples |
Speaking Style: Jupiter is intended to recognize and understand natural, conversational speech.
When you talk to Jupiter, it is actually best if you speak naturally to the system, as you would to
another person. In other words, you don't need to pause between words, overemphasize words (e.g.,
pronouncing them one syllable at a time), or speak in computerese (e.g. "weather boston" vs. "what's
the weather in boston"). The system also doesn't do as well if you shout, mumble, or speak softly. It
is best if you speak clearly, as you might to a young child. Also, if you speak softly (especially at
the end of your sentence), or pause extensively, the end point detector might cause your speech to be
clipped, which will make it harder for Jupiter to understand you.
Here are some example sentences you can speak to Jupiter. Notice that the system will remember some aspects of your previous queries.
- What cities do you know about in California?
- How about in Japan?
- What will the temperature be in Boston tomorrow?
- What about the humidity?
- Are there any flood warnings in the United States?
- Where is it sunny in the Caribbean?
- What's the wind speed in Chicago?
- How about London?
- Can you give me the forecast for Seattle?
- Will it rain tomorrow in Denver?
Here is how to call Jupiter.
The principal investigators |
Presentations and posters |
"Multilingual Conversational Interfaces: An NTT Collaboration", Stephanie Seneff, NTT, Musashino, Japan, January 2000.
Publications |
V. Zue, S. Seneff, J. Glass, J. Polifroni, C. Pao, T. Hazen, and L. Hetherington, "Jupiter: A telephone-based conversational interface for weather information," to appear in IEEE Trans. Speech and Audio Processing, Vol. 8, No. 1, January, 2000.
S. Seneff, E. Hurley, R. Lau, C. Pao, P. Schmid, and V. Zue, "Galaxy-II: A reference architecture for conversational system development," in Proc. ICSLP 1998, 931-934, Sydney, Australia.
V. Zue, S. Seneff, J. Polifroni, H. Meng and J. Glass, "Multilingual human-computer interactions: From information access to language learning," in Proc. ICSLP 1996, 2207-2210, Philadelphia, PA.
J. Glass, G. Flammia, D. Goodine, M. Phillips, J. Polifroni, S. Sakai, S. Seneff, and V. Zue, "Multilingual Spoken-language Understanding in the MIT Voyager System," Speech Communication, vol. 17, 1-18, 1995.
Proposals and progress reports |
Proposals:
NTT Bi-Annual Progress Report, July to December 1998:
NTT Bi-Annual Progress Report, January to June 1999:
NTT Bi-Annual Progress Report, July to December 1999:
NTT Bi-Annual Progress Report, January to June 2000:
NTT Bi-Annual Progress Report, July to December 2000:
NTT Bi-Annual Progress Report, January to June 2001:
For more information |