The long-term goals of this research project are to foster collaboration between MIT and NTT speech and language researchers and to develop language-independent approaches to speech understanding and generation. We will initiate this effort by developing the necessary human language technologies that will enable us to port our conversational interfaces from English to Japanese. The Jupiter weather information system will be used as the basis of this porting process. This work will involve the close collaboration with NTT researchers both in Japan and at MIT.

By the end of June, 1999, our goal is to have a hybrid Jupiter system which will allow for English input, and provide output in Japanese. This will require the ability to paraphrase English weather forecasts into Japanese, which has been the focus of Dr. Minami's work since he arrived in our group in January. This system will make use of the NTT synthesizer, which has been successfully integrated into the Galaxy architecture. The ability to cross languages serves two purposes. First, it tests the ability of our meaning representation to be language independent. Second, it will serve as a very valuable mechanism for developing a Japanese natural language (NL) capability, as Dr. Minami will be able to parse a Japanese sentence, convert it into a meaning representation, and then paraphrase the meaning in Japanese (or English). If the resulting paraphrase agrees with the original sentence, then we know the Japanese NL rules are operating properly. Thus, by starting with language generation, we set the stage for language understanding.

1. Develop an initial natural language capability using our TINA NL system. This can be aided by first translating many English Jupiter queries into Japanese, to help train and evaluation the NL system.

2. Once a Japanese NL capability is available, it will be possible to begin wizard-based data collection for weather queries from native Japanese speakers. The human wizard would listen to each user query in real time, and either speak or type to the system an abbreviated query that retains the core meaning of the original query. These data will be used initially to expand the NL coverage for Japanese Jupiter, and later to train the recognizer. We expect to data collection to be from local Japanese speakers initially.

3. Develop a speech recognition capability for the Jupiter domain. Both the existing NTT and MIT recognizers are being considered at this time. Using available telephone quality speech from Japanese speakers, or perhaps boot-strapping from English phonetic models, we can put together a Japanese recognizer for the Jupiter domain. One of the requirements will be data to train a language model, for which we propose to use the translated Jupiter queries, at least initially.

4. Once we have rudimentary speech recognition, we can begin to collect data from subjects talking to a complete system. These data are crucial for successive iterations to improve all aspects of the Japanese Jupiter system. We hope to be able to deploy such a system at NTT, in order to collect data from a wider variety of Japanese speakers than can be obtained in the Boston area.

It is our goal to have completed the first three steps by the end of December, 1999. At this point we will be able to demonstrate a prototype Japanese Jupiter system. The user should also have the ability to switch between English and Japanese Jupiter systems during a single conversation.

Once the system data-collection effort is under way in step 4, we hope to address other issues such as language independent acoustic-phonetic processing, dialogue control, and speech understanding. We can use the Japanese Jupiter corpus and data collection environment to evaluate research in these areas. In parallel with this series of four steps, we will also explore Japanese language content processing, in order to improve the quality and scope of the information the system can deliver concerning weather in Japan. For example, we will explore the feasibility of parsing weather reports available in Japanese from Web sites maintained in Japan, and incorporating the results into our weather database. The result will be improved weather information for both our English-based and Japanese-based systems.