Patching onto the Web:
An Emerging CL-HTTP Success Story

Byron Davies
Motorola Logic and Analog Technology Group
R15546@email.sps.mot.com (602) 655-2911 Mesa, Arizona
Victoria Bryan Davies
Industrial Lisp and Magic
vdavies@pobox.com (602) 759-8228 Phoenix, Arizona

Introduction


This paper reports on the successful application of dynamic object technology in rapidly putting a large factory database onto the web. MIT's Common Lisp web server software, CL-HTTP [Mallery 1994], was used to convert a standalone Lisp application into a nearly instant web server. Besides the CL-HTTP software itself, the conversion required a total of about 100 lines of new code, including one new class, two new explicitly exported URLs, and the modification of 5 existing methods and functions. The conversion did not require either recompiling or rebuilding the existing application. In effect, thanks to dynamic object technology, the conversion from single-user application to web server amounted to applying a patch.

The web application is part of an ongoing project to develop automated tools for scheduling wafer fabs (factories in which semiconductor wafers are fabricated) in Motorola's Semiconductor Products Sector. The project is grounded in the belief that manufacturing managers and associates need an explicit, accurate, detailed, and timely model of the factory and its past, present, and future behavior. When completed, the resulting tool set will have six major components: fab model, fab model extraction software, fab browser, historical data browser, scheduling system, and execution system.

The Fab Browser


The Fab Browser is a relatively mature application. Building on [Davies 91], the Fab Browser is a textual outline browser, implemented in the style of Dired within MCL's Fred version of Emacs. In the browser, an object is displayed as lines of text, each representing a component or attribute of the project. Some of the text lines represent other objects, which may in turn be expanded into multiline displays. Though very general, the Fab Browser has been highly customized for displaying wafer fab data, with specialized views of the fab model, scheduling data, and historical data. Using expansion methods that optionally depend on the customer for the display, the Fab Browser is further customized to the needs of each fab, and even to the needs of individuals within the fab,

The basic implementation of the Fab Browser is straightforward. The top level is a function which initializes the Fred buffer with lines of text representing major structural components of the wafer fab and operational data. Each line is inserted by a function called insert-line, which inserts a line of text in the buffer and associates an object with the line of text. When such a line is double-clicked, the Fab Browser invokes the expand method for the object associated with the line. Each object class in the Orchestra factory representation has an expand method, which inserts multiple lines describing the object into the buffer. Hence, information is displayed in the Fab Browser by repeated handoffs between the expand methods for objects and the insert-line function.

Besides the basic expand capability, the Fab Browser offers other ways of extracting information about a selected object, mechanisms for contracting previous expansions, and a mechanisms for creating a hardcopy report of data that has been expanded. The Fab Browser also offers mechanisms for text filtering of potentially large expansions.

Users find the Fab Browser powerful and easy to use. They like being able to get a high-level summary of fab data and then expand the summary to arbitrary levels of detail using only the mouse. They like the speed of the browser: expansions happen very quickly because tens of megabytes of manufacturing data are cached, in a network of CLOS objects, within the browser application.

Because of the large amount of data cached within it, the browser application runs in 70 to 100+ megabytes of memory, depending on the amount of historical or projected data. This was a significant roadblock to widespread use of the browser, both because of the amount of RAM needed to run such applications efficiently and because of problems of keeping large applications up to date on multiple desktops, some connected by modem lines. The situation demanded a client-server solution, but resources were not available to create either a customized server or a customized client.

Enter the Web


Late in 1995, we began investigating CL-HTTP as a possible means of providing web access to Orchestra data. For a time, because of problems with the MCL implementation of IP/TCP networking, we could not get CL-HTTP to operate properly in our large-address-space MCL applications. Once John Mallery reimplemented MCL's MacTCP interface in March 1996, we were able to make progress on converting Orchestra to the web. The goals for the web implementation were to (1) provide access to the same data, (2) preserve the look and feel, which had already demonstrated its effectiveness, (3) preserve performance, and (4) eliminate dependence on a heavyweight, expensive user platform in favor of lightweight, inexpensive client software.

The result: once the MacTCP problems were resolved, we ported the outline browser to CL-HTTP in about one person-week. The goals for the web implementation were achieved for the most part. Fab data is now accessible through a web browser to anyone in the wafer fab with a computer, whether Mac, PC, or Unix. The look and feel of the web interface are somewhat different from the original single-user version. The main difference is due to the inability to do in-place outline expansion. The single-user application uses multiple levels of nested indentation (like the twist-down behavior of the Mac Finder under System 7). The web version brings up a new window for each expansion (like the Finder under System 6). Per character displayed, the web interface is slower than the single-user interface, but feels like typical web access. Both network delay and CL-HTTP overhead appear to be factors.

Thanks to CL-HTTP, the implementation was almost a trivial undertaking, once we understood what we needed to do. After loading the CL-HTTP software into the application, the port from single-user to server required about 100 lines of new code, with significant changes to only five functions in the original implementation. As described, the original was implemented as recursive hand-offs between the expand methods, which know how to format the complete display of an object, and the insert-line function, which formats each line of the expansion. The main changes were to put the CL-HTTP server in charge of issuing the expand methods, through an appropriate "search URL," and to force insert-line to emit HTML to the appropriate stream when serving the web. The HTML sent by insert-line includes an URL which can be used to expand the object associated with the line.

To serve multiple users, the server generates, for each user, an instance of a new class, a connection object. The connection object maintains an index of the URLs to which the user has been given access and to the objects referenced by the URLs. URLs encode both the connection ID and an integer index to Lisp objects that have been made visible to the user by insert-line.

Although our web implementation of the Fab Browser is still in prototype form, it promises the client/server solution that we long needed. CL-HTTP and MCL have provided a strong foundation for rapid progress.

First, CL-HTTP provides - in our usual Common Lisp environment - the advantages of an embedded web server: (1) direct access to data, without the need for an intervening CGI-style interface, (2) access to existing Lisp software, including format, (3) easy compatibility between the single-user, Fred-based interface and the web interface, and (4) higher performance than a more loosely coupled solution.

Second, MCL (and Common Lisp in general) provides the advantages of dynamic object technology. Because of this, we were able to create a web server by patching the single-user application. We did not need to recompile or even rebuild the original application. We simply loaded the CL-HTTP software and the few needed changes into the existing application - and we were on the web. Dynamic object technology provided the most tangible benefit in modifying the properties of the expand methods to output to the web. To change all of the expand methods (for dozens of different classes), we needed only to change the default :before, :after, and :around methods for expand.

Our near-term plans include capturing additional aspects of the Fab Browser, such as filtering of output through text-matching and the transmission of graphical output (e.g., column and bar charts) across the web. The next step will be to recapture client-mediated, in-place, indented expansion, probably with Java.

Conclusion


Dynamic object technology wins again, with CL-HTTP as the vehicle. CL-HTTP, layered on top of an existing object-oriented Lisp application, has made it possible - even easy - to rapidly retarget a valuable data browsing application to the web.

Acknowledgments


The authors are grateful to John Mallery for CL-HTTP, Digitool for MCL, Steve Mitchell for ongoing discussions, and Bill Chapman at Motorola for providing an environment for innovation.

References


[Davies 1991] Byron Davies. Process Specifications and Process Knowledge in Semiconductor Manufacturing. Stanford University EE Ph.D. Thesis, 1991.

[Mallery 1994] John C. Mallery. "A Common LISP Hypermedia Server". In Proceedings of The First International Conference on the World-Wide Web, Geneva: CERN, May 25, 1994.