Fundus photographic and angiographic data will be superimposed on a real-time slit-lamp biomicroscopic image. The algorithm will initially be developed for macular applications. Specifically, images of the posterior fundus will be acquired and montaged into a single data set. Available angiographic data will be registered (off line, i.e. not in real time) with the montaged data set prior to biomicroscopic examination. Welch and coworkers (Markow et al 1993; Barrett et al 1994, 1996) and Becker et al. (1995) describe algorithms for real-time retinal tracking for automated laser therapy, however, the tracking algorithms require multiple templates to follow retinal vessels at several fundus locations, and therefore requires a large illumination area; tracked vessels must be visible at all times. This requirement is not consistent with our goal to allow for narrow beam illumination, and is more suitable for a fundus camera with monocular, wide angle viewing. Further, their tracking algorithms limit the search to a small area surrounding the previously tracked position, and is not tolerant of large changes in fundus position as might be encountered during a slit-lamp fundus, biomicroscopic examination.
We have implemented a modified, bidirectional Hausdorff-distance-based (Huttenlocher et al. 1993; Rucklidge, 1995) approach for image montaging. Multiple, partially overlapping fundus images are acquired. The images are then smoothed and edge detected (Canny, 1986).
To match two images together, the maximum Hausdorff fraction over
translation, rotation, and scale is found. Given that the fundus
images only partially overlap, if all points in both images are used
in the Hausdorff fraction calculation, then the resulting fraction
will be very low. In fact, it is likely that a random transformation
where the fundus images have more overlap will have a larger fraction
if all points are considered.
Therefore, when computing the Hausdorff fraction 
for a given
transformation, A and B are the sets of points from the two images
that are in the overlap region of the images. All points that fall
outside the overlap region for a certain transformation are ignored.
To build the montage, the position and scale of all images must be
determined relative to a fixed coordinate system (that of the
montage). Under the current design, a ``central'' image is chosen a
priori by the user, and the coordinate system of this image is used
for the montage. Generally, the ``central'' image is easily
identifiable by the user because it contains both the fovea (which
should be approximately centered) and the optic nerve. All the other
images are then matched to the central image, yielding a
transformation that maximizes the Hausdorff fraction. All images that
match the central image with a Hausdorff fraction above some threshold
(currently 
) are considered to be correctly placed in the
montage. Any images that do not match the central image are then
matched against the other images. Although worst case, this is
matches, in practice, most if not all of the images match the
central image.
Once the positions and scales of all the images are known relative to the central image, the montage image can be built. One simple method of building the montage is to average the intensity values of the overlapping images at every pixel. The montage image, I, is computed as follows:
where 
is 1 if 
lies inside image I and 0
otherwise.
However, in general, the average intensities of the images are not equal, so edge artifacts are introduced at the image borders. Furthermore, the fundus images are often brighter near the center and lose brightness and contrast near the image borders. The vessels we are using as landmarks disappear near the image borders. Thus, in addition to the edge artifacts, simply averaging images also degrades the visibility of the vessels in the montage near image borders.
Therefore, when blending all the images to form the montage, we use a convex combination that puts greater weight on pixels closer to the center of an image:
where 
is the distance from the point 
to the nearest border
of image i if 
is inside the image, and 0 if 
is
outside the image. This blending removes many of the artifacts due to
varying contrast and intensity. Figure 3 shows the result
of montaging five distinct, partially overlapping fundus images.
Figure: Montage of five distinct,
partially overlapping fundus images as determined by the Hausdorff
distance-based methods. The right image is the result of applying the
Canny edge detector on the left image.