% -*- Mode:Text -*-

\chapter{\STEP\ machine and controller}\sectlabel{machine}

\section{Overview}\sectlabel{machine-overview}

A complete \cam\ system consists of a three-dimensional array of \cam\
\tdf{boxes} driven by a \tdf{host} via a tree-shaped network
called the \tdf{\bus}, as sketched in \fig{cam-system} (where only two
dimensions of the array are shown).

\figonecol{cam-system}{\psfig{file=fig/cam-system.ps}}{A \cam\ system
consists of an array of \cam\ boxes (\emph{\cam\ machine}) driven
by a \sparcstation\ (\emph{host}). The \emph{\cam\ controller} interfaces the
host's Sbus with the machine's \bus.}

\noindent In the current implementation, the host is a \sun\ \sparcstation,
which features a high-performance external bus called the \sbus.  The \bus\
is driven at the root by the \cam\ \tdf{\ctrler}---a small custom board
plugged into one of the \sbus\ slots. From this board, a short cable---the
\bus\ \tdf{root cable}---leads directly to one of the boxes, located on a
corner of the array. From this \tdf{corner box} the \bus\ branches out
through the rest of the array via \bus\ \tdf{branch cables}.  We shall call
\cam\ \tdf{machine} the interconnected array of \cam\ boxes, when viewed as
distinct from the host and the \ctrler.

\section{Hardware hierarchy}\sectlabel{hierarchy}

Each \tdf{box} of the \cam\ machine array realizes, by means of eight identical
printed-circuit boards called \tdf{modules}, a functional subunit of the
\cam\ architecture---typically, a \emph{block} of eight \emph{sectors} (see
beginning of \chapt{box}; other ways to configure the contents
of a box as a functional subunit are possible).
	Besides, the box provides mechanical and electrical support
(backplane, enclosure, power supply and cooling) to the modules themselves,
and may house additional support circuitry. (In a multi-box machine, some of
these support functions need not be provided on a per-box basis; for
instance, it may be more practical to provide a centralized power supply, or
to house several boxes in one enclosure.)
	Communication between a \cam\ unit and the outside world (including
the \ctrler\ located in the host and optional \io\ devices such as \vga\
monitors, video cameras, and video monitors; see \sect{io-streams}) goes
through connectors located on the boxes' \tdf{backpanels} (\sect{box-overview}).

In turn, each module contains, besides conventional \dram\ and \sram\ chips,
sixteen copies of the \tdf{\CHIP}---a custom {\sc vlsi} chip that one may
view as embodying the \cam\ architecture. In fact, each \chip\ has complete
control over one \dram\ chip's data and control lines; and, collectively, the
sixteen \chip{}s have complete control over the \sram\ chips' data and
control lines---as well as over transactions over the \bus.

\medskip

Chapters \ref{sect.machine}--\ref{sect.chip} describe the \cam\
system at four levels of the design hierarchy, namely,
 \begin{enumerate}
	\item the \cam\ machine with its controller (this chapter),
	\item the \cam\ box (\chapt{box}),
	\item the \cam\ module (\chapt{module}), and
	\item the \chip\ (\chapt{chip}).
 \end{enumerate}
	For each level, formal interface specifications
(pinout and signal description) with the level above are given.

\medskip

The \chip\ (Level 4) is seen by the programmer as a set of registers
accessible via the \bus. \memo{Complete and update} Chapters \ref{sect.} and
following expand \chapt{chip} by giving a complete programmer's model of the
\chip: it systematically describes structure and functions of these
registers, giving, whenever appropriate, a functional model of the hardware
controlled by the registers themselves.

\medskip

One may view the host \sparcstation\ as representing Level 0 of
the hierarchy.  Its relevant hardware and systems-software aspects are
documented by \sun\ Microsystems. \Cam-specific software is
documented in the \SR\ and the \UG.


\section{The \cam\ controller}\sectlabel{controller}

The \cam\ controller is a small board with, on one edge, a connector
that plugs into one of the \sbus\ slots. At the opposite edge of the
card, a connector that shows through the rear of the \sparcstation\
represents the root of the \bus.

\medskip

The main \emph{raison d'\^etre} of this piece of hardware is to provide
an interface between the two buses, and to do so in such a way as to
lessen the requirements for real-time response from the host.

While the \sbus\ and the \bus\ are both synchronous, and at present
have roughly comparable bandwidths, they are timed by independent
clocks which are typically running at different rates.  Moreover, the
number and nature of the control and data lines, the protocols, and
the signals' electrical specifications are quite different in the two
buses.

	The \cam\ controller requires bus-master as well as bus-slave
capabilities.\footnote
	{On the \sparcstation-1, one of the three \sbus\ slots only
offers bus-slave capabilities, and cannot be used for this purpose.
At any rate, this slot is typically dedicated to the graphics interface.}
	In fact, in order to decouple \cam\ from the poor and rather
impredictable real-time response of standard {\sc unix}, the
controller's normal operating mode is to act as a bus master and
interpret according to the \cam\ machine's timing requirements a list
of commands (\tdf{step list}) directly off the \sparcstation's memory,
without going through the {\sc cpu}. These commands would previously have
been written to memory by the {\sc cpu} at its own leisure.

The controller is addressed as a slave only at initialization time or
to recover from runaway situations.
 
Clocking for the \cam\ machine is generated by the controller; a clock line
in the \bus\ root cable carries a clock reference signal from the
controller to the machine. Thus, the interface between circuitry
clocked by the \sbus\ and circuitry clocked by the \bus\ lies within
the controller, which has provisions for arbitrating between
the two clocks. See \sect{clock-sync} for details of clock
distribution and synchronization.  

\medskip

The principles of operation of the \cam\ controller (i.e., how it
interprets a script consisting of \steplist\ commands, initiating and
supervising any \sbus\ or \bus\ transactions that may be called for)
are explained in the \SR.


\subsection{\sbus\ signals}

The \cam\ controller interfaces with the \sbus\ by means of a standard
\sbus\ connector. Signal description and pinout can be found in {\em
The \sbus\ Specification} (specifically, Tables 2-1 and 3-8 of that
manual). The following \sbus\ signals are not used by the controller:
\memo{Any unused signals?}


\subsection{\bus\ signals}\sectlabel{controller-bus-signals}

The \bus\ runs from the \cam\ controller to the corner box
via a \tdf{\bus\ root cable}, and thence outwards from box to box (cf.\
\fig{cam-system} and \fig{2x2x2}) on \tdf{\bus\ branch cables}.
Each \bus\ cable segment consists of a bundle of tranmission lines
(\cf{interbox}) carrying the following signals
 \begin{equation}\eqlabel{machine-bus}
	\input +bus-signals
 \end{equation}

Within a box, the \bus\ is further distributed by
the backplane to the eight modules according to the pinout of table
\eq{module-bus-local}, and from there to the sixteen chips of each
module according to the pinout of table \eq{chip-bus}.

The various \bus\ lines are discussed in detail in
\chapt{bus}.

\medskip

The inner end of a \bus\ branch cable (the one closer to the tree root) plugs
into one of a box's three \tsy{\bus\ branch} ports (that's where a new
\bus\ branch is grafted onto the tree); the outer end plugs into a box's
\tsy{\bus\ root} port (that's where the box faces towards the tree
root).  According to this terminology, the port through which the \cam\
controller drives the \bus\ root cable is a \tsy{\bus\ branch} port.  Its
pinout is the following
 \input mod-pin-types
 \begin{pinout}{controller-tbus}{...}{\tsy{\bus\ branch} port}
	\input bus-branch
 \end{pinout}

In the \bus\ root cable (\sect{controller-bus-signals}), the \tsy{Scan*}
line carries master-clock (\sect{}) rather than scan-in-progress
(\sect{}) information.


\subsection{\bus\ Sync}\sectlabel{bus-sync}

The \bus\ \tsy{Sync} line carries, encoded as 5-bit bursts on a
background of all zeros, messages
used for achieving synchronization between sweeps and scans of the
\cam\ machine (\sect{}) and, respectively, horizontal and vertical
scans of a \crt\ monitor (or similar external equipment). The possible
messages are tabulated in \fig{sync-messages}.
 \figonecol{sync-messages}{
	\begin{tabular}{|c@{$\;$}c@{$\;$}c@{$\;$}c|l|l|}\hline
 	\multicolumn{4}{|c|}{code} & \multicolumn{1}{c|}{message} 
& \multicolumn{1}{c|}{recipient}\\\hline\hline
 	1&00&1&0 & \sym HMON & {\sc crt} monitor\\
 	1&01&0&0 & \sym VMON & {\sc crt} monitor\\\hline
	1&10&0&0 & \sym HCAM & \cam\ machine\\
 	1&11&1&0 & \sym VCAM & \cam\ machine\\\hline \end{tabular}
 }{The above codes, transmitted on a background of all zeros, signal
the appropriate times for a \crt\ monitor to start horizontal and
vertical scans, and for the \cam\ machine to start sweeps and scans.
Each message consist of a start bit (`1'), a two-bit field carrying
the intelligence of the message, a parity bit, and a stop bit (`0').}
 
The above messages are generated on the \cam\ controller as follows.
Two ganged programmable counters, namely, the \tdf{horizontal counter}
and the \tdf{vertical counter} (the latter is clocked by the rising
edge of the former), produce the waveforms of \fig{sync-counters}. For
each counter, ``mark'' and ``space'' lengths are independently
programmable; as a consequence, one can independently set the period
and the duty-cycle. \tsy{HCAM}, \tsy{VCAM}, \tsy{HMON}, and {\sym
VMON} are issued at the rising and falling edges of these waveforms,
as indicated in \fig{sync-counters}.

\figonecol{sync-counters}{\psfig{file=fig/sync.ps}}{Sync messages
are issued on the rising and falling edges of the waveforms generated
by the horizontal and vertical counters.}

\noindent The \tsy{HCAM} message that would occur concurrently with a
\tsy{VCAM} is superseded by the latter; the \chip\ will infer an
\tsy{HCAM} from the presence of a \tsy{VCAM}.

While the coincidence of \tsy{VCAM} with an \tsy{HCAM} is a logical
necessity, the fact that also \tsy{VMON} exactly coincides with an
\tsy{HCAM} (and supersedes the latter) is a nonessential feature of
the present version of the \cam\ controller.\footnote
	{Note that the \cam\ controller could be replaced by,
say, a video camera as a source of sync messages, in order to
have video-on-video.}
	Since the \chip\ does not respond to \tsy{HMON} and
\tsy{VMON} messages,\footnote
	{Each \chip\ monitors the integrity of all sync messages, by
checking the parity bit, but otherwise ignores messages directed
to the {\sc crt} monitor.}
	a \cam\ scan that extends past \tsy{VMON} and is synchronized
to \tsy{HCAM} (\cf{RMR}) will be suspended for one horizontal counter
period when \tsy{VMON} occurs, since the latter masks out the former.


\section{The \cam\ machine}\sectlabel{box-array}

The \cam\ machine is an assembly of boxes physically
arranged and interconnected as a three-dimensional array. The three
dimensions are labeled $x$, $y$, and $z$, and along
each dimension the elements of the box array are sequentially numbered
from 0 upwards.  Thus, the corner box, located
at the origin of the box array, has coordinates $\langle
0,0,0\rangle$, and in a system of 2\by4\by8 boxes the box on the
opposite corner would have coordinates $\langle 1,3,7\rangle$.

Along any of the three dimensions, the array need not extend for more
than one box; in fact, a \cam\ machine may consist of just a single box.

\medskip

Boxes communicate with one another by means of signals (\emph{glue} and
\emph{\bus} signals) which travel between boxes on transmission lines
(\sect{interbox}) and turn the collection of boxes into a tightly-knit whole.

The \tdf{glue signals} exchange site data between adjacent sectors in real
time, according to the current data-transport specifications for each
layer. Six glue ports are provided---each port corresponding to one of the
six ``faces'' of the box, i.e., the six spatial directions in which a box can
interconnect with other boxes, denoted by \tsy{XM} (``x minus''), \tsy{XP}
(``x plus''), and similarly \tsy{YM}, \tsy{YP}, and \tsy{ZM}, \tsy{ZP}.
Besides the data lines, each glue port includes two lines that exchange
clock-phase information between adjacent boxes (\sect{clock-sync}).

The \tdf{\bus\ signals} ultimately connect the boxes to the host. Each box
has one root port, facing inwards (that is, towards the host),
and three branch ports, facing outwards in the positive $x$, $y$,
and $z$ directions. The root port is always used: the one on the
corner box connects directly to the \bus\ root cable, the others, to branch
cables that come from intermediate boxes. On the other hand, a branch
port is used only if the box is the root of a subtree. Thus, for
example, in a machine of size 2\by2\by2 boxes (see \fig{2x2x2}), the
corner box $\langle 0,0,0\rangle$ uses all three branch ports, box
$\langle 0,0,1\rangle$ uses only $x$ and $y$, boxes $\langle
0,1,0\rangle$ and $\langle 0,1,1\rangle$ use only $x$, and the
remaining four boxes, with addresses of the form $\langle
1,y,z\rangle$, use none, since they are at the leaves of the tree.

 \figonecol{2x2x2}{%
 \setlength{\unitlength}{0.75pt}%
 \begin{picture}(224,240)(-72,-152)
    \thicklines
\put(0,0){\circle*{8}}\put(8,16){\makebox(0,0){$\langle0,0,0\rangle$}}
    \put(0,0){\vector(4,1){32}}\put(32,-4){\makebox(0,0){$x$}}
    \put(0,0){\line(4,1){128}}
    \put(0,0){\vector(0,-1){32}}\put(12,-32){\makebox(0,0){$y$}}
    \put(0,0){\line(0,-1){128}}
    \put(0,0){\vector(-3,2){16}}\put(-16,-2){\makebox(0,0){$z$}}
    \put(0,0){\line(-3,2){48}}
    \put(-48,32){\circle*{8}}\put(-48,48){\makebox(0,0){$\langle0,0,1\rangle$}}
    \put(-48,32){\vector(4,1){32}}\put(-16,28){\makebox(0,0){$x$}}
    \put(-48,32){\line(4,1){128}}
    \put(-48,32){\vector(0,-1){32}}\put(-36,0){\makebox(0,0){$y$}}
    \put(-48,32){\line(0,-1){128}}

    \put(0,-128){\circle*{8}}\put(0,-144){\makebox(0,0){$\langle0,1,0\rangle$}}
    \put(0,-128){\vector(4,1){32}}\put(32,-132){\makebox(0,0){$x$}}
    \put(0,-128){\line(4,1){128}}

    \put(-48,-96){\circle*{8}}\put(-48,-112){\makebox(0,0){$\langle0,1,1\rangle$}}
    \put(-48,-96){\vector(4,1){32}}\put(-16,-100){\makebox(0,0){$x$}}
    \put(-48,-96){\line(4,1){128}}

    \put(128,32){\circle*{8}}\put(128,48){\makebox(0,0){$\langle1,0,0\rangle$}}
    \put(128,-96){\circle*{8}}\put(128,-112){\makebox(0,0){$\langle1,1,0\rangle$}}
    \put(80,64){\circle*{8}}\put(80,80){\makebox(0,0){$\langle1,0,1\rangle$}}
    \put(80,-64){\circle*{8}}\put(80,-80){\makebox(0,0){$\langle1,1,1\rangle$}}
 \end{picture}%
}{\bus\ port usage in a 2\by2\by2 box machine. The dots represent boxes;
the arcs, \bus\ hops. The $x$, $y$, $z$ labels indicate to which
branch port of a box the inner end of a hop is connected.}


Through the \tsy{Flywheel} ports, external
equipment can selectively extract any of the data involved in the
simulation and optionally replace them with arbitrary new data; these
lines are thus used for something like ``local open-brain surgery''
(it is unlikely that even the most advanced user will connect external
equipment to more than one box).

Through the \tsy{Display} port, one can send to a color monitor and
view in real time the contents of a selected \cam\ sector.


\section{Clock distribution and synchronization}\sectlabel{clock-sync}

The current \cam\ implementation consists entirely of {\em
synchronous} digital circuitry.  Each box has its own \emph{local
master clock}, which drives the clock distribution tree for the entire
box.  The issue discussed here is the mutual synchronization of the
local masters within prescribed tolerances.

\medskip

Though the \cam\ array may extend arbitrarily in all three dimensions,
its elements (the boxes, in the present context) are only {\em
locally} interconnected. Thus, all that matters is that the clock
phase difference between a box and each of its six neighbors (the ones
with which the box exchanges \emph{glue} data) be within tolerances.
There will be no harm if, in moving from one box to the next along a given
direction, these differences add up---leading perhaps to a cumulative
phase difference of several clock cycles between widely separated
boxes.

In \cam, local master clocks in adjacent boxes keep within
tolerances with respect to one another by exchanging clock-phase
information. This provides the feedback by which the phase of each
individual clock is dynamically adjusted according to a {\em
phase-locked loop} scheme: i.e., this phase is continually made
to track (according to a suitable response function)
the mean of the phases of its six neighbors.


\section{Interbox communication}\sectlabel{interbox}

Within a box, signals run short distances; they have a stable and
uniform voltage reference provided by the modules' and the
motherboard's ground plane; they are shielded from one another by the
close proximity of the ground plane, and from external fields by the
box's enclosure. In this environment, a signal line, even though it
may extend and branch out over several inches, can still be usefully
treated as a voltage node.

Communication between boxes takes place in a rather different environment.
The physical length of the lines from driver to receiver may run up to
a few feet; glue lines run in parallel bundles of over a hundred
lines each;\footnote
	{Sixteen layers times eight strands gives 128 lines, to which
one must add two clock-phase feedback lines.}
	each box is the source or destination of about a thousand
30-Mbit/sec signals; and boxes will typically be tied together by a
mesh of ground bars or ground strips several inches in length rather
than by a continuous copper plane. For these reasons, it is desirable
to have interbox signals travel on well-characterized {\em
transmission lines}, which feature low dispersion and (when properly
terminated) low reflections, good common-mode noise rejection, and
good decoupling between adjacent signals.

\medskip

In the current implementation, the transmission lines are realized as
\emph{unbalanced twisted pairs}.  Accordingly, for each signal handled in
this we provide two separate pins---a signal pin and a ground pin
(\cf{mod-pin-types})---on the connectors at either end of a transmission
line.



