% -*- Mode:Text -*- 

\chapter{The \bus}\label{sect.bus}

\section{\sbus\ and \bus; the \controller}

The activity of the assembly of modules that make up a \step{} machine is
governed and monitored by a general-purpose computer called the {\em
host}. The host we are currently using is a \sun\
\sparcstation---which features a high-performance external bus called
the {\em \sbus}.

All exchanges between the host and the \step{} machine are mediated by
the {\em \controller} (\fig{two-buses}), which communicates with
the host on the \sbus, and drives the modules via the {\em \bus}---a
tree-like bus having the controller at its root and the individual
modules at its leaves.


  Physically, the \controller{} consists of a
small board plugged into one of the \sbus\ slots.

 \figonegiven{two-buses}{
 \begin{picture}(224,88)(0,0)
	\thinlines
	\put(0,0){\framebox(96,80){}}
	\put(32,32){\framebox(48,32){\shortstack{\step\\controller}}}
	\multiput(208,18)(0,16){4}{\framebox(12,12){}}
	\thicklines
	\put(16,8){\line(0,1){64}}

	\put(80,48){\line(1,0){64}}
%	\put(112,48){\vector(-1,0){0}}
%	\put(128,48){\vector(1,0){0}}

	\put(144,48){\circle*{4}}

	\put(144,48){\line(2,1){32}}
%	\put(152,52){\vector(-2,-1){0}}
%	\put(168,60){\vector(2,1){0}}

	\put(144,48){\line(2,-1){32}}
%	\put(152,44){\vector(-2,1){0}}
%	\put(168,36){\vector(2,-1){0}}

	\multiput(176,32)(0,32){2}{\line(4,1){32}}
%	\multiput(184,34)(0,32){2}{\vector(-4,-1){0}}
%	\multiput(200,38)(0,32){2}{\vector(4,1){0}}

	\multiput(176,32)(0,32){2}{\line(4,-1){32}}
%	\multiput(184,30)(0,32){2}{\vector(-4,1){0}}
%	\multiput(200,26)(0,32){2}{\vector(4,-1){0}}

	\multiput(176,32)(0,32){2}{\circle*{4}}

%	\put(16,48){\vector(1,0){16}}
	\put(16,48){\line(1,0){16}}
%	\put(16,48){\vector(-1,0){0}}
	\put(30,16){\makebox(0,0){\sbus}}
	\put(144,62){\makebox(0,0){\bus}}
	\put(0,0){\makebox(224,16)[r]{\step{} modules}}
	\put(0,80){\makebox(96,16){host}}
 \end{picture}
 }{The \controller{} drives the assembly of \step{} modules via
the tree-like \bus; it communicates with the host via the \sbus.}


\section{\bus\ usage}\label{sect.bus-usage}

Before discussing in more detail the structure of the \bus, it will
be useful to give some representative examples of its usage.

\medskip

\parhead{Trigger commands} The steady-state situation in which \step{}
requires least attention from the host is when all the structural
parameters (number of dimensions, neighbor connectivity, transition
function, etc.) remain constant from one updating scan to the next. In
this case, all that is needed to keep \step{} running at full speed is
simply a stream of ``Go!'' commands (\cf{RMR})---typically at
a rate of a few per second---to request a new scan every time that the
current one is finished.

Note that, even though new data are generated at a fairly high rate
(500 Mbits/sec per module), one does not have to make use of the
\bus\ to display them.  Each module's data appear on the module's
{\sym Video Out} lines in the same site-by-site order as they are
accessed for updating (\sect{}). If the current experiment uses an
appropriate scan format, these data can be directly displayed on a
dedicated {\sc crt} monitor.

\medskip

\parhead{\dram\ readback} Though direct visual monitoring may be
invaluable at an experiment's development stage, in most applications
one would eventually want to set up some form of automated
data-collection and analysis. Given \step's unprecedented throughput,
the choice is between looking at a lot of data once in a while and
looking at a little very often.

By ``a lot of data'' we mean the entire current state of the system
being simulated (or a substantial fraction of it).  Reading into the
host (via the \bus\ and the \sbus) the state of a single \step{}
sector takes a time comparable to that needed by \step{} to update the
entire array.

A much longer time, of course, will be needed for the host to perform
even the most trivial form of analysis on the data thus collected.
However, while this analysis is taking place within the host, the
updating of the \step{} array can progress at full speed.

In some circumstances, the monitoring might entail reading back only a
small area of \dram\ (for example, to check the status of a flag or
``indicator light'' within the simulated system). {\em
Localized} readback of this kind requires little \bus\ bandwidth, can
be performed between scans without substantially affecting the
updating rate, and can be digested by the host in real time. Analogous
considerations apply to the {\em writing} of small amounts of array
data, for instance as a way of applying localized stimuli or forcing
certain boundary conditions.

\medskip

\parhead{Event-counter readback} The \step{} event counters
(\sect{event-counter}) represent a powerful tool for performing a
fairly wide range of data analysis tasks {\em on line}, within the
\step{} modules themselves.  Suitable tasks for the event counters are
detecting the occurrence of specific events, or counting and averaging
local quantities. If desired, this counting can be done separately
over small ``patches'' ({\em coarse-grain averaging}) so as to yield
the spatial distribution of the quantities being measured instead of
just a system-wide average.

The information abstracted by the event counters about several
megabytes of system state reduces to just a few bytes per module.
Moreover, the event counters are double-buffered, and their readout is
streamlined by a special sequential-read mode
(\sect{sequential-read}). Thus, the entire complement of counters can
be read out while a new scan is in progress, and in a time much
shorter than the scan itself.

\medskip 

\parhead{Lookup-table downloading} Much of the flexibility of \step{}
derives from the ease with which one can set up time-dependent
dynamics, and especially {\em microprogrammed} dynamics---in which one
updating pass actually consists of several scans each associated with
a different neighborhood interconnection pattern and governed by a
different transition function. This mode of operation leads to the
most intensive usage of the \bus: while a scan is in progress, a new
set of \lut\ tables is downloaded via the \bus\ in preparation for
the next scan. Though the amount of data in a module's \sram\ lookup
table is not much smaller than that in the module's \dram, typically one
and the same table is broadcast to all modules, so that \lut\
downloading time is independent of system size.


\medskip

Other uses of the \bus---such as setting up the geometry of the array
at the beginning of an experiment, verifying the system's
functionality, performing parity or similar checks for the purpose of
error detection, and so on---are likely to be of an occasional
nature.

\section{\bus\ structure}\label{sect.bus-structure}

The \bus\ is a tree-like, synchronous, pipelined, bidirectional
communication channel having the \controller{} at its root and \step's
individual modules at its leaves.

The special structure of the \bus\ is dictated by the nature of the
\step{} architecture. In fact, a \step{} machine can in principle consist of
an arbitrarily large number of modules; as a consequence, the physical
distance between the host and a module---and consequently the transit
time of both data and control signals---can be unboundedly large.  In
order to achieve a modular bus design whose technological parameters
(clock rate, drive capability of line drivers, etc.) are independent
of the number of \step{} modules, an appropriate number of stages of
fan-out and of signal regeneration must be provided for.

\medskip

In the \bus, control and data states are {\em distributed}, and are
propagated by {\em local} means. This design permits maximum bus
throughput at a constant clock rate when both source and destination
are capable of sustaining this rate. At the same time, this design
makes it possible for the bus to automatically adjust to situations
where either source or destination become incapable of sustaining the
maximum rate, or where the data direction is changed ({\em bus
turnaround\/}, \sect{turnaround}).

What, of course, cannot be avoided is bus {\em latency}, which is
proportional to bus {\em length} (distance from node to leaves);
in turn, bus length is roughly proportional to the third root of the number
of modules---since modules will be physically arranged in a three-dimensional
array. \bus\ latency has no effect on \step{} throughput when the modules
are driven by the host in a ``blind reckoning'' mode---the most common
situation during a simulation run.  However, latency becomes important
when feedback from the modules is desired. In particular, latency
determines the bus turnaround time---during which the bus's
effective thoughput drops to zero.

In practice, owing to the third-root dependency on size, \bus\ latency
is negligible (a few clock cycles) up to sizes of a few hundred
modules. Even in an extremely large \step{} machine (say, 64K modules)
the \bus\ latency will not be greater than a few dozen clock cycles.
Moreover, special hardware optimization is provided to avoid frequent
bus turnaround in the most common situation requiring feedback from
the modules, i.e., readout of the event counters (see
\sect{sequential-read}).

\medskip
	
\figonegiven{unbal-bus}{\psfig{file=fig/unbal-bus.ps}}{(a) Balanced
and (b) unbalanced realizations of the \bus. \bus{} nodes are denoted
by dots, \step{} modules by solid square. In the current
implementation, a module board (dashed box) contains, in addition to a
module, also a full \bus{} node.}

The \bus\ specifications allow much latitude in the actual shape of
the \bus\ tree. The tree sketched in \fig{two-buses} is {\em
balanced}, i.e., all of the branches that start from a given node have
identical structure. Indeed, such a balanced tree could be embedded in
a regular way in a three-dimensional array of modules, as suggested by
\fig{unbal-bus}a (where only two dimensions are shown).  On the other
hand, the arrangement of \fig{unbal-bus}b, though unbalanced, is based
on a one-to-one correspondence between node machinery and module
machinery: nodes can be mounted ``piggyback,'' as it were, on the
modules themselves (dashed box in \fig{unbal-bus}b).  The \chip\
communication protocol is compatible with either type of tree
arrangement. What is more, in addition to the circuitry implementing a
\step{} module, a \team{} also contains all the circuitry for
optionally implementing a \bus{} node having the module itself as one
leaf. The current prototype model of \step{} makes use of this
circuitry at the box level: each box acts as \bus{} node which, in
addition to directly serving the eight modules in the box, provides
potential \bus{} branches for up to eight more boxes. Up to three
of these branches (i.e., one per dimension) are actually used, as
explained in \sect{box-array} (specifically, \fig{2x2x2}).

One disadvantage of the unbalanced arrangement is that the same
command will reach different modules at different moments. If the
command is one that requests a scan, where all modules are to operate
in lockstep, the modules that are closer to the root will have to
insert a calibrated delay (cf.\ {\sym Tree Balancing Delay},
\sect{HER}) before starting the scan, so as to start at the same time
as the farthest module.


\section{Signal direction}

Each segment of the \bus\ tree consists of sixteen {\em data} lines
and a number of {\em control} lines (see \sect{bus-signals} for
details). Control lines are unidirectional. Data lines are
bidirectional, in the sense that they can carry information in one
direction or the other, but not both at the same time ({\em
half-duplex} communication).

\medskip

For sake of clarity, it will be convenient to introduce the following
terminology.

The terms {\em in} and {\em out}---as well as {\em input} and {\em
output}---refer to signals that respectively enter or leave a node.
Thus, an output pin is one that drives the associated line, while an
input pin is one that senses it. (The same pin may be used as input or
output at different moments.)

The {\em inner} side of a node is the one facing towards the host;
the {\em outer}, towards the modules. The \controller{} acts
as a node having only a outer side, and each module as a node
having only an inner side.\footnote
	{As noted in the previous section, each module board also
contains the machinery for an additional full node.}

The terms {\em inbound} and {\em outbound} refer to the absolute
direction of travel: outbound signals travel from the host towards
the modules, and vice versa for inbound signals. Thus, an output pin
on the inner side of a node will be driving an inbound signal.

Finally, the terms {\em upstream} and {\em downstream} refer
to the current direction of travel; upstream is where a signal comes
from, downstream, where it goes.

\section{Fan-out and fan-in}\label{sect.fanout}

The bus nodes (cf.\ \fig{two-buses}) act as repeaters, allowing a
signal to propagate over an arbitrary distance; wherever the tree
branches out, they also act as {\em fan-out} or {\em fan-in}
elements---according to the direction of the signal. The following
discussion applies to each of the data lines individually.

Outbound data travel along the branches of the tree towards the
modules, with a delay of one clock cycle between one node and the
next. Fan-out occurs by each node copying its inner data to all its
outer data lines, as in \fig{fan}a.  Thus, outbound data eventually reach
all the modules; it is the responsibility of each module, as
instructed by previous commands, to accept or ignore the broadcast
data (\sect{modselect}).

Inbound data proceed along tree branches toward the root, with a delay
of one clock cycle between one node and the next.  Inbound data
originate from only one module at a time; previous commands will have
specified which module is selected for this purpose at any particular
time (\sect{modselect}). The unselected modules send 0s as their
data.\flag\footnote
	{This sending of valid, though dummy, data is done in order to
retain data synchronization between all modules; cf.\ \sect{}.}
	Fan-in of inbound data occurs by each node {\sc or}ing the
data from all outer lines and putting the result on the inner
line; since the 0s contributed by unselected modules are ignored by
the {\sc or} operation, in effect the data coming from the selected
module---denoted by $x$ in \fig{fan}b---are routed unchanged through the
node no matter which branch they come from.

\figonegiven{fan}{
 \setlength{\unitlength}{0.9pt}
 \begin{picture}(132,112)(-16,-8)
	\put(8,8){\makebox(0,0){(a)}}
	\thicklines
	\put(48,48){\circle*{4}}
	\put(0,48){\line(1,0){48}}	\put(24,48){\vector(1,0){0}}
	\put(48,48){\line(1,1){48}}	\put(72,72){\vector(1,1){0}}
	\put(48,48){\line(3,1){48}}	\put(72,56){\vector(3,1){0}}
	\put(48,48){\line(3,-1){48}}	\put(72,40){\vector(3,-1){0}}
	\put(48,48){\line(1,-1){48}}	\put(72,24){\vector(1,-1){0}}

	\put(-8,48){\makebox(0,0){$x$}}
	\put(104,96){\makebox(0,0){$x$}}
	\put(104,64){\makebox(0,0){$x$}}
	\put(104,32){\makebox(0,0){$x$}}
	\put(104,0){\makebox(0,0){$x$}}
 \end{picture}
 \quad\quad
 \begin{picture}(128,112)(-16,-8)
	\put(8,8){\makebox(0,0){(b)}}
	\thicklines
	\put(48,48){\circle*{4}}
	\put(0,48){\line(1,0){48}}	\put(24,48){\vector(-1,0){0}}
	\thinlines
	\put(48,48){\line(1,1){48}}	\put(72,72){\vector(-1,-1){0}}
	\thicklines
	\put(48,48){\line(3,1){48}}	\put(72,56){\vector(-3,-1){0}}
	\thinlines
	\put(48,48){\line(3,-1){48}}	\put(72,40){\vector(-3,1){0}}
	\put(48,48){\line(1,-1){48}}	\put(72,24){\vector(-1,1){0}}

	\put(-8,48){\makebox(0,0){$x$}}
	\put(104,96){\makebox(0,0){$0$}}
	\put(104,64){\makebox(0,0){$x$}}
	\put(104,32){\makebox(0,0){$0$}}
	\put(104,0){\makebox(0,0){$0$}}
 \end{picture}
}{Fan-out of outbound data (a), and fan-in of inbound data (b) on the
\bus.}

Unless otherwise specified, fan-out and fan-in of control signals occur
in analogous ways.

\section{\bus\ signals}\label{sect.bus-signals}

Each \bus\ {\em segment}---terminated by a node at each end---consists
of sixteen {\em data} lines and six {\em control} lines, as in
\fig{bus-segment}.

\figonegiven{bus-segment}{
 \def\sig{\small\sym}
 \def\dist{88}	% \def\dist{88}
 \def\prox{84}	% \def\prox{88}
 \def\far{162}	% \def\far{152}
 \def\diff{74}	% \def\diff{64}
 \def\zerom{-4}
 \def\distm{86}
 \def\farm{166}	%  \def\farm{162}
 \begin{picture}(246,170)(0,-32)
	\thinlines
	\put(\dist,-16){\line(0,1){144}}	%
	\put(\far,-16){\line(0,1){144}}		%
	\put(0,-16){\line(1,0){\dist}}		%
	\put(0,128){\line(1,0){\dist}}
	\put(\far,-16){\line(1,0){\prox}}	%
	\put(\far,128){\line(1,0){\prox}}
	\put(36,132){\makebox(0,0)[b]{\large node}}
	\put(\dist,130){\makebox(0,0)[b]{outer side}}
	\put(210,132){\makebox(0,0)[b]{\large node}}
	\put(\far,130){\makebox(0,0)[b]{inner side}}
	\put(44,-32){\vector(0,1){16}}		%
	\put(202,-32){\vector(0,1){16}}		%
	\put(46,-24){\makebox(0,0)[l]{\sig Clock}}	%
	\put(204,-24){\makebox(0,0)[l]{\sig Clock}}	%

	\put(\zerom,112){\makebox(\dist,0)[r]{\sig Outer Data[0:15]}}
	\put(\farm,112){\makebox(\prox,0)[l]{\sig Inner Data[0:15]}}
	\put(\distm,116){\makebox(\diff,0)[b]{\sig Data[0:15]}}
	\put(\dist,112){\vector(1,0){\diff}}
	\put(\dist,112){\vector(-1,0){0}}
	\put(152,112){\makebox(0,0){/}}
	\put(152,118){\makebox(0,0)[b]{\small 16}}

	\put(\zerom,88){\makebox(\dist,0)[r]{\sig Outbnd.\ Handsh.\ Out}}
	\put(\farm,88){\makebox(\prox,0)[l]{\sig Outbnd.\ Handsh.\ In}}
	\put(\distm,92){\makebox(\diff,0)[b]{\sig Outbnd.\ Handsh.}}
	\put(\dist,88){\vector(1,0){\diff}}

	\put(\zerom,72){\makebox(\dist,0)[r]{\sig Inbnd.\ Handsh.\ In}}
	\put(\farm,72){\makebox(\prox,0)[l]{\sig Inbnd.\ Handsh.\ Out}}
	\put(\distm,76){\makebox(\diff,0)[b]{\sig  Inbnd.\ Handsh.}}
	\put(\far,72){\vector(-1,0){\diff}}

	\put(\zerom,48){\makebox(\dist,0)[r]{\sig Frame Out}}
	\put(\farm,48){\makebox(\prox,0)[l]{\sig Frame In}}
	\put(\distm,52){\makebox(\diff,0)[b]{\sig Frame}}
	\put(\dist,48){\vector(1,0){\diff}}

	\put(\zerom,32){\makebox(\dist,0)[r]{\sig Reset Out}}
	\put(\farm,32){\makebox(\prox,0)[l]{\sig Reset In}}
	\put(\distm,36){\makebox(\diff,0)[b]{\sig Reset}}
	\put(\dist,32){\vector(1,0){\diff}}

	\put(\zerom,16){\makebox(\dist,0)[r]{\sig Sync Out}}
	\put(\farm,16){\makebox(\prox,0)[l]{\sig Sync In}}
	\put(\distm,20){\makebox(\diff,0)[b]{\sig Sync}}
	\put(\dist,16){\vector(1,0){\diff}}

	\put(\zerom,0){\makebox(\dist,0)[r]{\sig Status In}}
	\put(\farm,0){\makebox(\prox,0)[l]{\sig Status Out}}
	\put(\distm,4){\makebox(\diff,0)[b]{\sig Status}}
	\put(\far,0){\vector(-1,0){\diff}}
 \end{picture}
}{Makeup of a \bus\ segment.}

\noindent At any node where the \bus\ branches out into two or more segments,
the node's outer side will provide terminations as in
\fig{bus-segment} for each of these segments. The details of this
branching will be discussed in \sect{};\flag\ until then, the issue of bus
branching can be ignored.

All \bus\ signals are sensed at the rising edge of a common \step{} {\em
clock} which is made available to all \step{} modules and \bus\ nodes,
as well as to the \controller{} (\sect{controller},
\sect{clock-sync}).

\medskip

The direction of the data lines is not directly governed by a ``Read/Write''
control signal, but is established by a protocol discussed in
\sect{frames} and \sect{}.

The two {\em handhsake} lines ({\sym Outbound Handshake} and {\sym
Inbound Handshake}) regulate the rate of flow of data on the data
lines, as explained in \sect{data-readiness}.

The {\sym Reset} signal is originated by the \controller, and is used to reset
\step{} modules and \bus\ nodes to a known state after the occurrence
of errors or other unanticipated conditions.

The {\sym Sync} signal carries short binary-encoded messages (``sync pulses'')
providing horizontal and vertical sync information for a \crt\ monitor
and related information for the \step{} machine itself (see \sect{hvsync}).

The {\sym Frame} signal is used to delimit a \bus\ frame,
specifying which part of it contains address, data, or parity
information---as explained in \sect{frames}.

By the {\sym Status} line, modules or nodes send to the \controller{}
real-time status information, namely interrupt requests and ``scan in
progress'' reports (see \sect{} and \sect{}).

In the current implementation, in which \bus{} fanout is managed by a
module (cf.\ \fig{unbal-bus}b), the module hitches a ride on the
\bus{} for an additional outbound signal (not indicated in
\fig{bus-segment}), called {\sym Scan}, which is asserted precisely
when the module itself is scanning. Since in most circumstances all
\step{} modules are supposed to scan in lockstep in spite of any \bus\
tree unbalance (\cf{bus-structure}), this signal allows modules to
verify (by local comparison) that they are indeed scanning in
lockstep.


\medskip

We mentioned that a \step{} module acts as a \bus\ node having only a
inner side (since it is located at the end of a branch).  Besides one
bidirectional data pin, each \chip\ has one pin for each of the \bus\
control signals (\sect{tbus-interface}). When sixteen \step{} chips are
assembled together in a module, their sixteen {\sym Tbus Data} pins
(\sect{tbus-interface}) provide the terminations {\sym Inner
Data(0:15)} of \fig{bus-segment}.  The sixteen chips' {\sym Reset}
inputs are tied together, so as to present a single {\sym Reset In}
termination to the \bus\ {\sym Reset} line---and similarly for the
other outbound control signals. The inbound control signals, {\sym
Inbound Handshake Out} and {\sym Status Out} are supplied to the
\bus\ by just one of the sixteen chips, since all sixteen chips act in
lockstep.

\medskip

All of the \bus\ signals are {\em pipelined}, i.e., each node
samples a signal coming from upstream at the rising edge of the clock,
it logically combines the sampled signal with other sampled signals
and internal states, and puts out the result downstream early enough
during the clock cycle for this result to reach the next node
downstream in time to be sampled by the next rising edge of the clock.

In addition, the data signals are {\em double-buffered}, as explained
in \sect{data-readiness}.

\section{Data flow control}\label{sect.data-readiness}

In each \bus\ segment, the bundle of sixteen data lines is
accompanied by two {\em handshake} lines. The one pointing upstream signals
whether a node is {\em full}---and thus cannot accept the incoming data
bit. The one pointing downstream signals whether the node is providing
a {\em valid} data bit or just a dummy bit (whose value is of no
relevance); details will be given in a moment. Which line has which
role depends on the current data direction; that is, when the data are
outbound the {\sym Outbound Handshake} line (which in this case points
downstream, i.e., in the same direction as the data) plays the role of
{\sym Valid} signal, while the {\sym Inbound Handshake} line (which points
upstream) plays the role of {\sym Full} signal, as in
\fig{tags-out}.  The roles are reversed when the data are inbound, as
in \fig{tags-in}.

\figonegiven{tags-out}{
 \def\sig{\small\sym}
 \def\dist{88}	% \def\dist{88}
 \def\prox{84}	% \def\prox{88}
 \def\far{162}	% \def\far{152}
 \def\diff{74}	% \def\diff{64}
 \def\zerom{-4}
 \def\distm{86}
 \def\farm{166}	%  \def\farm{162}
 \begin{picture}(246,74)(0,64)
	\thinlines
	\put(\dist,64){\line(0,1){64}}		%
	\put(\far,64){\line(0,1){64}}		%
	\put(0,64){\line(1,0){\dist}}		%
	\put(0,128){\line(1,0){\dist}}
	\put(\far,64){\line(1,0){\prox}}	%
	\put(\far,128){\line(1,0){\prox}}
	\put(36,132){\makebox(0,0)[b]{\large node}}
	\put(\dist,130){\makebox(0,0)[b]{outer side}}
	\put(210,132){\makebox(0,0)[b]{\large node}}
	\put(\far,130){\makebox(0,0)[b]{inner side}}

	\put(\zerom,112){\makebox(\dist,0)[r]{\sig Outer Data[0:15]}}
	\put(\farm,112){\makebox(\prox,0)[l]{\sig Inner Data[0:15]}}
	\put(\distm,116){\makebox(\diff,0)[b]{\sig Data[0:15]}}
	\put(\dist,112){\vector(1,0){\diff}}
%	\put(\dist,112){\vector(-1,0){0}}
	\put(152,112){\makebox(0,0){/}}
	\put(152,118){\makebox(0,0)[b]{\small 16}}

	\put(\zerom,88){\makebox(\dist,0)[r]{\sig Valid Out}}
	\put(\farm,88){\makebox(\prox,0)[l]{\sig Valid In}}
	\put(\distm,92){\makebox(\diff,0)[b]{\sig Valid}}
	\put(\dist,88){\vector(1,0){\diff}}

	\put(\zerom,72){\makebox(\dist,0)[r]{\sig Full In}}
	\put(\farm,72){\makebox(\prox,0)[l]{\sig Full Out}}
	\put(\distm,76){\makebox(\diff,0)[b]{\sig Full}}
	\put(\far,72){\vector(-1,0){\diff}}

 \end{picture}}{When data are outbound, the {\sym Outbound Handshake},
which flows with the data, plays the role of {\sym Valid} signal, while
the {\sym Inbound Handshake}, which flows against the data,
plays the role of {\sym Full} signal.}

\figonegiven{tags-in}{
 \def\sig{\small\sym}
 \def\dist{88}	% \def\dist{88}
 \def\prox{84}	% \def\prox{88}
 \def\far{162}	% \def\far{152}
 \def\diff{74}	% \def\diff{64}
 \def\zerom{-4}
 \def\distm{86}
 \def\farm{166}	%  \def\farm{162}
 \begin{picture}(246,74)(0,64)
	\thinlines
	\put(\dist,64){\line(0,1){64}}		%
	\put(\far,64){\line(0,1){64}}		%
	\put(0,64){\line(1,0){\dist}}		%
	\put(0,128){\line(1,0){\dist}}
	\put(\far,64){\line(1,0){\prox}}	%
	\put(\far,128){\line(1,0){\prox}}
	\put(36,132){\makebox(0,0)[b]{\large node}}
	\put(\dist,130){\makebox(0,0)[b]{outer side}}
	\put(210,132){\makebox(0,0)[b]{\large node}}
	\put(\far,130){\makebox(0,0)[b]{inner side}}

	\put(\zerom,112){\makebox(\dist,0)[r]{\sig Outer Data[0:15]}}
	\put(\farm,112){\makebox(\prox,0)[l]{\sig Inner Data[0:15]}}
	\put(\distm,116){\makebox(\diff,0)[b]{\sig Data[0:15]}}
%	\put(\dist,112){\vector(1,0){\diff}}
%	\put(\dist,112){\vector(-1,0){0}}
	\put(\far,112){\vector(-1,0){\diff}}
	\put(152,112){\makebox(0,0){/}}
	\put(152,118){\makebox(0,0)[b]{\small 16}}

	\put(\zerom,88){\makebox(\dist,0)[r]{\sig Full Out}}
	\put(\farm,88){\makebox(\prox,0)[l]{\sig Full In}}
	\put(\distm,92){\makebox(\diff,0)[b]{\sig Full}}
	\put(\dist,88){\vector(1,0){\diff}}

	\put(\zerom,72){\makebox(\dist,0)[r]{\sig Valid In}}
	\put(\farm,72){\makebox(\prox,0)[l]{\sig Valid Out}}
	\put(\distm,76){\makebox(\diff,0)[b]{\sig Valid}}
	\put(\far,72){\vector(-1,0){\diff}}

 \end{picture} }{When data are inbound, the {\sym Inbound Handshake},
which flows with the data, plays the role of {\sym Valid} signal, while
the {\sym Outbound Handshake}, which flows against the data,
plays the role of {\sym Full} signal.}

Each of the data lines of the \bus\ is double-buffered as it passes
through a node.\footnote
	{\label{foot.squeeze}Traffic on a pipeline has some
``inertia,'' i.e., it cannot be stopped all at once by
action-at-a-distance. Since there is one clock cycle of delay from the
time a node realizes that traffic downstream is backed up to the time
this information is communicated to the node immediately upstream,
room must be available to accomodate the bit passed on during that
cycle.}
	The double-buffering mechanism, which is essentially a
2-bit {\sc fifo}, allows a node to use the {\sym Full} line as an
``XON/XOFF'' signal to regulate the rate of data flow. 
Let the flow proceed from node $A$ to $B$ and from $B$ to $C$,
as in the following diagram
 $$
	A\longrightarrow B\longrightarrow C\quad .
 $$
Normally node $B$ holds only one bit,\footnote
	{This may be a valid bit or a dummy bit: the {\sym Valid} tag
accompanies a data bit throughout its life.}
	$x$, which at the next clock pulse is passed on to $C$ and
replaced by the one supplied by $A$. If node $C$ asserts its {\sym
Full} line, then the $x$ bit is retained within $B$; $A$'s bit is still
accepted by $B$, which now contains two bits and consequently asserts
its {\sym Full} line. As soon as $C$ is no longer full, bit $x$ is passed
on to it and $B$ itself is no longer full.

Unlike other {\sc fifo} schemes, a \bus\ node will never contain less
that one bit. In fact, the ultimate sources of data bits (the
\controller{} for outbound data flow, and the \step{} modules for inbound
data flow) keep the data pipeline always primed: if at a given clock cycle
data could be accepted (i.e., the node immediately downstream
is not full) but the source does not have any valid data, a dummy bit
will be interpolated. Once thus created, a dummy bit will only be
consumed at the ultimate destination.\footnote
	{The alternative would have been to allow dummy-bit gaps
that may have developed between valid bits to be ``reabsorbed''
whenever earlier valid data were slowed down.}
	This ``incompressibility'' of the data stream was deliberately
introduced in order to maintain a satisfactory timing
relationship beween streams flowing towards the root from different
branches, even in the case of an unbalanced tree (\cf{bus-structure},
\sect{tree-balancing}).


\section{\bus\ frames}\label{sect.frames}

\flag Data on the \bus\ are transmitted in the form of self-contained {\em
frames}, delimited by the {\sym Frame} signal as in \fig{frame}.
A frame consists of three phases:
 \begin{itemize}
	\item An {\em address} phase of variable length (possibly of zero
length). The string transmitted during this phase specifies the
address of one of the \chip's registers (see
\sect{chip-regs}) and the direction of the requested data transfer
(see \sect{} for details). The address phase is terminated by
negating {\sym Frame}.
	\item A {\em data} phase, during which the \controller{} either
writes data to that register or reads data from it; the length of this
phase depends on the register that is being accessed (cf.\ \sect{chip-regs}).
The data phase is terminated by re-asserting {\sym Frame}.
	\item A {\em parity} phase, of exactly one bit, where the \controller{}
transmits the parity of the frame.
 \end{itemize}

\figonegiven{frame}{
 \def\sig{\small\sym}
 \begin{picture}(224,40)(-40,0)
	\thinlines
	\put(-40,8){\makebox(0,0)[l]{\sig Data}}
	\put(0,0){\line(1,0){184}}
	\put(0,16){\line(1,0){184}}
	\put(48,0){\line(0,1){16}}
	\put(160,0){\line(0,1){16}}
	\put(176,0){\line(0,1){16}}
	\put(24,8){\makebox(0,0){address}}
	\put(104,8){\makebox(0,0){data}}
	\put(168,6){\makebox(0,0){p}}

	\put(-40,32){\makebox(0,0)[l]{\sig Frame}}
	\put(0,24){\line(1,0){48}}
	\put(48,40){\line(0,-1){16}}
	\put(48,40){\line(1,0){112}}
	\put(160,24){\line(0,1){16}}
	\put(160,24){\line(1,0){24}}
 \end{picture}}{Framing of address, data, and parity on the {\sym
Data} lines.}

\fig{frame} represents the {\em sequence} of valid bits on a data
line, not the actual {\em timing\/}; depending on the status of the
{\sym Valid} line (\sect{data-readiness}), dummy bits may appear
anywhere between valid bits. Typically, several idle cycles,
represented by dummy bits, will elapse between the parity bit at the
end of one frame and the first address bit at the beginning of the
next frame.

\medskip

Note that writing or reading are done in parallel, via the sixteen data
lines, to or from all sixteen \chip{}s that belong to a module;
thus, the data flowing on the data lines during the data phase may
(and typically will) be different for each of the sixteen chips.
Similarly, sixteen different parity bits, one for each of the sixteen
\chip{}s of a module, are transmitted in parallel on the
data lines.

On the other hand, during the address phase the {\em same} address and
data-direction information is sent on all sixteen data lines (even though in
principle it would be possible to send different values), so that
homologous registers will be accessed (all read or all written) on
all sixteen chips of a module simultaneously.

\medskip

The entire frame will be called a {\em write} or a {\em read} frame
according to the direction of data movement (outbound or inbound) for
the data phase---which, as we have seen, is specified during
the address phase.


\section{Bus turnaround}\label{sect.turnaround}

The default data direction for the \bus\ is `write'.  The direction
turns to `read' only during the data phase of a read frame.

Data direction changes propagate from the \controller{} toward the
modules at the rate of one bus segment per clock cycle, with one
exception. If a change from write to read reaches a segment where
outbound data are being held up waiting for the {\sym Full} signal
($=\null${\sym Inbound Handshake}; \cf{data-readiness}) to be asserted, the
propagation of this change will be held up too, so as not to overcome
the stretch of write data still in the pipeline.

On the other hand, a change from read to write will propagate at full
speed regardless of whether there are inbound data still in the
pipeline.  These data, together with the information carried by the
associated handshake lines (cf.\ \sect{data-readiness}), will be lost.
The handshake lines themselves will immediately adapt to their new role
of accompanying outgoing data.

\medskip

\bus\ data direction and bus turnaround are discussed in more detail
in \sect{}.


\section{\chip\ registers}\label{sect.chip-regs}

As mentioned in \sect{frames}, the \bus\ represents the medium
through which the \controller{} writes data to or reads data from a
number of registers located on the \chip{}s (these registers
are detailed in \sect{regs}). During the address phase of
a \bus\ frame, a particular register is addressed and the data
direction for the ensuing tranfer is established; during the
data phase, data are serially written to this register from a \bus\
data line, or serially read from this register to the same data line.
This process goes on in parallel for the sixteen chips of a module and
the corresponding sixteen data lines.

The \chip\ registers come in different lengths. Most registers
have a fixed length, ranging from 1 to 120 bits. Two registers,
however, namely the {\sym LUT I/O Register} (\sect{LIOR}) and the {\sym Scan I/O
Register} (\sect{SIOR}), are actually serial data ports used to download or upload selected
regions of, respectively, the \dram\ and \sram\ areas of a module.
For these two registers, the length of the desired data
transfer---which plays the role of register length---must be
explicitly programmed by the \controller{} into the chip before the
transfer itself takes place.

\medskip

Many of the \chip's registers are logically collections of shorter
registers that have been put together under a single register address
for convenience and efficiency. For example, the six-bit {\sym Event
Counter Source Register} (sect{ECSR}) consists of (a) a two-bit field
selecting one of four possible data sources for the event counter and
(b) a four-bit field specifying how this source is to be combined with
another fixed source.

\medskip

For \io\ purposes, the \chip\ registers are organized as circular
shift-registers\footnote
	{These remarks do not apply to the data ports.}
	with an input/output tap at a distinguished position.  A
register is written by serially shifting new bits into it via the
input tap, while the register's entire contents rotates one
bit-position at a time; similarly, a register is read by rotating it
entire contents one bit-position at a time, letting its bits serially
appear at the output tap.

With this mechanism, it is clear that a read operation must clock out
of a register a number of bits equal to the register's length, in
order to leave the register in the original alignment at the end of the
operation; an equal number of bits must be clocked into the
register by a write operation, in order for each bit to have reached
its proper destination in the register at the end of the operation.

The length of each register is explicitly known to both the \chip\
and the host. This length is one of the parameters that is passed by
the host to the \controller{} in order to set up the transfer of a
frame of data, and is used by the \controller{} to know when to terminate
the data phase (cf.\ \sect{frames}) and proceed to the transmission of
the parity bit (cf.\ \sect{frames} and \sect{frame-parity}).

\medskip

The {\em register address space} of the \chip\ consists of 32
locations. Each register is uniquely identified by a five-bit address.
In the address phase of a \bus\ frame, leading zeros of the register
address need not be transmitted (that's why this phase has variable
length, as noted in \sect{frames}); thus, low addresses require less
time to send. Of course, the lowest addresses have been assigned to
the most frequently used registers (cf.\ \chapt{regs}).


\section{Frame parity}\label{sect.frame-parity}

The {\em parity} of a \bus\ frame is a single bit obtained by {\sc
xor}ing together the frame's address and data bits and complementing
the result (``odd parity''). This parity
is computed independently by the \controller{} and by the
individual \chip{}s.  At the end of the frame, the \controller{}
broadcast its version of the parity bit (\fig{frame}); it is the
responsibility of each \chip\ that was selected for that frame to
compare this bit with its own parity bit, and generate a parity
interrupt in case of mismatch (\sect{}).

\memo{Pointer to details of when the parity flags in the \controller{} and
the chips are cleared in preparation for a new frame.}\flag\

\medskip

The parity bit is handled in a similar way but with a somewhat different
meaning in sequential-read transactions (\sect{sequential-read}).

\section{Interrupts}\label{sect.bus-interrupts}

Interrupts generated by a \step{} module propagate inwards along the
\bus\ on the {\sym Status} line, which also carries Scan-in-Progress
information (\sect{}). On each module, only one of the sixteen \chip{}s
senses the incoming {\sym Status} line, and only one of the \chip{}s drives the
outgoing {\sym Status} line. A wire-{\sc and} rail, {\sym Int},
ties together the sixteen chips, allowing them to exchange interrupt
information (\sect{wire-and}, \sect{}).

Interrupts that are enabled (see {\sym IER}) will generate a host interrupt
when the corresponding error condition is signaled.  Once an
interrupt reaches the host interface, further interface operation is
suspended and the interface waits for the host to restart it.

Errors that are traceable to a specific \bus\ frame are signaled at
the end of that frame; thus Bus Parity (\sect{frame-parity})
(signaled by setting the {\sym BPIF}) and Unknown Register\footnote
	{This error occurs if an attempt is made to send data to (or
read data from) a non-existent register.  Note that data bits written to
a nonexistent register are always accepted (and simply discarded) by
the \chip, and are simply discarded; bits read are all zeros.}
	({\sym URIF}) errors are signaled at the moment when the
\chip\ checks the parity bit for the frame just completed (see also
{\sym VWE}) \memo{what is this?}\flag.  Thus, by the time the error is
signaled the next instruction is just being fetched by the \controller.
The erroneous instruction is therefore always the {\em previous}
instruction; a pointer to this instruction is available to the
interrupt service routine.

Because of the pipelined nature of the \bus, interrupts generated by
\step{} take some number of clocks to reach the controller.  It is
therefore necessary that the \controller{} guarantee a sufficient
delay so that a second data transfer over the \bus\ will not take place
before an interrupt from the previous transfer can reach the
\controller.

Contention errors (two drivers both trying to write onto the same
line) are not normally synchronized with a particular data transfer,
and so they are signaled as soon as they occur.  These are fatal
errors: the interrupt routines will not try to recover (as they would
from a Bus Parity error, for example).  While a given {\sym Contention
Interrupt Flag} is set, the \chip\ should probably turn off all
tri-state drivers on the affected wire, in order to avoid hardware
damage (at least if the associated enable flag is set).  Contention on
the \bus\ is of course a special case, since it could prevent one from
reading the {\sym IFR}.  This is not really a problem: if an interrupt
occurs but the {\sym IFR} is read as all zeros, then it must be that the
\bus\ drivers are disabled, and so the interrupt signals \step{} Bus
Contention.

\smallskip

In this implementation, the \controller{} treats \step{} interrupts as
being edge-triggered: once a \step{} interrupt has been detected, the
current instruction decode is aborted, a host interrupt is generated,
and the \controller{} goes into a halt (``waiting for host'') mode.  {\em No
new host interrupt will be generated until the \step{} interrupt(s)
have been cleared.} Thus the host interrupt service routine will be
free to interrogate the {\sym IFR}s, even though the interrupt is
still being asserted by \step.


\section{Module selection}\label{sect.modselect}

During the data phase of a write frame, data are physically fanned-out
by the \bus\ to {\em all} \step{} modules. In most cases, these data
are intended for use by all modules; in certain cases, however, it is
desirable that only certain modules be enabled as active listeners,
either on an individual basis or as members of a certain group of
modules.

During the data phase of a read frame, it is essential for the proper
functioning of the fan-in mechanism (cf. \sect{fanout}) that only {\em one}
module be enabled as a talker on the \bus.

\Step's module selection mechanism, described here, allows each
module to decide whether or not it should take active part in a given
write or read transaction. The unit of selection handled by this
mechanism is an entire module. Finer-grained selection, such as
write-enabling of individual layers within a module, may be achieved by
other methods (cf.\ \sect{VWR}).

\medskip

We have seen that a \step{} machine consists of an arbitrary number of
modules, and each module consists of sixteen layers each served by a
separate \chip. In most respects, the sixteen chips of a set act
independently of one another; they manage to achieve coordinated
action not because they exchange coordinating information between one
another, but because they are separately instructed by the host to
play in lockstep appropriately correlated roles.

However, the sixteen chips of a module act as a unit for the
purpose of deciding whether that module is {\em selected}, i.e., will
accept the data broadcast by the host on the \bus\ during a write
frame, or will put its own data on the \bus\ during a read frame.

\medskip

The decision of whether or not a module is currently selected is based
upon a match between an {\em internal name} and a {\em target name}.
Both of these names are sixteen bits long (one bit per chip).  For
each of the sixteen positions, internal- and target-name bits
are (directly or indirectly) specified by a three-bit {\em selection
code}, which is broadcast by the host and written to the {\sym Module
Select Register} (\sect{MSR}) of all modules.\footnote
	{This register is exceptional in the sense that data addressed to
it are accepted by {\em every} module regardless of the
module's current selection status.}

There are two primary sources available for the internal-name bit. The
{\em module ID}, stored in the {\sym Module ID Register} ({\sym MIDR})
is used to uniquely identify a module, and is assigned once and for
all at initialization time (\sect{id-init}); the {\em group ID},
stored in the {\sym Group ID Register} ({\sym GIDR}) is used for
grouping modules as needed for the convenience of particular
operations, and may be reassigned by the host from time to time. Both
IDs are sixteen bits long, with one bit contributed by each chip.
A third source is the {\sym Glue-AND} bit, used only at initialization
time (\sect{id-init}).

For the target-name bit, the selection code may supply directly a 0, a
1, or a `dont care'; or it may specify that the bit should be taken
from the {\sym LUT Index Register} (\sect{LIR}).
This fourth option is used during a special {\em sequential-read} mode
(\sect{sequential-read}).

\medskip

The following table lists what sources the different selection codes
specify for the internal bit and the target bit\footnote
	{The values 000, 010, and 111, which are probably the most
commonly used, happen to be of a form that can be packed as {\em immediate}
data in a step-list frame (cf.\ \sect{}).}
 \begin{equation}\label{eq.selection-codes}
 \input selection-codes.insert
 \;.
 \end{equation}

Of course, the individual \chip\ will be able to detect a match
between internal name and target name only in one of the sixteeen
bit positions. The overall name match is obtained by {\sc and}ing
together the sixteen partial matches.  This logical operation is
physically performed by a wired-{\sc and} line that ties together all
sixteen chips' {\sym ModSel} pins (\sect{wired-and}), collecting the
arguments of the {\sc and} from the individual chips and returning the
result to all chips. In this way, the individual chip not only gives
its contribution to the selection decision, but is also made aware of
the collective decision's outcome.

The above comparison between internal name and target name is made at
the end of the data phase of each frame (\fig{frame}), and
the result (`selected' or `not selected') remains valid
for the entire duration of the following frame.


\section{Module ID initialization}\label{sect.id-init}

As mentioned in \sect{modselect}, before a \step{} machine can function
properly each module must be given a distinct module ID. This initial
naming task presents a challenge, because there are no intrinsic
features in the architecture that distinguish one module from another.
In fact, (a) the modules are structurally identical and are physically
interconnected in a uniform way (i.e., they form a three-dimensional
torus), and (b) at system reset all modules are set to an identical
internal state.

To break this symmetry between modules, it is sufficient to ``tag''
one module; this is done at power up, as explained in \sect{}.  Once
the symmetry is broken, every module is in principle uniquely
identifiable by its $x$, $y$, and $z$ displacements (in the torus
coordinates) relative to the tagged one. This fact is exploited by the
module ID initialization procedure described below (which the host
will run immediately after power up): starting from the tagged module
and taking advantage of the glue lines that connect modules in the
$x$, $y$, and $z$ directions, this procedure successively steps from
module to module, selecting and uniquely naming each one in turn.

\medskip

What distinguishes the tagged module at power up is that it it is
accessible on the \bus (only the modules in the corner
box---\cf{machine-overview}---are initially accessible) and has a 1
rather than a 0 as the least-significant bit of its module ID.

\memo{Update the following procedure}

By appropriate use of the 000, 010, and 111 selection codes (table
\eq{selection-codes}), the host will select just this module and give
it a name consisting of all 0s ({\tt 0000} in hexadecimal notation),
and then select all the other modules at once and give them, for the
moment, a name consisting of all 1s (hexadecimal {\tt FFFF}).

\medskip

We shall now describe how naming information is propagated through
the array of modules by the initialization procedure.

Let us recall that, for each of the sixteen data layers, each module
has six glue pins which connect it, via as many glue lines, to the
six adjacent modules (\fig{glue-and}a).

 \figonegiven{glue-and}{
 \setlength{\unitlength}{0.75pt}
 \def\sig{\small\sym}
 \begin{picture}(144,128)(-72,-64)
	\thicklines
	\put(-40,-16){\line(0,1){48}} % the three vertical sides
	\put(-8,-32){\line(0,1){48}}
	\put(40,-32){\line(0,1){48}}
	\put(-40,32){\line(1,0){48}} % the three horizontal lines
	\put(-8,16){\line(1,0){48}}
	\put(-8,-32){\line(1,0){48}}
	\put(-40,32){\line(2,-1){32}} % the three diagonal lines
	\put(-40,-16){\line(2,-1){32}}
	\put(8,32){\line(2,-1){32}}
	\put(-72,0){\line(1,0){48}} % the y lines
	\put(40,0){\line(1,0){32}}
	\put(0,-64){\line(0,1){32}} % the z lines
	\put(0,24){\line(0,1){40}}
	\put(-64,32){\line(2,-1){24}} % the x lines
	\put(16,-8){\line(2,-1){48}}

	\put(56,-40){\makebox(0,0){\sig XP}}
	\put(-56,40){\makebox(0,0){\sig XM}}
	\put(64,8){\makebox(0,0){\sig YP}}
	\put(-64,8){\makebox(0,0){\sig YM}}
	\put(12,56){\makebox(0,0){\sig XP}}
	\put(12,-60){\makebox(0,0){\sig ZM}}

	\put(-64,-64){\makebox(0,0){(a)}}
 \end{picture}
 \quad
 \begin{picture}(120,144)(-16,-24)
	\thinlines
	\put(16,-24){\line(0,1){128}}
	\put(16,-24){\line(1,0){64}}
	\put(16,104){\line(1,0){64}}
	\thicklines
	\multiput(-16,0)(0,16){6}{\line(1,0){48}}
	\put(32,-4){\line(0,1){88}}
	\put(32,40){\oval(64,88)[r]}
	\put(64,40){\line(1,0){56}}
	\put(0,2){\makebox(0,0)[b]{\sig ZP}}
	\put(0,18){\makebox(0,0)[b]{\sig ZM}}
	\put(0,34){\makebox(0,0)[b]{\sig YP}}
	\put(0,50){\makebox(0,0)[b]{\sig YM}}
	\put(0,66){\makebox(0,0)[b]{\sig XP}}
	\put(0,82){\makebox(0,0)[b]{\sig XM}}

	\put(92,42){\makebox(0,0)[b]{\sig Glue-AND}}
	\put(48,40){\makebox(0,0){\sc and}}

	\put(-8,-16){\makebox(0,0){(b)}}
 \end{picture}}{(a) The six glue lines that connect one module
to its neighboring modules. (b) The module ID initialization sequence
makes use of the {\sc and} of the six glue lines.}

\noindent At power-up, the six glue pins of all modules are programmed
as inputs; these inputs are internally {\sc and}ed together into a
signal call {\sym Glue-AND} (\fig{glue-and}b). Since the glue inputs
are internally pulled up by resistors to logical 1, {\sym Glue-AND}
will have a value of 1.

We'll now turn the {\sym XP} pin of module {\tt 0000}, layer 0, into
an output\footnote
	{By appropriately setting the sign of the kick register;
cf.\ \sect{}.}
	and put a 0 on it. This 0 will travel on a glue line and
reach pin {\sym XM} of an adjacent module, forcing its {\sym Glue-AND}
to 0.  Making appropriate use of selection code 110, corresponding to
{\sym Glue-AND}=0 (cf.\ table \eq{selection-codes}), we can select
just this adjacent module,\footnote
	{With the present naming procedure, the only two modules
having {\sym Glue-AND}=0 are the module that emits 0 and the one that
senses it. But the former will already have been named, while the
latter still has the default name {\tt FFFF}. Since selection code 110
is only used with one of the sixteen chips, the remaining fifteen
name positions can use code 010 to discriminate between the emitting
module and the receiving module. (This scheme does not allow one to
discriminate between the last two modules in a 64K-module machine, but
in this case the last module does not have to be named, since it already
has the correct name {\tt FFFF}.}
	and write {\tt 0001} as its module ID.
At this point we'll ask module {\tt 0000} to turn
pin {\sym XP} of layer 0 back into an input, and module {\tt 0001}
to put a 0 on one of its glue pins, so as to ``touch'' another module.
We will name the new module {\tt 0002}, and so on until all modules
of the \step{} machine have been given unique IDs.


\section{Sequential read}\label{sect.sequential-read}

An optimized, sequential mode of data access is provided for use in
reading back event counts.  Normally, event-count readback is done
using literal data in the selection code to explicitly address
specific modules in turn (\sect{modselect}).

The sequential-read mode makes use of the {\sym LUT Index
Register} ({\sym LIR}; \cf{LIR}).  All {\sym LIR}s initially have the
same value, and all are incremented simultaneously to maintain this
property. In the sequential read mode, this common value of the {\sym
LIR}s is used to specify the target module for the
readback operation.

Within each module, the 16-bit {\sym LIR}---used as a target name\footnote
	{Entries 100 or 101 of \protect\eq{selection-codes}.}\relax
	---is compared with the module ID---used as an internal name
(\cf{id-init}).  The comparison is carried out cooperatively by each
module's \team: each \chip{} has a complete copy of the {\sym LIR} and
performs one bit of the comparison. As soon as target name and
internal name match, the selected module sequentially places on the
data lines event-counter data (least significant bits first) for as
many clock cycles as specified by the {\sym Event Counter Length}
(\sect{SFR}). After that, all modules increment their {\sym LIR} in
lockstep, and the next selection is performed.

\endinput


 \def\farm{162}
 \begin{picture}(246,170)(0,-32)
	\thinlines
	\put(\dist,-16){\line(0,1){144}}	%
	\put(\far,-16){\line(0,1){144}}		%
	\put(0,-16){\line(1,0){\dist}}		%
	\put(0,128){\line(1,0){\dist}}
	\put(\far,-16){\line(1,0){\prox}}	%
	\put(\far,128){\line(1,0){\prox}}
	\put(36,132){\makebox(0,0)[b]{\large node}}
	\put(\dist,130){\makebox(0,0)[b]{outer side}}
	\put(210,132){\makebox(0,0)[b]{\large node}}
	\put(\far,130){\makebox(0,0)[b]{inner side}}
	\put(44,-32){\vector(0,1){16}}		%
	\put(202,-32){\vector(0,1){16}}		%
	\put(46,-24){\makebox(0,0)[l]{\sig Clock}}	%
	\put(204,-24){\makebox(0,0)[l]{\sig Clock}}	%

	\put(\zerom,112){\makebox(\dist,0)[r]{\sig Outer Data[0:15]}}
	\put(\farm,112){\makebox(\prox,0)[l]{\sig Inner Data[0:15]}}
	\put(\distm,116){\makebox(\diff,0)[b]{\sig Data[0:15]}}
	\put(\dist,112){\vector(1,0){\diff}}
	\put(\dist,112){\vector(-1,0){0}}
	\put(152,112){\makebox(0,0){/}}
	\put(152,118){\makebox(0,0)[b]{\small 16}}

	\put(\zerom,88){\makebox(\dist,0)[r]{\sig Outbnd.\ Handsh.\ Out}}
	\put(\farm,88){\makebox(\prox,0)[l]{\sig Outbnd.\ Handsh.\ In}}
	\put(\distm,92){\makebox(\diff,0)[b]{\sig Outbnd.\ Handsh.}}
	\put(\dist,88){\vector(1,0){\diff}}

	\put(\zerom,72){\makebox(\dist,0)[r]{\sig Inbnd.\ Handsh.\ In}}
	\put(\farm,72){\makebox(\prox,0)[l]{\sig Inbnd.\ Handsh.\ Out}}
	\put(\distm,76){\makebox(\diff,0)[b]{\sig  Inbnd.\ Handsh.}}
	\put(\far,72){\vector(-1,0){\diff}}

	\put(\zerom,48){\makebox(\dist,0)[r]{\sig Reset Out}}
	\put(\farm,48){\makebox(\prox,0)[l]{\sig Reset In}}
	\put(\distm,52){\makebox(\diff,0)[b]{\sig Reset}}
	\put(\dist,48){\vector(1,0){\diff}}

	\put(\zerom,32){\makebox(\dist,0)[r]{\sig Sync Out}}
	\put(\farm,32){\makebox(\prox,0)[l]{\sig Sync In}}
	\put(\distm,36){\makebox(\diff,0)[b]{\sig Sync}}
	\put(\dist,32){\vector(1,0){\diff}}

	\put(\zerom,16){\makebox(\dist,0)[r]{\sig Frame Out}}
	\put(\farm,16){\makebox(\prox,0)[l]{\sig Frame In}}
	\put(\distm,20){\makebox(\diff,0)[b]{\sig Frame}}
	\put(\dist,16){\vector(1,0){\diff}}

	\put(\zerom,0){\makebox(\dist,0)[r]{\sig Status In}}
	\put(\farm,0){\makebox(\prox,0)[l]{\sig Status Out}}
	\put(\distm,4){\makebox(\diff,0)[b]{\sig Status}}
	\put(\far,0){\vector(-1,0){\diff}}
 \end{picture}
