% -*- Mode:Text -*-

\chapter{Module interconnection and addressing}\label{sect.addressing}


[Write an introduction here.]

\flag[This chapter is under construction]

\section{Module selection}\label{sect.modselect}

During the data phase of a {\em write} frame, it is the \step\ \ctrler\
that talks into the \bus. Data are physically fanned-out by the \bus\
to {\em all} \step{} modules. In most cases, these data are intended
for use by all modules; in certain cases, however, it is desirable
that only certain modules be enabled as active listeners, either on an
individual basis or as members of a certain group of modules.

During the data phase of a {\em read} frame, the \ctrler\ acts as a
listener, and it is essential that no more than one module be selected to
talk into the \bus.

\Step's module selection mechanism, described here, allows each
module to decide whether or not it should take active part in a given
write or read transaction. The unit of selection handled by this
mechanism is an entire module. Finer-grained selection, such as
write-enabling of individual layers within a module, may be achieved by
other methods (cf.\ \sect{VWR}).

\medskip

We have seen that a \step{} machine consists of an arbitrary number of
modules, and each module consists of sixteen layers each served by a
separate \chip. In most respects, the sixteen chips of a set act
independently of one another; they manage to achieve coordinated
action not because they exchange coordinating information between one
another, but because they are separately instructed by the host to
play in lockstep appropriately correlated roles.

However, the sixteen chips of a module act as a unit for the
purpose of deciding whether that module is {\em selected}, i.e., will
accept the data broadcast by the host on the \bus\ during a write
frame, or will put its own data on the \bus\ during a read frame.

\medskip

The decision of whether or not a module is currently selected is based
upon a match between an {\em internal name} and a {\em target name}.
Both of these names are sixteen bits long (one bit per chip).  For
each of the sixteen positions, internal- and target-name bits
are (directly or indirectly) specified by a three-bit {\em selection
code}, which is broadcast by the host and written to the {\sym Module
Select Register} (\sect{MSR}) of all modules.\footnote
	{This register is exceptional in the sense that data addressed to
it are accepted by {\em every} module regardless of the
module's current selection status.}

There are two primary sources available for the internal-name bit. The
{\em module ID}, stored in the {\sym Module ID Register} ({\sym MIDR})
is used to uniquely identify a module, and is assigned once and for
all at initialization time (\sect{id-init}); the {\em group ID},
stored in the {\sym Group ID Register} ({\sym GIDR}) is used for
grouping modules as needed for the convenience of particular
operations, and may be reassigned by the host from time to time. Both
IDs are sixteen bits long, with one bit contributed by each chip.
A third source is the {\sym Glue-AND} bit, used only at initialization
time (\sect{id-init}).

For the target-name bit, the selection code may supply directly a 0, a
1, or a `dont care'; or it may specify that the bit should be taken
from the {\sym LUT Index Register} (\sect{LIR}).
This fourth option is used during a special {\em sequential-read} mode
(\sect{sequential-read}).

\medskip

The following table lists what sources the different selection codes
specify for the internal bit and the target bit\footnote
	{The values 000, 010, and 111, which are probably the most
commonly used, happen to be of a form that can be packed as {\em immediate}
data in a step-list frame (cf.\ \sect{}).}
 \begin{equation}\label{eq.selection-codes}
 \input selection-codes.insert
 \;.
 \end{equation}

Of course, the individual \chip\ will be able to detect a match
between internal name and target name only in one of the sixteeen
bit positions. The overall name match is obtained by {\sc and}ing
together the sixteen partial matches.  This logical operation is
physically performed by a wired-{\sc and} line that ties together all
sixteen chips' {\sym ModSel} pins (\sect{wired-and}), collecting the
arguments of the {\sc and} from the individual chips and returning the
result to all chips. In this way, the individual chip not only gives
its contribution to the selection decision, but is also made aware of
the collective decision's outcome.

The above comparison between internal name and target name is made at
the end of the data phase of each frame (\fig{frame}), and
the result (`selected' or `not selected') remains valid
for the entire duration of the following frame.


\section{Module ID initialization}\label{sect.id-init}

As mentioned in \sect{modselect}, before a \step{} machine can function
properly each module must be given a distinct module ID. This initial
naming task presents a challenge, because there are no intrinsic
features in the architecture that distinguish one module from another.
In fact, (a) the modules are structurally identical and are physically
interconnected in a uniform way (i.e., they form a three-dimensional
torus), and (b) at system reset all modules are set to an identical
internal state.

To break this symmetry between modules, it is sufficient to ``tag''
one module; this is done at power up, as explained in \sect{}.  Once
the symmetry is broken, every module is in principle uniquely
identifiable by its $x$, $y$, and $z$ displacements (in the torus
coordinates) relative to the tagged one. This fact is exploited by the
module ID initialization procedure described below (which the host
will run immediately after power up): starting from the tagged module
and taking advantage of the glue lines that connect modules in the
$x$, $y$, and $z$ directions, this procedure successively steps from
module to module, selecting and uniquely naming each one in turn.

\medskip

What distinguishes the tagged module at power up is that it it is
accessible on the \bus (only the modules in the corner
box---\cf{machine-overview}---are initially accessible) and has a 1
rather than a 0 as the least-significant bit of its module ID.

\memo{Update the following procedure}

By appropriate use of the 000, 010, and 111 selection codes (table
\eq{selection-codes}), the host will select just this module and give
it a name consisting of all 0s ({\tt 0000} in hexadecimal notation),
and then select all the other modules at once and give them, for the
moment, a name consisting of all 1s (hexadecimal {\tt FFFF}).

\medskip

We shall now describe how naming information is propagated through
the array of modules by the initialization procedure.

Let us recall that, for each of the sixteen data layers, each module
has six glue pins which connect it, via as many glue lines, to the
six adjacent modules (\fig{glue-and}a).

 \figonecol{glue-and}{
 \setlength{\unitlength}{0.75pt}
 \def\sig{\small\sym}
 \begin{picture}(144,128)(-72,-64)
	\thicklines
	\put(-40,-16){\line(0,1){48}} % the three vertical sides
	\put(-8,-32){\line(0,1){48}}
	\put(40,-32){\line(0,1){48}}
	\put(-40,32){\line(1,0){48}} % the three horizontal lines
	\put(-8,16){\line(1,0){48}}
	\put(-8,-32){\line(1,0){48}}
	\put(-40,32){\line(2,-1){32}} % the three diagonal lines
	\put(-40,-16){\line(2,-1){32}}
	\put(8,32){\line(2,-1){32}}
	\put(-72,0){\line(1,0){48}} % the y lines
	\put(40,0){\line(1,0){32}}
	\put(0,-64){\line(0,1){32}} % the z lines
	\put(0,24){\line(0,1){40}}
	\put(-64,32){\line(2,-1){24}} % the x lines
	\put(16,-8){\line(2,-1){48}}

	\put(56,-40){\makebox(0,0){\sig XP}}
	\put(-56,40){\makebox(0,0){\sig XM}}
	\put(64,8){\makebox(0,0){\sig YP}}
	\put(-64,8){\makebox(0,0){\sig YM}}
	\put(12,56){\makebox(0,0){\sig XP}}
	\put(12,-60){\makebox(0,0){\sig ZM}}

	\put(-64,-64){\makebox(0,0){(a)}}
 \end{picture}
 \quad
 \begin{picture}(120,144)(-16,-24)
	\thinlines
	\put(16,-24){\line(0,1){128}}
	\put(16,-24){\line(1,0){64}}
	\put(16,104){\line(1,0){64}}
	\thicklines
	\multiput(-16,0)(0,16){6}{\line(1,0){48}}
	\put(32,-4){\line(0,1){88}}
	\put(32,40){\oval(64,88)[r]}
	\put(64,40){\line(1,0){56}}
	\put(0,2){\makebox(0,0)[b]{\sig ZP}}
	\put(0,18){\makebox(0,0)[b]{\sig ZM}}
	\put(0,34){\makebox(0,0)[b]{\sig YP}}
	\put(0,50){\makebox(0,0)[b]{\sig YM}}
	\put(0,66){\makebox(0,0)[b]{\sig XP}}
	\put(0,82){\makebox(0,0)[b]{\sig XM}}

	\put(92,42){\makebox(0,0)[b]{\sig Glue-AND}}
	\put(48,40){\makebox(0,0){\sc and}}

	\put(-8,-16){\makebox(0,0){(b)}}
 \end{picture}}{(a) The six glue lines that connect one module
to its neighboring modules. (b) The module ID initialization sequence
makes use of the {\sc and} of the six glue lines.}

\noindent At power-up, the six glue pins of all modules are programmed
as inputs; these inputs are internally {\sc and}ed together into a
signal call {\sym Glue-AND} (\fig{glue-and}b). Since the glue inputs
are internally pulled up by resistors to logical 1, {\sym Glue-AND}
will have a value of 1.

We'll now turn the {\sym XP} pin of module {\tt 0000}, layer 0, into
an output\footnote
	{By appropriately setting the sign of the kick register;
cf.\ \sect{}.}
	and put a 0 on it. This 0 will travel on a glue line and
reach pin {\sym XM} of an adjacent module, forcing its {\sym Glue-AND}
to 0.  Making appropriate use of selection code 110, corresponding to
{\sym Glue-AND}=0 (cf.\ table \eq{selection-codes}), we can select
just this adjacent module,\footnote
	{With the present naming procedure, the only two modules
having {\sym Glue-AND}=0 are the module that emits 0 and the one that
senses it. But the former will already have been named, while the
latter still has the default name {\tt FFFF}. Since selection code 110
is only used with one of the sixteen chips, the remaining fifteen
name positions can use code 010 to discriminate between the emitting
module and the receiving module. (This scheme does not allow one to
discriminate between the last two modules in a 64K-module machine, but
in this case the last module does not have to be named, since it already
has the correct name {\tt FFFF}.}
	and write {\tt 0001} as its module ID.
At this point we'll ask module {\tt 0000} to turn
pin {\sym XP} of layer 0 back into an input, and module {\tt 0001}
to put a 0 on one of its glue pins, so as to ``touch'' another module.
We will name the new module {\tt 0002}, and so on until all modules
of the \step{} machine have been given unique IDs.

\section{Corner box}\label{sect.corner-box}

Because of its unique position---at the origin of the \cam\ box array
and at the origin of the \bus\ distribution tree---the \cam\ machine's
corner box plays a distinguished role in certain setup and housekeeping
functions.

\medskip

The \cam\ modules come out of power-up with all activity on their {\em
\bus\ branch} ports disabled (see \sect{MPCR}); thus, while the
controller can communicate with the corner box, all other boxes are
cut off from the \bus\ for the time being. Moreover, module 0 of a box
is physically distinguished from the other modules of the same box by
having the {\sym Multipurpose Pin A} of \chip\ 0 pulled down by an
external resistor, so that its module ID at power up will be {\tt
FFFE} rather than {\tt FFFF}.\memo{Norm, confirm this}

Taking advantage of the above features, the host runs an
initialization procedure that systematically assigns module IDs,
starting with {\tt 0000} for module 0 of box 0 (\cf{id-init}). Once
the corner box's privileged geographical position has been recorded in
distingushed IDs for its modules, the other boxes are gradually
brought into communication with the \bus, proceeding outwards from
the corner box.


\section{Block wiring}\sectlabel{block-wiring}

Refer to \sect{} for the definition of \emph{block} and \emph{strand}. In the
present realization of the \cam\ architecture, a module consists of a
\emph{board} having all of its connectors on one edge, as in \fig{board}, and
a block consists of a \emph{box} with a backplane where eight of
these boards can be plugged in.

 \figonecol{board}{
 \setlength{\unitlength}{1.25pt}
 \begin{picture}(184,56)(-16,-8)
	\thicklines
	\put(-8,24){\oval(16,48)[l]}	% left oval side
	\put(160,24){\oval(16,48)[r]}	% left oval side
	\put(-8,48){\line(1,0){168}}	% top edge
	\multiput(0,0)(56,0){3}{\line(0,-1){4}}
	\multiput(16,0)(56,0){3}{\line(0,-1){4}}
	\multiput(24,0)(56,0){3}{\line(0,-1){4}}
	\multiput(40,0)(56,0){3}{\line(0,-1){4}}
	\multiput(-8,0)(56,0){3}{\line(1,0){8}}
	\multiput(16,0)(56,0){3}{\line(1,0){8}}
	\multiput(40,0)(56,0){3}{\line(1,0){8}}
	\multiput(0,-4)(56,0){3}{\line(1,0){16}}
	\multiput(24,-4)(56,0){3}{\line(1,0){16}}
	\put(8,0){\makebox(0,0){$\;x_-$}}
	\put(32,0){\makebox(0,0){$\;x_+$}}
	\put(64,0){\makebox(0,0){$\;y_-$}}
	\put(88,0){\makebox(0,0){$\;y_+$}}
	\put(120,0){\makebox(0,0){$\;z_-$}}
	\put(144,0){\makebox(0,0){$\;z_+$}}
 \end{picture}}
 {\Step{} module board, showing the six glue port connectors.}

\noindent For such a realization of a block, table
\eq{block-ports} can be interpreted as a template for the backplane,
indicating the eight slots (horizontal rows) and, for each slot, the
position of the six glue-port connectors.

Using this template, \fig{naive-wiring} shows in its essentials the
actual wiring between boxes. We only show the wiring for strand 0, and
then only for the $x$ and $y$ dimensions (the wiring for the $z$
dimension would go ``through the page'').

 \figonecol{naive-wiring}{\psfig{file=fig/naive-wiring.ps,height=2.5in}}
{Backplane wiring for an indefinitely extended array of \step{} boxes,
with no wraparound. Four boxes are shown, and only the wiring for
strand 0. The arrow direction goes from a `$+$' port to a `$-$' port
(cf.\ table \protect\eq{block-ports}).}
 \figonecol{naive-wrap}{\psfig{file=fig/naive-wrap.ps,height=2.5in}}{How
strands are connected at the edges of the array to form loops.}

Still using template \eq{block-ports}, \fig{naive-wrap} shows the
actual placement of the shorting plugs (cf.\ \fig{wrap-block}) at the
edges of the array (using an array that is two blocks deep in both
the $x$ and $y$ dimensions; as in \fig{naive-wiring}, the $z$
interconnections are not shown). With these plugs, strand 0 forms a
single loop with strand 1 in the $x$ dimension, and with strand 2 in
the $y$ dimensions (in the $z$ dimension, it would loop back as strand
4).


\subsection{Single-block wiring}

For a small \step{} system consisting of just one block
(eight modules), each dimension exhausts itself within
the block, as indicated by the following diagram for just the $x$
dimension
 $$
 \begin{picture}(124,64)(-112,-44)
	\thicklines
	\multiput(-88,16)(96,0){1}{\line(1,0){32}}
	\multiput(-88,16)(96,0){1}{\line(0,-1){32}}
	\multiput(-88,-16)(96,0){1}{\line(1,0){32}}
	\multiput(-56,16)(96,0){1}{\line(0,-1){32}}
	\multiput(-96,0)(96,0){1}{\line(1,0){8}}
	\multiput(-56,0)(96,0){1}{\line(1,0){56}}

	\multiput(-40,-8)(96,0){1}{\line(1,0){32}}
	\multiput(-40,-8)(96,0){1}{\line(0,-1){32}}
	\multiput(-40,-40)(96,0){1}{\line(1,0){32}}
	\multiput(-8,-8)(96,0){1}{\line(0,-1){32}}
	\multiput(-96,-24)(96,0){1}{\line(1,0){56}}
	\multiput(-8,-24)(96,0){1}{\line(1,0){8}}

	\put(-96,-12){\oval(24,24)[l]}
	\put(0,-12){\oval(24,24)[r]}
	\multiput(-88,-4)(96,0){1}{\makebox(32,8)[tl]{$\;x_-$}}
	\multiput(-88,-4)(96,0){1}{\makebox(32,8)[tr]{$x_+$}}
	\multiput(-40,-28)(96,0){1}{\makebox(32,8)[tl]{$\;x_+$}}
	\multiput(-40,-28)(96,0){1}{\makebox(32,8)[tr]{$x_-$}}
	\thinlines
	\multiput(-92,-44)(96,0){1}{\dashbox{2}(88,64){}}
 \end{picture},
 $$

\noindent and the complete wiring diagram will appear as in
\fig{naive-one-block}.

 \figonecol{naive-one-block}{
 \setlength{\unitlength}{1.5pt}
 \begin{picture}(72,72)(0,0)
	\thicklines
	\multiput(0,8)(24,0){3}{	% the dots
	 	\multiput(8,0)(0,8){8}{\circle*{2}}
		\multiput(16,0)(0,8){8}{\circle*{2}}}
	\multiput(8,8)(0,16){4}{\line(1,1){8}}
	\multiput(8,16)(0,16){4}{\line(1,-1){8}}

	\multiput(32,8)(0,8){2}{\line(1,2){8}}
	\multiput(32,24)(0,8){2}{\line(1,-2){8}}
	\multiput(32,40)(0,8){2}{\line(1,2){8}}
	\multiput(32,56)(0,8){2}{\line(1,-2){8}}

	\multiput(56,8)(0,8){4}{\line(1,4){8}}
	\multiput(56,40)(0,8){4}{\line(1,-4){8}}

	\thinlines
	\put(0,0){\dashbox{2}(72,72){}}
 \end{picture}}
 {Complete backplane wiring for a \step{} system consisting of a single
block.}

\sbox{\toy}{}

\subsection{Block wiring, improved}\sectlabel{block-improved}

Each of the ``wires'' shown in \fig{naive-wiring} or \fig{naive-one-block} is
actually a flat cable consisting of a collection of 16 twisted pairs (one per
layer).  Since such cables are stiff and bulky, it is desirable to route them
as neatly as possible, trying to keep them flat and parallel and avoiding
repeated crossings.

By systematically rearranging the order of modules within a box and the order
of ports within a module it is possible to arrive, for a single block, at the
wiring pattern of \fig{clever-one-block}, to be constrasted with that of
\fig{naive-one-block}.  The improved wiring is arrived at as follows.

\medskip

\parhead{Port swapping} \fig{wrap-block} suggests that, if strand 0
loops back as strand 1 in the $x$ dimension, it would be more
convenient if the modules of strand 1 had their $x_+$, $x_-$ ports
arranged in the opposite order as those of strand 0. More generally,
a similar argument holds for the $u$-dimension ports ($u=x,y,z$) of
any pair of strands whose labels differ only in the $s_u$ digit
(cf.\ \eq{block-ports}).

In order to respond to design requirements of the above kind, in a
\step{} module the correspondence between logical ports ($x_+$,$x_-$,
etc.) and physical ports (the six connectors at the bottom of
\fig{board}) is software assignable rather than hardwired (cf.\
\sect{GPCR}). Taking advantage of this feature, one can
make the two $x$ ports appear on the edge of the board in the order
$x_-,x_+$ or $x_+,x_-$, depending on whether the $s_x$ bit is 0 or 1
(and similarly for $y$ and $z$), leading to the following backplane
template

 \begin{equation}\label{eq.block-ports-better}
 \begin{array}{|cc||c@{}c|c@{}c|c@{}c|}\hline
	\multicolumn{2}{|c||}{\rm strand} & \multicolumn{6}{c|}{{\rm ports}}\\
	s_xs_ys_z& S & \multicolumn{2}{c}{x} & \multicolumn{2}{c}{y}
		& \multicolumn{2}{c|}{z}\\\hline\hline
	000 & 0 & - & + & - & + & - & +\\
	100 & 1 & + & - & - & + & - & +\\
	010 & 2 & - & + & + & - & - & +\\
	110 & 3 & + & - & + & - & - & +\\
	001 & 4 & - & + & - & + & + & -\\
	101 & 5 & + & - & - & + & + & -\\
	011 & 6 & - & + & + & - & + & -\\
	111 & 7 & + & - & + & - & + & -\\\hline
 \end{array}
 \end{equation}

\medskip

\parhead{Strand swapping} Finally, we shall permute the boards
in the backplane's eight slots so that the strand labels $S$
are ordered according to \emph{Gray code} rather than straight
\emph{binary} code:

 \begin{equation}\label{eq.block-ports-best}
 \begin{array}{|cc||c@{}c|c@{}c|c@{}c|}\hline
	\multicolumn{2}{|c||}{\rm strand} & \multicolumn{6}{c|}{{\rm ports}}\\
	s_xs_ys_z& S & \multicolumn{2}{c}{x} & \multicolumn{2}{c}{y}
		& \multicolumn{2}{c|}{z}\\\hline\hline
	000 & 0 & - & + & - & + & - & +\\
	100 & 1 & + & - & - & + & - & +\\
	110 & 3 & + & - & + & - & - & +\\
	010 & 2 & - & + & + & - & - & +\\
	011 & 6 & - & + & + & - & + & -\\
	111 & 7 & + & - & + & - & + & -\\
	101 & 5 & + & - & - & + & + & -\\
	001 & 4 & - & + & - & + & + & -\\\hline
 \end{array}
 \end{equation}

\medskip

It is easy to verify that the port-and-strand arrangement of template
\eq{block-ports-best} indeed leads to the improved backplane wiring
pattern given in \fig{clever-one-block}. Similar methods can be used
to streamline backplane wiring in more complex situations, such as
that of \fig{naive-wrap}.

 \figonecol{clever-one-block}{
 \setlength{\unitlength}{1.5pt}
 \begin{picture}(72,72)(0,0)
	\thicklines
	\multiput(0,8)(24,0){3}{	% the dots
	 	\multiput(8,0)(0,8){8}{\circle*{2}}
		\multiput(16,0)(0,8){8}{\circle*{2}}}
	\multiput(0,8)(0,16){4}{
		\multiput(8,0)(8,0){2}{\line(0,1){8}}}
	\multiput(0,16)(0,32){2}{
		\multiput(32,0)(8,0){2}{\line(0,1){8}}}
	\multiput(0,20)(0,32){2}{
		\multiput(32,0)(8,0){2}{\oval(8,24)[l]}}
	\multiput(56,36)(8,0){2}{\oval(4,24)[l]\oval(8,40)[l]\oval(12,56)[l]}
	\multiput(56,32)(8,0){2}{\line(0,1){8}}
	\thinlines
	\put(0,0){\dashbox{2}(72,72){}}
 \end{picture}}
 {Improved backplane wiring for a \step{} system consisting of a single
block.}

