Download Scalable Coherent Interface.pdf PDF

TitleScalable Coherent Interface.pdf
File Size5.5 MB
Total Pages255
Table of Contents
                            Title page
Copyright Page
Introduction
Participants
CONTENTS
1. Introduction
	1.1 Document structure
	1.2 SCI overview
	1.3 Interconnect topologies
	1.4 Transactions
	1.5 Cache coherence
	1.6 Reliability, availability, and support (RAS)
2. References, glossary, and notation
	2.1 References
	2.2 Conformance levels
	2.3 Glossary
	2.4 Bit and byte ordering
	2.5 Numerical values
	2.6 C code
3. Logical protocols and formats
	3.1 Packet formats
	3.2 Send and echo packet formats
	3.3 Logical packet encodings
	3.4 Transaction types
	3.5 Elastic buffers
	3.6 Bandwidth allocation
	3.7 Queue allocation
	3.8 Transaction errors
	3.9 Transmission errors
	3.10 Address initialization
	3.11 Packet encoding
	3.12 SCI-specific control and status registers
4. Cache-coherence protocols
	4.1 Introduction
	4.2 Coherence update sequences
	4.3 Minimal-set coherence protocols
	4.4 Typical-set coherence protocols
	4.5 Full-set coherence protocols
	4.6 C-code naming conventions
	4.7 Coherent read and write transactions
5. C-code structure
	5.1 Node structure
	5.2 A node's linc component
	5.3 Other node components
6. Physical layers
	6.1 Type 1 module
	6.2 Type 18-DE-500 signals and power control
	6.3 Type 18-DE-500 module extender cable
	6.4 Type 18-DE-500 cable-link
	6.5 Serial interconnection
7. Bibliography
Annex A (Informative) Ringlet initialization
Annex B (Informative) SCI design models
                        
Document Text Contents
Page 1

Recognized as an
American National Standard (ANSI)



IEEE Std 1596-1992



(Adopted by ISO/IEC and redesignated as
ISO/IEC 13961:2000)



IEEE Standard for Scalable Coherent
Interface (SCI)



Sponsor



Microprocessor and Microcomputer Standards Subcommittee
of the
IEEE Computer Society



Approved 19 March 1992



IEEE-SA Standards Board



Adopted by ISO/IEC and redesignated as
ISO/IEC 13961:2000

Authorized licensed use limited to: University of Waterloo. Downloaded on February 20, 2009 at 14:32 from IEEE Xplore. Restrictions apply.

Page 2

Abstract:



The scalable coherent interface (SCI) provides computer-bus-like services but, instead
of a bus, uses a collection of fast point-to-point unidirectional links to provide the far higher through-
put needed for high-performance multiprocessor systems. SCI supports distributed, shared
memory with optional cache coherence for tightly coupled systems, and message-passing for
loosely coupled systems. Initial SCI links are defined at 1 Gbyte/s (16-bit parallel) and 1 Gb/s
(serial). For applications requiring modular packaging, an interchangeable module is specified
along with connector and power. The packets and protocols that implement transactions are
defined and their formal specification is provided in the form of computer programs. In addition to
the usual read-and-write transactions, SCI supports efficient multiprocessor lock transactions. The
distributed cache-coherence protocols are efficient and can recover from an arbitrary number of
transmission failures. SCI protocols ensure forward progress despite multiprocessor conflicts (no
deadlocks or starvation).



Keywords:



bus architecture, bus standard, cache coherence, distributed memory, fiber optic,
interconnect,I/O system, link, mesh, multiprocessor, network, packet protocol, ring, seamless
distributed computer,shared memory, switch, transaction set



The Institute of Electrical and Electronics Engineers, Inc.
3 Park Avenue, New York, NY 10016-5997, USA

Copyright © 2001 by the Institute of Electrical and Electronics Engineers, Inc.
All rights reserved. Published 23 May 2001. Printed in the United States of America.



Print:



ISBN 1-55937-222-2 SH15255



PDF:



ISBN 0-7381-1206-2 SS15255



No part of this publication may be reproduced in any form, in an electronic retrieval system or otherwise, without the prior
written permission of the publisher.

Authorized licensed use limited to: University of Waterloo. Downloaded on February 20, 2009 at 14:32 from IEEE Xplore. Restrictions apply.

Page 127

SCALABLE COHERENT INTERFACE (SCI) IEEE Std 1596-1992

(potential)
. The last

rge UID
lization

ceived,
as

eans not
highest

UIDs.
es of this
ion.

ctions
Figure 3-67 —Reset-closure generates idle symbols

3.10.4.3 NodeId assignment

The distanceId value in each init packet is decremented by each nonscrubber node, then saved as that node's
nodeId value. The decremented distanceId is sent on to the next node as the reset packet is passed on
distanceId value received comes from a reset generated by the scrubber (with nodeId SCRUB_ID), so it correctly sets
all nonscrubber initial nodeId values. A distanceId value of zero indicates an error has occurred (perhaps a la
was erroneously generated and circulated, blocking every node from becoming the scrubber) and the initia
sequence is restarted.

3.10.4.4 Startup

After receiving the first of its own reset packets, the scrubber outputs idle symbols. After an idle symbol is re
the scrubber changes to a running state and injects idle.lg and idle.hg bits into the idle symbols that it generates,
illustrated in figure 3-68.

Figure 3-68 —Idle-closure injects go-bits in idles

3.10.5 Simple-subset ringlet resets

A simplified subset of this general model is also provided: one node on each ringlet may be configured by a m
specified by SCI (perhaps a jumper option) to be the scrubber. This node always considers itself to have the
UID, and it emits RESETH and CLEARH packets that cause the other nodes to consider that they have lower
Thus this node always wins the scrubber competition, even if many other nodes have scrubber capability. Nod
type that are configured not to be scrubbers, and nodes with no scrubber capability, always lose the competit

3.10.6 Ringlet resets

The previous sections illustrated how all nodes participate in the ringlet initialization activity. The following se
illustrate the behavior of state machines in the individual SCI nodes.
Copyright © 1992 IEEE All Rights Reserved 115

Authorized licensed use limited to: University of Waterloo. Downloaded on February 20, 2009 at 14:32 from IEEE Xplore. Restrictions apply.

Page 128

IEEE Std 1596-1992 IEEE STANDARD FOR

e 64-bit
e status

larger
nd saving
-69.

ing 80-
Ringlet-local initialization begins when primary power is turned on. Each node generates a (presumably) uniqu
random number and concatenates this after a 16-bit value provided by nonvolatile storage (or the backplan
signals, if nonvolatile storage is not provided) to form its UID. The node then sets its phase bit ph to zero, and sends a
reset packet (RESETL0) that contains this UID and the distanceId value SCRUB_ID, followed by sync (training)
packets.

A node continues sending its reset and sync packets (state reset) waiting for its own RESETL0 packet to be returned.
If its own RESETL0 packet is observed, the node changes its state (to winning) and outputs idle symbols. If a
effective UID value is observed, the node changes state (to losing) and forwards reset packets (decrementing a
their distanceId values) until an idle is received at its input, as illustrated in the reset state diagram of figure 3

Figure 3-69 —Initialization states

The UID comparison is performed as follows: Nodes with scrubber-competition capability compare the incom
bit UID with their own UID. If the incoming effective UID is greater than their own UID, the comparison is greater
than. Nodes that have no scrubber capability or that are configured not to be scrubbers always generate greater than.
Nodes that are configured to always be the scrubber generate less than until they receive a RESETH or CLEARH,
whereupon they generate equal.
116 Copyright © 1992 IEEE All Rights Reserved

Authorized licensed use limited to: University of Waterloo. Downloaded on February 20, 2009 at 14:32 from IEEE Xplore. Restrictions apply.

Page 254

IEEE Std 1596-1992 IEEE STANDARD FOR

aligned
uired. The
kaside

as QOLB,
, which is
consumer
/* Listing B-4; BoundedAdd4() illustration */
Quadlet
BoundedAdd4 (Quadlet *addr, Quadlet data, Quadlet test)
{ Quadlet register *rAddr, rOld, rTest, rData, rNew;
rAddr= addr; /* Put address in egister */
rTest= test; /* Fetch value for test */
rData= data; /* Fetch data for ddition */
SetLock (addr); /* Load line and lock */
rOld= Load4 (rAddr); /* Load old data value */
rNew= rOld+rData; /* Compute new data alue */
if (rOld!=rTest) /* compare old to bound */
Store4 (rAddr, rNew); /* and store if not qual */
Unlock (addr); /* Unlock cache line */
return (Rold); /* Return old data value */
}

/* Listing B-5: WrapAdd4() illustration */
Quadlet
WrapAdd4 (Quadlet *addr, Quadlet data, Quadlet test)
{ Quadlet register *rAddr, rOld, rTest, rData, rNew;
rAddr= addr; /* Put address in register */
rTest= test; /* Fetch value for test */
rData= data; /* Fetch data for ddition */
SetLock (addr); /* Load line and lock */
rOld= Load4 (rAddr ); /* Load old data value */
rNew= rOld+rData; /* Compute new data alue */
if (rOld!=rTest); /* compare old to bound */
Store4 (rAddr, rNew); /* store sum if not qual */
else
Store4 (rAddr, rData); /* else store argument alue */
Unlock (addr); /* Unlock cache line */
return (rOld); /* Return old data value */
}

To avoid instruction-cache faults, compilers are expected to align the SetLock() instructions on 16-byte-
addresses. To avoid cache faults, separate instruction and data caches or two-way associative caches are req
same effective two-way associativity is also required for the pagetranslation tables (called translation loo
buffers, or TLBs).

B.4 Coherence-performance models

B.4.1 Nonblocking message queues

Software-managed nonblocking message queues represent an alternative to hardware queue protocols (such
Queue on Lock Bit). Messages may be produced by any of several producers that share a message queue
serviced by a single consumer. One or more producers insert messages at the tail of the queue and one
extracts messages from the head of the queue. These messageflow structures are illustrated in Figure B-4.
242 Copyright © 1992 IEEE All Rights Reserved

Authorized licensed use limited to: University of Waterloo. Downloaded on February 20, 2009 at 14:32 from IEEE Xplore. Restrictions apply.

Page 255

SCALABLE COHERENT INTERFACE (SCI) IEEE Std 1596-1992

L value.
essage-
pdated.
essage-

s from the
ueue, as

o queued.
entry is

dequeue
eue and
Figure B-4 —Enqueuing messages

Before inserting a new message-queue entry (M1), the producer sets (1) the entry's forward pointer to the NUL
Using the swap operation (a special case of mask&swap), the tail pointer is updated (2) to point to the new m
queue entry. The enqueue operation is completed (3) when the forward pointer in the previous tail entry is u
Note that swap combining in the interconnect (see 1.5.4.2) could be used to improve the efficiency of the m
enqueue sequence.

Messages may be consumed (in a nonblocking fashion) by one consumer. The consumer deletes message
head of the queue. For a stable multi-entry list, reads and writes are sufficient to remove entries from the q
illustrated in figure B-5.

The address of the head (4) is read by the consumer. The first entry is checked (5), to see if there are others als
If others are queued, the head pointer is updated to point to the next queued entry and the previously-first
processed.

The removal of the last entry is more complex; one or two compare&swap operations are required to safely
the first entry while others are being added to the list. The C code illustrates more precisely how these enqu
dequeue routines, EnqueueEntry () and DequeueEntry (), could be implemented.

Figure B-5 —Dequeuing messages
Copyright © 1992 IEEE All Rights Reserved 243

Authorized licensed use limited to: University of Waterloo. Downloaded on February 20, 2009 at 14:32 from IEEE Xplore. Restrictions apply.

Similer Documents