CONTROLLER ARRAY MODULE
8
Figure 2 Plan of Mem OI:J! System
Owing to the l i m i ts of the <:xist i n g technol ogy, howeve r, the i n i ti a l m a c h i n e was i ntro duced with 3 2 MB for the 8800 and 8 7 0 0 sys tems, and 2 0 MB for the 8 5 0 0 and 8 5 5 0 systems. The 3 2 M B c o n f i g u r a t i o n c o n s i s ts of e i g h t 4 MB modu les w i t h 2 5 6K MOS dyn a m i c RAMs packaged in DIPs . To increase the density of the machi ne without using a d i fferent semiconduc tor technol ogy , a 2 MB d a u g h ter m o d u l e was developed after the initial announcement. This module uses double-sided surface-mount tech nology and plastic leadless chip carriers. Eight of these daughter modu l es are mou n ted o n a mother module to produce a 1 6MB array mod u l e . T h i s n e w m o d u l e h a s i n c re a s e d t h e machine's memory to 1 28MB for the 8800 and 8 7 0 0 systems, and to 80MB for the 8 5 5 0 and 8 5 0 0 systems.
Memory System Architecture
As shown i n Figures 1 and 2 , the m emory con trol l e r c o m m u n i ca tes w i t h the CPUs and the N B IAs over the memory interconnect , called the N M I b u s . C o m m a n d s , a d d re s s e s , a n d d a ta requests are a l l first received by the N M I i nter face and then passed to other sections of th<: m e m ory c o n t ro l l e r . Add resses and d a ta a rc srored i n custom m u l ti part RAMs, where eight locations arc reserved for addresses and eight for d a t a . The N M I i n t e rfa c e e n codes c o m m a n d informatio n , passing i t t o the command-control portion of the memory control ler.
Si nce the m e mory contro l l e r c o m m u n i cates with the N M I bus and the a rray bus, the N M I
Digital Technical journal No. 4 Febntary 1')87
protOcol has to be changed to that of the array bus. Reads and writes of data fi elds with various sizes are received by the N M I interface . The N M I b u s su pports a very robust s e t of c o m m a n d s . Reads and i n terlocked reads are su pported for longwords ( 4 bytes ) , octawords (4 longwords) , and hexworcls ( 2 octawords) . Masked writes and masked-write u n locks are supported for long word s , quadwords ( 8 bytes) , and octawords. Wri tes a re supported for longwords and acta words.
The r e a d - i n t e r l o c ked a n d m a s k e d - w r i t e u n lock commands are used r o i mplement VAX i ns t ru c t i o n s i n w h i c h m u t u a l e x c l u s i o n i s requ i red . For exa m p l e , t h e VAX i nstru c t ions A D AW J , B B C C I , B B S S J , I N S Q H I , I N S Q T I , I NSQUE, REMQHI , and REMQTI a l l need these c o m m a n d s . S i n c e a n i n terlocked i n structi o n locks t h e entire m e mory system , t h e i nterlock bit must reside i n the m emory controller. This bit restricts the execution of subsequent i n ter lock commands unti l the lock has been released by a masked-write u n lock i nstruction.
Aft e r re c e i v i n g a m e m o ry r e q u est fro m a nexus, the memory controller must transfer that req uest to the a ppropriate array modu l e . This transfer i s a c c o m p l ished using the a rray bus . This bus consists of
• A unid irectiona l set of command and address l ines from the memory control ler ro the array mod u l es
• Another unidirectional set of data l ines from the memory control ler to the array modules
5 3
The Memory System in the VAX 8800 Fam ily
• A set of data l ines (capable of assum i ng three states) that can be driven by any one of the array modules and recei ved by the memory control ler
• Various status and control l i nes that commu n icate in both directions
The a rray bus has a m i n i ma l repertoi re of commands, consisting of longword reads , acta word reads , and longword writes, but not hex word reads. Si nce the N M I su pports hexword reads, the memory controller must convert t hem i nto two octaword reads and then send them to the array modu les. Thus the two octawords of a hexword read can reside on d ifferent array mod u les. That fact i ncreases the memory bandwi dth because para llel accesses can be executed . The array bus supports only longword writes; t here fore, octaword writes must also be converted . As mentioned earlier, the array bus has one l ine for commands and addresses and another for data . Therefore, an octaword write , which takes five cycles to transfer on the N M I (one for the com mand , four for the data) , can be transm i tted i n five cycles o n the array bus to an array modu le. Figure 3 shows the corresponding actions dur ing each cycle on the N M I and on the array bus.
In addition to commands, the memory system must also execute maintenance tasks, i ncluding m em o ry refre s h , error report i n g , a nd battery backup .
Since physical memory is i m p lemented w i th MOS dynam ic RAMs , every array row m ust be
NMI ARRAY B U S COMMAND/ ADDRESS LINE DATA L I N E COMMAND OR ADDRESS CYCLE 2 3 DATA DATA COMMAND OR ADDRESS
refreshed once every 4 m i l liseconds . This func t i o n can be done by refreshi ng one row every
1 4 m icroseconds . To faci li tate this activity, the memory control ler sends signals to each a rray module from a 1 4 -m icrosecond osc i llator. Upon receiving a refresh signal , an array module w i l l h a n d l e t h e refresh arbitration and execute the operation .
Occasionally, a b i t w i l l be lost due to e ither alpha particles or a device fai lu re. In that case the memory controller must handle those errors and other types i n a gracefu l m a n n e r . To do that, the m e mory system uses a 7 -bit modified h a m m i n g code to g e n erate the E C C , w h i c h al lows a l l single-bit errors to be corrected and a l l dou ble-bit errors to be detected . After cor recting each error the memory system logs the error's physica l page add ress and the b i t . The memory system then i n terrupts the CPU to cal l a n error serv i ce rout i n e , w h i c h l ogs i n a VMS file the necessary information to i solate the fai l ure . The memory system can also i nterrupt the CPU to handle i nternal parity errors and i n ter locked t i me-outs. An i nterlocked ti me-out hap pens when a nexus executes a read i nterlock but never issues a masked-write unlock. The system software can enable or disable these i nterrupts.
Battery backu p , standard equipment on both t h e 8 8 0 0 a n d 8 7 0 0 syste m s , c a n power t h e refresh operation w h e n t h e system is down . That power a llows the memory system to continue to refresh the RAMs so that data w i l l not be lost . Note that the entire system is not backed up;
4 5 6 7
DATA DATA
COMMAND COMMAND COMMAND
OR OR OR
ADDRESS ADDRESS ADDRESS
DATA DATA DATA DATA
Figure 3 Cycles on NM! Bus and A rray Bus
5 4 Digital Technical Journal
BUS ENABLE ERROR CORRECTION LOGIC T A A M U LTIPORT RAM N M I MEMORY CONTROLLER ECC G E N ERATION LOGIC ARRAY MODULE
Figure 4 Datapaths in Memory Co ntroller and A rray Modules
therefore, a l l components must be in qu iescent states before the memory system enters battery mode. U pon sensing t hat power is erodi ng, the
8800 wi I I write a l l i ts data to the memory sys· tern . The memory control ler wi l l then complete all commands and send signals w t he array mod· ules i n form i n g them to enter battery mode. I n this mode o n ly five MSI c h i ps on the memory control ler and approx i mately half the control logic on the array module will be active .
Com mand Execu tion
The execution of any command received by the mem ory system is a j o i n t effor t between t h e memory controller and t h e array modules. Fig· ure 4 depicts the datapath in each memory com ponent. After a nexus places a command on the NMJ bus, the interface in the memory controller ascertains if the command is a val id memory ref· erence and, i f so, decodes i t . The i n terface then pl aces the command i n a q ueue of commands wai ting to be executed .
Si nce one array modu le can execute m u l t i ple write commands s i m u l taneously, and since m u l t i p l e array modu les c a n also execute commands, the memory control ler must m a intai n the status of the array modu les . The status control l ogic to
Digital Technical journal No. 4 February 1 987
monitor actiVIty must " remember" which par· tions of w h i c h a rrays a re " bu sy . " T h i s statll s control logic can best b e described b y showing how t he three basic operations, writes, reads, and masked writes, are executed .
Write Com mands
For a write command , the contro l portion of t he memory controller performs only three actions: it determines the capabi l ity of the array module to accept the command, it sends the command , and it wa i ts for the array mod u l e to signal i ts readiness to receive a nother comman d .
The write datapath is that portion o f t h e l ogic responsible for the flow of data from the NMI bus tO the array modules. This path comprises both e lectrical interconnects (buses and cables) and a considerable amount of logic . The major storage element for the data path is a 9-bit by 3 2 -location custom m u l tipart RAM ( MPR) with two ports for reads and two for writes. Data received from the NMI bus is p laced i n the next avai lable location of the MPR. Upon determini ng that the requ ired array module is ava i lable, the control logic sends the data from the M P R to that array module over t he array bus. Each array mod u l e ho lds the data u n t i l i t is s t r o b e d i n t o t h e d y n a m i c R A M s
5 5
The Memory Svstem in the VAX 8800 Family
( D RAMs) . The array module can load four long words of data with their associated ECC bits on four consecutive cycles.
Some writes are cal led masked because there is a 4 -bi t byte mask associated with each data word . The byte mask i n forms the memory sys
tem as to whi c h bytes arc to be written . The memory system executes this command by first doing a read and correcting a ny s ingle-bit errors that may exist . It then merges the memory data with the data received from the N M I bus, and fi n a l l y does a wri te command . This sequence easi l y a l l ows t he i mp lementation of longword and octaword masked writes. Masked writes for quadwords (8 bytes) are executed by perform ing an octaword masked write i n whic h the data of two of the longwords remains u ncha nged . Read Commands
For read commands, the memory controJler per forms fou r actions: it determi nes i f the selected array m od u l e is ready to a c c e p t t h e rea d , i t sends the com m a nd , i t wa i ts for a data-ready response, and i t transfers the data from the array module. I mbedded in the command field of the read are address b i ts that select the longword of the octaword that is req u i red first . This action a l l ows w r a p p e d r e a d s t o b e i m p l e m e n te d . (Wrapped reads are described later i n the sec tion " Impact of the Cache . " )
The react cla tapath origi n a tes a t t h e D RA M , wh ich sends the requested data . As in the case of wri te commands, each array m od u l e stores an octaworcl of read data. Once the data has been loaded i n to the l atches, the array module signals to the memory contro l ler that the data is ready. As mentioned earl ier, the read datapath between the array module and the memory controller is tristatabl e . Therefore , the memory control ler must ensure t h a t o n ly one array modu l e a t a t i m e drives t h i s d a t a pa t h . Once t h e d a ta has been requested by t he memory contro l ler, the array module must send the longwords sequen tially, beg i n ni n g with the starting aclclress t hat was sent with the command. This action a l lows the memory controller to request any one of the four longwords as the first to be read. The array modul e portion of the read data path can transfer one longword of data during every cycle.
The error-correction logic in the memory con troller receives each longworcl of data plus the seven ECC b i ts . This logic detects s i ngle- and double-bit errors, but only single-bit errors can
56
be correcte d . A sign i ficant feature of this pro cess is that error detection and correction is per formed as the read data is p i pelined through the memory control J er . Thus no a cl cl i t i on a l cycles are needed to correct read data .
Masked-write Com mands
The execution of a masked write i nvolves both a react and a write sequence. The m e mory con trol ler executes a masked-wr i te com mand by first iss u i ng a react to the selected array modul e . Assuming that there were no memory errors, the data r e t u r n e e! is s e n t to the M P R , where t h e bytes arc merged w i t h those sent t o the memory controller over the N M I bus . The memory con tro l l e r must ensure t h a t no commands to the same array come between t h e read and write portions of a m asked wri te . After all the bytes have b e e n m e rged i n to t h e d a t a b u ffe r , t h e memory contro l l er w i l l wri te the d a ta t o t h e array modu le. The array module then generates new ECC data , adds i t to t h e other data, and strobes the composite data i nto the D RAMs .
If a single-bit error is detected , the process is qu ite simi lar to the one with no errors, except that the data must be corrected . Since corrected data and N M I traffic both share the same data