A Hierarchical Multistage Interconnection Network

Transcription

1 A Herarchcal Multstage Itercoecto Networ Mohtar Aboelaze Dept. of Computer Scece Yor Uversty Toroto, ON. CANADA M3J P3 Kashf Al Dept. of Computer Scece Yor Uversty Toroto, ON. CANADA M3J P3 ABSTRACT Multstage tercoecto etwors are used may applcatos ragg from coectg processors to memory modules a parallel processg system, to hgh-speed etwor swtches ad routers. Oe of the major drawbacs multstage etwors s the lac of proxmty cocept. All odes are at the same dstace from ay other ode. I ths paper, we propose a ew herarchcal Multstage Networ (HMN) based o the mult-stage etwor. The ew etwor requres less hardware (slco area) tha the etwor, ad ejoys the same ease of routg as the etwor. Sce the HMN s herarchcal ature, ot all the odes are at the same dstace from ay other ode. The dstace betwee two odes that belog to the same cluster are less tha two odes two dfferet clusters. That maes the HMN sutable for applcatos where there are preferece amog the odes. We also troduce smple ad effcet algorthm for routg the HMN ad we compare the performace of the HMN to the etwor. Keywords Itercoecto etwors, multstage etwors, exteded bary cube etwor, etwor.. INTRODUCTION Multstage tercoecto etwors are used extesvely multprocessor systems [], [5]ad, hgh-speed etwor swtches[], [4], [9]. I ths paper, we propose a ew herarchcal tercoecto etwor called herarchcal multstage etwor (HMN). The HMN requres less ls ad swtchg elemets tha a comparable sze hypercube etwor. Aother advatage of the HMN etwor s that t s herarchcal etwor, whch maes t sutable for applcato that are clustered ature such as may dgtal sgal processg applcatos. Multstage etwors have the advatages of a costat ode degree. Usually the etwor s bult from smaller x swtchg elemets. These swtchg elemets are arraged log N, where N s the umber of puts (ad outputs) the swtch, ad s a small teger usually (f the swtchg elemet s a x swtch). Oe of the ma dsadvatages of the etwor s that there s a fxed dstace betwee ay two odes For bary swtches the dstace betwee ay two odes s log N, where N s the umber of odes. That meas there s o cocept of localty or two odes close to each other. That s sutable for systems that requre uform commucato. However, systems where there s clusterg, t s much more advatageous to use a etwor that has a cocept of localty. I ths paper, we propose a ew tercoecto etwor, the herarchcal multstage etwor HMN. The HMN etwor s sutable very large systems. It retas the ease of routg ad broadcastg ejoyed by the etwor, but t requres much less umber of ls ad much less swtchg elemets tha a comparable sze etwor. The HMN s also sutable for clustered systems where odes commucate wth a small subset of odes much more tha they commucate wth other odes the system. Aother advatage of the HMN etwor s that the dstace betwee two odes the same cluster (measured as the umber of stages to cross) s less tha the dstace a comparable sze etwor. We aalyze the HMN etwor ad calculate the average dstace betwee two odes uder two dfferet modes of commucato, we also troduce effcet routg algorthms for HMN.

2 The orgazato of ths paper as follows: I chapter we troduce ad defe the proposed etwor, Secto 3 states some of the propertes of the etwor. Routg the proposed etwor s troduced secto 4. We dscuss our smulato results secto 5, Secto 6 s a cocluso of our wor. 7 0 Networ. HMN Before descrbg the HMN, let us descrbe the Networ. The etwor (also equvalet to the drect bary cube) s a N x N etwor wth =log N stages. Each stage cossts of N/ x crossbar swtches [6] [7]. Oe of the ma advatages of the etwor (baya etwors geeral) s the ease of routg. Routg s completely dstrbuted. Swtches the dfferet stages are cofgured ether as straght or cross accordg to the bary represetato of the source ad destato. There s o eed for a cetral cotroller, whe the message reaches the th stage, the swtch exame the th bt the source ad destato addresses, accordg to these two bts, the swtch s set as ether straght or cross. Before we start the formal defto of the HMN, we wll expla a specal case of a levels HMN <3,>. The -level HMN wll be used to llustrate the dea of the HMN, the the formal defto of the HMN wll be troduced. A HMN of heght s defed as {, }-HMN wth puts ad outputs where = +. Iput ad output odes are umbered from 0 to -, or equvaletly, odes ca be represeted as s=<s, s > where 0. The {, }-HMN s s costructed by coectg together etwors each s a x etwor. The coecto s made va a x etwor such that ode 00 0 each x s a put to the x etwor. The output of the x s fed bac to oe of the dfferet put 00 0 each module. etwors va Fgure shows a <3,>-HCN where 4 etwors are coected together va a 4 x 4 etwor such that ode 000 (0) each etwor s the put to the 4 x 4 etwor. The output of the 4 x 4 etwor s fed bac to the puts of the etwor va ode 000 (0) each etwor.. Fgure <3,> HMN To llustrate the operato of such a etwor, f ode 0 seds a message to ode 5. Both of these two odes are the same frst stage etwor. The message s drected to ode 5 gog through 3 stages oly the 8x8 module. However, f the message s set from ode 0 to 8. The t s routed from put 0 to output 0 the frst module. The the 4x4 etwor drects t to output the 4 x 4 etwor. That output f feedbac to put 6 the stage (put 0 the thrd 8-by-8 module stage ). From ode 6 t s routed to ode 8 thus passg through (3++3=8 stages). Now, we ca formally defe the HMN. Defto: {, -,... }-HMN of heght wth N put ad N output odes where N =, s costructed by coectg N by etwor coected together va the puts of a etwor etwor etwor Level Level N N by HMN, where N =. = =

3 Aother way to loo at the HMN s as follows: stage there are modules, ( = N j ) each s a j = s a by N etwor. Every module at stage s a root of a tree wth modules at stage +, ad so o. That meas that two leaf odes share a root at stage f ad oly ff they share a th stage address, ad all the stages less tha A - 0 M U X 0 by Networ DE MU X To stage - To output A Stage A s N N a x etwor, m =, = = A m Fgure K-level HMN N x N HMN I a mathematcal form that could be expresses as follows: Lemma: Two odes F=<F, F -,, F > ad G=<G, G -,... G > share a module at stage as a root ff F + G +, ad F j = G j for j.. - feedbac paths Fgure 3 Iput ad output (de)multplexers at stage Fgure 3 shows a module the th stage. Each module the th stage s a by etwor. There are puts to ths module ad up to - feedbac paths from the prevous - stages. A multplexer s used to coect the - feedbac paths ad put 0 of the etwor to put 0 of the module. The multplexer s set to ether forward a pacet from the put of the HMN or from oe of the feedbac paths. Buffers are used to store pacets f the put to the multplexer s more tha oe pacet. Output 0 of the module s coected to a demultplexer. The two outputs of the demultplexer are ether the output of the HMN, or the put to stage -. Depedg o f the pacet s set to the output, or t set to stage - order to be routed bac to oe of the other th stage modules, the demultplexer s set accordgly. For modules stages,,..,- there s oly a output demultplexer at the output ode 0 of each module. to determe f the message wll be forwarded from stage to stage - or fedbac to stage. 3. HMN Propertes I ths secto, we wll dscuss some of the propertes of the HMN, ad ts cost. 3. Cost Oe measure of the cost of a etwor s the umber of x swtchg elemets. The umber of swtchg

4 elemets a {, -,... }-HMN ca be recursvely calculated as follows: The umber of swtchg elemets up to ad cludg these stage s equal to the umber of swtchg elemets up to ad cludg stage - plus the umber of swtchg elemets the th stage. By defto, at the th stage, j = j there are modules each s a by etwors. By usg ths defto, ad the fact that the umber of swtchg elemets at the frst stage s, we ca recursvely calculate the umber of swtchg elemets a -stage HMN. The followg equato descrbes how to calculate the umber of swtchg elemets a -stage HMN j j= C ( ) = C( ) + wth C () = ad the total umber os swtchg elemets a -stage HMN s C(). 3. Node umberg The odes {,, }-HMN are umbered as bary umbers usg = + + bts from 0 to The bts are grouped felds correspodg the levels of the HMN. Aother way to umber the odes s as a -tuple <F, F,,F > where 0 F < For example ode 7 Fgure ca be represeted as <0,7> or bary <000>, where the left-most bts represets t locato the x st stage whle the rght-most three bts represet ts posto the 8 x 8 etwor the secod stage 3.3 Dstace betwee odes I HMN, we defe the dstace betwee two odes to be the umber of stages to be crossed order to travel from the source ode to the destato ode. Clearly f the two odes belog to the same cluster, the dstace s less tha f the two odes belog to a dfferet cluster. I order to calculate the dstace betwee two ode F ad G, Assume the followg. F=<F, F,, F > ad G=<G, G,... G > The followg procedure could be used to calculate the dstace betwee F ad G procedure calculate _ dst( F, G) d= for(=i<;++) { f (F G) { } } d = d + j j= Nodes Orgazato #of swtchg elemets ; F=G} <3> 3 Average dstace betwee odes <,> <,> <,,> <5> 80 5 <,3> <,,> <,,,,> 3 8 <0) 50 0 <5,5> <3,3,4> <,,,,> 03 8 Table The cost ad the average dstace betwee two odes HMN Table shows the cost ad the average dstace betwee two odes dfferet szes HMN ad for dfferet cofgurato for every sze. The cost s represeted as the umber of -by- swtchg elemets. The dstace as stated before s the umber of stages to be crosses to reach from the source to the destato. Note that Table we cosdered a uform commucato where ay ode commucates wth ay other ode wth the same probablty. As we metoed before, HMN s sutable for clustered commucato where odes commucate wth other odes the same cluster wth a much hgher probablty tha wth odes outsde the cluster. That

5 of course teds to reduce the average dstace betwee the odes. Table shows two cases from the oes metoed Table whe the commucato s clustered. For every ode, p s the probablty of sedg a message to ay ode the same cluster, ad -p s the probablty of a message beg set to ay ode outsde the cluster. We ca see that whe p creases the average dstace betwee odes decreases eve gog below that of a etwor. P <5,5> <3,3,4> Table average dstace betwee two odes for dfferet values of p 4. Routg Routg the HMN s completely dstrbuted ad the oly formato eeded by ay swtch s the destato address oly. A routg tag s attached to each message. The tag cossts of + bts. N bts for the destato address, ad oe bt that we call the forward_bt. The use of the routg bt s explaed the ext secto. Frst, we wll expla the dea of the routg algorthm, ad the we formally preset t. For the th stage modules, the module has to chec f the destato the same module or ot. If t s the same module, t wll be routed to the fal destato. Else t wll be routed to ode 0 ths module wth a forward_bt set to dcate that ode 0 wll set the demultplexer to forward t to stage -. For modules stages to -, the module checs f the destato belog to the tree rooted at ths module, f yes, the there s o eed to forward the message ay further up the tree ad the message s set to the approprate output wth the forward_bt reset to dcate that the output multplexer wll feedbac ths message to stage, else the message s set to ode 0 wth the forward_bt set to dcate that would be forwarded up the tree to the ext stage. The routg tag each pacet cossts of the destato address dvded to felds, where the th feld cotas G ad oe extra bt forward_bt for a total of + bts. As show Fgure 3 Fgure 3:The routg tag HMN The forward_bt s used by the demultplexer after ode 0 each stage. If that bt s set, the pacet arrvg at output 0 of ths module s set to the lower output (that meas the ext stage). If the forward_bt s reset, the pacet s set to the upper output of the demultplexer (that meas the feedbac to stage f the stage s stage, - ad s set to the output f stage. We preset algorthm s a pseudo code format for the routg. We assume that there s already a fucto called route(a,) that routes the message from the curret ode to ode a - etwor. We assume that the routg s based o the routg tag whch clude bts arraged felds <G, G,.. G > ad a forward_bt. We also assume that the curret address of the ode s <F, F,.. F > The routg algorthm depeds o the stage, for the put stage (stage ) the routg algorthm s as follows Procedure routek(g, G -,.. G ) // Routg for ode stage f (G = F ) for all such that - The Else F F - F {Reset forward_bt; route(g, )} {Set forward_bt; route(0, )} For odes stages =,,..-, the routg procedure s as follows Procedure route(g, G -,.. G,) // Routg for stages =,,,- Forward_bt f (G j = F j ) for all j such that j - The

6 Else {Reset( forward_bt); route(g, )} {Set( forward_bt); route(0, )} 5. Smulato We used smulato to smulate our proposed etwor. We used CSIM [8] to smulate HMN wth 8,6, odes wth the followg cofgurato <3>, <,>, <4>, <,>, <3,> <,3>. We assume a sychrozed system where each ode seds a pacet every cycle wth a fxed probablty p., we also assumed a clustered system where each pacet s set to a ode the same cluster wth a probablty p c ad to ode outsde the cluster wth a probablty -p c. Our results dcates that as log as we are ot overloadg ode 0 each module, the delay s 0% more tha the delay for a smlar etwor, ad as you ca see Table the cost s much less tha the cost of a comparable sze etwor. 6. Cocluso I ths paper we preseted a herarchcal multstage tercoecto etwor. The proposed etwor ejoys the same ease of routg used the etwor ad has a less umber of swtchg elemets tha a comparable sze etwor. Our smulato ad aalytcal results show that the performace of the proposed system (both terms of the average dstace betwee odes, ad total delay) s better tha the etwor for clustered systems, ad slghtly worse tha the etwor for o-clustered systems. 7. Refereces [] Aboelaze, M; Maour, A; Elaggar, A; Dyamc cell allocato to put queues a combed I/O buffered ATM swtch, Proceedgs of Iteret computg 00, [] Bhuya, L.N.; Iyer, R.R.; Asar, T.; Nada, A.K.; Kumar, M Performace of multstage bus etwors for a dstrbuted shared memory multprocessor IEEE Trasactos o Parallel ad Dstrbuted Systems, Vol 8, No., Ja. 997 pp 8-95 [3] Chaey, T. Fgerhut, J. Fluce, M. ad Turer J. desg of a ggabt ATM swtch Proc. IEEE INFOCOM, Aprl 997. [4] Moro, H.; Bao, T T; Hoaso, N; Ada, H.; Sato, T. A scalable multstage pacet swtch for terabt IP router based o deflecto routg ad shortest path routg, IEEE Iteratoal Coferece o Commucatos, 00, vol. 4, pp [5] Omra, R.A.; Aboelaze, M.A., A effcet sgle copy cache coherece protocol for multprocessors wth multstage tercoecto etwors, Scalable Hgh-Performace Computg Coferece, 994, pp -8 [6] Segel, H. J. Hsu, W. T.-Y, ad Jeg, M. A troducto to the multstage cube famly of tercoecto etwors, The Joural of Supercomputg, Vol., No., pp 3-4, 987 [7] Segel, H.J. Itroducto to etwors for large scale parallel processg, McGraw-Hll 990. [8] Schwetma, H. CSIM User s Gude Mesqute Software Ic Aust TX 998. [9] Yag, Y, ad Wag J. A class of multstage coferece swtchg etwors for group commucato, Iteratoal coferece o parallel processg, 00 pp