Pactical Pftching Tchniqus fo Paalll Fil Systms David Kotz Cala Schlatt Ellis Dpt. of Math and Comput Scinc Dpt. of Comput Scinc Datmouth Collg Duk Univsity Hanov, NH 03755-3551 Duham, NC 7706 David.Kotz@Datmouth.du cala@cs.duk.du Abstact Impovmnts in th pocssing spd of multipocssos a outpacing impovmnts in th spd of disk hadwa. Paalll disk I/O subsystms hav bn poposd as on way to clos th gap btwn pocsso and disk spds. In a pvious pap w showd that pftching and caching hav th potntial to dliv th pfomanc bnts of paalll l systms to paalll applications. In this pap w dscib xpimnts with pactical pftching policis, and show that pftching can b implmntd cintly vn fo th mo complx paalll l accss pattns. W also tst th ability of ths policis acoss a ang of achitctual paamts. 1 Intoduction As computs gow mo powful, it bcoms incasingly dicult to povid sucint I/O bandwidth to kp thm unning at full spd fo lag poblms, which may consum immns amounts of data. Disk I/O has always bn slow than pocssing spd, and cnt tnds hav shown that impovmnts in th spd of disk hadwa a not kping up with th incasing aw spd of pocssos. This widning accss-tim gap is known as th I/O cisis [14, 0]. Th poblm is compoundd in typical paalll achitctus that multiply th pocssing and mmoy capacity without balancing th I/O capabilitis. Th most pomising solution to th I/O cisis is to xtnd paalllism into th I/O subsystm. On such appoach is to connct many disks to th comput in paalll, spading individual ls acoss all disks. Paalll disks could povid a signicant boost in pfomanc possibly qual to th dg of paalllism, if th a no signicant bottlncks in th I/O subsystm, and if th I/O qusts gnatd by applications can b mappd into low-lvl opations that div th availabl paalllism. Thus, th st challng to th dsigns of a multipocsso l systm is to congu paalll disk hadwa to avoid bottlncks (.g., shad busss), and to avoid futh bottlncks in th systm softwa. An ctiv l This sach was suppotd in pat by NSF gants CCR- 871781 and CCR-881809 and DARPA/NASA subcontact of NCC-560. systm fo a multipocsso must itslf b fully paalll to scal with additional pocssos o disks. Th scond challng is to mak this xtnsiv disk hadwa bandwidth asily availabl to application pogams. To mt ths challngs w popos a highly paalll l systm implmntation that incopoats caching and pftching as a mans of dliving th bnts of a paalll I/O achitctu though to th us pogams. W xpct a l cach to b usful in multipocsso l systms fo th sam ason as in unipocsso l systms: locality in l fnc bhavio. Indd, w xpct multipocsso l accss pattns to hav incasd oppotunitis fo locality. Intpocss locality can ais whn all pocsss in a multi-pocss pogam ad th sam l in som coodinatd fashion (.g., ach ading dint small cods fom th sam block). If th l accss pattn is squntial, th l systm can ad blocks into th cach bfo thy a qustd, making thm quickly availabl whn thy a qustd. This xtnsion to caching is known as pftching. Pftching dos not wok fo all accss pattns, of cous, but it should b bncial fo common squntial pattns. In [9], w showd that pftching has signicant potntial to impov ad pfomanc in multipocsso l systms. W masud th potntial using an idalistic pftching policy that was povidd with th complt l accss pattn in advanc. In pactic, of cous, th pftching policy dos not hav accss to th l accss pattn in advanc, and instad must bas its pftching dcisions on a al-tim viw of th accss pattn. This lads to sval qustions: Givn that w know pftching has potntial, is it possibl to dsign and implmnt pactical pftching policis? A pactical policy must b both ctiv, choosing th coct blocks to pftch, and cint, having low ovhad. This qustion is th pimay focus of this pap. Can ou pactical policis achiv thi full potntial, as dtmind in [9] by ou unalizabl \full-knowldg" policy? Can w dsign gnal policis that a pactical fo many dint typs of accss pattns? 1 Copyight 1991 by IEEE. Appad in Conf. on Paalll and Distibutd Infomation Systms, pags 18-189. Availabl at URL ftp://ftp.cs.datmouth.du/pub/cs-paps/kotz/kotz:pactical.ps.z
Do th pftching policis and implmntation scal wll, givn mo pocssos, mo disks, o a wid gap btwn pocsso spd and disk accss spd? To answ ths qustions, w usd th tstbd dvlopd fo [9]. Th tstbd implmntd many pftching and caching policis on a al multipocsso, and simulatd th paalll disk I/O. W valuatd many pftching policis on a wid vaity of wokloads and achitctual paamts. In th nxt sction w povid mo backgound infomation. In Sction 3 w dscib th tstbd, th wokload, and th xpimntal mthods. Sction 4 dns ou pactical pftching policis. In Sction 5 w psnt th xpimnts, pfomanc masus, and sults. Sction 6 concluds. Backgound Much of th pvious wok in I/O hadwa paalllism has involvd disk stiping. In this tchniqu, a l is intlavd acoss numous disks and accssd in paalll to simultanously obtain many blocks of th l with th positioning ovhad of on block [16, 7, 14]. All of ths schms ly on a singl contoll to manag all of th disks. Fo multipocssos, on fom of paalll disk achitctu is basd on th notion of paalll, indpndnt disks, using multipl convntional disk dvics addssd indpndntly and attachd to spaat pocssos. Th ls may b intlavd ov th disks, but th multipl contolls and indpndnt accss to th disks mak this tchniqu dint fom disk stiping. Exampls of this I/O achitctu includ th Concunt Fil Systm [15, 6] fo th Intl ipsc/ multipocsso, and th Bidg l systm [4, 3] fo th BBN Butty multipocsso. Caching commonly-usd disk blocks can signicantly impov l systm pfomanc [0], and indd is a tchniqu usd in most modn l systms. Pftching is also succssful in unipocsso l systms [0, 18, 19, 17]. Th cntal ida bhind pftching is to ovlap som of th I/O tim with computation by issuing disk opations bfo thy a qustd. With paalll disk hadwa, howv, w xpct pftching to also ovlap I/O with I/O, obtaining vn lag bnts. Fil accss pattns hav nv bn studid fo paalll computs, but hav bn studid xtnsivly fo unipocssos [5, 1]. Floyd [5] studid l accss pattns in a Unix systm, and found that 68% of ls opnd fo ading a compltly ad, usually squntially. Ov 90% of all ls opnd a opnd ad-only o wit-only. A classic Unix l systm study [1] found that 90% of all ls a pocssd squntially, ith though th whol l (70% of all accsss) o aft only on sk. Paalll l accss is discussd by Cocktt []. Although h did not study an actual wokload, h latd l accss pattns to possibl stoag tchniqus. Many of his basic l accss pattns a ctd in ou wokload modl. W concntat on scintic wokloads, chaactizd by squntial accss to lag ls [13, 11]. Dspit th lack of any paalll l accss study, w xpct th to b nough squntial accss in th paalll l accss pattns of scintic applications fo pftching policis that assum squntial accss to b succssful. 3 Modls and Mthods Ou mthodology is xpimntal, using a mix of implmntation and simulation. W implmntd a l systm tstbd calld RAPID-Tansit (\Rad- Ahad fo Paalll Indpndnt Disks") on an actual multipocsso. Sinc th multipocsso dos not hav paalll disks, thy a simulatd. Unfotunatly, fw paalll pogams us paalll I/O and so w did not hav accss to a al wokload. Thus, w w focd to us a synthtic wokload. Th synthtic wokload captus such nuancs of al wokloads as squntiality, gulaity, and int-pocss intactions. It consists of al paalll pogams that gnat l qusts and may incu synchonization dlays. Th tstbd xcuts th synthtic application, masuing th lapsd al tim and oth signicant statistics. This implmntation of th policis on a al paalll pocsso, combind with al-tim xcution and masumnt, allows us to dictly includ th cts of mmoy contntion, synchonization ovhad, int-pocss dpndncis, and oth ovhad, as thy a causd by ou wokload und vaious managmnt policis. This mthod allows us to valuat whth pactical pftching policis can b implmntd. 3.1 Modls and Assumptions Achitctu: Th achitctu on which w bas ou sach ots is a multipl instuction stam, multipl data stam (MIMD) shad-mmoy multipocsso. A subst of th poblms and many of ou poposd solutions (although not ou implmntation) may apply to mssag-passing achitctus as wll. W psnt th disk subsystm with paalll, indpndnt disks. W assum an intlavd mapping of ls to disks, with blocks of th l allocatd oundobin to all disks in th systm. Th l systm handls th mapping tanspantly, managing th disks and all qusts fo I/O. Th is a l systm manag unning on ach pocsso. This spads th I/O ovhad ov all pocssos and allows th us of all pocssos fo computation, ath than sving a st of pocssos xclusivly fo I/O. Wokload: Paalll l systms and th applications that us thm a not sucintly matu fo us to know what accss pattns might b typical. Paalll applications may us pattns that a mo complx than thos usd by unipocss vsions of th sam application. W wok with l accss pattns, ath than disk accss pattns. That is, w xamin th pattn of accss to logical blocks of th l ath than physical blocks on th disk. Th l accss pattn is th bst plac to look fo squntiality, sinc disk accss pattns a complicatd by th layout of logical blocks on th disk and by th activitis of multipl ls. Thus
w mak no assumptions of disk layout. Not also that th application is accssing cods in th l, which a tanslatd into accsss to logical l blocks by th intfac to th l systm. Th l systm intnals, which a sponsibl fo caching and pftching, s only th block accss pattn. In ou sach w do not invstigat ad/wit l accss pattns, bcaus most ls a opnd fo ith ading o witing, with fw ls updatd [5, 1]. W xpct this to b spcially tu fo th lag ls usd in scintic applications. This pap covs adonly pattns, whas wit-only pattns a covd in [10, 8]. All squntial pattns consist of a squnc of accsss to squntial potions. A potion is som numb of contiguous blocks in th l. Not that th whol l may b considd on lag potion. Th accsss to this potion may b squntial whn viwd fom a local pspctiv, in which a singl pocss accsss succssiv blocks of th potion. W call ths locally squntial accss pattns, o just local accss pattns. This is th taditional notion of squntial accss usd in unipocsso l systms. Altnativly, th pattn of accsss may only look squntial fom a global pspctiv, in which many pocsss sha accss to th potion, ading disjoint blocks of th potion. W call ths globally squntial accss pattns, o just global accss pattns. In this viw ach pocss may b accssing blocks within th potion in som andom o gula, but incasing od. If th fnc stings of all th pocsss a mgd with spct to tim, th accsss follow a (oughly) squntial pattn. Th pattn may not b stictly squntial du to th slight vaiations in th global oding of th accsss; it is this vaiation that maks global pattns mo dicult to dtct. In addition, th lngth of potions (in blocks) may b gula, so th l systm could pdict th nd of a potion and not pftch past it. Th dinc btwn th last block of on potion and th st of th nxt may also b gula (a gula skip), allowing th systm to pftch th st blocks of th nxt potion. W us ight psntativ paalll l accss pattns. Fou of ths a local pattns, th a global pattns, and on is andom. lw Local Whol l: vy pocss ads th nti l fom bginning to nd. It is a spcial cas of a local squntial pattn with a singl potion. lfp Local Fixd-lngth Potions: ach pocss ads many squntial potions. Th squntial potions hav gula lngth and skip, although at dint placs in th l fo ach pocss. lp Local Random Potions: lik lfp, but using potions of igula (andom) lngth and skip. Potions may ovlap by coincidnc. sg Sgmntd: th l is dividd into a st of nonovlapping contiguous sgmnts, on p pocss. Each pocss thus has on squntial potion. gw Global Whol l: th nti l is ad fom bginning to nd. Th pocssos ad distinct cods fom th l in a slf-schduld od, so that globally th nti l is ad xactly onc. gfp Global Fixd-lngth Potions: (analogous to lfp) pocssos coopat to ad what appas globally to b squntial potions of xd lngth and skip. gp Global Random Potions: (analogous to lp) pocssos coopat to globally ad squntial potions with andom lngth and skip. nd Random: cods a accssd at andom. This psnts all pattns that a too complx to b psntd as squntial in any way. Not that ths pattns a not ncssaily psntativ of th distibution of th accss pattns actually usd by applications. W fl that this st covs th ang of pattns likly to b usd by scintic applications. 3. Mthods Th RAPID-Tansit tstbd is a paalll pogam implmntd on a BBN GP1000 Butty paalll pocsso [1]. Th tstbd is havily paamtizd, and incopoats th synthtic wokload, th l systm, and a st of simulatd disks. Th l systm allocats and manags a bu cach to hold disk blocks. S [8] fo dtails. Pftching is attmptd whnv th pocsso is idl. Assuming a commonly usd pocsso-allocation statgy of on pocsso fo ach us pocss [1], th pocsso bcoms idl whnv its assignd pocss is idl, usually waiting fo disk activity o synchonization to complt. To dcid on a block to pftch, th pftching modul calls a pdicto, which ncapsulats a paticula policy, a pattn-pdiction huistic. Th pdicto maks its pdictions basd on th obsvd fnc histoy of th application. Th bas fo all of ou valuations of pftching policis is th simpl NONE policy, which is quivalnt to not pftching. W also us an o-lin pdicto calld EXACT, which is povidd with th nti accss pattn in advanc. (This is th appoach usd in [9].) Th advanc knowldg maks it a pfct pdicto, sinc it maks no mistaks and quis littl ovhad. Howv it is not alistic, sinc a al pdicto dos not know th nti accss pattn in advanc. In this sns, EXACT givs us a ough upp bound on th potntial of pftching. (EXACT dos hav som limitations, howv: in th lp and gp pattns, it dos not pftch past th nd of a potion until a dmand ftch has stablishd th location of th nxt squntial potion, and in th nd pattn, EXACT dos no pftching, sinc non is asonably possibl.) W us ths two simpl pdictos to valuat ou on-lin pdictos, dscibd blow. 4 Pactical Pdictos Ou statgy is to bgin with a coas compaison of many pdictos on all th pattns, fo a lativly 3
limitd st of paamts. Thn w valuat th most gnally pactical pdictos on a wid ang of paamts, xamining th scalability of th pdictos to oth achitctual situations. W bgin with pdictos fo local pattns, thn consid global pattns. 4.1 Local Pattn Pdictos W psnt fou pdictos that a dsignd fo pdicting local accss pattns. Th fouth is a hybid of th st th simpl pdictos. Ths pdictos monito th individual pocss fnc pattns, looking fo squntial accss. Sinc th pocss fnc pattns a indpndnt, ths pdictos a totally concunt. OBL On-Block Look-ahad: This algoithm (as in [0]) always pdicts block i 1 aft block i is fncd, and no mo. IBL Innit-Block Look-ahad: IBL pdicts that i ; i 3; : : : will follow a fnc to i, and commnds that thy all b pftchd in that od. Whth thy a actually pftchd dpnds on th cuntly availabl soucs. IBL is a logical xtnsion of OBL, and is dsignd fo th lw and sg pattns. PORT Potion Rcognition: This algoithm attmpts to cogniz squntial potions. Essntially, PORT tis to handl th lfp accss-pattn family. It watchs fo a gula potion lngth and gula potion skip. Lik IBL, it tis to pdict th pattn futh ahad than th nxt fnc, in od to pftch mo blocks. Unlik IBL, howv, it limits th numb of blocks that it pdicts into th futu (to limit mistaks), and it may also jump potion skips (if th potions a gula). In andom pattns (shot potions with igula skip) PORT pdicts nothing. IOPORT IBL/OBL/PORT: This pdicto is a hybid of th oth th, attmpting to combin th bst of ach. It bgins as IBL, to tat lw and sg pattns cintly, but switchs to OBL on th st non-squntial fnc. Th consvativ OBL is mo appopiat whn th pattn has unxpctd non-squntial accsss. If gula potions a dtctd, thn PORT is usd. 4. Global Pattn Pdictos To cogniz and pdict globally squntial pattns at untim is mo dicult. Th pdicto must collct and xamin th global fnc histoy by mging local fnc histois. Evn thn it is dif- cult to cogniz squntial accss, sinc th blocks in th pattn may b fncd in only a oughly squntial od du to vaiations in pocss spd. In addition, cint, concunt implmntations a dicult du to th nd fo global dcision making. To dtmin th impotanc of th tado btwn accuacy and cincy, w compa a highly accuat (but incint) pdicto with a lss accuat (but cint) pdicto. Both pdictos a concunt, in that sval pocssos may b activ simultanously, with intnal synchonization contolling accss to shad stat infomation. Th st, calld GAPS, woks had to dtct squntiality in th global accss pattn bfo doing any pftching. Th scond, calld RGAPS, assums that th pattn is squntial unlss it appas andom. Dtcting andom accss is much simpl and mo concunt, although lss accuat, than dtcting squntial accss. Onc thy dcid to pftch, both pdictos tack all accsss and pftchs, and suggst blocks fo pftching that hav not yt bn ftchd. In this mod thy a capabl of cognizing squntial potions, much lik PORT, with unxpctd non-squntial accsss quiing -valuation of th pattn. S [8] fo dtails on ths pdictos. 5 Expimnts W bgin with som dtails of ou xpimnts and masus, thn giv sults fom xpimnts that compa th pactical pdictos against EXACT and NONE. Finally, w valuat th scalability of th most gnal pdictos. 5.1 Expimntal Paamts In all of ou xpimnts, w x most of th paamts and thn vay on o two paamts at a tim. Th paamts dscibd h a th bas fom which w mak oth vaiations. Each combination of paamts psnts on tst cas. Th w 0 pocsss unning on 0 pocssos. W gnatd a st of accss pattns to b usd by all pdictos, including EXACT and NONE. Th pattns all containd xactly 4000 cod accsss, wh th cod siz was on block. Th block siz was 1 KByt. In local pattns this was dividd up as 00 fncs p pocss. Not that in most pattns this tanslats to 4000 blocks ad fom th disk, but in lw only 00 distinct blocks a ad sinc all pocsss ad th sam st of 00 blocks. Th cach containd 80 on-block bus. Aft ach cod was ad, dlay was addd in som tsts to simulat computation; this dlay was xponntially distibutd with a man of 30 msc. All oth tsts had no dlay aft ach ad, simulating an I/O-intnsiv pocss. Th l was intlavd ov 0 disks, at th ganulaity of a singl block. Disk qusts w quud in th appopiat disk quu. Th disk svic tim was simulatd using a constant aticial dlay of 30 msc, a asonabl appoximation of th avag accss tim in cunt tchnology fo small, inxpnsiv disk divs of th kind that might b plicatd in lag numbs. 5. Masus Th RAPID-Tansit tstbd cods many statistics intndd to masu and intpt th pfomanc of pftching. Th pimay pfomanc mtic fo masuing th pfomanc of an application is th total xcution tim. This, and all tim masus in th tstbd, is al tim, including all foms of ovhad. W also cod th avag tim to ad a block, th total synchonization tim, th cach hit atio, pftch ovhad, and many oths. In [9] w found that masus such as cach hit at and avag block 4
ad tim a impovd with pftching, but a not good indicatos of ovall pfomanc. Total xcution tim incopoats thos masus as wll as oth cts, such as synchonization dlays, and thus it is th bst masu of ovall pfomanc. A not on th data: Evy data point in ach xpimnt psnts th avag of v tials. Th cocint of vaiation (cv) is th standad dviation dividd by th man (avag). Fo all xpimnts in this pap, th cv was lss than 0.09 (usually much lss), maning that th standad dviation ov v tials was lss than 9% of th man. In many placs w giv th maximum cv fo a givn data st. Nomalizd Pfomanc: Du to limitd data spac w cannot psnt all of th xpimntal data (but s [8]). Instad, w us a summaizing masu. Sinc EXACT psnts th potntial fo pftching pfomanc, w valuat ou on-lin pdictos in tms of thi lativ pfomanc to EXACT. Ou masu is th nomalizd pfomanc, th ability of th on-lin pdicto to impov on NONE compad to EXACT's ability to impov on NONE. Thus, if t is th xcution tim fo EXACT, t n is th tim fo NONE, and t is th tim fo som oth pdicto, th nomalizd pfomanc of this oth pdicto is np = t?tn t?tn if t t 1 othwis In th nomal cas t t, so th nomalizd pfomanc is 1 whn th pdicto in qustion dos as wll as EXACT, zo whn it dos only as wll as NONE, and ngativ whn slow than NONE. If both EX- ACT and th on-lin pdicto a slow than NONE, th nomalizd pfomanc may also b gat than 1. Thus, it is bst to hav a nomalizd pfomanc na 1. Th cas t < t is considd an anomaly, sinc an on-lin pdicto should not un fast than EXACT (although it did somtims happn fo subtl asons [8]). W assign ths cass a nomalizd pfomanc of 1, sinc thy hav ctainly achd th full potntial of EXACT. Th nomalizd pfomanc is undnd fo th nd pattn, in which t t n. Th Idal Excution Tim: W also compa th xpimntal xcution tim to a simpl modl of th idal xcution tim. Th total xcution tim is a combination of th computation tim, th I/O tim, and ovhad. In th idal situation, th is no ovhad, and ith all of th I/O is ovlappd by computation o all of th computation is ovlappd by I/O. Thus, th idal xcution tim is simply th maximum of th I/O tim and th computation tim. This assums that th wokload is vnly dividd among th disks and pocssos and that th disks a pfctly utilizd. No al xcution of th pogam can b fast than th idal xcution tim. With th bas paamt valus, both th I/O and computation tims a 6 sconds, and thus th idal xcution tim is also 6 sconds. Th idal I/O tim fo lw is shot, only 0.3 sconds, sinc it only ads 00 blocks fom disk. 5.3 Rsults fo Local Pattn Pdictos W masud th pfomanc of th local pattn pdictos on th synthtic wokload, using th xpimntal paamts dnd in Sction 5.1, and vaying th pattn, pdicto, synchonization styl, and computation (ith som computation o no computation), ach vaiation foming a dint tst cas. Th pimay masu was total xcution tim, summaizd with th nomalizd-pfomanc mtic. Figu 1 plots th distibution of nomalizd pfomanc that ach pdicto achivd ov th st of tst cass, in th fom of a cumulativ distibution function (CDF). Rcall that th dsid nomalizd pfomanc is 1.0, indicating that th on-lin pdicto pfomd as wll as EXACT. IBL's xtm ngativ and positiv valus indicat that it was much slow than EXACT in som cass. OBL had lativly fw valus na on. IOPORT had th bst minimum valu, with only two ngativ points, and was within 5% of EXACT's pfomanc in ov half of all tst cass. In th nd pattn, which is not includd in Figu 1, PORT and IOPORT w within % of th xcution tim fo EXACT (NONE) in all tst cass. Thy cognizd th andom pattn as an igula st of on-block potions, and did no pftching. OBL and IBL, howv, pftchd blindly, unning up to 3.5 tims slow than NONE. Thus, IOPORT is a good gnal-pupos local pdicto: xcllnt pfomanc most of th tim, mdioc pfomanc som of th tim, and nv any tibl pfomanc. All of th abov xpimnts usd a on-block cod siz. With non-intgal cod sizs (i.., not a multipl of th block siz), som blocks a fncd. All of th abov pdictos handl such fncs by ignoing thm, and thus th pfomanc did not vay much with th cod siz (w xpimntd with IOPORT fo cod sizs vaying fom on-quat block to 10 blocks). Fo small cods (lss than on block) th ovhad of th fncs was nough to slow down xcution by a fw pcnt in som cass (NONE was th most actd, slowing down by 8% in on cas). 5.4 Rsults fo Global Pattn Pdictos Using a st of tsts simila to thos fo local pdictos, xcpt using global pattns, w masud th pfomanc of GAPS and RGAPS on th synthtic wokload. W plot th CDFs of th distibutions of th nomalizd pfomanc in Figu. Th lowpfomanc (ngativ) cass w all fom th gp pattn, wh GAPS and RGAPS w slow than NONE. In gnal, howv, half of th GAPS cass achd at last 0.6 nomalizd pfomanc (i.., 6% of th pfomanc impovmnt of EXACT), and half of th RGAPS cass achd at last 0.71 nomalizd pfomanc. In th nd pattn, which is not includd in Figu, GAPS and RGAPS w both within % of th th EXACT (NONE) tim, which is ssntially no dinc. Thus, thy both handld andom pattns cintly. All of th abov xpimnts usd a on-block cod siz. With long cods (multipl blocks), 5
CDF 1 0.8 0.6 0.4 Local Pdictos IBL OBL PORT IOPORT Full potntial 0. IBL to -18 0?4:0?3:0?:0?1:0 0:0 1:0 nomalizd pfomanc CDF 1 0.8 0.6 0.4 0. GAPS to -6 Global Pdictos GAPS RGAPS Full potntial c c c c c c cc c c cc c cc c cc c cc c cc c cc c cc c cc 0?4:0?3:0?:0?1:0 0:0 1:0 nomalizd pfomanc c Figu 1: Th nomalizd pfomanc fo th local pdictos on all pattns xcpt nd. A nomalizd pfomanc of 1.0 indicats that th pdicto matchd (o xcdd) EXACT's pfomanc, and a ngativ (o lag positiv) numb indicats that it was slow than NONE. IBL's ang was -18 to 0. (Total xcution tim cv < 0:058.) it bcam mo dicult to dtct squntiality in th block accss pattn. GAPS, in fact, faild fo cods lag than fou blocks, and an up to 10 tims slow than without pftching, bcaus of its faild ots to cogniz th squntiality. RGAPS had littl dif- culty with vaying cod siz, closly following EX- ACT's pfomanc. Thus RGAPS was a mo gnally succssful pdicto than GAPS. 5.5 Scalability Onc w knw that IOPORT and RGAPS w asonably gnal and succssful pdictos fo th vaious accss pattns in ou wokload, w valuatd thi pacticality acoss a wid ang of achitctual vaiations. In paticula, w vaid th numb of pocssos, th numb of disks, and th atio of pocsso spd to disk spd. W giv a sampl of th sults h, along with th ky conclusions; s [8] fo a full psntation. Numb of pocssos: W vaid th numb of pocssos to tst th scalability of th l systm softwa, including th pdictos. By holding th numb of disks constant at 0, this also allowd us to study th cts of having mo o fw pocssos than disks, sinc th pcding xpimnts always had 0 pocssos and 0 disks. (Essntially th sam conclusions w found whn holding th numb of pocssos at 0 and vaying th numb of disks fom 1 to 35.) Th total amount of wok (blocks ad, com- Figu : Th nomalizd pfomanc fo GAPS and RGAPS on all pattns xcpt nd. (Total xcution tim cv < 0:07.) putation tim) was also hld constant. Th idal xcution tim was thn max(6; C ) sconds, wh C p was th total computation tim in sconds, and p was th numb of pocssos. W usd ith C = 0 o C = 10 sconds, as bfo. Figu 3 shows th sults fo th lfp pattn with computation, fo vaious numbs of pocssos. Th idal xcution tim dcasd with mo pocssos until, limitd by I/O, it lvld o to 6 sconds at 0 pocssos. EXACT followd this cuv closly, and IOPORT naly matchd EXACT (nomalizd pfomanc 0.86{0.96 thoughout). NONE was much slow, paticulaly fo fw pocssos. NONE could not us mo disks than it had pocssos, so it was unabl to us th full paalll disk bandwidth o to ovlap computation and I/O. This gaph shows that pftching succssfully ovlappd computation and I/O, and scald wll (at last up to 3 pocssos). Th sults fo oth pattns with computation w simila (using RGAPS instad of IOPORT in global pattns). Figu 4 shows th sults fo th I/O-bound gfp pattn. Th idal xcution tim is a constant 6 sconds. NONE could not us mo disks than it had pocssos, and thus could not us th full paalll disk bandwidth. Howv, pftching was abl to us all of th disk bandwidth with only a fw pocssos. Th sults fo gw, lfp, and sg w simila. Pftching had mo diculty in th gp and lp pattns, though still fast than not pftching fo lss than 0 pocssos. In th lw pattn, NONE was limitd to on disk at a tim, gadlss of th numb of pocssos, whil pftching usd all of th disks. 6
Total Tim (sc) lfp with computation 50 NONE EXACT 00 IOPORT idal 150 100 50 0 1 5 10 15 0 5 30 Numb of pocssos Total Tim (sc) I/O-bound gfp 140 NONE 10 EXACT RGAPS 100 idal 80 60 40 0 0 1 5 10 15 0 5 30 35 Numb of pocssos Figu 3: Pocssos vaiation. (cv < 0:016) Whn th w mo pocssos than disks, NONE was somtims slightly fast than all oth pdictos. At this point th paalllism alon was nough to kp th disks occupid, whas pftching quid mo ovhad fo th sam task, and also mad mistaks. Sinc w xpct that most multipocssos will (and do) hav mo pocssos than disks, this is somwhat of a ngativ sult. Howv, th small slowdown causd by pftching whn th w mo pocssos than disks is a small pic to pay fo th many oth cass wh pftching had signicant bnts (.g., small cod sizs, fw pocssos than disks, th lw pattn, o unbalancd disk loads). In all, th IOPORT and RGAPS pdictos w pactical acoss th vaiation of th numb of pocssos (th is not nough vidnc to xtapolat RGAPS's scalability past 34 pocssos). Thy had paticulaly good pfomanc whn th w fw pocssos than disks, and only slightly ngativ pfomanc in som cass whn th w mo pocssos than disks. In any application, th bottlnck will limit pfomanc, so fo high pfomanc both th numb of pocssos and th numb of disks must b incasd, with th xact atio dpnding on th xpctd accss pattns and computational loads. Disk accss tim: It is xpctd that both pocsso spd and disk spd will incas with tim, but that th incas in pocsso spd will outstip any incass in disk spd, making disks appa slow to pocssos than thy a today. W w not abl to chang th pocsso spd, sinc w w using a singl typ of machin, but (sinc th disks w simulatd) w could asily chang th disk accss tim. Thus w could tst th bhavio of pftching as th accss-tim gap changd. Figu 4: Pocssos vaiation. (cv < 0:050) As an xampl, Figu 5 plots th total xcution tim fo gfp as a function of th disk accss tim. Th idal xcution tim is lina in th disk accss tim, sinc this pattn contains no computation. EX- ACT followd th idal cuv, and th oths at last matchd its slop xcpt fo th fastst disks, indicating only a constant ovhad. With fast disks lativ to th pocsso spd (an unlikly occunc givn achitctual tnds), RGAPS bok down and bcam slow than NONE. This is bcaus th bnts of pftching w ducd with th dcasd disk accss tim, but th costs of pftching (a function of pocsso spd) w unchangd. Fo slow disks, th succss of pftching scald dictly with th disk accss tim. Thus, as th accss-tim gap widns, pftching should continu to b bncial. Simila conclusions w achd fo oth pattns. 6 Conclusion W psnt a pactical pdicto fo gnal-pupos local-pattn wokloads (IOPORT), and a pactical pdicto fo gnal-pupos global-pattn wokloads (RGAPS). Th two pdictos w abl to impov on th non-pftching tim in many cass. In th fw cass wh thi pftching was not bncial, th sulting pfomanc loss was mino. Thy w makably succssful at aching th potntial fo pftching, as dtmind with th EXACT pdicto and oiginally potd in [9]. In addition, w found that ths pdictos w obust acoss vaiations in achitctual paamts, such as th numb of disks, numb of pocssos, and disk accss tim. Ths a impotant considations, bcaus w xpct to s an incasing gap btwn pocsso spd and disk accss tim, and w xpct to s machins with mo pocssos and mo disks. 7
Total Tim (sc) Disk accss tim vaiation fo gfp 11 10 9 8 7 6 5 4 3 NONE EXACT RGAPS idal 10 0 30 40 50 Disk accss tim (msc) Figu 5: Disk-accss tim vaiation. (cv < 0:087) Rfncs [1] BBN Advancd Computs. Butty Poducts Ovviw, 1987. [] Thomas W. Cocktt. Fil concpts fo paalll I/O. In Pocdings of Supcomputing '89, pags 574{579, 1989. [3] Pt Dibbl, Michal Scott, and Cala Ellis. Bidg: A high-pfomanc l systm fo paalll pocssos. In Pocdings of th Eighth Intnational Confnc on Distibutd Comput Systms, pags 154{161, Jun 1988. [4] Pt C. Dibbl. A Paalll Intlavd Fil Systm. PhD thsis, Univsity of Rochst, Mach 1990. [5] Rick Floyd. Shot-tm l fnc pattns in a UNIX nvionmnt. Tchnical Rpot 177, Dpt. of Comput Scinc, Univ. of Rochst, Mach 1986. [6] Jams C. Fnch, Tnc W. Patt, and Miganka Das. Pfomanc masumnt of a paalll input/output systm fo th Intl ipsc/ hypcub. Pocdings of th 1991 ACM Sigmtics Confnc on Masumnt and Modling of Comput Systms, pags 178{187, 1991. [7] Michll Y. Kim. Synchonizd disk intlaving. IEEE Tansactions on Computs, C-35(11):978{ 988, Novmb 1986. [8] David Kotz. Pftching and Caching Tchniqus in Fil Systms fo MIMD Multipocssos. PhD thsis, Duk Univsity, Apil 1991. Availabl as tchnical pot CS-1991-016. [9] David Kotz and Cala Schlatt Ellis. Pftching in l systms fo MIMD multipocssos. IEEE Tansactions on Paalll and Distibutd Systms, 1():18{30, Apil 1990. [10] David Kotz and Cala Schlatt Ellis. Caching and witback policis in paalll l systms. In 1991 IEEE Symposium on Paalll and Distibutd Pocssing, Dcmb 1991. [11] Ethan Mill. Input/Output bhavio of supcomputing applications. Tchnical Rpot UCB/CSD 91/616, Univsity of Califonia, Bkly, 1991. Submittd to Supcomputing '91. [1] John Ousthout, Hv Da Costa, David Haison, John Kunz, Mik Kupf, and Jams Thompson. A tac divn analysis of th UNIX 4. BSD l systm. In Pocdings of th Tnth ACM Symposium on Opating Systms Pincipls, pags 15{4, Dcmb 1985. [13] John Ousthout and Fd Douglis. Bating th I/O bottlnck: A cas fo log-stuctud l systms. ACM Opating Systms Rviw, 3(1):11{ 8, Januay 1989. [14] David Pattson, Gath Gibson, and Randy Katz. A cas fo dundant aays of inxpnsiv disks (RAID). In ACM SIGMOD Confnc, pags 109{116, Jun 1988. [15] Paul Pic. A concunt l systm fo a highly paalll mass stoag systm. In Fouth Confnc on Hypcub Concunt Computs and Applications, pags 155{160, 1989. [16] Knnth Salm and Hcto Gacia-Molina. Disk stiping. In IEEE 1986 Confnc on Data Engining, pags 336{34, 1986. [17] Alan Jay Smith. Squntial pogam pftching in mmoy hiachis. IEEE Comput, pags 7{1, Dcmb 1978. [18] Alan Jay Smith. Squntiality and pftching in databas systms. ACM Tansactions on Databas Systms, 3(3):3{47, Sptmb 1978. [19] Alan Jay Smith. Cach mmois. Computing Suvys, 14(3):473{530, Sptmb 198. [0] Alan Jay Smith. Disk cach-miss atio analysis and dsign considations. ACM Tansactions on Comput Systms, 3(3):161{03, August 1985. [1] Andw Tuck and Anoop Gupta. Pocss contol and schduling issus fo multipogammd shad-mmoy multipocssos. In Pocdings of th Twlfth ACM Symposium on Opating Systms Pincipls, pags 159{166, Dcmb 1989. 8