Approximate Dimension Equalization in Vector-based Information Retrieval

Transcription

1 Approximat Dimnsion qualization in Vctor-basd Information Rtrival Fan Jiang Dpartmnt of Computr Scinc, Duk Univrsity, Durham, NC USA ichal L. Littman AT&T Labs Rsarch, Florham Park, NJ USA Dpartmnt of Computr Scinc, Duk Univrsity, Durham, NC USA Abstract Vctor-basd information rtrival mthods such as th vctor spac modl (VS), latnt smantic indxing (LSI), and th gnralizd vctor spac modl (GVS) rprsnt both quris and documnts by high-dimnsional vctors larnd from analyzing a training corpus of txt. VS scals wll to larg collctions, but cannot rprsnt trm trm corrlations, which prvnts it from bing usd in translingual rtrival. GVS and LSI can rprsnt trm trm corrlations, but do not scal wll to vry larg rtrival collctions. W prsnt a novl mthod w call approximat dimnsion qualization (AD) that combins idas from VS, LSI, and GVS to produc a mthod that prforms wll on larg collctions, scals wll computationally, and can rprsnt trm trm corrlations. W compar th prformanc of AD to th othr mthods on both larg and small collctions of both monolingual and bilingual txt. AD outprforms all othr mthods on larg bilingual collctions, and prforms clos to th bst in all othr cass. 1. Introduction Thr ar thr basic vctor-spac mthods for information rtrival (IR): vctor spac modl (VS), latnt smantic indxing (LSI), and gnralizd vctor spac modl (GVS). ach has its own uniqu strngths. VS is simpl, scals xtrmly wll, and givs xcllnt prformanc on larg txt collctions. LSI xtnds VS by larning a rducd dimnsional rprsntation that modls trm trm associations, thus allowing a qury to hav a positiv similarity to a documnt with which it shars no trms. This is spcially important in cross-languag IR applications. Whil LSI has shown imprssiv prformanc on som txt collctions, its prformanc on xtrmly larg and divrs txt collctions has laggd bhind that of VS. W prsnt som vidnc that this is bcaus of th larg numbr of dimnsions ndd for larg txt collctions. GVS also modls trm trm associations, and scals much mor asily than LSI. Unfortunatly, its prformanc also lags substantially bhind that of VS and oftn LSI. W argu that this is bcaus, although GVS uss mor dimnsions than LSI, it puts trmndous wight on th largst ons, rsulting in an ffctiv dimnsionality that is substantially smallr. W show how to combin idas from VS, LSI and GVS to larn a vctor rprsntation w call approximat dimnsion qualization (AD) that scals wll and supports trm trm associations. On larg monolingual tst collctions, its prformanc approachs that of VS. On larg cross-languag collctions, to which VS cannot b applid, its prformanc is a substantial improvmnt ovr both GVS and LSI. 2. Vctor-asd Information Rtrival Information rtrival (IR) is th task of locating information itms (documnts) that ar rlvant to a usr qury from a collction. Cross-languag (or translingual) information rtrival is th problm of slcting usful documnts from collctions that may contain svral languags. Vctor-basd information rtrival mthods rprsnt both quris and documnts by high-dimnsional vctors and comput thir similarity by th vctor innr product. Whn th vctors ar normalizd to unit lngth, thn th innr product is qual to masuring th cosin of th angl btwn th two vctors in th vctor spac. Th componnts of thos vctors can b trm wights, as in VS, dvlopd by Salton Salton t al., 1975], or thy can b transformd into anothr spac whos dimnsions bar som nw manings, as in, for xampl, GVS Wong t al., 1985] and LSI Drwstr t al., 1990]. athmatically, vctor-basd IR mthods can b formu-

2 C latd in th following way:! whr "# $%& rprsnts th similarity btwn a documnt and a qury, is th vctor rprsntation of documnt, is th vctor rprsntation of documnt, and is a transformation matrix. Th vctor in quation 1 is usually obtaind this way: a documnt ( collction )+*-,/ is indxd into a trm-by-documnt matrix, whr )+* is th wight of th 8 - is th siz of th vocabulary th trm in th 9 -th documnt, : and ; is th numbr of documnts in th corpus. Thn, th 9 -th column of th matrix is an : -dimnsional vctor of trm wights. A usr qury can b rprsntd in a similar way. In th convntional VS, th transformation matrix is simply th :=<: idntity matrix: " VS A A criticism of VS is that it trats trms as unrlatd in that thy occupy orthogonal dimnsions in th smantic spac. A nonzro similarity scor btwn two vctors will rsult only if thy both hav nonzro valus in at last on dimnsion, i.., thir original documnts contain at last on word in common. This is crtainly far from idal bcaus, for xampl, for a qury that contains th sarch word computr but not digital, a documnt that uss xclusivly th word digital will b dmd no mor rlvant than a documnt that talks about gardningc. It is also obvious that th xact trm matching schm of VS is not suitd for cross-languag information rtrival. Tchniqus that addrss this problm oftn us a training corpus to driv statistical information about trm trm corrlations, and rlatd trms can b addd to th original qury to nrich rtrival. Idally, th training corpus would b a thsaurus that lists for vry word ntry all th rlatd words and how strong th rlationships ar. At th othr xtrm, this thsaurus would only contain word ntris followd by no rlatd words at all, and this is xactly analogous to convntional VS. In cross-languag information rtrival, a bilingual corpus can b usd for training, whr corrsponding documnts in two collctions ar translations of ach othr or ar on th sam or rlatd subjcts. Th Gnralizd Vctor Spac odl (GVS) Wong t al., 1985] of information rtrival, also known as th dual spac approach Shridan and allrini, 1996], is a mthod that capturs trm trm corrlations from th documnts (or matching documnt pairs, in th cas of D Qury xpansion tchniqus such as global analysis and local fdback can b usd to dal with th word mismatch problm in VS Xu and Croft, 1996]. (1) cross-languag rtrival).f thy co-occur in. In monolingual rtrival, lt b th trm documnt matrix of th training corpus. atching th vctor lmnts in and rows of by th trms thy rprsnt, transforms into a nw vctor whos lmnts corrspond to th ; documnts in th training corpus. Th qury vctor can also b transformd by in th sam way. Thn, th qury documnt similarity is masurd btwn th transformd vctors: GVS-L whr th :J<K: $%&AG matrix L H I (2) has a nonzro valu in its row 8 and column 9 if and only if thr is a documnt in that contains both th 8 -th and 9 -th trms. Th xtnsion of GVS to translingual IR was proposd by Yang t al. Yang t al., 1997]. Using a bilingual corpus for training, two matrics and ar formd, whr is a trm documnt matrix for th training documnts in th languag of th rtrival documnts, and is a trm documnt matrix for th training documnts in th languag of th quris. Whil th numbr of uniqu trms in th two languags ar diffrnt, th numbr of training documnts ar th sam, and ar rprsntd by th corrsponding columns of and. Thus, whn documnt is transformd by and qury innr product: $%& GVS-TL H by, w can comput thir L Latnt smantic indxing (LSI) Drwstr t al., 1990] is anothr vctor-basd mthod that taks this ida of training corpora on stp furthr. Givn a trm documnt training matrix, it uss th Singular Valu Dcomposition (SVD) Golub and Van Loan, 1989] to factor it into thr parts ON5P3Q RNTSU WVXY$Z &Z%\--!&Z]^_Q whr N and Q ar unitary matrics whos columns ar th lft and th right singular vctors of, P is a diagonal matrix whos diagonal lmnts ar non-ngativ and arrangd in dscnding ordr, and ` is th rank of. Th valus Z --\a&z%] C ar known as th singular valus of, and ar th squar roots of th ignvalus of (3) and I. Th first b (bcd` ) columns of N that corrspond to th b largst singular valus of togthr ar th transformation matrix for LSI " LSI-L $%&AG$?\N AH?-N N?-HN whr? is lik th idntity matrix? xcpt that only its first b diagonal lmnts ar nonzro (on). Whn bgft`, w say w prformd dimnsion rduction on N (or ). (4)

3 C Th us of th columns of N as th transformation matrix amounts to an orthogonal projction of quris and documnts into th row spac of th training matrix. In this way, LSI larns th structur of th training matrials, and uss this in making nw judgmnts of qury documnt rlatdnss. 3. Dimnsion qualization W can obtain th LSI similarity formula (quation 4) from a diffrnt prspctiv, in trms of a spcial form of th original matrix. For bdc `, dfin th rducd dimnsional form of to b TN5P Q TNTSU WVX $Z Z--\a&Z --\a _Q Thn, is th closst rank-b matrix to in Frobnius norm (dfind as sum of squars of lmnt-wis diffrncs) Golub and Van Loan, 1989]. Now, w dfin th opration of dimnsion qualization as rcomposing with N and Q but giving all columns unit qual wight. Lt b th rsult of such an opration: TNRSU WVXY H\-\ ^&Q ON? Q TN5Q is th closst unitary matrix to. For thn also hav th rducd dimnsional form: TN? Q TNRS7 "VHX H -\-Y \-\a &Q, w can Now, through th following drivation, w s that also a transformation matrix for N Q N Q?-HN Q?-HN?-Q?-HN?-HN $%& LSI-L To us LSI for translingual rtrival, w could simply substitut for on th qury sid: " LSI-TL G AH Not that this formulation is diffrnt from th original translingual LSI formulation by Landaur and Littman Landaur and Littman, 1990]. Thir crosslanguag LSI formula includs a singl SVD of th matrix that combins alignd documnts from both training corpora into singl documnts: and TNAPAQ Th vctors can b xtndd (with all zros) to covr trms in th languag of th othr. Compard to this is (5) (6) GVS VS LSI onolingual H H H Translingual H H H Tabl 1. asic vctor-basd information rtrival mthods in trms of th training matrix. rprsnts a paralll training corpus in th translingual cas. traditional translingual LSI, our nw approach computs two sparat SVDs of smallr matrics instad. This is crtainly mor usful whn th combind matrix bcoms too larg to analyz via SVD. Whn b ;, th numbr of documnts in th training collction, for b in quation 6 (i.., no dimnsion rduction), this nw translingual LSI is a spcial form of th Procrusts mthod Littman t al., 1998]. As shown in quations 2, 3, 5, and 6, GVS and LSI us forms of th training matrix. VS can b xprssd in a similar fashion as wll. First of all, w can viw VS as using th rtrival corpus dirctly for training. In othr words,, whr is th trm documnt matrix for th rtrival collction. Now, if w us as th transformation matrix, w hav AH Q N N Q N Q NQ N5N whr th last quality holds bcaus, as is a column of, its projction onto th column spac of is itslf ( N5N ). All thr mthods xprssd in a normal or spcial form of ar summarizd in Tabl 1. Not that th formula w giv for translingual VS is not xactly VS but fulldimnsional LSI, bcaus in VS w would us a paralll corpus for both training and rtrival, in which cas a trivial xtnsion of th monolingual mthod would do th rtrival task. 4. Comparison of Singular Valus If th singular valus of th training matrix wr all th sam, thn th formula for GVS and VS in Tabl 1 giv th sam valus. Similarly, LSI and VS ar th sam if th dimnsionality b is chosn qual to th rank ` of. Thrfor, th diffrncs w obsrv btwn VS, GVS, and LSI in practic will dpnd on th distribution of th singular valus of th training matrix. Figur 1 shows th singular valus of th small 1,121 UNICF nglish tst documnts usd by Yang t al. Yang t al., 1997]. Th collction is indxd with th

4 P singular valu singular valu singular valu numbr singular valu numbr Figur 1. Plot of all singular valus of th UNICF nglish tst collction of 1,121 documnts. Figur 2. Plot of 480 singular valus of TRC Frnch corpus of 141,643 documnts. SART ntc wighting schm and a complt spctrum of 1,103 singular valus is asily found from th matrix. Th charactristic of this plot is that it posssss th so-calld low-rank-plus-shift structur, rportd by Zha and Zhang in thir study of LSI Zha and Zhang, 1998]: th singular valus ar rlativly larg but dcrasing sharply at th bginning, lvling off noticably for th most part in th middl, and dipping again at th nd. W hav plottd th singular valus of a numbr of corpora, varying siz and indxing schm, and all sm to hav this spcial proprty. Whn th matrix siz gts too larg to comput th complt spctrum using currnt softwar, w still s th trnd of initial dropping and lvling off of singular valus. Plottd on Figur 2 ar th first 480 singular valus of th TRC Frnch collction of 141,643 documnts with Okapi trm wighting Robrtson t al., 1995]. What dos this imply for th bhavior of th various vctorbasd IR mthods? All thr transformation matrics that appar in Tabl 1 (,, and ) hav th sam lft and right singular vctors ( N and Q ) and only diffr in thir singular valus ( P,?,? ). Figur 3 dpicts th singular valus of th thr transformation matrics (and a fourth, xplaind in th nxt sction). xtrapolating from studis using smallr collctions, w d prdict that th idal singular valu plot for th transformation matrix for a larg collction would b flat up to som valu b lss than th full rank of th training matrix, but much largr than what can b computd using currnt softwar. This is dpictd as a dottd lin in Figur 3. Whil non of VS, GVS, and LSI match th singular valus of th idalizd transformation matrix, th following lmma is usful in judging which givs th bst approximation. Lmma 4.1 Lt b a monolingual training corpus with singular valu dcomposition N5P3Q. Lt N Q N 3Q and b two transformation matrics that shar lft and right singular vctors with. Compar th matrics of similaritis obtaind by comparing all pairs of documnts in to ach othr using " and ". Th Frobnius norm of U L is qual to th Frobnius norm of P. Proof: Using to rprsnt th Frobnius VH_ norm (sum of squars of th matrix componnts) and to rprsnt th trac (sum of diagonal lmnts of th matrix), w hav " Q P N N 3Q Q N N5P3Q Q P Q Q P Q Q P &Q V_Q P _Q Q P VQ# P _ Q V_ P & Q Q _Q & What this tlls us is that th closr two transformation matrics ar in thir singular valus, th closr ar th similaritis thy produc. Furthr, th dimnsions that mattr

5 A A GVS VS LSI AD onolingual Translingual H H H H H H H H GVS VS Tabl 2. Unifid viw of th four vctor-basd mthods of information rtrival. LSI A k AD A k Figur 3. Wights usd on orthogonal dimnsions of th smantic spac by th four diffrnt mthods. Ak A Ak Ak σ σ = k k k k k Figur 4. Using th dimnsion wights of, and to form. th most ar th ons that corrspond to th largst singular valus in th training matrix. asd on this, w d xpct VS to produc th most accurat similarity scors, followd by LSI, which is accurat for th dimnsions with th highst singular valus. Howvr, VS cannot b usd dirctly for translingual rtrival. In th nxt sction, w dvlop a novl approximation to th idalizd transformation matrix. 5. Approximat Dimnsion qualization Dfin k Z% Z% has th sam lft and right singular vctors of, and consists of th first b qualizd dimnsions of and th rmaining ` b dimnsions of normalizd by Za, as illustratd in Figur 4. This matrix taks advantag of th spcial shap of th singular valu plots of all th trm documnts w hav sn; it flattns out th first b vry larg singular valus, and attachs th rst of th ral singular valus, which ar a rlativly lvl and long middl portion with a small dipping tail. This way, w obtain rlativly qualizd dimnsions of th training matrix until clos to th nd of all dimnsions. Now, for monolingual rtrival, w simply calculat U AD-L $%&A AH and for translingual rtrival, w rplac th by : " G AD-TL A in front of W call this approach Approximat dimnsion qualization (AD). Th dimnsion wights of th AD mthod ar also plottd in Figur 3; it is clar that AD approximats th idal singular valus bttr than ithr LSI or GVS. From on prspctiv, AD is trying to xtnd th limitd ability of LSI to comput th singular vctors and valus of a larg training matrix by implicitly adding additional ons with rlativly qual wights. From anothr, AD maks cross-languag VS possibl, obtaining most if not all th dimnsions of both training matrics with qualization. From a third, it uss GVS s approach to scalably capturing trm trm corrlations, modifid to prvnt ovrmphasizing th first handful of dimnsions. Computationally, AD only nds to comput th first b dimnsions of th training matrix, not much diffrnt from a b -dimnsional LSI. orovr, in practical application, th dns matrix is nvr dirctly formd. First, w comput by forming a transformd qury vctor at ach stp: Z% Q?-HN Z Z Z Q P3HN Not that thr is no intrmdiat rsult (matrix) biggr than th siz of th numbr of quris by th numbr of documnts (or trms, whichvr is biggr). vn if w procss quris in batchs of tns or hundrds, thos matrics ar not vry larg. Thn, sinc " AD-L $%& can b computd in a similar way.

6 Cranfild TRC AP 1990 GVS VS LSI AD Tabl 3. onolingual avrag prcision rsults for th Cranfild and TRC AP 1990 collctions. 6. Rsults In our xprimnts, w tstd all four mthod on various documnt collctions of diffrnt siz and in diffrnt languags. W hav usd both th traditional tf < idf and th mor rcnt Okapi wighting schms, and hav prsntd our rsults for whichvr wighting schm works bttr with th convntional VS. Tabl 3 shows th rsults for monolingual rtrival on two diffrnt collctions: Cranfild and TRC AP Th numbrs in th tabl ar th avrag prcision scor ovr all quris bing tstd, a standard IR masur. Th Cranfild corpus is a classic IR tst collction that consists of 1,400 documnts and 225 tst quris. This collction dmonstrats th ability of LSI to driv trm trm corrlations at th rducd dimnsions of th smantic spac. Th LSI scor of is achivd at 300 dimnsions, with th full rank bing 1,400. AD s nar bst scor of is obtaind at 100 dimnsions, confirming that it approximats LSI with fwr dimnsions. Th 78,321-documnt TRC AP 1990 collction is a subst of th TRC AP collction. This siz is alrady too larg to comput a complt SVD of its trm documnt matrix. For this collction, only 800 singular triplts wr calculatd, which is only about 1% of th total numbr of dimnsions. Hnc, LSI can only obtain a modrat scor compard to VS, and this is by using all th availabl 800 dimnsions. ut, AD partially bridgs th gap btwn LSI and VS, achiving rsults clos to VS with th sam numbr of dimnsions as LSI. As w hav prdictd, GVS bhavs lik a vry low dimnsional LSI on both collctions. Our scond st of rsults, listd in Tabl 4, shows ths vctor-basd mthods on both monolingual and translingual rtrival and with th complt SVD bing computabl. Th xprimnts ar don on th UNICF collction cratd and dscribd by Yang t al. Yang t al., 1997]. Thr ar 1,134 training documnts in addition to 1,121 tst documnts, ach in both nglish and Spanish, and ths documnts ar alignd in both sts. In Tabl 4, mans th training is don on th tst collction itslf (possibl only in monolingual rtrival), and mans othrwis. Sinc th complt SVD can b computd, w tstd LSI and AD with diffrnt dimnsionalitis and found th bst possibl rsults, and onolingual Translingual GVS VS LSI AD Tabl 4. Rsults for th CU UNICF nglish-spanish collction. onolingual Translingual Fr Gr Fr Gr Gr Fr GVS VS N/A N/A LSI AD Tabl 5. Rsults for TRC Frnch-Grman Corpora with TRC-6 cross-languag topics. w wr also abl to do full-dimnsional cross-languag LSI (or cross-languag VS as w discussd arlir). As w can s, whil LSI gts its bst rtrival scors at rducd dimnsions (180, 900, and 900 for th thr avrag prcision scors in th row for LSI), AD achivs similar rsults with lowr dimnsions (at 80, 800, and 400). Whn th siz of rtrival collction bcoms vry larg, finding a complt SVD of th training matrix is byond th limitation of currnt computing rsourc that w hav. Thn, w can no longr try various dimnsionalitis to find th idal on for rtrival. Instad, w gt th two nds of th spctrum: first fw hundrds or thousands singular triplts that can b computd with currntly availabl hardwar and softwar, or th full st of dimnsions usd implicitly and only as a whol in VS, whil th idal dimnsionality w sk is probably somwhr in th middl. In thos cass, VS probably is th mthod to us unlss w can comput mor dimnsions. ut, whn it coms to translingual rtrival, VS is no longr applicabl, and this is whn AD coms in handy. This cas is illustratd in Tabl 5, with rsults for larg TRC Frnch and Grman collctions of 185,082 and 141,643 documnts rspctivly. Th training documnts ar a subst of 40,000 roughly alignd ons from th two collction as usd by Rhdr t al. Rhdr t al., 1997]. W computd a maximum of 1,700 dimnsions (which is still only a littl mor than 4% of th full dimnsionality), and both LSI and AD ar lagging bhind VS in monolingual rtrival, with AD improving upon LSI. ut, whil VS is impossibl in th cas of cross-languag rtrival, AD dos obtain vry dcnt rsults in that trritory. In fact, ths rsults compar favorably with th vry bst rsults obtaind for this collction Franz t al., 1999].

7 7. Conclusion This papr shows that th thr traditional vctor-basd mthods of information rtrival, VS, LSI, and GVS, can b viwd as blonging to a family of algorithms that diffr only in th singular valus of thir transformation matrix. Ignoring computational concrns, LSI is th most dsirabl mthod to us bcaus, lik VS, it projcts quris and documnts into th row spac of th training corpus, and, unlik VS, it taks advantag of trm trm corrlations. ut, as th training collction siz incrass, so dos th numbr of dimnsions rquird by LSI, and computational concrns dominat. With approximat dimnsion qualization (AD), a nw mthod that combins idas from VS, LSI, and GVS, w can approximat th idal transformation matrix mor accuratly using availabl computational rsourcs. Our rsults show that AD can improv upon LSI in both monolingual and translingual information rtrival. Rfrncs Drwstr t al., 1990] Drwstr, S., Dumais, S. T., Furnas, G. W., Landaur, T. K., and Harshman, R. A. (1990). Indxing by Latnt Smantic Analysis. Journal of th Amrican Socity for Information Scinc, 41(6): Franz t al., 1999] Franz,., ccarly, J. S., and Roukos, S. (1999). Ad hoc and multilingual information rtrival at I. In Voorhs,.. and Harman, D. K., ditors, Procdings of th Svnth Txt Rtrival Confrnc (TRC-7). Dpartmnt of Commrc, National Institut of Standards and Tchnology. Golub and Van Loan, 1989] Golub, G. H. and Van Loan, C. F. (1989). atrix Computations. Th Johns Hopkins Univrsity Prss, altimor, aryland, 2nd dition. Landaur and Littman, 1990] Landaur, T. K. and Littman,. L. (1990). Fully automatic cross-languag documnt rtrival using latnt smantic indxing. In Procdings of th Sixth Annual Confrnc of th UW Cntr for th Nw Oxford nglish Dictionary and Txt Rsarch, pags UW Cntr for th Nw OD and Txt Rsarch, Watrloo Ontario. Littman t al., 1998] Littman,. L., Jiang, F., and Kim, G. A. (1998). Larning a languag-indpndnt rprsntation for trms from a partially alignd corpus. In Shavlik, J., ditor, Procdings of th Fiftnth Intrnational Confrnc on achin Larning, pags organ Kaufmann. Rhdr t al., 1997] Rhdr,., Littman,. L., Dumais, S., and Landaur, T. K. (1997). Automatic 3-languag cross-languag information rtrival with latnt smantic indxing. In Th Sixth Txt Rtrival Confrnc Notbook Paprs (TRC6), pags National Institut of Standards and Tchnology Spcial Publication. Robrtson t al., 1995] Robrtson, S.., Walkr, S., Jons, S., Hancock-auliu,.., and Gatford,. (1995). Okapi at TRC-3. In Harman, D., ditor, Th Third Txt Rtrival Confrnc (TRC-3), pags National Institut of Standards and Tchnology Spcial Publication Salton t al., 1975] Salton, G., Wang, A., and Yang, C. S. (1975). A vctor spac modl for information rtrival. Journal of th Amrican Socity for Information Scinc, 18(11): Shridan and allrini, 1996] Shridan, P. and allrini, J. P. (1996). xprimnts in multilingual information rtrival using th SPIDR systm. In Fri, H.-P., Harman, D., Schäubl, P., and Wilkinson, R., ditors, Procdings of th 19th Annual Intrnational AC SIGIR Confrnc on Rsarch and Dvlopmnt in Information Rtrival, Nw York. Association for Computing achinry. Wong t al., 1985] Wong, S. K.., Ziarko, W., and Wong, P. C. N. (1985). Gnralizd vctor spac modl in information rtrival. In AC SIGIR Confrnc on Rsarch and Dvlopmnt in Information Rtrival (SI- GIR 85), pags Xu and Croft, 1996] Xu, J. and Croft, W.. (1996). Qury xpansion using local and global documnt analysis. In Procdings of th 19th Annual Intrnational AC SIGIR Confrnc on Rsarch and Dvlopmnt in Information Rtrival (SIGIR 96), pags 4 11, Zurich, Switzrland. Yang t al., 1997] Yang, Y., rown, R. D., Frdrking, R.., Carbonll, J. G., Gng, Y., and L, D. (1997). ilingual-corpus basd approachs to translingual information rtrival. In Th 2nd Workshop on ultilinguality in Softwar Industry: Th AI Contribution (ULSAIC 97). Zha and Zhang, 1998] Zha, H. and Zhang, Z. (1998). On matrics with low-rank-plus-shift structurs: Partial SVD and latnt smantic indxing. Tchnical Rport CS , Th Pnnsylvania Stat Univrsity, Univrsity Park, PA.