2. Hubs and authorities, a more detailed evaluation of the importance of Web pages using a variant of

Transcription

1 5 Web Serch Outline: 1. Pge rnk, for discovering the most ëimportnt" pges on the Web, s used in Google. 2. Hubs nd uthorities, more detiled evlution of the importnce of Web pges using vrint of the eigenvector clcultion used for Pge rnk. 5.1 Pge Rnk Intuitively, we solve the recursive deænition of ëimportnce": pge is importnt if importnt pges link to it. Crete stochstic mtrix of the Web; tht is: 1. Ech pge i corresponds to row i nd column i of the mtrix. 2. If pge j hs n successors èlinksè, then the ijth entry is 1=n if pge i is one of these n successors of pge j, nd 0 otherwise. The intuition behind this mtrix is: æ Imgine tht initilly ech pge hs one unit of importnce. At ech round, ech pge shres whtever importnce it hs mong its successors, nd receives new importnce from its predecessors. æ Eventully, the importnce of ech pge reches limit, which hppens to be its component in the principl eigenvector of this mtrix. æ Tht importnce is lso the probbility tht Web surfer, strting t rndom pge, nd following rndom links from ech pge will be t the pge in question fter long series of links. Exmple 5.1 : In 1839, the Web consisted on only three pges tscpe, Microsoft, nd zon. The links mong these pges were s shown in Fig. 9. Figure 9: The Web in 1839 Let ën; m; ë be the vector of importnces for the three pges: tscpe, Microsoft, zon, in tht order. Then the eqution describing the symptotic vlues of these three vribles is: 0 0 1=2 1=2 1 0 For exmple, the ærst column of the mtrix reæects the fct tht tscpe divides its importnce between itself nd zon. The second column indictes tht Microsoft gives ll its importnce to zon. We cn solve equtions like this one by strting with the ssumption n = m = = 1, nd pplying the mtrix to the current estimte of these vlues repetedly. The ærst four itertions give the following estimtes: 16

2 n = 1 1 5è4 9è8 5è4 m = 1 1è2 3è4 1è2 11è16 = 1 3è2 1 11è8 17è16 In the limit, the solution is n = =6=5; m =3=5. Tht is, tscpe nd zon ech hve the sme importnce, nd twice the importnce of Microsoft èwell this ws 1839è. 2 æ Note tht we cn never get bsolute vlues of n, m, nd, just their rtios, since the initil ssumption tht they were ech 1 ws rbitrry. æ Since the mtrix is stochstic èsum of ech column is 1è, the bove relxtion process converges to the principl eigenvector. 5.2 Problems With Rel Web Grphs 1. Ded ends: pge tht hs no successors hs nowhere to send its importnce. Eventully, ll importnce will ëlek out of" the Web. 2. Spider trps: group of one or more pges tht hve no links out of the group will eventully ccumulte ll the importnce of the Web. Exmple 5.2 : Suppose Microsoft tries to duck chrges tht it is monopoly by removing ll links from its site. The new Web is s shown in Fig. 10, nd the mtrix describing trnsitions is: 0 0 1=2 The ærst four steps of the itertive solution re: Figure 10: Microsoft becomes ded end n = 1 1 3è4 5è8 1è2 m = 1 1è2 1è4 1è4 3è16 = 1 1è2 1è2 3è8 5è16 Eventully, ech of n, m, nd become 0; i.e., ll the importnce leked out. 2 Exmple 5.3 : Angered by the decision, Microsoft decides it will link only to itself from now on. Now, Microsoft hs become spider trp. The new Web is in Fig. 11, nd the eqution to solve is: The ærst steps of the solution re: 0 1 1=2 17

3 Figure 11: Microsoft becomes spider trp Now, m converges to 3, nd n = =0. 2 n = 1 1 3è4 5è8 1è2 m = 1 3è2 7è4 2 35è16 = 1 1è2 1è2 3è8 5è Google Solution to Ded Ends nd Spider Trps Insted of pplying the mtrix directly, ëtx" ech pge some frction of its current importnce, nd distribute the txed importnce eqully mong ll pges. Exmple 5.4 : If we use 20è tx, the eqution of Exmple 5.3 becomes: =0: =2 The solution to this eqution is n =7=11; m =21=11; =5= :2 0:2 0:2 æ Note tht the sum of the three vlues is not 3, but there is more resonble distribution of importnce thn in Exmple Google Anti-Spm Devices ëspmming" is the ttempt by mny Web sites to pper to be bout subject tht will ttrct surfers, without truly being bout tht subject. æ Google, like other serch engines, tries to mtch the words in your query to the words on the Web pges. However, Google, unlike other engines tends to believe wht others sy bout you in their nchor text, mking it hrder for you to pper to be bout something you re not. æ The use of Pge rnk to mesure importnce, rther thn the more nive ënumber of links into the pge" lso protects ginst spmmers. The nive mesure cn be fooled by the spmmer who cretes 1000 pges tht mutully link to one nother, while Pge rnk recognizes tht none of the pges hve ny rel importnce. 5.5 Hubs nd Authorities Intuitively, we deæne ëhub" nd ëuthority" in mutully recursive wy: hub links to mny uthorities, nd n uthority is linked to by mny hubs

4 æ Authorities turn out to be pges tht oæer informtion bout topic, e.g., the Quest home pge bout the IBM dt-mining project. æ Hubs re pges tht don't provide the informtion, but tell you where to ænd the informtion, e.g., the CS345 home pge. æ Uses mtrix formultion similr to tht of Pge rnk, but without the stochstic restriction. We count ech link s 1, regrdless of how mny successors or predecessors pge hs. æ Repeted ppliction of the mtrix leds to divergence, but we cn introduce scling fctors nd keep the computed vlues of ëuthority" nd ëhubbiness" for ech pge within ænite bounds. Deæne mtrix A whose rows nd columns correspond to Web pges, with entry A ij = 1 if pge i links to pge j, nd 0 if not. æ Notice tht A T, the trnspose of A, looks like the mtrix used for computing Pge rnk, but A T hs 1's where the Pge-rnk mtrix hs frctions. Let ~ nd ~ h be vectors, whose ith component corresponds to the degrees of uthority nd hubbiness of the ith pge. let ç nd ç be suitble scling fctors to be determined lter. Then we cn stte: 1. ~ h = ça~. Tht is, the hubbiness of ech pge is the sum of the uthorities of ll the pges it links to, scled by ç. 2. ~ = ça T~ h. Tht is, the uthority of ech pge is the sum of the hubbiness of ll the pges tht link to it, scled by ç. We cn derive from è1è nd è2è, using simple substitution, two equtions tht relte vectors ~ nd ~ h only to themselves: ~ = çça T A~; ~ h = ççaa T~ h As result, we cn compute ~ h nd ~ by relxtion, giving us the principl eigenvectors of the mtrices AA T nd A T A, respectively. Figure 12: Web for Exmple 5.5 Exmple 5.5 : Consider the Web of Fig. 12. The relevnt mtrices re: 2 A = A T = AA T = A T A = If we use ç = ç = 1 nd ssume tht the vectors ~ h =ëh n ;h m ;h ë nd ~ =ë n ; m ; ë re ech initilly ë1; 1; 1ë, the ærst three itertions of the equtions for ~ nd ~ h re: 19

5 n = m = = h n = h m = h = For instnce, the vector ~, properly scled, will converge to vector where n = m, nd ech of these is greter thn in the rtio 1 + p 3 : 2, or bout