Testing the BRIEF Helth Litercy Screening Tool Jolie Hun, PhD, Virgini Nolnd-Dodd, PhD, MPH, Jill Vrnes, EdD, John Grhm-Pole, MD, Brbr Rienzo, PhD, nd Ptrici Donldson, RN Due to the oppressive strins lid on the ntion s helth cre system, efforts to prevent dverse outcomes relted to ptients indequte helth re wrrnted. By identifying ptients helth skills with this tool, these uthors strive not only to prevent personl nd system loss but lso to provide ptient-centered cre. Indequte helth is mjor problem in the United Sttes. Ptients helth skills ffect their bility to communicte with helth cre providers, dhere to helth cre regimens, ccess nd nvigte helth cre services, nd mnge helth issues. 1,2 Yet the Institute of Medicine estimted in 2004 tht nerly hlf of ll Americn dults (bout 90 million people) hve difficulty understnding nd cting upon helth informtion. 2 Ech yer, indequte helth results in pproximtely $73 billion in unnecessry helth cre costs. 3 In order to fcilitte effective nd timely interventions tht cn promote high qulity, cost-efficient helth cre experience, helth cre providers re in need of brief, effective tool for detecting indequte helth mong their ptients. At present, there re vlidted tools for ssessing At the time this rticle ws written, Dr. Hun ws doctorl student t the University of Florid (UF) College of Helth nd Humn Performnce, Ginesville. She is currently Ntionl Institutes of Helth postdoctorl fellow in the deprtment of fmily nd community medicine t the University of Arizon, Tucson. Dr. Nolnd-Dodd is n ssocite professor nd grdute coordintor nd Dr. Vrnes is professor, both in the deprtment of helth eduction nd behvior t UF. Dr. Grhm- Pole is professor emeritus in the deprtment of peditrics nd pllitive cre t the UF College of Medicine. Dr. Rienzo is n ssocite den for fculty ffirs nd professor in the deprtment of helth eduction nd behvior t UF. Ms. Donldson is nurse t the Mlcom Rndll VA Medicl Center, Ginesville, FL. helth, including the Short Test of Functionl Helth Litercy in Adults (STOFHLA) 4 nd the Rpid Estimte of Adult Litercy (REALM). 5 Current tools, however, either tke too long to dminister or re potentilly embrrssing to ptients. 6 Recent studies hve ttempted to crete brief nd effective helth screening tool. Chew nd collegues developed 16 helth screening questions nd dministered them, long with the STOFHLA, to 332 prticipnts. 6 They found tht, s determined by the STOFHLA, three of the new questions were effective t detecting indequte helth, tht these three questions were weker t detecting indequte/ mrginl helth, nd tht no combintion of the three questions ws more effective t identifying indequte helth thn ny of those individul questions. Wllce nd collegues dministered the sme three questions, long with the REALM, to 305 prticipnts who differed demogrphiclly from the prticipnts in Chew nd collegues study. 7 Their results indicted tht, s determined by the REALM, one of the questions ws ccurte in detecting limited nd limited or mrginl helth nd ws more ccurte thn ny combintion of the three questions. Lter, Bker sserted tht the inconsistencies between Chew nd collegues study nd Wllce nd collegues study indicte need for further reserch, 8 while Prker nd Kindig clled for more reserch on the mesurement of both individul nd popultion-level helth skills. 9 With these issues in mind, we developed the Brief Helth Litercy Screening Tool (known s the BRIEF) nd conducted study to determine its efficcy. The BRIEF consists of the three questions evluted by Chew nd collegues nd Wllce nd collegues, long with fourth question tht we developed. We investigted the tool s efficcy t detecting indequte nd indequte/mrginl helth in VA mbultory cre setting by using the STOFHLA nd the REALM s stndrds of ctul helth. In this rticle, we present our findings nd discuss how we hve implemented the BRIEF within our own helth cre system. Methods Our study involved the dministrtion of the BRIEF, the STOFHLA, the REALM, nd self-dministered survey to convenience smple of ptients presenting for cre in VA mbultory setting between Mrch nd August 2006. We employed n ction reserch method, through which 21 trined, volunteer helth cre providers collected ptient dt on site in the clinicl, mbultory cre setting. The dt collectors were from eight rurl 24 FEDERAL PRACTITIONER DECEMBER 2009
Tble 1. Ptients generl skills ccording to level nd score of the STOFHLA nd the REALM b Level Score Generl skills STOFHLA Indequte 0 16 Able to perform uncomplicted tsks involving brief nd uncomplicted texts nd documents Mrginl 17 22 Able to locte informtion in text, mke low-level inferences using printed mterils, nd integrte esily identifible pieces of informtion Able to integrte informtion from reltively long or dense texts or documents, determine pproprite rithmetic opertions bsed on informtion contined in the directive, nd identify quntities needed to perform opertion Adequte 23 36 Demonstrtes proficiencies ssocited with long nd complex documents nd text pssges; ble to determine nd interpret qulittive nd quntittive dt needed to perform n opertion REALM Limited 0 44 Not ble to red most low helth mterils; will need repeted orl instructions; mterils should be composed of illustrtions or video tpes; will need low mterils; my not be ble to red prescription lbel Mrginl 45 60 Struggles with most ptient eduction mterils Adequte 61 66 Able to red nd comprehend most ptient eduction mterils STOFHLA = Short Test of Functionl Helth Litercy in Adults. b REALM = Rpid Estimte of Adult Litercy. nd nonrurl VA mbultory cre clinics nd hospitls in northern Florid nd southern Georgi nd included 17 nurses, the principl investigtor, nutritionist, dentl technicin, nd nurse eductor volunteer. Once ptients greed to prticipte in the study, their dt were collected either by one of the provider dt collectors in their mbultory cre setting or by the principl investigtor in n exmintion room in the sme setting. Inclusion criteri required the ptients to ssent to their prticiption verblly nd in English. No incentives for prticiption were provided. The instruments The BRIEF consists of the following questions: (1) How often do you hve someone help you red hospitl mterils? ; (2) How confident re you filling out medicl forms by yourself? ; (3) How often do you hve problems lerning bout your medicl condition becuse of difficulty understnding written informtion? ; nd (4) How often do you hve problem understnding wht is told to you bout your medicl condition? For the first, third, nd fourth questions, response options re offered in the following five-point Likert scle: 1 = lwys, 2 = often, 3 = sometimes, 4 = occsionlly, nd 5 = never. For the second question, the following five-point Likert scle is offered: 1 = not t ll, 2 = little bit, 3 = somewht, 4 = quite bit, nd 5 = extremely. The first three questions were evluted in the studies by Chew nd collegues nd Wllce nd collegues. 6,7 We dded the fourth question to ssess difficulties with uditory helth informtion nd, thus, increse the tool s vlidity. The BRIEF cn be dministered nd scored in less thn two minutes. BRIEF scores rnge from 4 to 20. Bsed on the previous evlutions of the first three questions, 6,7 we set three criterion levels for BRIEF scores: indequte (scores of 4 to 12), mrginl (scores of 13 to 16), nd dequte (scores of 17 to 20). The STOFHLA nd the REALM were used to test the BRIEF s efficcy (Tble 1). The STOFHLA is shortened version of the Test of Functionl Helth Litercy, 10,11 nd it mesures ptients bility to red nd understnd two pssges. It includes 36 items, hs score rnge of 0 to 36, tkes up to seven minutes to dminister, nd tkes bout two dditionl minutes to score. STOFHLA scores divide helth skills into three criterion levels: indequte (0 to 16), mrginl (17 to 22), nd dequte (23 to 36). Continued on next pge DECEMBER 2009 FEDERAL PRACTITIONER 25
Continued from previous pge Tble 2. Demogrphic chrcteristics of the study prticipnts (N = 378) Chrcteristic Study popultion Age in yers Averge (SD) 61.5 (11.9) Rnge 23 89 Gender, no. (%) Mle 356 (94.2) Femle 19 (5) Not reported 3 (0.8) Eduction level, no. (%) Elementry school (grdes 1 5) 4 (1.1) Junior high school (grdes 6 8) 11 (2.9) Some high school (grdes 9 12) 56 (14.9) High school/ged 98 (25.9) Some college 126 (33.3) College degree 80 (21.2) Trde school 1 (0.3) Not reported 2 (0.5) Ethnicity, no. (%) Blck 69 (18.3) White 278 (73.5) Hispnic/Ltino 12 (3.2) Ntive Americn 12 (3.2) Asin Americn 1 (0.3) Other 3 (0.8) Not reported 3 (0.8) GED = Generl Eductionl Development. The REALM ssesses helth by sking respondents to red list of 66 words loud. REALM scores rnge from 0 to 66 nd re divided into three criterion levels: limited (0 to 44), mrginl (45 to 60), nd dequte (61 to 66). Although the REALM hs been described s tking one to two minutes to dminister, 5 the pce of its dministrtion is set by respondents nd, in our experience, cn exceed two minutes considerbly. The self-dministered survey consisted of 19 items. Seven items ssessed demogrphic dt: ge, gender, mritl sttus, rce/ethnicity, lnguge, home ownership, nd eduction. One item ssessed English s first lnguge nd nother sked prticipnts to rte their reding bility on five-point Likert scle (1 = excellent nd 5 = poor). Three items pertined to helth sttus, ssessing whether ptients currently hd high blood pressure or dibetes or hd history of stroke. The lst eight items gthered miscellneous dt bout ccess to helth informtion resources. Dt nlysis Dt were mnged nd nlyzed using the SPSS 12.0 (SPSS Inc, Chicgo, IL) softwre pckge. The level of significnce ws set t 95% level of confidence, with P vlue of less thn.05. Univrite sttisticl nlyses were conducted to provide preliminry sttisticl informtion nd ssess dt ptterns. A Person product moment correltion coefficient ws clculted to determine the vlidity of the BRIEF in comprison to the REALM nd the STOFHLA. A principl component nlysis ws conducted to determine if, in fct, the BRIEF mesures one distinct construct helth. Our study utilized the mesures of sensitivity defined s the proportion of ptients who were positive for the test mong ll ptients with the disese 12 s the primry indictor of ccurcy of the BRIEF. Receiver opertor chrcteristic (ROC) curves were clculted to plot sensitivity. Ares under the ROC curve (AU- ROCs) were clculted to test the efficcy of ech of the BRIEF s four items individully nd the items grouped s the BRIEF, using the STOFHLA nd the REALM s stte vribles or true indictors of helth skills. Agin using the STOFHLA nd the REALM s stte vribles, AUROC curves were clculted to determine the degree to which the BRIEF identifies respondents with indequte helth skills nd those with indequte/mrginl helth skills ( composite grouping of respondents with indequte skills nd those with mrginl skills). An nlysis of vrince (ANOVA) lso ws conducted to test for group differences between the proposed BRIEF score levels to indicte optiml cutoff points for determining levels of helth. Results A totl of 378 ptients greed to prticipte, giving the study response rte of pproximtely 90%. Ptients resons for refusl were not ssessed. Of the prticipnts, 94% were mle nd 74% were white (Tble 2). 26 FEDERAL PRACTITIONER DECEMBER 2009
The men (SD) ge of the totl study cohort ws 61.5 (11.9) yers; by ethnicity, it ws 62.7 (11.4) yers for whites, 56.7 (12.1) yers for blcks, 61.9 (12.6) yers for Hispnics/Ltinos, nd 68.6 (11.4) yers for Ntive Americns. When sked to self-rte their reding bility, 85% of prticipnts sid it ws good to very good nd 15% sid it ws fir to poor. In ddition, 97% of prticipnts reported speking English s their first lnguge, 77% reported owning their home, 64% reported hving high blood pressure, 35% reported hving dibetes, nd 11% reported hving hd t lest one stroke. Helth Of the prticipnts, 20% showed indequte helth skills on the BRIEF, 7% showed indequte skills on the REALM, nd 9% showed indequte skills on the STOFHLA. Approximtely 37% of the prticipnts showed mrginl skills on the BRIEF, 30% showed mrginl skills on the REALM, nd 8% showed mrginl skills on the STOFHLA. Finlly, 43% of the prticipnts showed dequte skills on the BRIEF, 64% showed dequte skills on the REALM, nd 83% showed dequte skills on the STOFHLA (Figure). The verge (SD) scores for the three screening tools were s follows: BRIEF = 15.39 (3.67); REALM = 59.42 (8.96); nd STOFHLA = 29.83 (8.03). % of ptients 90 80 70 60 50 40 30 20 10 0 9 STOFHLA REALM BRIEF Indequte Mrginl Adequte Level of helth skills Figure. Study prticipnts levels of helth s indicted by the Short Test of Functionl Helth Litercy in Adults (STOFHLA), the Rpid Estimte of Adult Litercy (REALM), nd the BRIEF. BRIEF ccurcy Person correltion results were: r (378) =.40 (P <.01) for the BRIEF nd the REALM; r (378) =.42 (P <.01) for the BRIEF nd the STOF- HLA; nd r (378) =.61 (P <.01) for the REALM nd the STOFHLA. Since generl rule of thumb for determining if there is reltionship between vribles is minimum correltion coefficient of.3, 13 compred to stndrds, our results indicte there is n ssocition between ll three tools nd tht the ssocition between the BRIEF nd the vlidted tools is moderte. 14 The ddition of the fourth item to the BRIEF, How often do you hve problem understnding wht is told to you bout your medicl condition? incresed the internl vlidity of the screening mesure s indicted by the incresed correltion coefficient (Tble 3). The principl component nlysis findings suggested tht the BRIEF mesures one distinct construct, helth (eigenvlue = 2.388), ccounting for 60% of score vrince. The remining eigenvlues were less thn 1 nd, thus, were not retined. These findings vlidte the BRIEF s helth screening tool s compred to the REALM nd the STOFHLA. Using the REALM s the stte vrible, the BRIEF ws slightly more 7 20 8 30 37 83 64 43 ccurte t identifying respondents with indequte skills thn it ws t identifying respondents with indequte/mrginl skills, s indicted by higher sensitivity (AUROC) scores for indequte versus indequte/ mrginl skills (Tbles 3 nd 4). Specificlly, the BRIEF demonstrted.79 sensitivity (95% confidence intervl [CI],.70.87) for detecting indequte skills nd.69 sensitivity (95% CI,.64.75) for detecting indequte/mrginl skills. Still using the REALM s the stte vrible, ll four BRIEF items together hd higher AUROC thn ny of the individul items (Tble 3), indicting tht the tool s whole is more sensitive thn its components for identifying indequte helth skills. With the STOFHLA s the stte vrible, the BRIEF remined slightly more sensitive for indequte helth skills thn for indequte/ mrginl helth skills (Tbles DECEMBER 2009 FEDERAL PRACTITIONER 27
Tble 3. Person product moment correltion nd AUROC vlues for the BRIEF s whole nd s individul items, with the REALM b nd the STOFHLA c s the stte vribles AUROC vlue (95% CI e ) Indequte Indequte/ Test vrible r d mrginl BRIEF compred to REALM BRIEF overll.40.79 (.70.87).69 (.64.75) BRIEF item 1.34.73 (.62.84).63 (.57.69) BRIEF item 2.38.71 (.59.84).68 (.63.74) BRIEF item 3.28.69 (.59.79).65 (.59.70) BRIEF item 4.21.68 (.58.78).59 (.53.65) BRIEF compred to STOFHLA BRIEF overll.42.76 (.69.83).74 (.67.80) BRIEF item 1.32.66 (.56.77).64 (.56.72) BRIEF item 2.42.75 (.65.84).69 (.61.77) BRIEF item 3.28.65 (.56.74).66 (.59.73) BRIEF item 4.28.68 (.59.78).66 (.59.74) AUROC = re under the receiver operting chrcteristic curve. b REALM = Rpid Estimte of Adult Litercy. c STOFHLA = Short Test of Functionl Helth Litercy in Adults. d r = Person product moment correltion coefficient. e CI = confidence intervl. 3 nd 4), with vlues of.76 (95% CI,.69.83) nd.74 (95% CI,.67.80), respectively. All of the items hd n AUROC greter thn.5 t 95% CI. And with the STOFHLA s the stte vrible, the BRIEF tool retined higher AUROC thn ny of its individul items (Tble 3). ANOVA results indicted tht the three proposed BRIEF levels were significntly different from one nother on the REALM (F score = 28.63, P <.0001) nd the STOFHLA (F score = 35.32, P <.0001). Posthoc Tukey nlyses suggest ll levels were significntly different t P <.0001. DISCUSSION Does the brief mesure up? Our findings indicte tht the BRIEF, the STOFHLA, nd the REALM re positively correlted nd tht the BRIEF cn ccurtely identify prticipnts with indequte nd indequte/mrginl helth skills. The BRIEF tkes shorter time thn either the STOFHLA or the REALM to dminister nd score. In ddition, we believe tht the BRIEF my offer n dvntge over the other two tools which involve testing ptients skills in tht it is less likely to be embrrssing to ptients. Despite the BRIEF s correltion with the other tools, however, its estimte tht 20.1% of prticipnts hd indequte helth ws much higher thn the REALM s estimte of 6.6% nd the STOFHLA s estimte of 9%. It should be noted tht the BRIEF estimte is closer to previous findings of lrger studies, which suggest tht 33% of ptients hve indequte helth skills. 1,2,15 Nevertheless, it is possible tht the BRIEF s ccurcy is hindered by flse positives. At present, it cnnot be determined if the discrepncies resulted from error in the BRIEF or from error in the REALM nd the STOFHLA, lthough these discrepncies probbly re relted to the tools mesurement of slightly different spects of helth. Findings from this study should be interpreted with cution. Further reserch is needed to vlidte the BRIEF s effectiveness t determining rtes of indequte helth in the generl popultion nd in priority subpopultions. Although the three mesures used in this study focus on different spects of the concept of helth, contributing to error in mesurement, the REALM nd the STOFHLA currently re the most studied nd vlidted helth screening mesures vilble. Efforts to mesure helth in diverse popultions remin impertive, s reserchers seek to understnd the interpersonl nd culturl contexts of communiction in medicl encounters. 16 Severl study limittions should be noted when interpreting our results. The homogeneous nture of our smll smple my hve influenced our findings, which cnnot be generlized to other popultions or geogrphicl regions of the United Sttes. Becuse the study did not use rndomized, controlled design, it my hve filed to ccount for confounding vribles tht introduced mesurement errors. The fct tht our dt were gthered through self-reports on the BRIEF nd on the prticipnt survey lso limits the inferences tht cn be mde from the study findings. The BRIEF in prctice Bsed on the outcomes of this study, the BRIEF ws pproved for clinicl Continued on pge 30 28 FEDERAL PRACTITIONER DECEMBER 2009
Continued from pge 28 Tble 4. Performnce of the BRIEF in detecting indequte nd indequte/mrginl helth skills, by score, using the STOFHLA nd the REALM b s the stte vribles Indequte helth STOFHLA Indequte/ mrginl helth Indequte helth BRIEF 1 1 1 1 score Sensitivity specificity Sensitivity specificity Sensitivity specificity Sensitivity specificity 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 5 0.03 0.02 0.02 0.02 0.04 0.01 0.03 0.01 6.5 0.15 0.02 0.08 0.02 0.12 0.02 0.06 0.01 7.5 0.15 0.02 0.10 0.02 0.16 0.02 0.07 0.01 8.5 0.18 0.04 0.13 0.04 0.16 0.05 0.09 0.03 9.5 0.21 0.06 0.19 0.05 0.20 0.07 0.14 0.04 10.5 0.24 0.08 0.21 0.08 0.24 0.09 0.17 0.05 11.5 0.35 0.12 0.32 0.11 0.44 0.12 0.25 0.08 12.5 0.50 0.17 0.43 0.16 0.60 0.17 0.33 0.13 13.5 0.59 0.24 0.52 0.22 0.72 0.24 0.44 0.17 14.5 0.74 0.33 0.63 0.31 0.76 0.34 0.58 0.25 15.5 0.79 0.41 0.78 0.38 0.84 0.42 0.65 0.33 16.5 0.88 0.54 0.86 0.51 0.96 0.54 0.75 0.46 17.5 0.97 0.65 0.92 0.63 0.96 0.66 0.82 0.60 18.5 1.00 0.75 0.95 0.74 0.96 0.76 0.88 0.72 19.5 1.00 0.83 0.97 0.83 0.96 0.84 0.90 0.82 21 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 STOFHLA = Short Test of Functionl Helth Litercy in Adults. b REALM = Rpid Estimte of Adult Litercy. REALM Indequte/ mrginl helth tril in April 2007 within the prticipting North Florid/South Georgi Veterns Helth System (NFSGVHS). This tril resulted in the implementtion of the BRIEF throughout prticipting fcilities in the NFSGVHS. At these fcilities, the BRIEF is used to generte clinicl reminders within the electronic ptient medicl record system. When ptient s BRIEF score indictes indequte helth, this fct is entered into the electronic record. Therefter, screen prompt pops up in the record to provide clinicins with steps to help the ptient understnd helth informtion. From July 1, 2007 to July 1, 2009, the BRIEF ws used in NFSGVHS fcilities to screen 112,442 ptients 6,466 (6%) of whom were found to hve indequte helth. Recommendtions for the future We recommend tht other helth cre fcilities use the BRIEF to screen ptients helth levels, document these levels in ptient records, nd tilor their pproch to ptients ccordingly. It is impertive tht clinicins support ptients by creting shme-free, empowering pproch to ssessing nd responding to their helth needs. 17,18 Empowering ptients cn minimize the stigm ssocited with low nd cn help ptients to tke leding role in their helth cre. The prcticl nd legl implictions of identifying nd responding to individuls needs bsed on the documented results of the BRIEF need to be ddressed s prt of implementing the screening nd posting process. 30 FEDERAL PRACTITIONER DECEMBER 2009
Acknowledgements This reserch ws conducted in coopertion with the VA nd ws mde possible, in prt, by Grnt Number T32 AT001287 from the Ntionl Center for Complementry nd Alterntive Medicine t the Ntionl Institutes of Helth. Author disclosures The uthors report no ctul or potentil conflicts of interest with regrd to this rticle. Disclimer The opinions expressed herein re those of the uthors nd do not necessrily reflect those of Federl Prctitioner, Qudrnt HelthCom Inc., the U.S. government, or ny of its gencies. This rticle my discuss unlbeled or investigtionl use of certin drugs. Plese review complete prescribing informtion for specific drugs or drug combintions including indictions, contrindictions, wrnings, nd dverse effects before dministering phrmcologic therpy to ptients. References 1. Prtnership for Cler Helth Communiction. Wht is helth? Ask Me 3 web site. http:// www.npsf.org/skme3/pchc/wht_is_helth.php. Accessed November 17, 2009. 2. Nielsen-Bohlmn L, Pnzer AM, Kindig DA, eds. Helth Litercy: A Prescription to End Confusion. Wshington, DC: Ntionl Acdemies Press; 2004. 3. Center on n Aging Society, Georgetown University. Low helth skills increse nnul helth cre expenditures by $73 billion. Center on n Aging Society web site. http://ihcrp. georgetown.edu/gingsociety/pubhtml/helthlit. html. Accessed November 17, 2009. 4. Bker DW, Willims MV, Prker RM, Gzmrrin JA, Nurss J. Development of brief test to mesure functionl helth. Ptient Educ Couns. 1999;38(1):33 42. 5. Dvis TC, Long SW, Jckson RH, et l. Rpid estimte of dult in medicine: A shortened screening instrument. Fm Med. 1993;25(6):391 395. 6. Chew LD, Brdley KA, Boyko EJ. Brief questions to identify ptients with indequte helth. Fm Med. 2004;36(8):588 594. http://www.stfm. org/fmhub/fm2004/september/lis588.pdf. Accessed November 19, 2009. 7. Wllce LS, Rogers ES, Roskos SE, Holidy DB, Weiss BD. Brief report: Screening items to identify ptients with limited helth skills. J Gen Intern Med. 2006;21(8):874 877. http://www. ncbi.nlm.nih.gov/pmc/rticles/pmc1831582/pdf /jgi0021-0874.pdf. Accessed November 19, 2009. 8. Bker DW. The mening nd the mesure of helth. J Gen Intern Med. 2006;21(8):878 883. 9. Prker RM, Kindig DA. Beyond the Institute of Medicine helth report: Are the recommendtions being tken seriously? J Gen Intern Med. 2006;21(8):891 892. 10. Prker RM, Bker DW, Willims MV, Nurss JR. The test of functionl helth in dults: A new instrument for mesuring ptients skills. J Gen Intern Med. 1995;10(10):537 541. 11. Nurss JR, Prker RM, Willims MV, Bker DW. TOFHLA: Test of Functionl Helth Litercy in Adults. Snow Cmp, NC: Peppercorn Books & Press; 2001. 12. Espllrdo NL. Decisions on dignosis in fmily prctice: Use of sensitivity, specificity, predictive vlues nd likelihood rtios. Asi Pc Fm Med. 2003;2(4):229 232. http://www.pfmj-rchive.com /fm2.4/fm_95.pdf. Accessed November 17, 2009. 13. Hinkle DE, Wiersm W, Jurs SG. Applied Sttistics for the Behviorl Sciences. 2nd ed. Boston, MA: Houghton Mifflin Compny; 1988. 14. Frnzblu AN. A Primer of Sttistics for Non-Sttisticins. New York, NY: Hrcourt, Brce & World; 1958. 15. Kirsch IS, Jungeblut A, Jenkins L, Kolstd A. Adult Litercy in Americ: A First Look t the Findings of the Ntionl Adult Litercy Survey. Wshington, DC: Ntionl Center for Eduction Sttistics, US Dept of Eduction; 1993. 16. Street RL Jr. Communiction in medicl encounters: An ecologicl perspective. In: Thompson TL, Dorsey AM, Miller KI, Prrott R, eds. Hndbook of Helth Communiction. Mhwh, NJ: Lwrence Erlbum Assocites; 2003:63 89. 17. Prikh NS, Prker RM, Nurss JR, Bker DW, Willims MV. Shme nd helth : The unspoken connection. Ptient Educ Couns. 1996;27(1):33 39. 18. Bker DW, Prker RM, Willims MV, et l. The helth cre experience of ptients with low. Arch Fm Med. 1996;5(6):329 334. DECEMBER 2009 FEDERAL PRACTITIONER 31