Proficiency Testing Manual

Transcription

1 Proficiecy Testig Maual cap.org

2 Table of Cotets Surveys & Educatioal Aatomic Pathology Programs Maual 1 Order Cofirmatio, Specime Hadlig, ad Customer Service 3 7 Cofirmatio 3 Regulatory Reportig 3 Biders/Glossaries 3 Alertig the Mailroom 3 CAP Idetificatio Number 4 Replacemet Specimes 4 Testig Istructios ad Completio Time 4 Chagig Results ad CMS Reportig Istructios 4 Fax-Back Respose Programs 5 Evaluatio Reports 5 Correctios 5 Customer Support 5 Program Certificates 6 Letters 6 Limitatios of Proficiecy Testig 6 Hadle With Cautio 6 Laboratory Accidets 6 Limitatios of Proficiecy Testig Letter 7 2 Geeral Commets 8 9 Completig the Result Form 8 Teleforms 8 Preprited Method Summary Page 8 Exceptio Codes 8 Idetificatio Master Lists 9 Method/Istrumet Master Lists 9 Hadwritig 9 Decimal Poits ad Box Positios 9 Less Tha or Greater Tha Values 9 3 e-lab Solutios TM Usig Olie Features 10 Security 10 Olie Data Submissio 11 Iteractive Evaluatios 11 Olie Reports 11 4 How to Iterpret the Evaluatio Report Geeral Guidelies for Evaluatio 12 Selectio of a Target Value 12 Calculatio of Summary Statistics 12 Comparative Statistics 14

3 Surveys & Educatioal Aatomic Pathology Programs Maual 5 Cotiuig Educatio CME Category 1 15 CE for Cytotechologists 15 Cotiuig Educatio (CE) for Nophysicia Laboratory Persoel 16 EXCEL CE Program 16 6 Disciplie-Specific Reportig Iformatio How to Complete the Result Form 18 7 The Evaluatio ad Participat Summary Reports Your Evaluatio Report 20 Reviewig Your Evaluatio Report 20 The Participat Summary 21 How to Perform a Self-Evaluatio 21 8 Laboratory Legislatio ad Ceters for Medicare ad Medicaid Services (CMS) Reportig Provisio of Results to CMS ad State Agecies 25 Use of Reaso Codes for Noevaluated Specimes 26 CMS Performace Summary Data 27 Iterlaboratory Compariso Program Cycle 1CAP Catalog Delivery September: The catalog is set to your laboratory. 2 Your Laboratory Subscriptios September December: Your laboratory places its order. 3CAP Order Processig September December: Order quatities are reserved. 4CAP 5 Order Cofirmatio Kits September December: Cofirmatio reports are set to your laboratory after your order is processed. 7Scietific Resource Committee Evaluatio Criteria The scietific resource committee reviews results ad the impact of evaluatio criteria. 8 Reports Surveys Mailig Mailed Kits are prepared ad set from the maufacturer to your laboratory. CAP Mailed/ Available Olie Reports are set to or made available olie for your laboratory, regulatory agecy, ad/or cosultats. 6Your Laboratory Result Form Completed result forms are retured via mail, fax, or olie submissio to the CAP where data are summarized. 9Certificate Certificate of Participatio As a CAP Surveys ad Aatomic Pathology Educatioal Programs ad EXCEL subscriber, your laboratory is etitled to receive a Certificate of Participatio. The certificate will be issued at the begiig of the program year.

4 1Order Cofirmatio, Specime Hadlig, ad Customer Service Cofirmatio After your order is received, a order cofirmatio report is set that cotais the followig iformatio: Shippig address Billig address Telephoe ad fax umber List of products ordered List of agecies ad/or cosultats to whom you have requested copies of your evaluatio report be set Please review your laboratory order cofirmatio report carefully. If you have chages, retur the form withi two weeks of receipt to: Customer Data Maagemet College of America Pathologists 325 Waukega Road Northfield, Illiois (fax) Regulatory Reportig The CAP will automatically forward results for aalytes regulated for proficiecy testig to the Ceters for Medicare ad Medicaid Services (CMS) for laboratories that have provided a CLIA idetificatio umber. To request that o results be forwarded or to make chages to your laboratory s aalyte selectio report for the iformatio provided to CMS, please cotact the Customer Cotact Ceter at optio 1 or access your report olie at cap.org. Documetatio will be requested ad may be faxed to A explaatio of regulatory reportig ad curret laboratory legislatio is icluded i Chapter 8 of this maual. Biders/Glossaries Please see your catalog for istructios o how to obtai three-rig biders for filig results ad reports. Surveys results are prited o threehole-puched sheets for storage i the biders. If you are erolled i hematology ad/or cliical microscopy Surveys cotaiig photographs for morphologic idetificatio, you will be able to access a olie glossary of terms for your geeral use. If you eed a hardboud copy, please call the CAP Customer Cotact Ceter. Alertig the Mailroom Refer to the shippig schedule ad advise your receivig departmet accordigly. A shippig schedule is provided to your laboratory with the orderig catalog, or you ca obtai a customized shippig caledar through e-lab Solutios at cap.org. The receivig departmet of your hospital or laboratory should be advised how to hadle the CAP kits. Isist o prompt trasfer to the laboratory. (Kits received by the hospital but ot delivered to the laboratory are ot eligible for free replacemet.) Uless otherwise specified, store the specimes i the refrigerator.

5 Order Cofirmatio, Specime Hadlig, ad Customer Service CAP Idetificatio Number Each participat receives a CAP idetificatio umber that is prited o all result forms. This umber will also appear o each evaluatio report received by your laboratory. It is helpful to have this umber available whe cotactig the College. Replacemet Specimes The kits cotai a result form ad specimes for aalysis. Check the cotets of the kit agaist the istructios. EXCEL participats should verify that all modules listed o the mailig page are icluded i the kit. If the kit is icomplete or cotais broke or ulabeled specimes, cotact the College withi the umber of days idicated i the kit istructios followig the actual shippig date for a free replacemet. Additioal replacemet specimes may be purchased withi the same time frame. Because proficiecy testig materials must be procured i advace of shipmet, o occasio, additioal ivetory is available for a omial fee. To purchase these materials cotact the Customer Cotact Ceter at optio 1. All literature associated with this product icludig the summary data would be provided. This optio does ot replace routie proficiecy testig. These materials may be used for but ot limited to: Competecy assessmet Istrumet troubleshootig Traiig Educatio Research I the evet that a replacemet specime is required, retai your origial result form while awaitig the arrival of the replacemet specimes. The replacemet specimes will be set i the same maer as your origial specimes. Whe you receive the replacemet specimes, you will be allowed the same amout of time for aalysis as was allowed with the origial shipmet. You are esured a evaluatio. Occasioally, it may ot be possible for the maufacturer to replace your specime(s) kit. I this case, fill the exceptio code 33 bubble o the result form. Specime Problem will appear o your evaluatio report, ad you will ot be pealized. Testig Istructios ad Completio Time Per the Federal Register, proficiecy testig (PT) specimes must be tested with the laboratory s regular workload, usig routie methods ad testig the PT specimes the same umber of times it routiely tests patiet specimes. Whe hadlig PT specimes, laboratories must ot commuicate results or share or refer specimes for tests ot o the laboratory s meu. If referral for testig is routiely performed for patiet specimes, the practice caot be followed for PT specimes. Referral is cosidered to be movemet of the specime from a laboratory with a CLIA idetificatio umber to aother laboratory that has a differet CLIA idetificatio umber. Laboratories must esure that persoel do ot share results or refer PT specimes for ay reflex or testig outside their CLIA idetificatio umber. The Surveys ad EXCEL programs are used for certificatio of certai laboratories. Sice promptess is cosidered i determiig certificatio, we caot accept late etries. Results are due by the date oted o the result form. Result forms received beyod the date oted will ot be evaluated. Participats will receive a evaluatio idicatig that the results were received past the evaluatio cut-off date alog with a Participat Summary that ca be used for self-evaluatio. Chagig Results ad CMS Reportig Istructios Chages to submitted data caot be made after the due date listed o the result form. Review all etries made o the result form for accuracy prior to submissio. For results approved olie, correctios must also be doe olie. Faxed or mailed correctios will ot be accepted. If you have registered your lab o the CAP website, you may use Result Form Verificatio to verify submitted results. For ay testig that you do ot routiely perform i your laboratory, leave all reportig areas for that test blak, icludig method iformatio. Please ote, a pealty will ot be applied for blak resposes i the case of 4 College of America Pathologists Proficiecy Testig Maual

6 Order Cofirmatio, Specime Hadlig, ad Customer Service educatioal challeges, challeges ot formally graded, or the proper use of exceptio codes. If you do ot perform specific testig i a Surveys program, please refer to the kit istructios ad result form for the appropriate istructios. For ay regulated aalytes that your laboratory does ot report or may have discotiued, you must otify the CAP i writig. Please fax ay chages i writig o your CMS report to to avoid receivig a zero score o your ext PT evaluatio. (Your reportig prefereces are outlied o the CMS Aalytes Reportig Selectios documet, which is available olie at cap.org o the Accreditatio ad Laboratory Improvemet tab.) If you have ay questios, please call the Customer Cotact Ceter at optio 1. Fax-Back Respose Programs The College offers immediate (fax-back) resposes for the submissio of the laboratory forms for the followig educatioal aatomic pathology programs: Iterlaboratory Programs i Cervicovagial ad Nogyecologic Cytopathology (PAP/NGC) Fie-Needle Aspiratio Glass Slide Program Idividual result forms ca be submitted olie (preferred method) or faxed to the CAP. CME/CE forms will be self claimed olie through the Educatio tab. Evaluatio Reports The evaluatio report will be posted olie ad mailed approximately two to five weeks after the ship date of the kit. This time is eeded for processig data, establishig evaluatio criteria, ad preparig the summary report. Correctios Occasioally, icorrect etry of submitted data occurs. If this is due to your trascriptio error or failure to complete the result form appropriately, your etry caot be re-evaluated. If the error is made by the CAP, please cotact the Customer Cotact Ceter at optio 1 for further assistace. Customer Support Customer Cotact Ceter Hours 7:00 am 5:30 pm CT Exteded Customer Cotact Ceter hours esure coverage for all time zoes. Call optio 1 to speak with a Cotact Ceter represetative. For iteratioal customers, please call optio hour Messagig Service The CAP s 24-hour voice mail system allows you to leave a message after hours. A respose will be provided the followig busiess day. 24-hour Service Fax Hotlie ( ) As with the 24-hour voice mail system, the Service Fax Hotlie makes it easy for you to fax requests ay time, day or ight. Respose is guarateed by the ext busiess day. Cotact the CAP via the Website You ca submit your request or questio coveietly olie via the CAP Web page. Log o to cap.org ad click o the Cotact Us ico located at the top of the mai page. CAP Mail The CAP has implemeted a otificatio service, CAP Mail, desiged to keep you iformed of our receipt of your order form(s) ad result forms. For each documet type, the CAP will otify the appropriate idividuals that we have received your iformatio. All result form receipt ackowledgmet messages iclude a result form receipt. Lik to the CAP website page where you ca review detailed iformatio for each kit o the umber ad specific pages received ad method of receipt. To take advatage of this service, esure the CAP has the appropriate addresses for your laboratory. For documet ackowledgmet, iclude the appropriate address o the first page of your order form i the sectio titled, PT Shippig Cotact. To choose ot to participate i this program, participats ca cotact the CAP Customer Cotact Ceter at optio Optio 1 cap.org 5

7 Order Cofirmatio, Specime Hadlig, ad Customer Service Cotact the CAP via Geeral iquiries may be routed to the College at the followig address: Program Certificates After the completio of the program year, participatig laboratories will receive a program certificate that recogizes each istitutio s participatio i the CAP proficiecy testig program ad its commitmet to patiet care. Certificates are siged by the CAP presidet ad are suitable for framig. Letters All letters received by the College are reviewed ad, if appropriate, forwarded to a medical techologist or a scietific resource committee member for respose. Your iput is ecouraged ad has always served as a valuable vehicle for chages ad improvemets to the iterlaboratory compariso programs. The College does ot require that you submit documetatio for all proficiecy testig deficiecies. However, it is recommeded that such documetatio be retaied i your laboratory. The CAP Laboratory Accreditatio Program issues a separate report, the Proficiecy Testig Exceptio Summary, that addresses proficiecies for CAP-accredited laboratories. Istructios for respose will be icluded with the report. Limitatios of Proficiecy Testig Hadle With Cautio Proficiecy testig specimes must be hadled with cautio. Each shipmet icludes a biohazard warig statemet explaiig proper hadlig. Laboratory Accidets Icidets of persoel exposure to ifectious specimes, through eedlesticks, cotamiatio of the mucous membraes through splashes or aerosolizatio, or cuts from cotaiers, should be reported immediately to the CAP. 24-hour hotlie: Please try to have the followig iformatio available: CAP umber Phoe umber Name of istitutio/city/state Name of perso affected, if other tha caller Date ad time of icidet Where ad how affected Survey ad specime umber Name ad telephoe umber of laboratory director This iformatio will be relayed to a pathologist member of the appropriate resource committee who will cotact the participat s laboratory director or hospital employee health services physicia with istructios cocerig prophylaxis. Due to the maufactured ature of the specimes ad the logistics of shippig, proficiecy testig does ot always correlate with the maer i which fresh, cliical specimes are hadled. A letter addressig these differeces is icluded o page 7 for geeral use by your laboratory. 6 College of America Pathologists Proficiecy Testig Maual

8 Limitatios of Proficiecy Testig Letter The College of America Pathologists (CAP) Surveys program is the largest exteral quality assessmet program i the world. As such, it provides a uparalleled selectio of challeges ad offers the largest database i existece for iterlaboratory compariso. The CAP has accumulated sigificat experiece i maagig this type of program ad is kowledgeable i its uses ad limitatios. Performace o CAP Surveys is ot to be take as the sole idicator of a laboratory s abilities. A proficiecy testig Survey is but oe of a umber of programs that laboratories should employ to assess, maage, ad improve quality. I additio to Surveys, proper method validatio, quality cotrol testig, periodic calibratio ad istrumet maiteace, employee competecy testig, ad laboratory ispectio ad accreditatio provide importat tools for measurig laboratory performace ad esurig quality. The Surveys program, although outstadig, is ot a perfect measurig device. A umber of factors limit this tool s ability to measure laboratory accuracy. Specific limitatios iclude requisite use of matrix materials that may impact test systems differetly tha patiet specimes; the appropriateess of groupig resposes accordig to methodology, istrumetatio, ad test platforms; varyig size of compariso groups with attedat variability of statistical parameters; regulated limitatios i samplig of laboratories testig systems; difficulties i quatitatio at the extremes of aalyte cocetratio; ad usuitability of certai federally-madated evaluatio limits. Thus, a certai umber of resposes that are graded as uacceptable i Surveys will i fact be acceptable, ad a certai umber of resposes graded as acceptable will i fact be uacceptable. Although usuccessful or usatisfactory Surveys performace may reflect problems withi a laboratory, it does ot costitute proof of iadequate performace or a iability to meet patiet eeds. Sicerely, R. Bruce Williams, MD, FCAP Chair, Coucil o Scietific Affairs

9 2Geeral Commets Completig the Result Form The result form is a prepared form o which you record your methods of aalysis ad results of laboratory testig. The completed result form must be retured olie or by fax to the CAP for evaluatio. Prepare your result form carefully accordig to the istructios. Double-check your aswers for accuracy ad completeess. It is importat to photocopy or prit a copy of the completed forms for your records before mailig or submittig them olie to the CAP. Per directive from the Ceters of Medicare ad Medicaid Services (CMS), chages to submitted data caot be made after the due date prited o the result form. Review all iformatio o the method summary page ad all etries made o the result form for accuracy prior to submissio. Oce you have opted i, you ca use the Result Form Data Etry ad Receipt Verificatio optio o the CAP website (uder e-lab Solutios) to verify the submitted data. Teleforms Teleforms are scaable forms. Because these forms are scaed, please refer to the kit istructios for more detailed istructios o completig the result form. Preprited Method Summary Page The computer system is desiged to ehace result reportig from your laboratory. Oce you have iitially provided a master list code for a method, istrumet, ad/or reaget, the computer will maitai these codes, edig your eed to report them throughout the year. Please check each master list to esure the correct codes are listed. Should you eed to chage a code, eter it i the appropriate boxes o the result form. Exceptio Codes If it is ecessary for you to report a aalytical problem for a etire test or idividual specimes withi a test, leave the result area blak ad fill the bubble for the appropriate exceptio code. Select the appropriate two-digit code from those listed ad fill the appropriate bubble. Exceptio Code* Reaso 11 Uable to aalyze (documetatio to be provided by laboratory). 22 Result is outside method/ istrumet reportable rage. 33 Specime determied to be usatisfactory after cotactig the CAP. * It is the laboratory s resposibility to documet the appropriate use of these exceptio codes should this be requested durig a laboratory ispectio. Please refer to the kit istructios for more iformatio.

10 Geeral Commets Idetificatio Master Lists Master lists of possible idetificatios are provided for microbiology, blood cell idetificatio, urie sedimet, cliical microscopy, ad provider-performed microscopy. Select your aswers from the appropriate master lists provided. Record the appropriate master list code for your choice o the result form. For blood cell idetificatio, urie sedimet, cliical microscopy, ad provider-performed microscopy, all possible idetificatios are icluded o the master lists. Do ot use the code 010, Other, Specify. The use of this code will be evaluated as a uacceptable respose. Method/Istrumet Master Lists Method/istrumet master lists are provided i the kit istructios. Choose the appropriate istrumet ad/or method ad provide this iformatio o the result form. It is importat that you otify the CAP ad the maufacturer if your istrumet or method is ot listed o a master list. This may be doe by listig this iformatio i the Use of Other sectio of the result form. You may also cotact the CAP directly while completig the result form to see if a code has already bee established for your method ad/or istrumet but was ot available i time for pritig of the result form. Hadwritig Decimal Poits ad Box Positios The computer is programmed to accept oly those aswers coformig to the boxes ad decimal poits o the result form. If a umber is ot large eough to fill the boxes, isert zero i the remaiig spaces. Results should be right-justified. Whe submittig results olie, this will be doe automatically Example: Correct Icorrect Glucose 73 mg/dl Urea Nitroge 12.8 mg N/dL Less Tha or Greater Tha Values Do ot attempt to add less tha or greater tha to the value you submit uless this optio is provided o the result form. Where provisio is made to report less tha or greater tha results, you must fill i the bubble of the appropriate box to idicate your respose is a less tha or greater tha value. All other results will be cosidered equal to values. Where o optio to report greater tha or less tha is give, refer to exceptio codes o page 8 or i your kit istructios. The result forms are desiged for quick, easy scaig by our computers. If you fax your results, the iformatio o your result form must be clearly readable Optio 1 cap.org 9

11 .. 3e-LAB Solutios S O L U.. T.. -LAB I.. O.. N.. S... TM.... Usig Olie Features Powerful Iteret-based tools are available to give participatig laboratories may excitig choices. Simple istructios for use of these tools are provided. Ad as always, CAP Customer Cotact Ceter represetatives stad ready to offer assistace. I order to take full advatage of the fuctioalities available olie, users ca selectively eable laboratories ad persoel by udertakig erollmet procedures as outlied below: Self Registratio This is the process of creatig a accout with the College for olie activities. All users of the advaced features of the CAP website are required to have a persoal Web accout ad be logged i to be able to use the site. This process oly eeds to be doe oce per user; it allows the user to select a ID that ca be easily recalled ad that belogs to the user, regardless of laboratory affiliatio. Both members ad omembers are required to be registered, thereby allowig the etire laboratory commuity to participate olie. This registratio is also valid for CME/CE olie learig activities. Loggig I Oce users have established a accout with the College, they will be prompted by the system to eter their user ID ad password. This must be doe every time users visit the site, or users ca choose to have the system remember their logi. Both members ad omembers utilize the same logi fuctioality. Optig I A user (the laboratory admiistrator, by default) eters a CAP # ad Collegeprovided PIN #, thereby eablig the laboratory to access advaced features olie. This process oly eeds to be doe oce per lab; ad oce accomplished, the lab will have access to all of the features curretly accessible olie as well as those that will be offered i the ear future. Security The ature of the idividualized accouts for laboratory users allows for flexibility i determiig what levels of access each user should have. Because users are idepedet etities from the laboratory, the users ca be associated with multiple laboratories ad their security ca be admiistered i a differet maer at each site. This is carried out by the delegated security admiistrator at the laboratory who is resposible for overseeig the access ad site privileges for each user associated with that lab. Because of the chagig ature of the employee-employer relatioship i the laboratory, this feature allows the most flexibility for the admiistrator to add, modify, ad/or remove users access privileges as the admiistrator sees fit. This has the overall effect of elimiatig the eed to have to call the Customer Cotact Ceter to carry out some of the more routie maiteace of a accout. The delegated security admiistrator(s) ca cofer as may or as few permissios as they see fit, ad the admiistrator(s) ca cofigure persoel security so that it closely matches idividuals roles withi the laboratory. This will eable the lab to replicate the same workflow model that it employs for paper-based submissio of results, thereby helpig to icrease the itegrity ad cofidece i the data etered olie.

12 e-lab Solutios Olie Data Submissio Oce the lab has opted i ad security for employees has bee admiistered, oe of the first features a lab ca take advatage of is the ability to submit its data results olie. A lab will receive its PT material ad paper result forms as usual ad will test them i its customary maer. Most laboratorias may prefer to record their results o the paper result form for ease of trascriptio to the olie system. The users will log i to the site, avigate to their lab ad kit, ad they will be preseted with a Adobe PDF versio of the same paper result form they have i had. The form is desiged to prevet errat data from beig saved to the form. Users will be able to select method ad qualitative codes from drop dow lists rather tha havig to refer to a separate documet for the code. Ivalid codes ad data will prevet the user from savig the form. Users will eter the data ito the virtual form ad save the data as they fiish. For Surveys i which multiple laboratorias perform the testig ad resultig, the flexible ature of the data etry system will allow multiple users to log i to the system at differet times ad eter their data, so as to miimize ay iterruptio to their ormal laboratory work. Depedig o the ormal workflow withi the lab, the admiistrator has a choice to desigate a additioal approver role for aother user whose job it is to verify that all data was etered as iteded ad to submit the data. Oce this step is fiished, the data is committed to the College s system ad is ready for ormal processig. The advatages to this system lie i speed, efficiecy, ad clarity of the resultat data. I additio to these beefits, labs who cotiue to fax their result forms i ca also take advatage of the olie etry sice the system will be able to preset the data as it has bee iterpreted by computerized scaig equipmet. Users ca access their iterpreted data ad will have the opportuity to correct ay scaig errors that may have occurred due to the ature of errors iheret i these media. Givig the participats the opportuity to correct for these scaig errors also icreases the accuracy of their data which, if allowed to propagate through the system, could have egative cosequeces i their performace iterpretatio. Iteractive Evaluatios As soo as the College receives a statistically sigificat populatio of the results from kits mailed for each Survey mailig, the data are processed ad graded accordig to striget govermetal ad committee criteria. The lab results ad gradig iterpretatios are displayed i a idividualized report that will be available to all users via paper or olie. The paper report that labs are used to seeig has a updated graphical format. The ew format also allows for more data to be displayed per page, thereby cuttig dow o the amout of paper that must be stored for future retrieval. The olie versio is available i a static format that ca be prited or dowloaded ad stored locally, thereby egatig the eed to create a separate library of evaluatios for future referece. I additio, the olie evaluatio will allow the user to avigate through the evaluatio ad browse the data aalyte by aalyte. Also, users will beefit by the iclusio of detailed images that will be hyperliked from the olie evaluatio report oly. Laboratory maagers will fid a useful tool i the All Aalyte Scorecard, which will allow them to filter ad customize the scorecard data for their laboratory as a meas of idetifyig deficiecies or treds i performace. Olie Reports Other acillary reports that accompay the evaluatio report will be available olie, as well, for review ad to dowload for future referece. This icludes participat summary reports, fial critiques, ad aual summaries, which cotai useful data ad educatio that ca be reviewed ad accessed by all users with appropriate security. These documets may be dowloaded ad stored locally for future use. This will help to icrease the dissemiatio of the iformatio cotaied i these documets to multiple persoel, especially ow that the iformatio cotaied therei is beig used icreasigly for educatioal ehacemets for which idividuals ca obtai cotiuig medical educatio/cotiuig educatio (CME/CE) credits Optio 1 cap.org 11

13 4How to Iterpret the Evaluatio Report This chapter icludes geeral iformatio regardig evaluatios. A sectio explaiig how to iterpret your idividual evaluatio will be icluded with each mailig. Geeral Guidelies for Evaluatio O February 28, 1992, the Secretary of Health ad Huma Services published the fial rules implemetig the Cliical Laboratory Improvemet Amedmets (CLIA) of These regulatios established evaluatio criteria limits for may of the aalytes icluded i the CAP Surveys programs. The target values are determied by the scietific resource committees of the CAP. For those aalytes ot icluded i the proficiecy testig portio of CLIA, the target values ad evaluatio criteria are determied solely by the scietific resource committees. Selectio of a Target Value To miimize the effect of method differeces ad to allow compariso of all methods, participats results are combied ito comparable method/istrumet groups called peer groups. It is importat for participats to provide complete iformatio regardig the method or istrumet used i order to be combied i the appropriate peer group. For most aalytes, the peer group mea is desigated as the target value for evaluatio. The peer group mea is the preferred target if o sigle target value exists that ca provide a accuracy-based target that is traceable to the true value as determied by a defiitive or referece method aalysis. The peer group must cosist of greater tha ie results after outlier exclusio, ad the variability of the peer group data must ot be too great. If peer group data are ot available or are too variable, method group statistics may be used. The method group must also cosist of greater tha ie results ad demostrate acceptable variability before it is used as the target group. If peer ad method group statistics are ot available, a comparative method group may be desigated as the target mea. The comparative method is ot the method recommeded by the College. However it is established as a historically reliable method ad is used for evaluatig results from methods that have a isufficiet umber of participats to geerate a separate peer group ad/or method group statistics. If o comparative method exists for the aalyte, results will ot be evaluated. For some aalytes, a sigle target value is used i which cosistet results are demostrated across all peer groups. Calculatio of Summary Statistics Peer Group Results Results are grouped accordig to the method used for aalysis ad screeed for outliers. Various statistics are calculated from the remaiig data that summarize the peer group s resposes. These summary statistics iclude the followig: the mea (the average of the reported results); the stadard deviatio (a measure of the variability of the participat results, ofte abbreviated as SD);

14 How to Iterpret the Evaluatio Report the coefficiet of variatio (CV); the media (the middle value i a ordered list of the o-outlier results); the low value; the high value; the fial cout of reported results that were ot excluded as outliers. Outlier Detectio Techique Outlier exclusio is ecessary because a large series of results frequetly will iclude some aberrat values. These may arise from istrumet malfuctio, techical errors, specime mix-ups, misplaced decimals, icorrect uits of measure, or data etry errors. If ay results are excluded, the outlier process is repeated usig the remaiig values. The summary statistics that appear o your reports do ot reflect results that were cosidered to be outliers durig either outlier pass. Quatitative Procedures/Roudig All quatitative resposes are evaluated based upo a rage of acceptability. This rage is determied usig a target value ad a limit. The limit will be either a fixed iterval (eg, ± 5 mg/dl), a percetage of the mea (eg, ± 25%), a SD (eg, ± 3 SD), or a variable rage (eg, ± 6 mg/dl or 10%, whichever is greater). The Participat Summary icluded with your evaluatio will list the criteria used to evaluate your performace. The followig sectios provides specific examples of how to calculate the rage of acceptability depedig upo the criteria used. Fixed Rage Example Your laboratory reports a sodium result of 138 mmol/l. The peer group mea is mmol/l. The evaluatio limit for sodium is ± 4 mmol/l. The acceptable rage is determied by the formula mmol/l ± 4 mmol/l, which is 135 to 144 mmol/l. Therefore, your reported result of 138 mmol/l is withi the calculated acceptable rage of 135 to 144 mmol/l whe usig beefitof-the-doubt roudig. Percetage of the Mea Example Your laboratory reports a albumi result of 3.1 mg/dl. The peer group mea is 3.39 mg/dl. The evaluatio limit for albumi is ± 10%. Te percet of 3.39 mg/dl is 0.34 mg/dl. The acceptable rage is determied by the formula 3.39 mg/ dl ± 0.34 mg/dl, which is 3.0 to 3.8 mg/dl whe usig beefit-of-the-doubt roudig. Therefore, your reported result of 3.1 mg/dl is withi the calculated acceptable rage of 3.0 to 3.8 mg/dl. Stadard Deviatio Example Your laboratory reports a TSH result of 16.4 µu/ml. The peer group statistics are as follows: mea = 15.7 µu/ml, SD = 1.5, ad CV = 9.6. The evaluatio limit for TSH is ± 3 SD. 3 x 1.5 = 4.5. The acceptable rage is determied usig the formula 15.7 µu/ml ± 4.5 µu/ml, which is 11.2 to 20.2 µu/ml whe usig beefit-of-the-doubt roudig. Therefore, your result of 16.4 µu/ml is withi the acceptable rage of 11.2 to 20.2 µu/ml. Variable Rage Example Your laboratory reports a total bilirubi result of 4.5 mg/dl. The peer group mea is 4.68 mg/dl. The evaluatio limit for total bilirubi is ± 0.4 mg/dl or 20%, whichever is greater. Twety percet of 4.68 is Sice the percetage limit of 0.94 is greater tha the iterval limit of 0.4, the percetage limit is applied to the target value. The acceptable rage is determied usig the formula: 4.68 ± 0.936, which is 3.7 to 5.7 mg/dl whe usig beefit-of-the-doubt roudig. Therefore, your result of 4.5 mg/dl is withi the acceptable rage of 3.7 to 5.7 mg/dl. The Participat Summary Report icluded with your evaluatio report will list the criteria used to evaluate your performace. To determie the acceptable rage, a beefit-of-the doubt roudig procedure is utilized whe gradig. The upper limit of acceptability is obtaied by roudig up to the ext reportable result, while the lower limit is determied by trucatig. Calculatio of the Stadard Deviatio Idex (SDI) The computer-prited evaluatio report lists your results ad the evaluatio statistics for your peer group. It also lists your ormalized results as a SDI. This value is obtaied by subtractig the group mea from your result ad the dividig by the SD. The SDI is expressed i terms of the umber of SDs from the mea, with a arithmetic sig idicatig the directio of the differece. The calculatio of the SDI ormalizes your result ad, therefore, allows for a compariso of results from specimes of differet cocetratios of a aalyte Optio 1 cap.org 13

15 How to Iterpret the Evaluatio Report Whe a comparative method has bee desigated for Surveys aalysis, a secod set of statistics is listed o the pritout comparig your results with those obtaied usig the comparative method. The figures show are the mea ad SD for the comparative method ad your result as a SDI usig these statistics. It is possible for your result to be defied as good performace i your method group ad yet produce a comparative SDI greater tha two. This will occur if your method has a large aalytic bias or a large SD. It is possible to receive a comparative SDI lower tha the method group SDI, although this rarely occurs. I practice, most participats receive similar method group ad comparative SDIs. Comparative Statistics Quatitative Procedures Your evaluatio report will display plots of the relative distace of your reported results as a percetage of allowable deviatio from the target value. The umeric digit idicates the umber of results at a plot locatio. The allowed deviatio may be calculated as follows: If your result is greater tha the target mea: Percetage of your result-target Acceptable Deviatio =100 x upper limit-target mea If your result is less tha the target mea: Percetage of your result-target Acceptable Deviatio =100 x target mea-lower limit Qualitative Procedures For qualitative resposes, cosesus agreemet of referee or participatig laboratories is used for evaluatio. Geerally, 80% agreemet is required. 14 College of America Pathologists Proficiecy Testig Maual

16 5Cotiuig Educatio CME Category 1 The College of America Pathologists (CAP) is accredited by the Accreditatio Coucil for Cotiuig Medical Educatio (ACCME) to provide cotiuig medical educatio for physicias. The CAP desigates these educatioal activities for a maximum of the stated umber of AMA PRA Category 1 Credits. Physicias should oly claim credits commesurate with the extet of their participatio i the activity. The America Medical Associatio has determied that physicias ot licesed i the US who participate i these CME activities are eligible for AMA PRA Category 1 Credit. See the curret Surveys catalog for available CME programs. CE for Cytotechologists Cytotechologists may apply the credits from the FNAG, FNA, PAPCE1/PAPJE1/PAPKE1/PAPME1, PAP PT ad NGC programs toward the required educatioal activities for the America Society for Cytopathology (ASC) Cotiuig Educatio Credit Program. Olie Virtual Microscopy Educatio Programs The CAP offers olie educatio programs that use advaced imagig techology to preset images from actual glass slides for a variety of sites ad specime types. This techology simulates a microscope, allowig you to sca the image ad use multiple magificatios to view the material. From the images ad cliical iformatio provided, you select a diagosis, aswer learig assessmet questios, ad receive immediate feedback olie. See the curret Surveys catalog for available olie virtual microscopy educatio programs. 15

17 Cotiuig Educatio Cotiuig Educatio (CE) for Nophysicia Laboratory Persoel Disciplie Chemistry Coagulatio Hematology Histology (HistoQIP) Immuology Microbiology Therapeutic Drug Moitorig/Edocriology Toxicology Trasfusio Medicie Surveys CE Programs Maximum CE Credits The umber of credits are specific to the program mailig. Please go to cap.org ad click o the Educatio tab for up-to-date activity listigs. EXCEL CE Program For the EXCEL program, each idividual may receive cotiuig educatio (CE) credits for participatig i the educatio activities. Selected mailigs of EXCEL will iclude a educatio activity desiged to provide techical ad otechical iformatio to keep your staff o the leadig edge to esure quality patiet care. The educatio activity cosists of a related readig foud i the Participat Summary ad learig assessmet questios olie at cap.org. All laboratory professioals i your lab may ow ear idividual CE credits by completig the related educatio readig ad olie learig assessmet questios o the CAP website. Upo completio of the educatio activity, you will receive a CAP cotiuig educatio certificate. Desiged by the CAP Poit-of-Care Testig Committee, each activity combies the uique perspective of the committee s pathologist, cliicia, ad medical techologist members. The educatio activities are desiged to challege ad educate while providig the coveiece of earig free CE credits ad fulfillig licesure ad educatio requiremets without leavig your laboratory. The educatio activities are also valuable tools for i-house cotiuig educatio programs. Learig Cycle Iformatio Each educatio activity provides iformatio o commo techical ad otechical issues ecoutered i all laboratory settigs. To receive cotiuig educatio credit, you must complete the educatio readig provided i your Participat Summary ad aswer the olie learig assessmet questios. Each educatio activity will be available for six moths ad must be completed withi that time frame. Cotiuig educatio credit will be applied toward the year i which the activity is completed. Detailed iformatio o how to access the olie compoets will be icluded i each Participat Summary. 16 College of America Pathologists Proficiecy Testig Maual

18 Cotiuig Educatio Overall Learig Objective Upo completio of these educatio activities, the learer will be able to: 1. List preaalytic, aalytic, ad postaalytic variables that affect patiet testig. 2. Idetify quality improvemet opportuities withi his/her ow laboratory settig. 3. Apply appropriate quality assurace measures to esure accurate patiet results. CE (Cotiuig Educatio) This activity is acceptable to meet the cotiuig educatio requiremets for the ASCP Board of Registry Certificatio Maiteace Program. This activity is approved for cotiuig educatio credit i the states of Califoria ad Florida Optio 1 cap.org 17

19 6Disciplie-Specific Reportig Iformatio How to Complete the Result Form To esure statistically valid data, it is essetial that participats provide all ecessary method, reaget, ad/or istrumet iformatio as required. Please remember, oce you have provided accurate iformatio, our computer system will retai this iformatio. Verify that the correct method, reaget, ad/or istrumet code is oted o the preprited method summary page icluded with your result form or listed o the olie result form. If a code is ot oted or you ve chaged istrumets, you must eter the correct code o the result form. Hematology To report your blood cell idetificatio, select the best idetificatio code from the Hematology Blood Cell Idetificatio Master List provided i the kit istructios. To assist you with blood cell idetificatio, the Hematology, Cliical Microscopy, ad Body Fluids Glossary is availiable olie at cap.org. If results are reported for both blood cell idetificatio ad auto differetials, the blood cell idetificatio will be reported to CMS. The Hematology Blood Cell Idetificatio Master List choice, Immature cell or abormal cell, would refer for idetificatio, must be reserved for cells you rarely ecouter ad are uable to specifically idetify. Gradig of this respose will follow the guidelies set forth i the July 26, 1993, Federal Register Notice. Coagulatio For plasma-based coagulatio testig (PT, APTT, Fibrioge), a istrumet ad reaget code are required for proper evaluatio. Participats erolled i whole blood proficiecy testig for PT eed oly idicate a istrumet (if requested) ad their results. For all prothrombi time modules, reportig of Iteratioal Normalized Ratio (INR) results is optioal. Plasma-based ad whole blood INR are evaluated. Urialysis There is a separate urialysis ad specific gravity method master list ad istrumet master list. To esure a accurate peer group evaluatio of your results, it is critical to provide accurate method ad istrumet iformatio. A specific list of reportig optios is provided for each urialysis procedure. It is ot feasible to provide a list of reportig choices specific for every possible dipstick beig marketed to laboratories. Subsequetly, the result rages listed may ot exactly correlate with the rages used with your istrumet/dipstick. I these few cases, choose the rage that most closely matches your iteded result. To esure a accurate peer group evaluatio of your results, it is critical to select a appropriate result that the method allows. To report urie sedimet, cliical microscopy, or provider-performed microscopy, select the best idetificatio code from the Urie Sedimet/ Cliical Microscopy Master List provided. To assist with idetificatio, the Hematology, Cliical Microscopy, ad Body Fluids Glossary is available olie at cap.org.

20 Disciplie-Specific Reportig Iformatio Urialysis Dipstick Tests For qualitative procedures i urialysis, evaluatio is based o participat cosesus by method ad istrumet. For each aalyte, a miimum of two, but ot more tha four, resposes will be give a passig score. Aalyte results graded good performace must have 80% participat cosesus. Eighty percet participat cosesus ca be determied by groupig the mode with the ext oe or two most frequet resposes. This group will be give good performace. Acceptable performace will be give to additioal resposes util a miimum of 90% of participat results are give a passig score. I the case of a egative specime, egative resposes must costitute 90% participat cosesus. Specimes with results for oe or more methods distributed over both egative ad positive respose (ocosesus) will ot be evaluated. Specimes for which there is greater tha 90% of participat resposes distributed over more tha four resposes will be graded as ocosesus. Microbiology Where appropriate, a cliical diagosis, age, ad source are listed to simulate a true cliical situatio ad to allow laboratory persoel to select appropriate media or methods for processig these specimes. However, as the pathogeic bacteria preset i ay of these samples may be isolated from multiple sources of the body, all participats should attempt idetificatio of the orgaisms preset i all these specimes. Per the Federal Register, a Survey must grade a laboratory s ability to distiguish betwee a pathoge ad a cotamiat. Culture challeges will be desigated i the istructios to be hadled as idetify pricipal pathoge or idetify all orgaisms challeges. Participats must report i this maer eve whe this differs from their laboratory s routie practice. For example, a urie specime cotais Klebsiella peumoiae ad Staphylococcus epidermidis. If the istructios idicate to Idetify all orgaisms, both orgaisms should be reported. If the istructios idicate to idetify pricipal pathoge, oly the Klebsiella peumoiae should be reported. If the Staphylococcus epidermidis is reported, it would be pealized. Specime results will be evaluated if 80% or more of the participat laboratories agree o the idetificatio of the test orgaism(s) to geus or to geus ad species. I the absece of participat cosesus, referee laboratories will be used. The CLIA regulatios state that a laboratory must perform a miimum of five specimes i each testig evet for the subspecialty of bacteriology. The five challeges ca iclude a combiatio of the followig specimes: bacterial atige detectio bacterial idetificatio (culture) Gram stai atimicrobial susceptibility Please ote that procedures assayed with waived methodologies will ot cout toward the five-challege miimum. The laboratory is resposible for maitaiig the five specimes per testig evet for its remaiig owaived tests i the subspecialty whe a test is waived by the FDA midyear. Atimicrobial Susceptibility Testig Participats will be asked to perform susceptibility tests usig the atimicrobial agets ad techiques i routie use i their idividual laboratories. The laboratories should report oly atibiotics appropriate for therapy of ifectios at the site idicated i the patiet history. See curret Cliical ad Laboratory Stadards Istitute (CLSI) documets M2, M7, M100, or other appropriate documets for guidace. Iterpretatio results (susceptible, itermediate, ad resistat) will be pealized for: Agets that are ot cliically appropriate for the site of ifectio (meigitis, peumoia, uriary tract, etc) Use of methods CLSI advises agaist Use of methods that the maufacturer recommeds agaist usig, due to poor performace Selective reportig for the presumed site of ifectio helps improve cliical relevace, ecourages appropriate therapy, ad helps to miimize selectio of resistace Optio 1 cap.org 19

21 7The Evaluatio ad Participat Summary Reports Your Evaluatio Report Shortly after you submit your proficiecy testig results to the CAP, a evaluatio report evaluatig your submitted results will be available olie or mailed back to you. Your evaluatio report ca be used as a quality assurace tool to assess how you performed compared to other participats. To beefit from this report, it is importat that you review ad uderstad the iformatio preseted. Here is a review of the iformatio cotaied o your evaluatio report: Demographic Iformatio: Provides iformatio about your laboratory, icludig the ame of your istitutio, your CAP Idetificatio Number, ad ay agecies or cosultats you have desigated to receive copies of your evaluatio. Result Area: Cotais all results reported for a particular mailig ad statistical data used for evaluatio purposes. A detailed descriptio of evaluatio data specific for each disciplie is preseted i each Participat Summary. CMS Performace Summary Report: Icludes iformatio o curret ad cumulative performace for regulated aalytes to be set to CMS. Reviewig Your Evaluatio Report 1. Review the demographic iformatio o the evaluatio report. If ay iformatio is icorrect or has chaged, cotact the CAP at optio Compare iformatio o your evaluatio report with results o your photocopy or prited copy of the result form. If ay of your data was etered by the CAP icorrectly, cotact us immediately. Correctios due to data etry errors made by the CAP must be requested withi four weeks after the first evaluatio was mailed. 3. Look for ay uacceptable results. Commo, easily corrected reasos for uacceptable results iclude: Icorrect or icomplete method/ istrumet data Clerical error Decimal poit placemet Specime hadlig error Remember, whatever the cause, CLIA states that all PT deficiecies must be documeted ad corrective actio take to resolve the deficiecy. 4. Although the results may ot be formally evaluated, you ca compare your results with the data provided i the Participat Summary. You ca use the all method mea or media, low, ad high values to compare your results for a self-assessmet of your performace. To truly realize the beefit of proficiecy testig, it is importat that you take the time to carefully review your evaluatio. You ca gai valuable isight ito your laboratory s overall processes by followig these easy steps i reviewig this report.