Audit Power Tools: Understanding and Using Reports and Graphs

Transcription

1 2017 Diabetes in Indian Country Conference Audit Power Tools: Understanding and Using Reports and Graphs Karen Sheff IHS Division of Diabetes Treatment and Prevention 9/20/2017

2 Objectives 1. Generate Audit reports using the WebAudit. 2. Review and understand Audit reports. 3. Interpret results in Audit trends graphs.

3 Audit Report Basics

4 What are Audit Reports (capital R) The Audit Report (capital R) provides a basic summary of the data elements in the Audit for each year. Results are presented as the number and percent of patients who meet each report item. IHS and Area results are provided on reports from the WebAudit for comparison. Order of report items does not match order on form. Audit Reports can be obtained from the WebAudit or RPMS/DMS (known as Cumulative Audits). Results for both reports should be the same IF run on the same patients and at the same date and time (exception is minor differences due to rounding).

5 Other Audit reports (lowercase R) Other Audit summary reports (lowercase r) are available via the WebAudit and/or RPMS. These reports are available for each year and include: SDPI Key Measures: 2015 and 2016 versions (WebAudit and RPMS) Cardiovascular Disease (WebAudit only) Renal Preservation (WebAudit only) Means (WebAudit only) Graphs of Audit results over time for selected measures are available via the WebAudit - Trends Graphs and Means Graphs

6 How to get Audit reports NOTE: WebAudit access is limited to individuals who participate in conducting Audits or as determined by facility staff. RPMS/DMS access is determined by facility staff. 1. If you do have access to the WebAudit and/or RPMS: WebAudit: See DDTP website and training for info RPMS: See RPMS/DMS manual and training for info 2. If you do not have WebAudit or RPMS access: Request a report from your facility Contact your Area Diabetes Consultant

7 First several items from page 1 of 7: Audit Report - WebAudit

8 Audit Report (Cumulative Audit) RPMS/DMS First several items from page 1 of 6:

9 Audit Report General Info Results are presented as the number and percent of patients who meet each report item. Items may be reported for: 1. All patients 2. A subgroup of patients: Items reported for subgroups are indicated by keywords In or If (e.g., In patients aged 40-75). Some are also indented. Look at the denominator. 3. Both of the above Patients with missing values for an item are counted in the denominator for most items. Audit reports items and calculations are the same for Annual and Interim Audits. There are some changes to Audit Report items every year. Some years these are minor, others more extensive.

10 Different Time Periods for Different Items Most Audit items are reviewed for the past 12 months, including: -Tobacco/ENDS screening -Blood pressure -Medications Exceptions include: -Tobacco/ENDS use -TB test/results/tx -Weight -Education -Labs -Height -Health conditions (e.g., HTN, depression, CVD) Look for key words: Audit period, ever -Immunizations (except flu)

11 Audit Report Changes When: Audit Report (capital R) changes are made each year for the Annual Audit. RPMS patch is released and WebAudit updated in February. Why 1. Currency - To align with IHS and other national standards and reports, including American Diabetes Association (ADA) guidelines and IHS GPRA measures. 2. Clarity - To facilitate understanding of each item on the report. 3. Completeness - To provide all the necessary information for interpreting the report items. 4. Consistency To be consistent with other materials and products from IHS and the Division of Diabetes.

12 How to read Audit Reports

13 Examples

14 Audit Form Example 1: One response Foot exam Audit Report - WebAudit

15 Example 1 continued Percent calculation for foot exams: 118/196 = *100 = 60% Numerator = 118 = number of patients Audited that had a foot exam Denominator = 196 = number of patients Audited All patients either had a foot exam or not Generally, higher Percent is better outcome

16 Example 2: Number put in a category A1C Audit Form Audit Report - WebAudit

17 Example 2 continued Percent calculation for A1C<7.0: 75/196 = *100 = 38% Numerator = 75 = number of patients Audited with A1C<7.0 Denominator = 196 = number of patients Audited All patients in only one category for blood sugar control/a1c Sum of Percents for all categories = 100% Generally (important to individualize A1C goals): Higher Percent is better outcome for HbA1c<8.0 Lower Percent is better outcome for HbA1c 9.0 or higher

18 Example 3: Skip pattern Depression Audit Form Audit Report - WebAudit

19 Example 3 continued Percent calculation for depression screening: 158/158 = *100 = 100% Numerator = 158 = number of patients Audited that were Screened Denominator = 158 = number patients Audited with No Depression (NOT total number of patients Audited) All patients in only one category Sum of Percents for all categories = 100% Generally, Higher Percent is better outcome for Screened Lower Percent is better outcome for Not screened

20 Audit Form Example 4: All that apply DM Therapy

21 Example 4: DM Therapy continued Audit Report - WebAudit Page 3 Page 4

22 Example 4: DM Therapy continued Three sections for this item: 1. Diet and exercise alone means none of listed meds 2. Otherwise, on one or more meds 3. If on at least one med, then how many 1 2 Notes: Sum of percentages for sections 1 and 3=100 Sum of percentages for section 2 is >100 if any patients are on more than one med 3

23 Annual Audit Report Review General Guidance 1. Print: Annual Audit Reports from the WebAudit for this year (2017) and the previous year (2016). 2. Number of patients: a. Check (i) Number included in Audit should be equal to or smaller than (ii) number in registry and eligible. b. Check (i) and (ii) vs. previous year: Are they similar or different with good reason? 3. Missing data: Are there unexpectedly large amounts of missing data for any items? If so, review charts and/or taxonomies and make changes as needed.

24 Annual Audit Report Review Guidance (continued) 4. Are there rows with unexpectedly low (near 0%) or high (near 100%) values? If so, review charts and/or taxonomies and make changes as needed. 5. Compare the results for each report item for current vs. previous year. i. If there are large differences in any items, is there a good reason why? ii. If not, review charts and/or taxonomies for these items for Correct data as needed in RPMS or in the WebAudit. i. For Electronic Audits, best to correct in RPMS (or other system), create a new data file, and reupload to the WebAudit. ii. For Manual Audits, make changes to individual records in the WebAudit.

25 Trends Reports Review - Guidance 1. Download Trends Reports (Excel) file from the WebAudit for your facility. Note that there is only one and it contains data for multiple years. 2. Look for spikes, big changes from one year to another in graphs.

26 Trends Reports Review Guidance (continued) 3. Select the Data tab and look at the following columns. a. DIFF : Look for big (positive or negative) numbers b. Trend Line (only displays if Excel 2013 or later): look for spikes

27 For more information Audit webpages: Audit team: Area Diabetes Consultants: Additional Audit training:

28 Audit Website and WebAudit Demo

29 Questions?

30 Diabetes Audit Team Ann Bullock Director, IHS Division of Diabetes Lori Butcher RPMS Developer Cecilia Butler WebAudit Support Karen Schellenberger WebAudit Developer Karen Sheff Biostatistician, Support, Training Mark Williams Visual DMS Developer Chris Lamer RPMS Support, Clinical Expert

31 Thank you! Want more Audit training? Making the Case: Sharing Diabetes Data with Tribal Leaders and Other Stakeholders Wednesday at 2:20p - Pavilion VI Don t File that Audit Report Away Yet! Use Your Diabetes Audit for Improvement Activities Thursday at 9:05a - Fiesta I-II (same room) Outside the RPMS Box: Using non-rpms Systems for Diabetes Data Wednesday at 3:35p - Fiesta III-IV