HOW TO EVALUATE ACTIVITIES INTENDED TO INCREASE AWARENESS AND USE OF COLORECTAL CANCER SCREENING. Using your toolkit to conduct an evaluation

Transcription

1 EVALUATION TOOLKIT HOW TO EVALUATE ACTIVITIES INTENDED TO INCREASE AWARENESS AND USE OF COLORECTAL CANCER SCREENING Using your toolkit to conduct an evaluation

2 Welcome Mary Doroshenk, MA Director National Colorectal Cancer Roundtable (NCCRT)

3 Evaluation toolkit

4 Toolkit resources Sample tools/worksheets Tip sheet

5 Webinar overview Webinar 1 Introduction, mapping the program, prioritizing what you need to know Webinar 2 Designing the evaluation and creating tools Webinar 3 Collecting data, analyzing and using data

6 Today s speakers Robert Smith, Ph.D., NCCRT Co-Chair, Senior Director, Cancer Screening, American Cancer Society Cheryl Holm-Hansen, Ph.D., Senior Research Manager, Wilder Research Laura Martell Kelly, MPA, Research Scientist, Wilder Research Andi Dwyer, Program Director, Cancer Prevention and Control Research Network - Colorado

7 Screening for the Prevention and Early Detection of Colorectal Cancer Robert A. Smith, PhD Senior Director, Cancer Screening Cancer Control Science Department American Cancer Society

8 Colorectal Cancer: Paradox Colorectal cancer is the #2 cause of cancer death in the US Colorectal cancer can be detected early & is often preventable

9 About Colorectal Cancer Includes cancers of the colon and rectum Often referred to as colon cancer Affects both men and women Most colon cancers occur in adults age 50 and older but younger people can get this disease. Certain risk factors, such as a family history of colon polyps or cancer, increase the risk. Most colon cancers occur in people with no family history.

10 About Colorectal Cancer One of few cancers for which we have screening tests that not only detect cancer early, but often can prevent the cancer

11 The Window of Opportunity Some types of polyps, called adenomatous polyps, can potentially progress into cancer. It can take years for a polyp to progress into cancer. This is our window for prevention. Colon polyps and early colon cancers often cause no symptoms. Screening is done before symptoms develop.

12 Colorectal Cancer: Concept of Early Detection Early Stage: - Often no symptoms - Surgery alone - High rate of cure Late Stage: - Symptoms - Surgery and chemotherapy - Lower rate of cure

13 Take-Home Messages: A polyp can take several years to turn into a cancer. Polyps & early colon cancers often cause no symptoms. Screening is done before symptoms occur. Screening allows for both the early detection and often the prevention of colon cancer. Screening remains underutilized.

14 COLORECTAL CANCER SCREENING

15 ACS Screening Guidelines Options for Average risk adults age 50 and older Tests That Detect Adenomatous Polyps and Cancer Colonoscopy every 10 years, or Flexible sigmoidoscopy (FSIG) every 5 years, or Double contrast barium enema (DCBE) every 5 years, or CT colonography (CTC) every 5 years Tests That Primarily Detect Cancer Guaiac-based fecal occult blood test (gfobt) with high test sensitivity for cancer, or Fecal immunochemical test (FIT) with high test sensitivity for cancer, or Stool DNA test (sdna), with high sensitivity for cancer

16 AVERAGE RISK INCREASED RISK HIGHEST RISK

17 Percent Change in the use of different CRC screening tests over time by age group Colorectal Cancer Tests Within Recommended Time Intervals. Adults 50 and Older, US Colonoscopy in the past 10 years Combined FOBT / Endoscopy* Home fecal occult blood test in the past year Sigmoidoscopy in the past 5 years *Either a home fecal occult blood test within the past year or a sigmoidoscopy within the past 5 years or a colonoscopy within the past 10 years. Rates are age-adjusted to the 2000 US standard population. Source: National Health Interview Survey Public Use Data File 2000, 2005, and National Center for Health Statistics, Centers for Disease Control and Prevention American Cancer Society, Inc. No

18 Colorectal Cancer Screening Prevalence by State among Adults 50+,

19 Stool Tests: gfobt, FIT, sdna

20 High Quality Stool Testing CRC screening by FOBT should be performed with highsensitivity FOBT - either FIT or a highly sensitive gfobt (such as Hemoccult SENSA). - Older, less sensitive guiaic tests (such as Hemoccult II) should not be used for CRC screening. Tests should be repeated yearly In-office FOBT is essentially worthless as a screening tool for CRC and must be strongly discouraged. All positive screening tests should be evaluated by colonoscopy

21 Colonoscopy Polypectomy prevents Colorectal Cancer The National Polyp Study observed a 76-90% reduction in CRC incidence after polypectomy Winawer et al, NEJM 1993

22 Influence of colonoscopic polypectomy on risk of death from colorectal cancer Colonoscopic polypectomy was associated with a 53% reduction in colorectal cancer mortality

23 CT Colonography 2-D view 3-D view Polyp Courtesy of Beth McFarland, MD

24 Factors Associated with Adult Uptake of Cancer Screening Education Health Insurance Even better---health insurance and no copays Risk awareness Younger age A regular source of health care A personal physician A physician s recommendation Reminder systems A Checkup

25 Before we begin why evaluate? Reasons to conduct evaluations Guide programming decisions Show effectiveness Reach target audience Compare outcomes with similar programs Seek funding Cultural/ethical considerations

26 Who should do the evaluation? Inventory internal resources Consider staff experience with: Evaluating Budgeting Managing data Analyzing data Communicating information

27 Overview of the evaluation process

28 STEP 1: Describe and map your program Who does your intervention target? How are services delivered? Why do you provide these specific services? How do you hope your activities benefit participants or the community?

29 Program theory You are here. What needs to happen to get from here to there? You need to be there. Activity IF the activity is provided, THEN what should be the result for participants? WHY do you believe the activity will lead to this result? What evidence do you have that this activity will lead to this result (data from your own or other programs, published literature, etc.)?

30 Mapping your program Inputs Activities Outputs Short-term outcomes (changes in knowledge, attitudes) Intermediate outcomes (changes in behaviors or practices) Long-term outcome/ Overall Impact One approach to a logic model

31 AN EXAMPLE: St. Joseph s Church Better understanding about screening

32 PROGRAM THEORY: St. Joseph s Church Activity IF the activity is provided, THEN what should be the result for participants? WHY do you believe the activity will lead to this result? What evidence do you have that this activity will lead to this result (data from your own or other programs, published literature, etc.)? Group education sessions Parishioners gain knowledge about screening options, make a pledge to be screened, and are more likely to get screened Awareness and knowledge can be a key step in deciding to get screened In the area of breast cancer, group education has been shown to be effective in increasing screening

33 Mapping your program Inputs Activities Outputs Short-term outcomes (changes in knowledge, attitudes) Intermediate outcomes (changes in behaviors or practices) Long-term outcome/ Overall Impact Group education sessions

34 Mapping your program Inputs Activities Outputs Short-term outcomes (changes in knowledge, attitudes) Intermediate outcomes (changes in behaviors or practices) Long-term outcome/ Overall Impact Materials Location Trainer Group education sessions

35 Mapping your program Inputs Activities Outputs Short-term outcomes (changes in knowledge, attitudes) Intermediate outcomes (changes in behaviors or practices) Long-term outcome/ Overall Impact Materials Location Trainer Group education sessions # of trainings # of people attending # of materials provided

36 Mapping your program Inputs Activities Outputs Short-term outcomes (changes in knowledge, attitudes) Intermediate outcomes (changes in behaviors or practices) Long-term outcome/ Overall Impact Materials Location Trainer Group education sessions # of trainings # of people attending CRC screening rates increase CRC incidence rates decrease # of materials provided CRC mortality rates decrease

37 Mapping your program Inputs Activities Outputs Short-term outcomes (changes in knowledge, attitudes) Intermediate outcomes (changes in behaviors or practices) Long-term outcome/ Overall Impact Materials Location Trainer Group education sessions # of trainings # of people attending # of materials provided Parishioners learn about the importance of screening Parishioners pledge to be screened Parishioners make appointments to be screened for CRC Parishioners get screened CRC screening rates increase CRC incidence rates decrease CRC mortality rates decrease

38 Why do we need a logic model? Logic models help you: Describe the program Train new staff or volunteers Control program drift Facilitate program management

39 STEP 2: Prioritize what you need to know Program impact (outcomes) Program implementation (process) Satisfaction

40 Prioritize outcome questions Which outcomes will be the most... Useful in understanding success and guiding improvements? Important to the participants? Important to other stakeholders, including funders?

41 Prioritize process questions How much will a process issue... Influence participant outcomes or satisfaction? Concern staff members or other key stakeholders? Help with planning or improvement decisions?

42 Prioritize satisfaction questions Do elements of client satisfaction make a difference in positive outcomes? Will you be able to do anything with your satisfaction results, or is it beyond your resources or control? Are there key stakeholders whose satisfaction will strongly influence your program?

43 Questions

44 Next steps An online evaluation survey will be distributed shortly The second webinar is scheduled for July 10 The third webinar is scheduled for September 3 If you have any additional questions that you d like addressed in upcoming webinars, please them to Cheryl at Wilder (cheryl.holmhansen@wilder.org)

45 Thank You! Today s speakers Wilder Research NCCRT evidence-based education & outreach task group National Comprehensive Cancer Control Program Centers for Disease Control and Prevention (CDC) CDC s Colorectal Cancer Control Program American Cancer Society

46 The evaluation toolkit and this webinar series were made possible by funding from the Centers for Disease Control and Prevention Cooperative Agreement Number 1U38DP The views expressed in the materials and by speakers and moderators do not necessarily reflect the official policies of the Dept. of Health and Human Services.

47 PRESENTED BY: Cheryl Holm-Hansen & Laura Martell Kelly Wilder Research DIRECT ANY QUESTIONS TO: Mary Doroshenk and Tamar Wallace National Colorectal Cancer Roundtable