Types of Group Comparison Research. Stephen E. Brock, Ph.D., NCSP EDS 250. Causal-Comparative Research 1

Transcription

1 Causal-Comparative Research & Single Subject Research Stephen E. Brock, Ph.D., NCSP California State University, Sacramento 1 Correlation vs. Group Comparison Correlational Group Comparison 1 group 2 or more groups 2 or more variables 1 independent variable Extent to which 2 or more variables are related to each other Identifies relationships among variables Extent to which 2 or more groups are different from each other Makes comparisons between groups 2 Types of Group Comparison Research Causal-comparative AKA Ex Post Facto (Latin for after the fact). Researcher does not form the groups. Groups to be compared are formed before the study begins. A preexisting variable defines the group. Quasi Experiment Researcher forms the groups. Groups to be compared are not formed before the study begins. Individuals are not randomly assigned. Intact groups are randomly assigned to a treatment condition. True Experiment Researcher forms the groups. Groups to be compared are not formed before the study begins. Individuals are randomly assigned. 3 Causal-Comparative Research 1

2 Portfolio Activity #8 Mini-proposal 3 Briefly describe a causal-comparative research project relevant to one of your identified research topics. In small groups discuss your mini-proposal ideas and be prepared to share your discussions with the rest of the class 4 Causal-Comparative Research Groups are defined by their difference on some pre-existing variable (the independent variable). Causal Comparative - group difference(s) exist(s) before the study begins (e.g., SES, Gender, ADHD). Group differences often necessarily exist before the study begins Group membership is the independent variable Experiment - group difference(s) are assigned by the researcher (e.g., type of instruction, an approach to counseling). Group differences do not necessarily exist before the study begins 5 Causal-Comparative Research The question being asked is whether, and to what degree, groups also differ on another variable (the dependent variable or measure). Causal Comparative - Do children from high SES (IV) backgrounds attain higher achievement levels (DV) than children from low SES backgrounds? Experiment - Do children who learn to read via Reading Mastery (IV) attain higher achievement levels (DV) than children who learn to read via a whole language approach? 1. What would make this Experiment a Causal Comparative Study? 2. Why might an educational researcher want to make this into such a study (i.e., turn it into a causal comparative study)? 6 Causal-Comparative Research 2

3 Reasons for Employing a Causal- Comparative Approach Causal-Comparative methods are typically used because the variable under study (the IV) cannot be directly manipulated. Gender Age Others? should not be manipulated. Destructive habits Disease or disorder Others? Why else would a causal-comparative method be used???? 7 Reasons for Employing a Causal- Comparative Approach These methods are also sometimes used to help determine if the more complicated and expensive experimental design is worthwhile. Did our prior discussion identify this as a possible reasons for conducting a Causal-Comparative study of Reading Mastery? 8 Variables Often Examined in Causal-Comparative Studies Internal Organismic Ability Personal Characteristic External Family-related School-related Identify examples in each of these five categories. These would be the IV in a causal-comparative study 9 Causal-Comparative Research 3

4 The Two Basic Research Designs Group IV DV Case A E X O C X O Case B E X 1 O E X 2 O Symbols: E = Experimental group C = Comparison group X = Independent variable O = Dependent variable 10 Control Procedures In an ex post facto study, if is difficult to make conclusions about whether there is or is not a causal relationship between two variables. One cannot be sure that the two groups do not differ with respect to variables other than the variable under study (Crowl, 1996). We need to consider the possibility that changes in the dependent measure (results) are due to factors other than the independent variable (group membership). 11 Control Procedures Sometimes you are aware of these alternative explanations for group differences before you begin a study. For example, in my study of the effect of ADHD on reading comprehension I was aware of the fact that ADHD often co-exists with reading disabilities. The presence of ADHD children with reading disabilities in my sample would have been a confounding variable. 12 Causal-Comparative Research 4

5 Control Procedures Confounding Variables Any variable on which groups in an experiment systematically differ, other than the variable whose effect the research is interested in determining, is a confounding variable (Crowl, 1996, p. 274). Because of its inability to randomly assign participants, confounds are especially problematic when conducting an ex post facto study. The random assignment of an experiment minimizes such confounding effects. 13 Is ADHD Associated with Relative Reading Comprehension Difficulties? ADHD Poor rding. comp. IV - Group membership DV -Test Results No ADHD Relatively high rding. comp. 29% SLD Confound 10% SLD How might a causal-comparative study attempt to address this confound? 14 Do learning disabilities cause low self concepts? LD Poor self-concept IV - Group membership DV -Test Results No LD Relatively high self-concept. Pull out Low tchr. Expectations Teased Confounds Not pulled out High tchr. Expectations Not teased How might a causal-comparative study attempt to address these confounds? 15 Causal-Comparative Research 5

6 Sample Selection Control Procedures Matched Pair Design Systematically select participant pairs who are similar in all important ways other than the independent variable. Homogenous Grouping Design With the exception of the independent variable (group membership) make sure that participants in both groups are very similar in all important ways. 16 Data Analysis Control Procedures Factorial analysis of variance. A statistical way to assess the effects of potential confounds on the dependent measure. Analysis of Covariance Adjusts scores on the dependent variable for initial differences on some other variable related to the dependent variable. Pretest Score IV Group Membership DV Post-Test Score 110 X O 112 O 17 Data Analysis Descriptive Statistics Mean Standard Deviation Inferential Statistics t-test The difference between 2 dependent measure means ANOVA The difference between 3 or more dependent measure means Chi Square The difference between the frequency of occurrence of the dependent measure. 18 Causal-Comparative Research 6

7 Single Subject Research involves multiple measurements of the behavior of a single individual at different points in time prior to, during, and following the use of some intervention designed to change the individual s behavior (Crowl, 1996, p. 324). Differs from case studies in that this research attempts to control some aspect of the environment. 19 Single Subject Research The objective is to determine if an intervention has significantly affected the behavior of the subject. The previously discussed observational strategies are often used to provide the data to be analyzed. The design used in FAAs and in RTI See handout for an example of a Single subject research / RtI data presentation. 20 Single Subject Research Single-Subject versus Group Designs Unlike an experiment there is no control group in single-subject research Validity determined by Repeated and consistent measurement Baseline stability The single variable rule 21 Causal-Comparative Research 7

8 Types of Single Subject Research A-B-A Withdrawal Multiple Baselines Alternating Treatments 22 A-B-A Withdrawal A-B Design O O O O O Baseline Phase A A-B-A Design X O X O X O Treatment Phase B O O O O O Baseline Phase A X O X O X O Treatment Phase B O O O O O Baseline Phase A NOTE: O = measurement, X = treatments 23 A-B-A-B Design See handout 24 Causal-Comparative Research 8

9 Multiple Baseline Employed when it is impossible to return to the baseline (e.g., the intervention has resulted in permanent change in behavior), or when there are several interventions to be implemented 25 Alternating Treatments The Alternating Treatments Design is used to directly compare the effects of two or more different experimental variables across the same span of time in the same subject. Effective in controlling for systematic changes in the subject or setting across time. Disadvantages inability to deal with irreversible effects potential generalization from one condition to the other interpretation problems due a variety of interactions, carryover, and order effects. 26 Single Subject Research CREATING SINGLE-SUBJECT DESIGN GRAPHS WITH MICROSOFT EXCEL by James E. Carr & Eric O. Burkholder artid= Causal-Comparative Research 9

10 Single Subject Research More about single subject designs: Single Subject Research Example of an intervention appropriate for a single subject research project. A Behavioral Intervention for Increasing Ontask Behavior. 29 Increasing On-task Behavior A token economy program for increasing on-task behavior that can be used during seatwork and other learning situations. Primary components of this program include immediate reinforcers, several daily mini-conferences with the teacher, and daily and weekly rewards. Some have criticized token economies because of their reliance on extrinsic reinforcers. Others have suggested that the use extrinsic reinforcers do not negatively impact the intrinsic motivation of students as it relates to classroom tasks. 30 Causal-Comparative Research 10

11 Beginning the Program Ensure that students understand program expectations and procedures. Behaviors to be rewarded should be operationally defined and understood by both student and teacher. Specific on-task target behaviors may include begin work immediately, work quietly, remain seated, ask good questions, complete work and follow instructions. A teacher may choose to target some or all of these behaviors. Regardless, the behaviors should be framed in positive language, focusing on desired student behaviors. 31 Immediate Reinforcers As frequently as possible the student should be given immediate behavior-specific verbal praise whenever one of the targeted on-task behaviors is observed. Comments such as Good or Nice should be expanded to include a statement specifying the specific behavior for which the student is being praised. Such behavior specific verbal praise is most effective when given immediately following display of appropriate behavior(s). Particular attention should also be given to increasing the amount of praise relative to the amount of negative comments. 32 Mini-Conferences At several times during the day the teacher has a one to two minute mini-conference with the student. The number of mini-conferences held is a decision made by the teacher. As a general rule, the more conferences held the better. However, it is essential that it be feasible for the teacher to consistently participate in all scheduled conferences. A natural time for these conferences to be held is just before each recess and lunch period, and just before the end of the school day. 33 Causal-Comparative Research 11

12 Mini-Conferences During the mini-conference the teacher gives verbal praise for each on-task behavior demonstrated during the just completed period. Making use of an on-task tally sheet, the teacher would place a mark or sticker on the sheet to further reinforce on-task behavior. Also during the mini-conference, encouragement and instruction regarding on-task behaviors not displayed should be offered. In addition to providing reinforcement, mini-conference focus should be on finding solutions to the off-task behavior problems. 34 Chart for Use During Mini-Conference Work Period Begin work immediately Work quietly Daily Tally Sheet Remain Ask good seated questions Work carefully Follow instructions Complete Assignments Total My daily total My daily total goal for this week 35 Daily Rewards Before implementing the program, the teacher and student should set a daily goal. This goal should be written on the on-task tally sheet. As appropriate, the student should be encouraged to set his or her own on-task behavior goals. During the initial stages of the program the daily goal should be set low and at a level that ensures success and rewards relatively small approximations of the desired behavior. Expectations should be gradually increased until rewards are only given for the completion of the entire assignment. 36 Causal-Comparative Research 12

13 Daily Rewards If the student reaches the on-task behavior goal, one or more daily rewards, specified in a previously written behavior contract, would be given. One reward possibility, that would also facilitate homeschool communication, is to send home a positive note to the student s parent(s). It is critical that the agreed upon reward be meaningful to the specific student. If a variety of desirable rewards can be identified, an effective method for delivering reinforcement is to make each reward be a surprise. 37 Daily Reward Option Date: Dear, SUPER WORKER REPORT Your child met the on-task behavior goal today. It would be appropriate for you to do something special for your child tonight to reinforce this good work. Sincerely, Classroom teacher 38 Weekly Rewards An optional component of this program involves setting a weekly goal and reward. An example of such a reward might be lunch with the teacher, or a special in class activity. During each mini-conference, the teacher may also want to graph each week s goal attainment. It is important to note that especially among younger children, these long-term rewards are less effective and should not replace immediate and daily rewards and tracking of progress for any student. 39 Causal-Comparative Research 13

14 Weekly Reward Chart WEEKLY CONTRACT Day Monday Tuesday Wednesday Thursday Friday Daily Total If I meet my weekly goal, I will earn My weekly total goal is This week s total 40 Next Week Next class meeting: Experimental Research Read Educational Research Chapter 10. Portfolio Element #9 Due: Mini-proposal 4. Briefly describe an experimental research project relevant to one of you identified research topics. 41 Causal-Comparative Research 14