Behavior Progress Monitoring Rating Rubric 1

Transcription

1 Behavi Progress Moniting Tools Chart Rubric Please note that the following rubrics are applied separately f each sub-scale, grade level/span, infmant targeted by the tool. Tools Chart Tab 1: Foundational Psychometric Stards Note: F all stards in Tab 1, it is expected that evidence is drawn from a sample that is representative of students across all perfmance levels. Tools not intended f use with students across all perfmance levels may receive a rating of N/A in these categies. 1A. Reliability (a) A model based approach to reliability was repted with at least two sources (b) At least two other types of reliability were repted appropriate f the purpose of the tool, drawn from at least two samples that are representative of students across all perfmance levels interval around the median estimate met exceeded A model-based approach to reliability was repted with at least two sources At least two other types of reliability were repted appropriate f the purpose of the tool, drawn from at least one sample that is representative of students across all perfmance levels / interval around the median estimate fell below 0.70 but met exceeded Behavi Progress Moniting Rubric 1 10/2017

2 1B. Validity There are at least two types of appropriately justified validity analyses* from a sample representative of students across all perfmance levels The lower bound of the confidence interval around the each estimate met exceeded 0.60 ( if not, within an acceptable range given the expected relationship with the criterion measure(s)). Analyses, measures, sample were appropriate, but evidence was mixed, with one me measures either not meeting exceeding 0.60 not within an acceptable range given the expected relationship with the criterion measure(s). * Appropriately justified analyses must include criterion measures that are external to the progress moniting system theetically linked to the underlying construct measured by the tool. 1C. Bias Analysis Conducted Yes No One me of the following three types of analyses were conducted: 1. Multiple-group confirmaty fact models f categical item responses. 2. Explanaty group models such as multiple-indicats, multiple-causes (MIMIC) explanaty IRT with group predicts 3. Differential Item Functioning from Item Response They (DIF in IRT). 4. Testing differential classification accuracy across demographic groups Fails yes. Behavi Progress Moniting Rubric 2

3 Tools Chart Tab 2: Progress Moniting with Intensive Population Note: F all stards in Tab 2, it is expected that evidence is drawn from a sample of student in need of intensive Convincing evidence that children were in need of intensive intervention may include one me of the following: students have ED label; students are placed in an alternative school/classroom; students have demonstrated non-response to moderately intensive intervention (e.g., Tier 2); students have demonstrated severe problem behavis (e.g., Tier 3), accding to an evidence-based tool (e.g., systematic screening tool direct observation). 2A. Sensitive to Behavi Change The basis f assuming that the data are sensitive to incremental change is strong (e.g., the range of possible sces is sufficient to detect small changes documentation of sensitivity to change consistent with another criterion). The basis f assuming that the data are sensitive to incremental change is moderate (e.g., the range of possible sces is sufficient to detect a change documentation of sensitivity to change). 2B. Reliability: Intensive Population A model based approach to reliability was repted with at least two sources At least two other types of reliability were repted appropriate f the purpose of the tool, drawn from at least two samples that are representative of students in need of intensive intervention interval around the median estimate met exceeded Behavi Progress Moniting Rubric 3

4 A model-based approach to reliability was repted with at least two sources At least two other types of reliability were repted appropriate f the purpose of the tool, drawn from at least one sample that is representative of students in need of intensive intervention / interval around the median estimate fell below 0.70 but met exceeded C. Validity: Intensive Population There are at least two types of appropriately justified validity analyses* from a sample representative of students in need of intensive intervention The lower bound of the confidence interval around the each estimate met exceeded 0.60 ( if not, within an acceptable range given the expected relationship with the criterion measure(s)). Analyses, measures, sample were appropriate, but evidence was mixed, with one me measures either not meeting exceeding 0.60 not within an acceptable range given the expected relationship with the criterion measure(s). Behavi Progress Moniting Rubric 4

5 2D. Data to Suppt Intervention Change The data provided to suppt decisions about intervention change is (1) strong (2) based on analysis of progress moniting measurement collected at least weekly over the period of time that is deemed necessary f the decision rules; (3) from a sample of students that is in need of intensive The data provided to suppt decisions about intervention change has (1) moderate empirical suppt f targeted behavi; (2) is based on analysis of progress moniting measurement collected at least weekly over the period of time that is deemed necessary f the decision rules; (3) is from a sample of students that is in need of intensive 2E. Decision Rules f Changing Instruction The data provide guidance on intervention choice (e.g., a class of relevant interventions a specific intervention), that is (1) strongly evidence-based; (2) based on analysis of progress moniting measurement collected at least weekly over the period of time that is deemed necessary f the decision rules, (3) from a sample of students that is in need of intensive The data provide guidance on intervention choice (e.g., a class of relevant interventions a specific intervention), that is (1) moderately evidencebased; (2) based on analysis of progress moniting measurement collected at least weekly over the period of time that is deemed necessary f the decision rules, (3) from a sample of students that is in need of intensive Behavi Progress Moniting Rubric 5