Likert Scaling: A how to do it guide As quoted from

Likert Scaling: A how to do it guide As quoted from www.drweedman.com/likert.doc Likert scaling is a process which relies heavily on computer processing of results and as a consequence is my favorite method of attitude scaling. However, it was not so long ago that this process of scaling was considered too time consuming and other methods, like Thurstone methods, were preferred. I will present Likert methods by tracing through an example. You should realize, of course, that what I will present is my interpretation of Likert methods. The first step in this scaling method, after you have selected the attitude to be assessed, is to assemble a group of judges. These judges have as their task the development of potential statements which would tap the attitude domain. In short, you want these judges to compose a list of statements which can be responded to on a five point scale. This five point scale will range from "strongly agree" to "strongly disagree", with the middle of the scale identified by the response alternative "undecided" or "neither agree nor disagree". I have chosen for our example to develop a test of self-attitude. The list of the statements that I developed to be potential members of our final questionnaire can be found in the first draft of the field test questionnaire on the following page. You should note that I did not utilize a panel of judges to get these statements, I just made them up myself. You would, of course, use judges. Your judges should have some knowledge of the topic area. In assembling your judges, usually 3 to 5 are sufficient, you instruct them to develop items which would be positive and items that would be negative. By positive items I mean any items on which a strongly agree response would indicate a favorable disposition toward the attitude and by negative I mean items on which a strongly agree response would indicate a negative disposition. For an example of these types of statements, see statements 1 and 9 on our field test questionnaire. You should instruct your judges to avoid items which appear to be ambiguous (ask for two or more opinions in one statement). I included one of these ambiguous statements in our questionnaire to demonstrate how the scaling methods is able to systematically eliminate the statement (see statement 6). It is important that your potential statements represent a large range of possible opinions and are relevant to the attitude domain. I have attempted to include some irrelevant items, see statements 5 and 7. You should develop about 40 items and aim for a final test of about 20 items (in our example we will use less items in our original pool). Once you have finished this initial phase, you should have your field test questionnaire typed and administered to as many people as practical. I administered our field test questionnaire to 25 people, mostly friends. The next step is to score each of the items and calculate a total score by summing the various items. What you will end up with, in our example, is 13 scores. You always give 5 for "strongly agree"; 4 for "agree"; 3 for "neither agree nor disagree"; 2 for "disagree"; and 1 for "strongly disagree". You assign the scoring in this manner regardless of whether you think the item is a positive item or a negative item. We are now ready for our first set of calculations. Your task is to calculate the

correlation between each item and the total for the test. In other words, what is the correlation between responses to item 1 and the total of all items? Item 2 and the total? etc. I have performed these calculations and they are summarized in the correlation table on the following page and are listed under Run 1. The correlations that identify items as being negative will need to have the polarity of the scale coding reversed. Therefore, for questions 3, 6, 9 and 10 we will need to reverse the scoring, assigning a 1 to the response "strongly agree" and 5 to "strongly disagree." Can you see that reversing the scoring polarity of these items will change the total score for the test? You will need to recompute the total score also. The next step is to calculate the 12 correlations again. These correlations are presented in the second column of correlation coefficients. We want to continue reversing the polarity of the items with negative correlations until we reach the point where all the correlations are positive. We will reverse the scoring polarity of questionnaire items 2, 3, 4, 7, 9, 10, and 11 since they gave negative correlations on our second run. Notice that we had reversed the polarity of item 4 in the previous step and at this step we are going to reverse the polarity back to its original scoring. Column 3 in our correlation table gives the correlations for our third run. (Again, note that we had to recompute the total for this third run). Notice that in our third set of correlations we need, again, to reverse the polarity of some items back to their original polarity. We will continue this process until we reach some stable pattern, reversing the polarity and recalculating the correlations. At the end of the fourth set of correlations it appears that we have some questionnaire items (number 1, 10, and 11) which seem to want to continue to alternate their polarities.

Likert Version of The Self-Concept Questionnaire Below are 12 statements regarding attitudes which I would like you to rate on a five point scale. I want you to circle one of the symbols SA through SD to represent your opinion on each of the statements. The scale is defined as follows: SA = Strongly agree with the statement A = Agree? = Neither agree or disagree D = Disagree SD = Strongly disagree with the statement 1) I like myself. 2) I feel I will make a significant contribution to mankind. 3) I have a good relationship with my mother. 4) I have a good relationship with my father. 5) I enjoy math/science courses. 6) I like my name, but I am concerned by my weight. 7) I am good at drawing things. 8) I wish I were someone differ. 9) I have a low opinion of myself. 10) I have had a good life? 11) My friends have a good opinion of me. 12) I do not have many friends.

Correlations Between the 12 Test Items and the Total for the 12 Items Items Correlation Runs 1 2 3 4 5 6 7 8 1 -.13.12.18.18.56.54.57.55 2.08 -.36.39.39.45.58.64.77 3 -.10.08 -.02.00 --- --- --- --- 4 -.08 -.04 -.16.21.15.13 --- --- 5.08.05.07.10.15.17 --- --- 6.25 -.32 -.34 -.29 -.14 -.26 -.35 --- 7.14 -.11 -.03.00 --- --- --- --- 8.17.05.28.29.38.42.53.60 9 -.18.24 -.17.15.44.46.70.68 10 -.16 -.17 -.19 -.18.08 --- --- --- 11.29 -.12.16 -.17.06 --- --- --- 12.00.21.13.16.47.52.63.65 At this point, we need to begin the process of eliminating questionnaire items from consideration in the final form of the instrument. We will begin the elimination process by discarding questionnaire items with the lowest correlations (items 3 and 7). The new correlations, with items 3 and 7 eliminated, are presented in column 5. As you can see we now have two additional items which we can disregard (items 10 and 11). The new correlations, now with items 3, 7, 10, and 11 eliminated, are presented in column 6 (notice that we have continued to reverse the polarity of item 6, in a futile attempt to achieve a positive correlation). At this step we will eliminate items 4 and 5. Have you noticed that the correlations for the items that have not been eliminated continue to become larger? The correlation, with items 3, 4, 5, 7, 10, and 11 eliminated, are presented in column 7. Now we will eliminate our final item (fickle item 6) and run the last set of correlations. Through this process, we have arrived at a five-item test of self-attitude which is composed of items 1, 2, 8, 9, and 12 from our original pool of items. Critique of the Likert Scaling Process

There are a number of criteria on which we can evaluate the outcomes of a scaling process. The first of these guidelines is zero point. In other words, with our Likert version of a self-attitude scale do we have a number such that scores below this number would reflect negative self-attitude and score above this number would reflect positive self-attitude? The neutral point for each item is a score of 3, "neither agree or disagree." Since we have a five item test, if a respondent achieved a total score of 15 then logically he/she should have a neutral self-attitude. Therefore, with respect to the zero point guideline it would appear that the Likert scaling process satisfies the requirement. If I were to rate the degree to which Likert scaling accomplishes this task on a five point scale where 1 would indicate no compliance and 5 would indicate very good compliance, I would give Likert scaling a 4 on zero point compliance. Our next criteria on which we evaluate a scale is equality of units. In other words, does our test provide scores which have interval-like characteristics, therefore enabling us to use more sophisticated statistics with the test results? Surely, a larger total score on our test would indicate a more positive self-attitude than a smaller total score. Therefore, it would appear that we have, at minimum, an ordinal score. If every item on our self-attitude scale correlated perfectly with the total score for the test and if there were zero correlations among the scale items we would have excellent equality of units since each questionnaire item would be contributing uniquely to the total score. However, no test I know of has fulfilled such a stringent requirement. The intercorrelations among our five items are summarized below. As you can see our inter-item correlations are not extremely high except for the correlation between items 9 and 12. This pattern of intercorrelations would suggest good equality of units, I will give our scale a rating of 3 on our five point rating system. Unidimensionality of items refers to characteristics of the test items. In short, is there a process to insure that the test items assess only one attitudinal Intercorrelations Among Questionnaire Items 1 2 8 9 12 1 1.00.35.34.05.08 2.35 1.00.48.34.41 8.34.48 1.00.16.16 9.05.34.16 1.00.61 12.08.41.16.61 1.00 dimension? The assurance is satisfied first by using judges to develop the test items and

instructing them to review the items to insure that they assess only one content. Secondly, unidimensionality is supported by the correlation process. If an item were not uni-dimensional then its correlation pattern should indicate an unstable pattern, like we found for item 6 on our trial instrument. I would give Likert scaling a rating of 4 on our five point scale of compliance in terms of unidimensionality of items. Unidimensionality of scale is concerned with whether or not the total score for the test is an index of one attitudinal component or several attitude sub-components. In order to determine the degree to which Likert scaling violates this uunidimensionality criteria, a factor analysis procedure would have to be employed. However, employing judges to develop the various item contents would seem to support acceptable total score - unidimensionality for Likert scaling. So I will give Likert scaling a score of 2 on our compliance rating system for total score unidimensionality. An index of the reliability of our Likert scale can be determined from the item inter-correlations presented above. The mean correlation among the 5 test items was.44 and the estimated reliability of the test was.91. k x ra reliability = 1 + (k-1) x ra where k = number of items ra = mean item inter-correlation 5 x.44 2.5 reliability = = =.91 1 + (5-1) x.44 2.76 What is important here is to realize that the Likert scaling procedure provides you with a method for determining the reliability of the test that is developed. Therefore, on our five point rating scale of compliance I will give Likert scaling a 5 on the reliability criteria. Remember that our rating of 5 means that the scale development process provides a very good way of estimating reliability and does not necessarily imply that our particular test has good reliability. The last criteria on which we evaluate a scaling process is validity, more specifically content validity. As you may recall, content validity, in a sense, is an evaluation of the amount of rigor which has been used in the test item development. Those procedures which involve

more rigor are viewed as more content valid than those procedures which utilize less rigor. In our case I would give Likert scaling a 5 on our rating scale of compliance. A final rating needs to be made of the ease of application of Likert scaling. In other words, compared to other scaling methods, how much effort is required to develop a Likert type attitude test? I would consider the degree of effort to be minimal, if you have the availability of a computer and if you are versed in computer programming. I would rate Likert scaling as a 4 in ease of application. The summary of the ratings of Likert scaling is presented below. Compliance Ratings of Likert Scaling* Criteria Rating Zero Point 4 Equality of Units 3 Unidimensionality of items 4 Unidimensionality of scale 2 Reliability 5 Content Validity 5 Ease of application 4 * 1 = no compliance, through 5 = very good compliance