KCP learning. factsheet 44: mapp validity. Face and content validity. Concurrent and predictive validity. Cabin staff

Size: px

Start display at page:

Download "KCP learning. factsheet 44: mapp validity. Face and content validity. Concurrent and predictive validity. Cabin staff"

Jason Garrett
5 years ago
Views:

mapp validity Face and content validity Careful design and qualitative screening of the questionnaire items in the development of MAPP has served to maximise both the face validity and the content

1 mapp validity Face and content validity Careful design and qualitative screening of the questionnaire items in the development of MAPP has served to maximise both the face validity and the content validity of the questionnaire as an assessment instrument for use at managerial, professional, graduate and administrative levels. All of the questionnaire scales have been defined in terms of behaviours, preferences or values that are strictly relevant to the incumbent s performance and/or job satisfaction at work. The high internal consistencies (see Factsheet 23) of the MAPP scales provide further evidence that the questionnaire items relate to scale definitions in a homogenous, consistent and unambiguous fashion. Concurrent and predictive validity Cabin staff One hundred and twenty Cabin Staff were asked to participate in this study by completing the MAPP questionnaire. Three different Cabin Staff Managers provided performance data on the 120 job incumbents. Performance ratings for a number of different aspects of the job were collected, as well as overall performance ratings. In order to summarise the results of this study, overall performance ratings from the three different managers were averaged to provide validation criteria. The MAPP scales were then correlated with mean overall performance ratings to produce validation coefficients. The inter-rater reliability of the overall performance ratings was also computed and used to correct validation coefficients for attenuation. Data were also collected on the length of time between recruitment into a basic grade cabin staff position and first promotion to a more senior grade. Complete data were collected for 85 of the 120 participants in the study. The following results are based on this sample of 85 job incumbents. Correlates of Performance (Cabin Staff) Listed overleaf are the MAPP scales that correlated significantly with mean overall performance ratings. Arrows indicate the direction of the relationship between the scale and the performance criterion (e.g. Distractable indicates that incumbents with lower scores on Distractable performed better, and the validation coefficient of this scale was -0.21). 1

2 PEOPLE Affectionate Confident Convincing Candid Uncompromising Systematic Cautious Free-thinking Distractable Self-assured Work A multiple regression using the scales listed above to predict overall performance yielded a multiple correlation coefficient of 0.60 (corrected for shrinkage), demonstrating that MAPP may be used as a highly effective instrument for predicting high performance in the cabin staff role. 2

3 Correlates of Promotion (Cabin Staff) MAPP scales were also correlated with the length of time between recruitment and first promotion. The main predictors of speed of promotion are listed below. Again, arrows indicate the direction of relationships (e.g. Results indicates that incumbents with a higher value for Results achieved promotion more rapidly, and the validation coefficient of this scale was +0.30). PEOPLE Candid Assertive Systematic Expressive Results Responsibility Novelty Self-expression Work A multiple regression using the scales listed above to predict speed of promotion resulted in a multiple correlation coefficient of 0.66 (corrected for shrinkage). We may therefore conclude that MAPP may also be used to identify individuals who have the potential to be more rapidly developed and promoted. 3

4 Correlates of Attendance (Cabin Staff) Attendance data were collected in the form of number of days off work in a six month period, and number of spells away from work in a six month period. MAPP correlations with both criteria were very similar. However, correlations with number of spells away from work were slightly stronger and it is these which have been used to compile the list presented below. Arrows indicate the direction of relationships between the scale and attendance (e.g. Anxious indicates that incumbents with lower scores on Anxious have better attendance records; i.e. have fewer spells away from work). Free-thinking Distractable Systematic Anxious Personal authority Novelty Multiple regression of the scales listed above onto attendance data gave a multiple correlation coefficient of 0.28 (corrected for shrinkage). 4

5 Correlates of Length of Service (Cabin Staff) Length of service within the group ranged from 6 months to 10 years. The main MAPP scales which correlated with length of service are given below. Arrows indicate the direction of relationships (e.g. Novelty indicates that incumbents with a lower value for Novelty are likely to stay for longer). Systematic Cautious Sensitive Competition Security Novelty Multiple regression using these 6 scales to predict length of tenure gives a multiple correlation coefficient of 0.43 (corrected for shrinkage), demonstrating that MAPP can be used as a powerful means of predicting length of service and hence reducing future levels of staff turnover. Summary The results of this validation study indicate that scores on MAPP scales are related to performance in the cabin staff role, rapidity of promotion, length of tenure and, to some extent, attendance. Indeed, the multiple validation coefficients for the performance, promotion and tenure criteria are extremely impressive and compare very favourably with other self-report questionnaires: 0.60: Overall performance ratings 0.66: Rapidity of promotion to No : Attendance 0.43: Length of tenure Inspection of the specific MAPP scales which correlate with the criterion data suggests that most of the discovered relationships are consistent with intuitive predictions. However, we were surprised to find that some of the PEOPLE characteristics, such as Affectionate, Sociable and Perceptive were not more strongly related to performance data, given the strong customer contact element of the cabin staff job. This may be attributed to the fact that the existing selection procedures focus on people orientation and may be quite effective in identifying suitability for this aspect of the role. The incumbent sample used in this study was comprised of individuals who had met the existing selection criteria and would thus be restricted in range. 5

6 Financial consultants - a predictive validity study Part Two of the MAPP questionnaire, which provides scores on the 14 Values scales, was used by a Building Society as part of an assessment procedure specially designed for the selection of Financial Consultants (FCs). This was a newly created selling role. Candidates were highly experienced in the financial services sector and would typically have proven success in new business development. 325 applicants were assessed, of whom 94 were offered positions. The 94 successful candidates were followed up after their first 6 months in the FC role by investigating their objective sales performance in three areas: Fee Income (FI) Mortgage (M) Investment (I) This is generated by sales of life assurance schemes. This represents a relatively new product area for the Society and as such requires the development of new markets. This is a traditional market for the Society. Also a well established market for the Society. The MAPP scores were restricted in range because of the selection process. Correction for restriction of range has therefore been performed on the basis of the standard deviations of the scales for the full applicant sample (N=325). Presented below are the Part 2 scales that correlated significantly (2-tailed) with the sales criteria at the 5% level: Part 2 Scale Fee (FI) Mortgage (M) Investment (I) V2 Competition V3 Results V9 Self-expression V11 Intimacy V13 Security

7 Construct validity It is important that we know whether MAPP is measuring the attributes that it is supposed to measure, and one useful way of establishing this is to compare MAPP data with that obtained from other trait based questionnaires. Two studies have been conducted comparing MAPP with the 16PF (Form A) and the OPQ (Concept Model 5.2). These questionnaires are important as yardsticks since both are extensively used in personality assessment. In both studies, we have primarily considered how Part 1 of MAPP compares with these instruments since MAPP Part 2 is a survey of work-related values as opposed to personality dimensions. Nonetheless, there are some OPQ dimensions, for example, that are perhaps more closely associated with certain MAPP Part 2 scales, and where this is the case we have included them in the study. The findings are reported in detail in the MAPP Manual. However, in summary, MAPP correlated with all but one of the 16PF factors (Factor I) at the 0.05 level or better, and correlated with all the OPQ scales at the 0.05 level or better. 7

Craft Personality Questionnaire

Craft Personality Questionnaire Evidence of Reliability and Validity 888-298-6227 TalentLens.com Copyright 2008 Pearson Education, Inc., or its affiliates. All rights reserved. Copyright 2008 by Pearson