Comparison with: OPQ

Transcription

1 Comparison with: OPQ Norman L. Buckley BSc (Psych) Hons C.Psychol Rebekah J. Williams BA Hons (Psych.) MSC (Beh. Med), MSC (Org Psych), C.Psychol. N. L. Buckley All rights reserved, including translation. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, recording or duplication, in any information storage and retrieval system, without permission in writing from N. L. Buckley, and may not be photocopied or otherwise reproduced.

2 EXECUTIVE SUMMARY Facet5 and the OPQ are both normative personality questionnaires designed to assess personality in the occupational space. They were designed at roughly the same time but to slightly different theoretical models. Both are normative, web based, multi-lingual and have extensive data to support them. Analysis shows that there is a great deal of overlap between the two approaches and technical data suggest they have similar reliabilities. Validity studies have shown both to be valid predictors of performance. Differences appear in the scale content. The OPQ has additional scales covering elements of cognitive style (e.g. Cognitive, Conceptual and Data Rational) which cannot be easily extracted from a Facet5 profile. On the other hand Facet5 has a broader measure of elements of Affection which is related to situations where relationships (especially long term relationships) are important. Technically therefore the two questionnaires measure similar things in similar ways. However they have very different presentations and Facet5 has been described as being more user-friendly and flexible in its application. Page:2

3 Application Theoretical Position For a test to be broadly useful it needs to be rooted in sound psychological theory. Failure to do this can limit its credibility and breadth of application. OPQ One of the most widely applied tools in the world. Same application areas as Facet5 Created by Peter Saville, Roger Holdsworth and the team at SHL. Designed to be an advance on the 16PF where Saville had experience. Items and clusters were developed rationally original version had 30 factors. Also produced multiple other versions using different Factor Structure and Item response format. Structure has been extended to, now, 32 factors and format has been modified. Latest version is OPQ32r which measures the same 32 factors as OPQ32n (normative) but uses different presentation method and analyses response using Item Response Theory (IRT). For an intense critique of the OPQ see Facet5 Facet5 can be applied in any setting from targeted selection to self-awareness programmes and team builds. Based on Big5 Theory that is current and academically sound. Big5 is accepted worldwide as providing the foundation for understanding individual differences. There are many references to the Big5 theory in psychological literature. What does it measure? A psychometric profile needs to be sufficiently broad based (also known as wide-band ) to be sure you are covering the relevant areas and still remain simple enough to be accessible to people outside the psychological elite. How does it measure? Most different response methods will deliver similarly effective results. However there is one exception and that is the Ipsative format. This is where people are presented with mutually exclusive sets of items and are asked to choose between Page:3 Most popular are the 32 Factor models OPQ32n, OPQ32i and OPQ32r. Factors measured are: Persuasive, Controlling, Outspoken, Independent, Outgoing, Affinitive, Socially Confident, Modest, Democratic, Caring, Data Rational, Evaluative, Behavioural, Conventional, Conceptual, Innovative, Variety Seeking, Adaptable, Forward Thinking, Detail Conscious, Conscientious, Rule Following, Relaxed, Worrying, Tough Minded, Optimistic, Trusting, Emotionally Controlled, Vigorous, Competitive, Achieving, Decisive 5 main factors, 13 sub-factors, 17 families. OPQ32n Normative scales using Likert format OPQ32i Ipsative using quadrads of behaviours OPQ32r - measures the same 32 factors as OPQ32n (normative) but uses different presentation method and analyses response using Item Response Theory (IRT). Main factors are: Will, Energy, Affection, Control and Emotionality. All are independent factors with high reliability, high consistency and low intercorrelations in-line with internationally accepted Best Practice. Sub-factors are: Determination, Confrontation, Independence, Vitality, Sociability, Adaptability, Altruism, Support, Trust, Discipline, Responsibility, Tension, Apprehension Normative items are presented individually and a respondent is asked to rate it s applicability to him/her using a 5 point scale. Facet5 uses a semantic differential format where pairs of antonymous (opposite in meaning) statements are presented and the respondent is asked to show a preference for the one on the left or the one on the right. This is not ipsative since both statements load on the same dimension so the person is not asked to choose between mutually exclusive

4 them. Comparisons between people (Norms) AND OPQ OPQ Many norm groups are available for OPQ32n. Also for OPQ32i but there is much controversy over the use of norms for the ipsative test. OPQ32r is newest and claims that although presentation is ipsative using triads, sophisticated analytics produce interpretable norms. Facet5 constructs. Numerous language and country norms are available so Facet5 allows you to compare a person with other relevant employee groups. Similarities and differences are real and reliably measured. Specific norms can be added if required. Completion Format Completion time The time taken to complete a questionnaire has a complex impact on the attitude of the respondent. Too long and it becomes tedious and irritating. Too short and it is seen as superficial. Outputs and applications The value of a psychometric assessment process lies in its applicability. People need to use the information to assist day-to-day decisions about people. Paper, computer or web based. OPQ32n and OPQ32i about 45 minutes OPQ32R claims to halve the time. Has a variety of commentaries under different headings depending on vendor. A lot of emphasis is placed on self development. Uses combination of charts and text but does not directly compare profiles. Can be linked to defined competencies from SHL Work Profiling System Paper, computer or web based. 17 minutes web based. Approximately minutes on paper. Makes use of graphic output to paint a picture. Minimal text, messages are as visual as possible, particularly where comparisons are being made. Can represent multiple profiles simultaneously. Has separate outputs covering: 1. Overall behaviour 2. Searchlight - Competency structure 3. Audition - Fit to specific role 4. Leading Edge - Management guide 5. Style as a leader (with links to 360 f/b unique to Facet5) 6. Work preferences preferred job elements 7. TeamScape team integration Delivery process SHL offices worldwide. Many SHL partners acting as resellers. Available from distributors worldwide. See for details. Page:4

5 Price Training/support Job relatedness of items People are much happier where they can see a link between the element being measured and the purpose of the assessment. Fakeability of items People do not generally lie outright but they will try to present themselves in the best way possible. Invasiveness of items People become very irritated if they feel the questions impose on their private life without good reason. AND OPQ OPQ Varies with volume Requires Accreditation. Assumes Level A already. Comprehensive 3 day training to BPS Level B. If no Level A then 5 day course Designed to be work related although see Barrett and Paltiel. Complex to fake. OPQ32i has always been claimed to be the most resistant due to Ipsative format. OPQ32r is claimed to combine fake resistance with proper norms. Not personally invasive. Facet5 Varies with volume Requires accreditation then permanent support is provided. Redfield Consulting is actively involved in ongoing work based research with Facet5. Highly work related. Uses combination of behavioural and attitudinal items linked to work. Quite difficult. Uses Item balancing, Item ambiguity and Response Latency Analysis (unique to Facet5). Response Latency Analysis shows not only if faking is occurring but also which areas are affected. See Buckley NL and Williams RJ, Testing on the web - Response patterns and Impression Management, Selection and Development Review, February 2002 Items are job related therefore not personally invasive. Relate to specific role? For selection it is important that the link between a specific role and the profile is made. Has a process called Work Profiling System that creates a job template which relates to both OPQ scales and SHL cognitive tests. Individual profiles can be matched to this. Has Audition process that provides a detailed template for any role along with behavioural and situational interview questions. Candidates can be matched to Audition templates. Redfield works with clients continuously to ensure that templates are and remain reliable predictors of performance. Page:5

6 RELATIONSHIP TO FACET5 Sample A sample of 125 people who were attending a series of assessment/development centres in a financial institution. Facet5 had been administered previously for other purposes. OPQ32n was administered as part of the centre. There were 107 cases where both Facet5 and OPQ data were available. Of these 83 were male and 24 were female. Results Table 1 shows the raw correlations between the 32 OPQ scores and the 5 main Facet5 factors. Table 1: n=107 Facet5 Factor OPQ Factor Will Energy Affection Control Emotionality Persuasive Controlling Outspoken Independent.439 Outgoing.577 Affiliative.441 SociallyConfident Modest Democratic Caring DataRational Evaluative Behavioural Conventional Conceptual Innovative VarietySeeking Adaptable ForwardThining DetailConscious.321 Conscientious.317 RuleFollowing.408 Relaxed Worrying ToughMinded Optimistic Trusting.331 EmotionallyControlled Vigorous Competitive Achieving Decisive.352 Note that values less than an absolute value of 0.3 have been omitted for clarity. All values Page:6

7 significant at level To simplify this table we can look at which OPQ factors relate to Facet5 in the table below: Facet5 factor OPQ scales that correlate > 0.3 (-) means negative relationship Will Independent, Outspoken, Decisive Energy Outgoing, SociallyConfident, Worrying(-), Affiliative Affection Caring, Trusting Control Conventional, Rule Following, Conscientious, Detail Conscious Emotionality Relaxed(-), Worrying, Optimistic(-), Controlling(-) More detail can be found in the table below which show the relationship between OPQ factors and Facet5 sub-factors. OPQ Scale W1 W2 W3 E1 E2 E3 A1 A2 A3 C1 C2 Em1 Em2 Persuasive Controlling Outspoken Independent Outgoing Affiliative SociallyConfident Modest Democratic Caring DataRational Evaluative Behavioural Conventional Conceptual Innovative VarietySeeking Adaptable ForwardThining DetailConscious Conscientious RuleFollowing Relaxed Worrying ToughMinded Optimistic Trusting Emot Controlled Vigorous Competitive Achieving Decisive Note that values less than an absolute value of 0.3 have been omitted for clarity. All values significant at level Page:7

8 Legend W1 Determination W2 Confrontation W3 Independence E1 Vitality E2 Sociability E3 Adaptability A1 Altruism A2 Support A3 Trust C1 Discipline C2 Responsibility Em1 Tension Em2 Apprehension OPQ scales that correlate > 0.3 (-) means negative relationship Independent, Decisive, Controlling Outspoken, Independent, Emotionally controlled(-) Outspoken, Decisive, Independent Socially Confident, Outgoing, Controlling, Worrying(-), Affiliative Socially Confident, Outgoing, Affiliative, Worrying(-), Controlling, Optimistic, Caring Socially Confident, Outgoing, Worrying(-), Controlling, Affiliative Caring, Trusting Trusting, Caring, Optimistic Caring, Trusting, Optimistic Conventional, Rule Following, Conscientious, Detail Conscious Rule Following, Conventional, Variety Seeking(-) Relaxed(-), Worrying, Socially Confident(-), Optimistic Relaxed(-), Worrying, Controlling(-), Relaxed(-), Achieving(-) Note that OPQ scales are in order of decreasing correlation Page:8

9 THE DIFFERENCE BETWEEN FACET5 AND OPQ32 Because of the different structure of Facet5 and the OPQ differences such as shown in the previous tables make it difficult to really understand where the two questionnaires are similar and where they are different. To try and make this clear we did a final analysis where we analysed the 32 OPQ scales and the 13 Facet5 scales simultaneously. This can be seen in the table below. OPQ Scale Facet5 Scale Group 1 Forward Thinking(-) Apprehension Achieving(-) Worrying Persuasive(-) Relaxed(-) Controlling(-) Innovative(-) Optimistic(-) Tension Group 2 Affiliative Sociability Outgoing Socially Confident Caring Adaptability Vitality Group 3 Detail Conscious Responsibility Conventional Conscientious Rule Following Variety Seeking(-) Discipline Group 4 Independent Confrontation Decisive Outspoken Determination Independence Group 5 Trusting Support Adaptable Trust Altruism Note: To produce this table we produced a Principle Components analysis with the number of factors constrained to 5 to match the Facet5 structure. We rotated it using a Promax rotation for clarity and have shown the scales with a primary factor loading > 0.5 and a secondary factor loading of <0.4. This table shows a general consistency between Facet5 and the OPQ in terms of what they are measuring. There are some OPQ scales that do not link closely to Facet5 (Evaluative, Modest, Conceptual, Competitive, Behavioural, Toughminded, Data Rational, Vigorous, Democratic, Emotionally Controlled) although some of them can be estimated from a combination of Facet5 scores. This can be seen in the table below. Page:9

10 Relationship between selected OPQ scales and Facet5 sub-factors. OPQ Factor Multiple R Sig W1 W2 W3 E1 E2 E3 A1 A2 A3 C1 C2 Em1 Em2 Emotionally Controlled ** ** ** ** ** * Modest * ** ** ** *.357 ** Democratic **.212 *.238 * * Competitive Behavioural **.229 *.250 ** * Tough Minded ** Vigorous * Evaluative * Conceptual ** ** Data Rational * * Interpretation The following OPQ scales can be predicted reasonably well from a combination of Facet5 sub-factors. Emotionally Controlled: Low Will, Low Energy, High Apprehension Modest: Low confrontation, Low Energy, High Emotionality Democratic: Low Independence, High Vitality and Adaptability, Low Tension Competitive: Independence, Energy, Low Affection, Low Emotionality Behavioural: High Energy, Low Tension Tough Minded, Vigorous, Evaluative, Conceptual, Data Rational do not appear to relate strongly to Facet5. Page:10

11 SUMMARY As would be expected, the OPQ and Facet5 are measuring similar domains. It does appear that there are some things that the OPQ covers that are difficult to extract from Facet5. Conversely, the OPQ seems to have a very narrow coverage of Affection which might be unfortunate in evaluating those situations where Affection is a significant factor such as: Selection In validation studies Affection is frequently linked to high performance in Contact centre roles where Affection is associated with customer relations. Leadership We also know that Affection is a significant contributor to the Individual Consideration element of Transformational Leadership. Leaders with higher Affection generally are rated as more likely to value people as individuals and to provide individual support and encouragement. Affection also contributes to the Development element of Transactional Leadership. Transactional leaders not only monitor performance and provide feedback to staff but they also focus on the development of their staff. Team Development In team building activities differences linked to Affection are frequently associated with two broad areas: 1. High Affection contributes to a team by providing a supportive and non-judgemental atmosphere. It is this that assists in the initial, blue-sky, brain storming phase. It is not that Affection directly produces a wide range of ideas themselves but they create the environment where people feel free to contribute without fear of being shut down and criticised. 2. Affection is a major contributor to conflict resolution. While Will is clearly associated with Assertiveness Affection is linked to Cooperativeness. These are the two core drivers of the resolution process so a solid evaluation of both is important. Corporate Culture It is also clear from research that Affection is a core driver of culture. Schein, Handy and opthers have all pointed to the importance of a supportive, people oriented culture which relates very clearly to Affection. They emphasise harmony and consensus. They avoid diktat and project themselves as caring, considerate and understanding. They achieve through the development of close relationships with all stakeholders. They succeed in long term coalitions where strong personal relationships are seen as more important than short term advantage. In salesman parlance they are the farmers rather than hunters and are known for their tolerance and consideration. In trying to evaluate culture it is therefore important to have a broad analysis of the domain of Affection. Page:11