The Assessment of Competence

Transcription

1 jeanslangdir: HECAC.DOC GRID 1: A MODEL OF COMPETENCE TO BE REPRINTED FROM CHAPTER 17 ALSO NEED FIGS 28.1, and 28.2a&b, (CURRENTLY KNOWN AS FIGS 1, 2a, & 2b, BUT TO BE R NUMBERED BY PRINTER) AS CRC CHAPTER 28 The Assessment of Competene John Raven Version Date: 19 September 2000 In previous hapters, I have shown that new forms of assessment are required so that: 1. Leturers an manage individualised ompeteny oriented eduational programmes. If high level ompetenies are to be developed, students must be able to pratise doing the things they need to do while undertaking ativities they are about. If they are to be able to do this, leturers must be able to help eah student to identify his or her onerns, interests, and patterns of ompetene and monitor his or her reations to his or her experienes. 2. Students an identify their own distintive talents, monitor their progress as they develop them, and get redit for their aomplishments. New forms of assessment are also required to enable people to get redit for the talents they have developed at work and in the ommunity, for this is where people obtain their most important developmental experiene and information. Only in this way will it be possible to break the stranglehold that eduational institutions urrently have on job entry qualifiations. 3. Leturers an get redit for their aomplishments in aountability exerises and evaluation studies. They, like the other publi servants they really are, need to be able to get redit for doing suh things as paying attention to their lients needs and inventing better ways of meeting them. 4. Evaluators an design studies whih will enable administrators to find out how to improve eduational programmes and poliies. 5. It beomes possible to implement a more effetive manpower poliy based on more sensible guidane, plaement, and development proedures and seletion poliies that are better at getting the right people into, and exluding the wrong people from, important positions. 6. It beomes possible to enourage and reognise diversity and thus break the oneptual stranglehold urrently held by single fator onepts of ability, monoultures of mind, redutionist siene, and authoritarian soial ontrol more generally. In this hapter I will first suggest a basis on whih an alternative measurement paradigm might be built. Later I will desribe the ways in whih this paradigm has already been operationalised and ways in whih its implementation ould be improved. The plae to begin to build an alternative measurement paradigm must be with the question What is ompetene? This has been disussed in earlier hapters and at greater length in my Competene in Modern Soiety. The disussion that follows will be limited to issues that are essential as a basis on whih to build an alternative assessment paradigm and will pass over other 28. 1

2 aspets of the topi, important though they are in themselves. We will begin by taking an example one that we have mentioned before namely initiative. As we have seen, initiative is a quality whih it is vital for eduators to foster. It is seen as essential to both ompetitive apitalism and suessful soialism. What is initiative? To take a suessful initiative, people have to be self motivated. Self starting people must be persistent and devote a great deal of time, thought, and effort to the ativity in whih they are engaged. They need to initiate innovative ation, monitor the effets of that ation, and learn from those effets more about the problem they are trying to takle; the soial, politial, and environmental ontext in whih it is situated; and what is effetive and ineffetive about the strategies they are using. To sueed, they must antiipate obstales in the future and invent ways of irumventing or overoming them. They will need to build up their own, unique, store of speialist knowledge. They will have to get help from others. More often than not, it will be neessary to establish oalitions with others to gain ontrol over soial and politial fores that would otherwise deflet them from their goals. Perhaps the ruial point to be emphasised in attempting to larify the nature of ompetene is that no one is going to do any of these things unless they are about the ativity they are undertaking. Values are, therefore, entral. 1 In pratie it turns out that what is valued may be a partiular outome (suh as stopping a fatory polluting a river) or it may be a partiular style of behaviour (suh as finding better ways of doing things or getting people to work together effetively). What has been said has major impliations for psyhologial and eduational measurement. It means that one must know someone's values, preoupations, or intentions before one attempts to assess his or her abilities. The exerise of important abilities demands time, energy, and effort. As a result, people will only display them when they are undertaking ativities that are important to them. It does not make sense to attempt to assess abilities (inluding, as we saw in previous hapters, suh things as "ognitive ability") exept in relation to valued goals. These observations are in sharp onflit with many traditional anons of psyhometry. I have argued that one annot assess abilities independently of values. This means that it is essential to adopt a two stage approah when assessing ompetene. We must first find out whih types of behaviour someone values, and then, and only then, assess his or her ability to bring to bear a wide variety of potentially important ognitive, affetive, and onative omponents of ompetene to undertake the ativity effetively. It is important to emphasise that the widely held view that one an use one set of measures to assess values and another set of measures, independently, to assess knowledge, skills, abilities or ompetenies, simply does not make sense. The latter will only be developed and displayed when the situation in whih the individual finds him or herself triggers or releases the former. Furthermore, sine people often annot tell one what their distintive preoupations and onerns are (sine they do not know what other people's are) one of the best ways of finding out what people are about is to ask: "In the ourse of pursuing what kinds of ativity does this person display multiple and high level talents?" Our example initiative also highlights another way in whih the assumptions on whih the dominant measurement paradigms in psyhology and eduation are based fail to engage with important aspets of ompetene. Conventional psyhometri theory plaes great stress on internal onsisteny or fatorial purity. Sores derived from tests omposed of items whih do not orrelate with eah other are said to be meaningless. Yet it would seem from our example that 28. 2

3 this assertion is not orret. People's initiatives are more likely to be suessful the more independent and different omponents of ompetene they bring to bear to ahieve their goals effetively. For example, they are more likely to be suessful if they re oneptualise the problem, obtain the help of others, persist over a long period of time, and so on. Yet their inlination and ability to do any one of these things in pursuit of their goals is unlikely to be losely related to their inlination and ability to do others. Furthermore, if they do any one of them partiularly well that will, to some extent, ompensate for their failure to do others. It follows from the observations made in the last paragraph that, if we are to assess suh qualities as initiative, instead of trying to develop measurement tools whih are as internally onsistent as possible, we need to try to develop indies made up of items whih are as little orrelated with eah other as possible. 2 This is atually not so heretial as at first sight it appears, beause it is standard pratie to make use of multiple regression equations whih involve summing over maximally independent variables in order to obtain the best preditions of behaviour. The insights we have developed so far may be summarised as follows: If we are to find ways of assessing important human traits it will be neessary to abandon our desire to develop value free, internally onsistent measures. Instead, we will need to develop value based, maximally internally heterogeneous indies whih do justie to the psyhologial omplexity of these qualities. Cognitions of Institutional Strutures This is an appropriate point to introdue one more disturbing insight whih has emerged in the ourse of our work: value based ognitions of soial proesses are entral to ompetent behaviour and need to be doumented in any meaningful assessments of ompetene. Behaviour is very muh determined by suh things as people s beliefs about how things should be done and who should relate to whom about what. It is very muh influened by their pereptions of roles by what they think it is appropriate for someone in their position to do, by what they think other people expet them to do, and by how they think other people will reat to their behaviour. It is determined by their understanding of what is meant by terms suh as management, partiipation, majority deision taking, managerial responsibility, wealth, and demoray. The disturbing onlusion is that if we are to assess ompetene in any meaningful way, it will be neessary to assess suh beliefs. Beause this onlusion raises the spetre of soial ontrol and brainwashing, it is neessary to reinfore it by reminding the reader that, as I showed in an earlier hapter, we initially ame to this onlusion from exatly the opposite starting point: When we ompared more with less ompetent farmers, teahers, bus drivers, blaksmiths, managers, and military offiers we found that, in eah ase, it was what the more effetive people did to influene the soial ontext in whih they worked viz. other people s expetations, the legal (regulatory) ontext, the eonomi ontext, and so on that was most important. Put the other way round, the most important soure of inompetent oupational behaviour in modern soiety is the inability and unwillingness to do something about the wider soial, institutional, and politial onstraints arising from outside one s job beause it is these fators that overwhelmingly determine what one an do within it

4 The Need to Desribe the Situation in Whih an Individual Finds Himself or Herself as an Integral Part of the Assessment Although the way in whih people define the situation in whih they find themselves has a marked effet on their behaviour, that ontext has other diret and indiret effets. It influenes their behaviour diretly through the onstraints it plaes on what they an do, and it influenes it indiretly through the onepts, understandings, and ompetenies that people are able to pratise and develop. It therefore emerges that, if one wishes to assess ompetene, it is neessary to assess both the pereived and the atual institutional ontext in whih it ours. As we saw earlier, it is either meaningless or wildly prejudiial to say that people lak the ability to do something that they have never had the opportunity to pratise doing. That is why a bak to basis approah reinfores a single fator model of ability. The only way out of the dilemma is to make assessment of the ontext part of the assessment of the individual. Identifiation of Values and Cognitions Although satisfatory measures of ompetene must be value based and inlude the wider soial and ivi pereptions and understandings just mentioned, one unfortunately annot disover these simply by asking people to identify the behaviours they value and their beliefs about how soiety works and their role in it. Beause they do not know muh about the values, preoupations, and thoughtways of others, they annot pereive, still less identify, the ways in whih they themselves are distintive. That is why it is impossible for students who have ome through one type of eduational programme to tell one how the issues on whih they will in future tend to fous, and their ways of approahing those problems, differ from those of others who have ome through other programmes. Not only are people unable to pereive and identify their own distintive values and beliefs, they are also often unable to identify important shared value based soial ognitions beause they are ommon to all members of their ultural group. Reapitulation and Restatement In the ourse of these remarks I have introdued some ideas that my olleagues and I have taken many years to stumble upon and make expliit and that ontrast sharply with many traditional assumptions in psyhology and eduation. For this reason, many people have found it helpful for me to re present the same ideas in different way. I will now do this, making use of a three dimensional diagram proposed by Ron Johnson, shown in Figure

5 Johnson argues that behaviour is the result of three sets of variables: skills and abilities, motivation, and the situation in whih people find themselves. For our purposes we an substitute omponents of ompetene for skills and abilities and values for motivation. So far, so good. But I have also argued that: 1. Components of ompetene will only be developed and displayed whilst those onerned are undertaking tasks they are about. They annot be abstrated in the way suggested by the diagram and assessed independently of motivation. Motivation is an integral part of ompetene. 2. Effetive performane the resultant is muh more dependent on the number of independent and substitutable ompetenies that are brought to bear in a wide variety of situations in order to reah a goal than it is on the level of ompetene or ability displayed in relation to any one of them in a partiular situation. It is the total number 28. 5

6 of ompetenies that individuals display in many situations over a long period of time in order to reah their valued goals that we need to assess, not their level of ability in relation to any one of them. Any overall index of a person s ability or motivation is virtually meaningless. 3. The situation in whih an individual is plaed influenes the values whih are aroused and the ompetenies whih are pratised and developed diretly quite apart from its influene on the behaviour that emerges at the end when a person with a partiular pattern of motivation and abilities is plaed in a partiular situation. Not only do environments have the power to transform people, people atively selet themselves into, and attend and respond to different features of, partiular environments. Johnson s diagram does not reognise this. It gives the impression that a hange in some feature of the environment will lead to an inrease (or derease) in the quality or frequeny of a partiular behaviour and that the motivation and ability of the ator will remain the same. Despite these limitations, Johnson s diagram is useful beause it emphasises (1) that it is important to assess all three sets of variables, (2) that behaviour is a produt of all three sets of variables, (3) that the omponents of ompetene an only be assessed in relation to a task the individual ares about, (4) that behaviour is influened by people s pereptions of the situation in whih they find themselves, their understandings of the way the organisation works, and the reations they expet from others, and (5) that people will only display the levels of ompetene of whih they are apable if they define the situation in whih they are plaed as one that will enable them to undertake ativities they are about. Above all, the diagram emphasises that the ompetene with whih people perform tasks they are given the resultant annot, on its own, be treated as a meaningful index of their urrent ompetene to perform those tasks, let alone as an index of the ompetenies they possess. The diagram an also be used to illustrate the fat that other people s ratings of observed behaviour the resultant are even less valid indies of the ratee s ompetene than is the behaviour itself, for what raters pereive depends on their own values and priorities, what they take to be the demands of the task and situation, and their subjetive ability to manage the ratee who has values, priorities, and talents that may well differ from their own. Many leturers (and managers) lak onfidene in their own ability to manage independent, thoughtful, questioning students. This makes them unwilling to reate situations in whih suh qualities ould be developed and displayed. And it has a marked effet on the interpretation they plae on suh behaviour when it ours. Having said that, it is important to note that it is only the (already ontaminated ) resultant behaviour, further ontaminated by their own values and abilities, that any observer an see with the unaided eye. The only way round this diffiulty involves, on the one hand, getting inside the ratee s head, and, on the other, making the values, priorities, assumptions, and ompetenies of the rater as expliit as possible. A Formal Model of Competene, Motivation, and Behaviour and Its Assessment We return now to the task of elaborating our model of ompetene and the way in whih its omponents are to be assessed. We have seen that it is inappropriate to try to assess the self motivated ompetenies that make for effetive behaviour exept in relation to ativities 28. 6

7 whih the person onerned ares about. We have also seen that there are many omponents of ompetene, that many of them are relatively independent of eah other, and that these ompetenies are umulative and substitutable. This way of thinking about ompetene may be made more onrete by referene to Grid 1. On it, some of the types of behaviour whih people value have been listed aross the top. These behaviours have been grouped into the three lusters (ahievement, affiliation, and power) identified by MClelland in 1958 and onfirmed empirially in our own previous work. 3 Down the side are listed a number of omponents of ompetene whih, if present, are likely to result in the ativity being suessful. These omponents of ompetene inlude ognitive ativities suh as making plans and thinking about obstales to goal ahievement, affetive ativities suh as enjoying the ativity or wishing that a neessary but distasteful task was ompleted, and onative ativities suh as exerising will, being determined, and persisting. However, also listed are a number of other fators whih ontribute to suessful performane suh as having the support of others and believing that one s behaviour is onsistent with both one s own and others views of what it is appropriate for someone in one s position to do. 4 The importane of separating these value and effiay omponents in assessment an be re emphasised by taking another example. An individual who values suess at football may show a great deal of initiative in relation to football, be very sensitive to feedbak from the environment, seek the help of others to improve performane, monitor, and ontinuously improve his or her style, seek out new tehniques and ideas, be sensitive to minor ues that suggest ways to improve, be sensitive to the approval or disapproval of his or her peers, have the willpower to persist in the fae of diffiulty, and be able and willing to persuade loal politiians to provide a pith or field. Nevertheless, if the ability of this same person to engage in these omplex, ognitive, affetive, soial, and onative ativities is assessed in relation to performane at mathematis a goal whih, for the sake of argument, we may assume this individual does not value then one might erroneously onlude that he or she is unable (and not just unmotivated) to engage in the ativities that have been mentioned. Teahers, psyhologists, and managers have, in the past, too frequently been guilty of drawing suh erroneous onlusions

8 GRID 1 A MODEL OF COMPETENCE Examples of Potentially Valued Styles of Behaviour Ahievement Affiliation Power Examples of omponents of effetive behaviour. Doing things whih have not been done before. Inventing things. Doing things more effiiently than they have been done before. Developing new formal sientifi theories. Providing support and failitation for someone onerned with ahievement. Establishing warm, onvivial relationships with others. Ensuring that a group works together without onflit. Establishing effetive group disussion proedures. Ensuring that group members share their knowledge so that good deisions an be taken. Artiulating group goals and releasing the energies of others in pursuit of them. Setting up domino like hains of influene to get people to do as one wishes without having to ontat them diretly. Cognitive Thinking (by opening one's mind to experiene, dreaming, and using other sub onsious proess) about what is to be ahieved and how it is to be ahieved. Antiipating obstales to ahievement and taking steps to avoid them. Analysing the effets of one's ations to disover what they have to tell one about the nature of the situation one is dealing with. Making one's value onflits expliit and trying to resolve them. Consequene antiipated: Personal: e.g. "I know there will be diffiulties, but I know from my previous experiene that I an find ways round them. Personal normative beliefs: e.g. "I would have to be more devious and manipulative than I would like to be to do that." Soial normative beliefs: e.g. "My friends would approve if I did that": "It would not be appropriate for someone in my position to do that." Affetive Turning one's emotions into the task: Admitting and harnessing feelings of delight and frustration: using the unpleasantness of tasks one needs to omplete as an inentive to get on with them rather than as an exuse to avoid them. Antiipating the delights of suess and the misery of failure. Using one's feelings to initiate ation, monitor its effets, and hange one's behaviour. Conative Putting in extra effort to redue the likelihood of failure. Persisting over a long period, alternatively striving and relaxing. Habits and experiene Confidene, based on experiene, that one an adventure into the unknown and overome diffiulties, (This involves knowledge that one will be able to do it plus a stokpile of relevant habits). A range of appropriate routineised, but flexibly ontingent behaviours, eah triggered by ues whih one may not be able to artiulate and whih may be impereptible to others. Experiene of the satisfations whih have ome from having aomplished similar tasks in the past

9 Attention should be drawn to the fat that, while this model is readily omprehended as a model designed to help us to understand and assess motivation the styles of behaviour someone values and his or her ability to pursue those goals effetively it is, in reality, a model of ompetene. Desriptive Statements and Profiles In priniple, Grid 1 an be used to identify the behaviours that people value and the omponents of ompetene they tend to display in pursuit of them. For any one person, an assessor ould, after having made relevant observations, enter tiks in the appropriate ells under the behaviours the person values. By adding up the tiks in any one olumn, the assessor an obtain an index of how likely it is that the person onerned will undertake that kind of behaviour effetively. By summing the sores obtained in adjaent olumns under eah of the overall headings, sores whih indiate the probability that a person will reah ahievement, affiliation, and power goals an be obtained. This yields a profile whih is diretly omparable with those published by MClelland, and whih he (in the present ontext, misleadingly) refers to as profiles of motivation. It is important to note, however, that beause, as has been indiated, the Grid should be onsiderably extended, the proedure would beome umbersome if it were applied whole heartedly. A way round this problem will be suggested shortly. Heterogeneous Indies or Internally Consistent Fator Sores? Not only must values be assessed as an integral part of the assessment of ompetene, the omponents of ompetene we have identified annot be meaningfully analysed or identified in fatorial or dimensional terms. The sores obtained by summing down the olumns in Grid 1 are, quite obviously, not unidimensional. Indeed, the more independent and heterogeneous the ompetenies that are omposited, the better provided, of ourse, eah relates to goal ahievement. At this point many readers will (as a result of their training in the dominant internal onsisteny, fator analyti, measurement paradigm) be thinking Suh sores are not meaningful! It is therefore important to note that, while the fator analysts laim that suh heterogeneity shows that the sores whih are obtained are not unidimensional is orret, the assumed orollary that they are not meaningful does not neessarily follow. No one would argue that multiple regression oeffiients are meaningless simply beause they are alulated by summing aross as many maximally independent preditors of performane as possible. Overall Indies Versus Detailed Desriptive Statements In pratie, an aount of the types of behaviour whih a person values and the ompetenies they display in the ourse of arrying out those ativities provides muh more useful information than a single total sore. Suh a desription is radially different from a profile of sores aross a series of fatorially independent dimensions. The assumptions behind a fatorial profile are that behaviour is best desribed and understood in terms of people s relative sores on a small number of dimensions. The assumption behind the model developed here is that behaviour is best to be understood by identifying people s values, ompulsions, pereptions, and 28. 9

10 expetations and the omponents of ompetene they tend to display spontaneously in pursuit of their valued goals. Atomi Versus Variable Models The differene between fatorial profiles and desriptive statements an be illustrated by using examples from physis and hemistry. Physiists have shown that the behaviour of a projetile is best desribed by some suh equation as: s = ut + 1 / 2 ft 2 (The distane travelled at a partiular time is determined by the initial veloity multiplied by the time elapsed plus half the aeleration multiplied by the square of the elapsed time.) The fator analysts model is analogous. For example, it may assert that the degree of leadership whih will be displayed is a funtion of the person s sores on variables suh as extroversion and intelligene. Unlike physiists, hemists have found a very different type of equation to be most useful in their work. They argue that substanes and the environments in whih they are plaed are best desribed by listing the elements of whih they are omposed and the relationship between these elements. The desriptors (elements) are drawn from a large set known to all hemists. The elements that are not present do not need to be listed. The behaviour of the substane in a partiular environment is then desribed by equations that make it possible to desribe transformations as well as monotoni ombination: Cu + 2H 2 SO 4 = CuSO 4 + 2H 2 O + SO 2 (Copper plus sulphuri aid yields opper sulphate, water, and sulphur dioxide.) It is being argued here that human beings might best be desribed and understood by adopting a model that has more in ommon with that used by hemists than that used by physiists. Suh a framework would enable us to indiate people s values and the omponents of ompetene they show a spontaneous tendeny to display in pursuit of them, and together with the relevant and signifiant features of their environments without restriting us to the small number of variables that haraterise fator analyti models. 5 We will now push the hemial analogy further. If we were to pursue this model we would find ourselves writing summary desriptions of people and the environments in whih they live and work. This might take the following form (the symbols that are used are exemplary only and should in no way be taken to suggest that we have developed even a preliminary version of a more omplete table of human elements ): Ahs 4 Pow 3 ;Auth 4 PartCit 2 ;NuP 4 HostP 3 ;DP(T) 1 Suh a statement might be interpreted to mean that the individual onerned showed a spontaneous tendeny to display four omponents of ompetene in pursuit of ahievement goals and three omponents of ompetene in pursuit of power goals. Four items that ontribute to the set dealing with authoritarian pereptions of soiety, and only two of the set dealing with partiipatory itizenship, were endorsed. Four aspets of the environment were supportive of the

11 individual s goals: The manager modelled ahievement behaviour but did not delegate, enourage partiipation, or reate developmental tasks for his or her subordinates. There was hostile press from other people in the individual s environment. Conern with effiieny and effetive leadership were sorned. The task that the individual was set had little developmental potential: It was a routine task that prevented the person onerned from developing pereptions and expetations appropriate to innovation. If the equation were written in some way that permitted movement, one would onlude that the individual would be likely to beome frustrated and lose motivation to engage in ahievement and leadership behaviours. In fat, of ourse, suh summary statements ould be filled out in a great deal more detail, and very usefully too. One ould identify exatly what type of ahievement or power behaviour the individual thought it was important to engage in, one ould identify exatly whih ompetenies were brought to bear in pursuit of eah interest, one ould identify the partiular pereptions and expetations that enouraged and prevented the person onerned from engaging in suh behaviour, and one ould say more about the role models to whom he or she was exposed by managers, olleagues, and subordinates. The next point to be made is that suh statements about people an be extended to inlude statements about their environments. One an identify the way in whih the motives and ompetenies of other people in the environment engage with those of the individual and result in emergent and vitally important group harateristis not possessed by any individual within it or ahievable by summing the parts ; one an identify the way in whih the presene or absene of other people and partiular working arrangements whih failitate or inhibit ertain types of behaviour transform the individual and result in motives, ompetenies, and behaviours that would not otherwise be displayed; one an say something about the tasks set and their probable effets on the person s future development and motivation. Suh statements enable us to desribe (or, more orretly, model) the transformational proesses that our in homes, eduational institutions, and workplaes. By insisting that statements about the individual are aompanied by statements about both the situation in whih the observations were made and the relevant previous experiene of the individual and thus prior opportunities to learn we an also overome the serious hallenges to faith in the possibility of objetive psyhologial and eduational assessment that are typially levelled at work in this area. In previous hapters we have seen that: (1) values are triggered, and ompetenies thus released or suppressed, by the situations in whih people find themselves; (2) people an only have developed high level ompetenies if the situations in whih they have previously found themselves have tapped their values (or, put the other way round and in a more onrete form, they may be perfetly apable of learning to do something they are urrently unable and unwilling to do if they are plaed in a situation that engages their values); and (3) people may be able to unleash high level ompetenies that they urrently do not display if they ome to value the task they are being expeted to undertake. Our position is, therefore, that a desription of the situations that have in the past tapped people s motives, the ompetenies they displayed in those situations (and would therefore probably transfer to any new task they might now ome to value), and whether the situation in that they urrently find themselves (and are now perhaps being observed and assessed) taps their values must form an integral part of any meaningful assessment of their ompetene

12 ***** For the sake of larity, I will now briefly reapitulate the argument that has just been presented. We first noted that people s areas of ompetene an be identified by putting tiks in the ells of a two dimensional grid whih has valued behaviours aross the top and omponents of ompetene down the side. We then noted that the internally heterogeneous summary sores (analogous to multiple regression orrelation oeffiients) that an be obtained by summing the tiks in adjaent olumns of the grid are oneptually idential to MClelland s motivation sores. However, we also noted that the original (tik based) desription of the behaviours the individual valued and the ompetenies displayed whilst undertaking those ativities was muh more revealing than the profile of summary sores. We further noted that sine the printed grid was only an illustrative sample drawn from a muh larger theoretially definable grid (with the result that putting tiks on the grid would beome umbersome if pursued wholeheartedly), we ould ahieve the desired effet by writing hemist style desriptive statements about people. These identify the behaviours they value and the ompetenies they display whilst undertaking those ativities. We then noted that this very same proedure would enable us to desribe the relevant (and only the relevant) features of the environments in whih people live and work and those in whih they had previously lived and worked. We would be able to identify different types of group having different emergent properties not derivable from summing the parts. (But we also noted that this would make the identifiation of the basi harateristis of the individual diffiult.) We finally noted that this way of proeeding would enable us to both model the transformational proesses that have proved so intratable in developmental psyhology and eduation and to handle the problems that the situational speifiity of behaviour pose for onventional (trait based) onepts of ability. ***** One final observation may be made about our researh and the nature of the future senario that would stem from its adoption. The ruial almost idiosynrati feature of what we have been doing has been that we have been mapping and sampling relevant domains of ompetene inluding their motivational basis and their ognitive, affetive, and onative omponents. At the present stage in the development of our siene, this has been no routine, ativity. Quite the opposite: It is only possible to arry it out effetively after one has developed a thorough understanding of the area one is dealing with. To pursue suh work one needs not so muh a new methodology as a limate that emphasises that sientists should devote a onsiderable amount of time to what is, after all, the ruial phase of any sientifi enquiry worth the name namely developing onepts and understanding. However, as the framework for thinking about and mapping the domains of ompetene beomes learer, the task of assessing people will beome more like arrying out a hemial analysis than measuring their height with a ruler or taking their temperature with a thermometer after the manner of a physiist. Implementing Generi Competene Assessments Assessment performs many funtions. Teahers and leturers need it to monitor the effets of their ations. Students need it to find out how well they are doing and improve their performane. Administrators need it to study the effets of individual teahers and leturers, eduational institutions, and groups of suh institutions. Summative assessments are required at

13 the point of interfae between eduational institutions and soiety so that students an get reognition for the ompetenies they have developed and thus get an opportunity to use them and develop them further in the ourse of employment. The preoupation with traditional tests and the riteria established to assess test quality has not only resulted in invalid, unreliable, and dysfuntional tests, it has also resulted in the failure to develop diagnosti and presriptive tools more suited to suh purposes as evaluating and improving eduational programmes on the one hand, and to diagnosing and remedying students learning diffiulties (e.g., in reading) and offering individualised programmes of ompetene based eduation geared to eah student s interests, values, and talents, on the other. To either assist with reading diffiulties or implement ompeteny oriented eduation, teahers and leturers need to be able to obtain information about the motives and potential interests of eah student, invent a possibly developmental experiene for that student (i.e., one that harnesses the students motives, builds on the ompetenies that have already been developed, and addresses the problems the student has in pursuing his or her own goals), monitor the student s reations to that experiene (espeially his or her speifi diffiulties), and take orretive ation when neessary. In addition, authentiating and governing Boards need to be able to assess partiular poliies and programmes of study: They need to be able to doument the distintive features of the programmes and demonstrate that they have distintive onsequenes for those who pass through them. They also need to be able to find out whether individual leturers are identifying and developing at least some of the talents of eah of the students enrolled in their ourses. Eduational offiials, administrators, leturers, students, and employers need to be able to undertake stok taking exerises to look at the human resoures available. To do these things they must, both individually and olletively, assess the quality of the developmental environments and experienes that are available and their probable onsequenes. For these purposes there is a need for a set of mirrors that enable people, individually or olletively, to take stok of what is happening so that they an, if appropriate, deide to hange it. In the remainder of this hapter I will summarise what we have been able to do, using the model developed above, to fill some of these gaps. In the ourse of so doing, the methodologies we have employed to operationalise the model will be illustrated. However, if the reader is not to be too disappointed with what is to follow, it is important for him or her to approah the material with realisti expetations. Virtually all the work on whih this disussion has been based has been arried out in spare time, on an unfunded basis, as private skirmishes on the edges of a series of unrelated and non umulative projets that were ommissioned for reasons having little to do with the entral theme of this book. Given the unquestioning aeptane of the dominant paradigm by those who ontrol funds and review researh proposals, and given the desire for quik returns and immediate answers among those who ommission researh and evaluation studies, it has proved impossible to obtain funds for researh that would have addressed these issues diretly. In setting appropriate expetations it is also important to say that, preisely beause there has been no ontinuity in funding or projets, there has been no ontinuity in staffing either. No sooner have those onerned been soialised into (earlier versions) of the way of thinking presented here than they omplete with their hard won insights and expertise have had to move on. In this ontext, the progress that has been made looks less insignifiant. It has proved possible to use the measurement model outlined above without diffiulty in programme evaluation

14 It was used in both our evaluation of the Lothian Region Eduational Home Visiting projet (whih was a Levenstein like programme of adult eduation designed to emphasise the unique and irreplaeable role of the mother in promoting the development of her hildren 6 ), and in our evaluation of the links established between primary shools and agenies of non formal eduation, suh as zoos and museums. 7 In both ases, it enabled us to show that, ontrary to the reeived wisdom, adults (whether parents or teahers) had, for better or worse, dramati effets on both hildren s and adults values and on their ompetene to undertake valued ativities effetively. It has also been employed without diffiulty when assessing what might loosely be alled national and organisational limates and patterns of ompetene assoiated with eonomi and soial development and deline. 8 We have had more diffiulty in using it for individual assessment purposes. However, even here, one set of proedures (Behavioural Event Interviewing and Reords) provides relevant and useful information in an elegant and ost effetive way, and other proedures (based on value expetany methodology) have been shown to have onsiderable potential. In this hapter, methods based on externally generated statements will be reviewed first, followed by behavioural event interview methods, and then methods based on value expetany instrumentality theory. Statements There are two essential prerequisites to obtaining meaningful external assessments of ompetene. First: assessors should be thoroughly familiar with the oneptual framework summarised above and developed more fully in Competene in Modern Soiety. 9 Seond, they, like good mothers 10 and managers, 11 should both have gone out of their way to pay attention to what their hildren and subordinates say and do (and to the meanings of their gestures and innuendoes) and thereafter have reated situations in whih students or subordinates an enthusiastially pursue ativities that they are about, growing in onfidene and ompetene in the proess. If they have done these things, teahers, leturers, and managers will, if they are good observers, find it relatively easy to put tiks in the ells of an extended version of Grid 1 to indiate whih ativities their students or subordinates value and the ompetenies they display spontaneously whilst pursuing them. An alternative is for assessors simply to list, after the manner of a hemist (or dotor, when writing a presription), the behaviours that those being assessed value and the ompetenies they display while pursuing those valued ativities. The lists of values and omponents of ompetene published in Competene in Modern Soiety may be used as aides mémoire for this purpose. If this approah is adopted, teahers and managers an also usefully desribe the situations in whih students and subordinates have worked, using the framework presented for desribing lassroom and organisational limates (in terms of the motives they tend to arouse and the behaviours they tend to enourage) presented in my Eduation, Values and Soiety and Competene in Modern Soiety. It is important to note that whereas most external assessments of people take the form of ratings (going from, e.g. intelligent to stupid ), what one gets by following the proedures desribed above is a series of statements about, or desriptions of, people and the environments in whih they have been observed. Ratings are made on a small number of sales assumed to be adequate to map the totality of the individual s ompetene. Statements draw on a vast pool of potential desriptors to make suint statements about the individual and ontext. It will be readily apparent that this proedure requires teahers, leturers, and managers

15 first to beome thoroughly familiar with the ideas summarised above (a task no more diffiult than that required of every student who aspires to be a hemist) and then to devote a onsiderable amount of time to the proess of (1) studying students or subordinates interests and talents and (2) reating situations in whih those talents an be expressed. (If leturers, teahers, or managers have failed to reate appropriate individualised developmental environments, or failed to make their observations in suh environments, any statements made about or ratings made of high level ompetenies will be meaningless.) Beause of the time required, the use of rating systems suh as are often found in staff appraisal systems is not a feasible, or at least a sensible, proposition in many settings. On the other hand, familiarity with (indeed, day to day use of) the ompeteny framework is ruial to the development, release, and effetive deployment of human resoures. It is therefore essential that teahers, leturers, and managers develop the habit of thinking more arefully about their students and subordinates talents and how best they an be developed and deployed. This objetive might best be ahieved, however, not by pithing them diretly into assessing these qualities, but by enouraging them to use the results of the more student based assessment proedures and limate surveys to be desribed below. In the past we have experimented with, indeed advoated the use of, behaviourally anhored rating sales. 12 In essene, this proedure requires raters to agree on, for example, preisely what level of initiative is indiated by a speifi behaviour of a partiular ratee. At first, the approah appeared to be very promising. However, we enountered serious diffiulties when trying to implement the neessary proedures. The reason for this took some time to emerge. Although it was obvious from our earliest trials that behaviour that one teaher would desribe as initiative would be desribed by another as the student trying to ingratiate himself with his teaher, it was not until we had reognised the entrality of values in the assessment of ompetene that we were able to appreiate that this problem ould not be resolved without first finding out what the student s values were and then respeting those values, whatever they were. One that was done, we ould begin to get some agreement about what was meant by suh qualities as initiative in relation to the student s own priorities. But, even then, if one wished to assess his or her ompetene, one had to develop behaviourally anhored sales for all the ompetenies listed in Competene in Modern Soiety in relation to all possible goals. The task beame even more umbersome than putting tiks in an extended version of Grid 1. We baked off. Behavioural Event Interviews Behavioural Event Interviews 13 or their development as olletions of personal reports on ritial inidents (or reords of behavioural events) require teahers and leturers to share more of the responsibility for assessment with students. Students are asked to think of or keep reords of times when things went partiularly well and partiularly badly for them; they are asked to report both events that they were partiularly pleased about and events that led them to feel frustrated and unomfortable. They are asked to reord what happened, what led up to the situation, and what the outome was. They are asked to say what they were trying to do or aomplish. (In this onnetion are has to be taken to reassure them that it is both appropriate and important to reord unaeptable goals suh as passing the time as pleasantly as possible in warm friendly onversation beause workplaes and soiety need people who value suh behaviour and do it well.) They are asked to desribe their thoughts and feelings while they were

16 engaged in the ativity. And they are asked to say what others did, what they did, and how others reated. These reords an then be sored by the teaher or leturer, or by an external ageny, using a variant of Grid 1. The student s or subordinates values and the ompetenies displayed when pursuing them are very apparent to anyone familiar with the oneptual framework developed above. The basi interview or reord sheets remain available should those being assessed wish to hallenge the overall statements that are made about their values and pattern of ompetene. When the interviewing and soring are arried out jointly by student and teaher and possibly by the students peers a wealth of information is available to guide future plaement and development. The methodology is elegant and, provided all onerned are prepared to take personal development seriously, it has the potential progressively to initiate both staff and students into ways of thinking about human resoures and their development and utilisation that are essential to the future development both of the eduational system and soiety. Variants of this methodology have been developed independently by Stansbury 14 and by Burgess and Adams. 15 Their work is important for two reasons. On the one hand, it indiates that it is feasible to envisage that suh assessments might be muh more widely employed in shools and olleges. On the other, it like the experiene at NELP alerts prospetive users to the amount of time that is required if students are to be offered the guidane and ounselling that is required as a basis for effetive ompeteny oriented eduation, whih is itself a prerequisite to meaningful assessments of multiple talents. Adams and Burgess 16 went on to produe what is, in a sense, an even more important variant of their methodology. They enouraged teahers to keep reords of oasions on whih they felt pleased with what they had done, perhaps oasions on whih they had ontributed in worthwhile ways to eduation and the eduational system. The teahers were then enouraged, but not obliged, to disuss these reords with olleagues with whom they felt omfortable and, in due ourse, with head teahers and administrators. What then happened was remarkable. It beame apparent to all that all teahers and not just a few wished to ontribute to the effetive funtioning of shools. Furthermore, they had all done so but had done so in very different ways. Their previously invisible onerns and talents surfaed. Morale improved dramatially. It beame apparent that the shools needed a wide variety of people who were onerned to do, and were good at doing, very different things. And a wide variety of talents was available. There was no suh thing as a model teaher to whih all teahers needed to approximate. The proess ontributed in a fundamental way to the reognition and utilisation of high level ompetenies. It provided virtually the only viable pratial solution to the widely aknowledged but seemingly insoluble problem of the need to aredit workplae learning. As Wolf 17 shows, most attempts to solve this problem set out to doument ourses taken or assess formal, low level, general (as distint from idiosynrati and tait) knowledge aquired. Adams and Burgess s sheme ends up reognising the high level ompetenies members of staff have developed. Although widely ignored as just another staff appraisal system, their sheme is in fat a development of the greatest importane. It is my belief that its effetiveness would be further improved if the disussions whih are involved were to be, at least in part, onduted with the aid of the framework for thinking about ompetene, its development, and its utilisation outlined here. The Assessment of Competene Using Value Expetany Methods