Measurement, Theory, and Current Issues in Metacognition: An Overview

Transcription

1 Chapter 1 Downloaded via on October 14, 2018 at 15:08:47 (UTC). See for options on how to legitimately share published articles. Measurement, Theory, and Current Issues in Metacognition: An Overview Tyler M. Miller * Department of Psychology, South Dakota State University, Brookings, South Dakota 57007, United States * tyler.miller@sdstate.edu. People thinking about their own thinking, a phenomenon known as metacognition, has a long history. Records of metacognitive thinking date back to ancient Greece, but people thinking about their own thoughts and beliefs certainly existed before then. However, actual metacognitive research in psychology only began in the mid-nineteenth century. In this chapter, I provide a very brief account of metacognition in history and present an overview of some topics in measurement, theory, and current research from my perspective as an experimental psychologist. Topics included have psychological, neuropsychological, and educational relevance. I pay particular attention to the varieties, causes, and consequences of metacognitive bias. I also include a sample of interventions from the literature used by teachers and researchers to improve metacognitive monitoring accuracy in the classroom and in the lab. Finally, the reasons why metacognitive sophistication should be promoted are found throughout the chapter American Chemical Society

2 Suppose we try to recall a forgotten name. The state of our consciousness is peculiar. There is a gap therein; but no mere gap. It is a gap that is intensely active. A sort of wraith of the name is in it, beckoning us in a given direction, making us at moments tingle with the sense of our closeness, and then letting us sink back without the longed-for term. If wrong names are proposed to us, this singularly definite gap acts immediately so as to negate them. They do not fit into its mould. And the gap of one word does not feel like the gap of another, all empty of content as both might seem necessarily to be when described as gaps. -William James, The Principles of Psychology, 1890 (1) Introduction Metacognition is thinking about thinking, knowing about knowing, or first in the literature as cognition about cognitive phenomena (p. 906; (2)). Eighty-nine years earlier, William James was writing about metacognition in his classic twovolume book, Principles of Psychology (1). In the quote above, James referred to one of the three interrelated processes that make up metacognition monitoring. The other two are knowledge and control of cognition. The person described in the quote is in the unenviable position known as a tip of the tongue state. He is sure he knows the name but is unable to produce it. In other words, he has monitored his memory and determined the name is available just not accessible. Perhaps if he were given more he could retrieve the name. Or, perhaps the name would spontaneously appear in his consciousness with more time. When asked about something else, he may monitor his memory again for the longed for information, and confess he does not know the answer and will never know the answer no matter how much time passes. These scenarios both describe metacognitive monitoring. Another component of metacognition is control and refers to a person intentionally directing her thinking. A student may prepare for an upcoming exam by allocating her study time to the material that is perceived to be the most difficult, easiest, or somewhere in between. Finally, metacognitive knowledge is a person s belief about cognitive processes. For example, an astute student will know that preparing for an exam in a distracting environment will interfere with learning. These three metacognitive processes operate in concert at each phase of learning (i.e., encoding, storage, and retrieval). John Flavell, the American developmental psychologist, is credited with the term metacognition itself (2). In his seminal works, Flavell discussed metacognition and tracked the development of metamemory in school-aged children (3). The term metacognition is a broad term that encompasses a person s awareness of all cognitive processes. Metamemory refers to a person s awareness of their own memory. Although all types of metacognition have received attention from researchers, the focus of this overview is considered metamemory. Even before William James, metacognitive processes were top-of-mind for researchers in psychology and other figures in history. People were very likely thinking about their own mental representations since the beginning. Evidence of 2

3 metacognitive behavior even exists in non-human primates (e.g., rhesus monkeys) and suggests an evolutionary pressure for this advanced form of thinking perhaps co-occurring with the appearance of humans earliest ancestors (4). Ancient Greek orators, like Simonides (c BC), exhibited metacognitive knowledge and control by recognizing the limitations of memory and devised strategies (e.g., the method of loci) to allow them to deliver long speeches and recite poetry without notes. Today, the method of loci and other strategies to scaffold memory are called mnemonic devices. The German psychologist, Herman Ebbinghaus ( ) also exhibited metacognitive knowledge and used only nonsense syllables as stimuli in his classic memory research. He used nonsense syllables, rather than meaningful syllables, because he knew meaningful stimuli would be more memorable and thus contaminate his results (5). The entire system known as structuralism promoted by the English psychologist Edward Titchener ( ) was based on monitoring one s own thinking through analytic introspection. Through introspection, the structuralist s goal was to create a periodic table of consciousness of simple characteristics like brightness, loudness, clarity and others that could be combined to form an individual s perception (6). Today, changes in measurement, theory, and technology have significantly changed the scope of metacognitive research since the work of James, Ebbinghaus, and Titchener. Researchers in many disciplines measure a wide variety of metacognitive judgments using many research designs to answer both basic and applied questions. Early Modern Research and Theory in Metacognition Measuring metacognition, like some other psychological constructs, is not as straightforward as measuring tangible objects. In early modern research, researchers began comparing peoples metacognitive judgments against an objective standard such as a person s performance on a memory test. Joseph Hart was among the first in the modern era to evaluate a person s metacognitive state against some basis of fact. In this research, Hart asked participants to make a feeling-of-knowing (FOK) judgment after failing to retrieve a question. For example, an experimenter may first ask a participant to name the prime minister of Great Britain. If the participant cannot name the prime minister, he would then report how likely he would be to select the correct name from a list of names. Hart then compared participants FOK reports to their actual ability to select the correct name. As it happened, participants were accurate at predicting their ability to select the correct name (7). Similarly, Brown and McNeill, in what may be considered a 75-year later epilogue to James quote to start this chapter, gave definitions of rare words (e.g., What is the name of the instrument that uses the position of the sun and stars to navigate? ) to participants and asked them to produce the term [sextant] (8). For some participants, the term was unknown, but for others the definition produced a state in which they felt they knew the term and they were close to producing it but simply could not. These latter participants were experiencing a peculiar state of consciousness known as the tip-of-the-tongue (TOT) state. What was most intriguing about this state of 3

4 consciousness was that participants were often able to produce the first letter of the term, the number of syllables, or similar sounding words. Flash forward to 1990, and a complete framework of the monitoring and control judgments that could be made at each stage of learning was proposed by Thomas Nelson and Louis Narens (9). This framework, which included the encoding, storage, and retrieval stages of learning connected researchers to a large body of literature investigating numerous types of judgments. Not only were researchers investigating participants FOK and TOT states, there had been intense interest in the types of monitoring and control judgments at the time of encoding (10). For example, even before learning, a student will exert control by deciding how to study by selecting a type of processing (e.g., reading, testing, highlighting). Furthermore, they decide what to study either by starting with already learned information or information that is deemed most difficult or easiest (i.e. allocation of study time or item selection). Finally, the student decides when to quit studying. These control judgments are linked to monitoring judgments that are made at the same time and much research has investigated the relationship between the two. The relationship between monitoring and control is exemplified in a model by Nelson & Narens (9). An important assumption of the model is that people have the ability to be self-reflective about their cognitive activity. The model also splits cognitive processes into two inter-related components. The first component is the object-level and encompasses any ongoing cognitive process. For example, memory is an object-level process. The second component is the meta-level and contains a model of the object-level cognitive process. It is also assumed there is a communication scheme by which the meta-level receives information about the object-level via monitoring processes, defined as subjective reports about his or her introspections (p. 127). Once updated, the meta-level can compare the object-level state to another state (such as mastery of the material) and then communicate to the object-level processes via control processes. The goal of control communication is to modify the object-level process and could include continuing an on-going process, terminating an on-going process, or changing the process (9). The Nelson and Narens (1990) model of metamemory does not assume that monitoring processes provide a veridical account of object-level processes. In fact, there are many biases that exist in metacognitive monitoring. For example, people often believe they will remember more than they actually do, a metacognitive bias known as the better-than-average effect or simply overconfidence (11, 12). This bias and others are discussed in detail in a later section. Exactly how a person monitors, or subjectively reports his or her introspections, is still the topic of debate. One early theory was the direct access theory of metacognition (13). In the theory, a person would directly assess the associative strength of to-beremembered items. In other words, the same information that is used in memory forms the basis of metacognitive judgments (14). For example, imagine a person recognizes the person standing next to them in line at the grocery store but cannot remember her name. Even though the person cannot remember her name, he is sure he knew it at one time. His metacognitive state is based on knowing the name is stored in memory, but the memory trace for that name is not strong enough to be retrieved. In this way, the direct access view claims the memory trace and 4

5 the metamemory state are based on the same cognitive process. The direct access theory is included here because it was influential in metacognition s history. Today, though, researchers have dismissed the idea that people have direct access to their own memory traces. In contrast to the direct access theory, the indirect or inferential theory of metacognition claims people do not have direct access to a memory trace, and therefore can neither assess the strength of memory nor base a metacognitive judgment on that assessment. Rather, people assess proxies of the memory trace to make metacognitive judgments (15, 16). For example, if the person at the grocery store was asked to make a FOK judgment about the likelihood of retrieving her name, he may make the judgment based on recalling other information about the person, maybe, information about where she works, where he saw her last, the names of people she is with and other related information. If he were able to retrieve more related information, he would give a higher FOK judgment. If he could retrieve little related information, he would report a lower FOK judgment. Further, the sources of related information in the inferential theory of monitoring will vary depending on the type of metacognitive judgment (17). Research on the judgment of learning (JOL), a monitoring judgment people make at the time of encoding, began almost as early as any topic in the modern era (13). A JOL is a person s report of his or her own likelihood of recalling a studied item on a later test. The exact form of the report for a JOL can vary, but essentially a person studies an item and then immediately after or sometime later reports if they will remember it on a test. Like many other metacognitive judgments, the JOL can be dichotomous the person reports they will or will not remember it on the test or the judgment can be made on a scale of for example. Importantly, the JOL report is compared to performance on the test. Comparing a metacognitive judgment, in this case, a JOL, to performance can yield two types of accuracy relative and absolute accuracy. Relative accuracy, typically measured by calculating a gamma correlation, is a person s ability to discriminate between well-learned information from poorly-learned information within a list of to-be-remembered material. Like other correlational coefficients, 0 values indicate no relationship and extreme values indicate strong relationships. Absolute accuracy on the other hand, can be measured using a difference score by subtracting performance from one s prediction. In this way, 0 values indicate perfect metacognitive accuracy, positive values indicate overconfidence and negative values indicate underconfidence. Using both relative and absolute measures of metacognitive accuracy can lead to different conclusions (18). While the vast majority of researchers use traditional measures of metacognitive accuracy (i.e., gamma correlations and/or difference scores), there is growing interest and work in the area of improving measurement. For example, Higham and colleagues pointed out that traditional conceptualizations of metacognitive accuracy fail to account for the variety of ways participant s translate their subjective metacognitive experiences to overt scale values, a process they have named mapping 1 (19). Failing to account for mapping 1 in their view can lead to multiple problems, all of which hamper a study s validity. Newer approaches to measurement that can attenuate these validity 5

6 concerns include using receiving operating characteristics and signal detection theory concepts. In another paradigm, participants rate how difficult items would be to remember later. When compared to their ability to remember the items, these difficulty ratings predicted participants memory ability (20). Today, participants make ease-of-learning judgments (21). Classic metacognitive research and most research since indicates peoples metacognitive monitoring judgments have above-chance predictive accuracy. While there is substantial room for improvement, people know what they know and know what they do not know. That people s predictions are not perfectly accurate opens up investigation into the varieties and causes of inaccuracy as well as the consequences and potential interventions to improve metacognitive accuracy these topics are discussed in detail in a later section. Research on metacognitive control has gained a significant understanding of what students choose to study if they have a goal of study. For example, if a student needs to study for an upcoming exam, she must decide if her goal is to get an A or perhaps just to pass with a D. The goal of study then, may contribute to her reasoning about how long to study and what material to study. There are two prominent models that have been proposed to explain how people decide how long to study, the discrepancy-reduction model (22) and the region-of-proximal learning model (23). In the discrepancy-reduction model, learners have a goal, which is known as the norm-of-study. If the norm-of-study is complete mastery, then learners would continue to study items until they believe they have completely memorized the material. In other words, the goal of studying is to reduce the discrepancy between what is known and the norm-of-study (22). Another model of study time allocation, proposes that, perhaps in addition to reducing the discrepancy between what is known and the goal, learners prioritize their study time from the subjectively easiest material to the hardest material. According to this model, learners terminate study as soon as they believe they are no longer learning (23). Ideally, students study for a sufficient amount of time to perform well on an upcoming test. Yet, students often do not study for enough time to perform well on tests. There may be several reasons why students stop studying prematurely. For example, they may simply run out of time, especially if they have busy schedules or if they start studying at the last minute. Students may also stop studying prematurely because they falsely believe that they are prepared for the exam (12, 24), or because they make a reasonable guess about their preparation (25). Given metacognitive control decisions, such as study time allocation, are based on metacognitive monitoring, they can only be as good as the quality of the monitoring (9). Benefits of Accurate Monitoring and Control Metacognition refers to the understanding that a person has about his or her own cognition. It is preferable in most cases to have accurate metacognition. Certainly, in educational settings, it is preferable for students to be able to 6

7 accurately assess their own knowledge. If students know that they do not understand the course material they can decide to continue to study for the exam. Several laboratory and classroom studies demonstrate that accurate metacognition is associated with better performance (26 31). Conversely, metacognitive inaccuracies can lead to poor academic performance (26), under-preparedness for exams, and inefficient study decisions (32). Metacognitive accuracy is associated with and causally related to more efficient learning and improved performance outcomes. In one such study, participants studied paired words from different languages (e.g., ardhi - soil), reported judgments of learning for each word pair and took a cued-recall test (e.g., ardhi -?; (29)). Researchers also told participants the study would conclude when the participant could recall all 36 word pairs or after a set number of trials. Following the first study-test phase, participants were able to self-select items to re-study. Participants with the most accurate JOLs and the most effective item-selection also had the highest cued-recall test performance. When metacognitive monitoring is experimentally manipulated by asking some participants to generate keywords and self-select material to re-study, the data lead to the same conclusion more accurate monitoring leads to more effective control which, in turn, leads to better performance outcomes (30). Biases in Metacognition When differences in monitoring accuracy emerge, one immediately wonders why there are differences. Countless studies show that people show inaccurate metacognition in the direction of being overconfident in their abilities and characteristics and this overconfidence extends to a variety of domains. People are often overconfident about their dating attractiveness (33), driving ability (34, 35), performance on college course exams (36), humor recognition (12), and gun safety knowledge (37). Low performers in particular are prone to being overconfident about their abilities. It has been suggested that low performers suffer from a double curse, they do not know the material they will be tested on and they do not know that they are underprepared (12). Indeed, David Dunning likened low performers inflated self-assessments to a form of brain damage (i.e., anosognosia), and suggested that people performing poorly cannot be expected to recognize their ineptitude and that the ability to recognize the depth of their inadequacies is beyond them (38). It follows from the double-curse account that if low performers lack knowledge and awareness then, in addition to making inaccurate performance predictions, they would also be unduly confident in these predictions. However, there is some evidence that people may not be entirely unaware of their metacognitive shortcomings (26). In one study, participants made performance predictions about an upcoming test and then reported their confidence in the accuracy of the prediction. For example, a participant could have reported that she would earn an 86% on the test and either high or low confidence that the prediction was accurate. Some research groups have coined this confidence judgment a second-order judgment the performance prediction is the first-order 7

8 metacognitive judgment and confidence about the performance prediction is the second-order metacognitive judgment (39). Miller and Geraci showed that while students are sometimes overconfident, they predict that they will perform much better than they do, their confidence in the accuracy of their prediction can be low, suggesting a dissociation between the two types of judgments (26). The finding that overly high performance predictions of low performers were associated with low confidence was taken as evidence that some participants may have awareness that their judgments are inaccurate. Other possible explanations for low performers overconfidence are that they wish to look good to the experimenter (37), they are motivated to be overconfident (40), they attribute their poor performance to external factors (41), or their overconfidence is simply a statistical artifact (42). Two recent papers have provided support for the idea that students may base their predictions for upcoming exams on desired grades (43, 44). In one such series of studies, students in upper and lower level college courses reported scores they hoped to earn on an upcoming test (desired grade) and reported scores they thought they would earn (predicted grade; (43)). In another series of studies, students in upper and lower level college courses reported predictions and indicated what factors they considered when making the predictions. The factors were categorized around two major themes: 1) educational factors like exam preparation, or 2) motivational factors like desired grades (44). In both sets of studies, motivational factors explained more variance in predictions than educational factors. One clear implication is that any attempt to improve metacognitive accuracy that does not address motivational factors, like desired grades, are not likely to eliminate overconfidence. However, as some have noted, there could be adaptive reasons for people to believe they can perform better than they actually do (40, 45). Future research should assess metacognitive accuracy versus wishful thinking to determine their beneficial and/or detrimental contributions to performance and other outcomes in educational, occupational, social, and other important areas of functioning. Overconfidence is not the only bias in metacognitive monitoring judgments. Another systematic distortion is the hard-easy effect and occurs when people overestimate their learning on hard items and underestimate their learning on easy items (46). Participants can also show underconfidence with practice (UWP). When participants have multiple study opportunities and make multiple JOLs, their JOLs may underestimate the amount of learning that has taken place (47). In other words, people do not believe they have learned as much as they have. This fact, coupled with the fact that people believe their memories will remain accessible over time has been referred to as a stability bias in human memory (48). Biological Bases of Metacognition Metacognitive research using special population samples and imaging technology has made the understanding of the brain areas associated with metacognitive processes more complete. For example, younger college student 8

9 participants display more overconfidence than their older student counterparts (49). Compared to older adults, some studies show younger people display more overconfidence than older adults (50) or equivalent metacognitive resolution (51). Older adults spared metacognitive ability has even been used to improve test performance (52, 53). Older adults with dementia attributed to Alzheimer s disease show little insight into their cognitive impairment. In fact, lack of insight is an early symptom of dementia (54). Importantly, individuals with vascular dementia have relatively preserved insight. The difference in metacognitive insight between the two types of dementias may be explained by the differences in pathophysiology. Whereas significant neocortical, frontal lobe atrophy is common with dementia of the Alzheimer s type and fronto-temporal dementia, frontal-lobe atrophy is not as common in vascular dementia (55). Similarly, patients with frontal lobe lesions due to surgical removal of brain tumors or arteriovenous malformation treatment exhibited greater metacognitive inaccuracy (i.e., overconfidence) than control subjects, particularly those with left frontal lesions (56). Research using imaging technology has corroborated the important role of the pre-frontal cortex in metacognitive processing (57, 58). In one such study, participant s grey matter volume in the anterior prefrontal cortex was associated with more metacognitive awareness (59). In another imaging study, participants viewed pictures and were asked to make dichotomous predictions about their future memory performance (60). Participants reported either they would remember or would not remember the picture on a recognition memory test while fmri data were collected. The authors reached several conclusions from the data. Among them was the conclusion that medial-temporal lobe activity, long implicated in memory formation, was not associated with metacognitive processing. Additionally, ventro-medial and lateral prefrontal cortical activity was associated with greater metacognitive accuracy. Experimentally, repetitive trans-cranial magnetic stimulation (rtms) has been used to depress activity in the dorsolateral prefrontal cortex resulting in less insight into cognitive processes (61). Training Metacognition Given the benefits of accurate metacognition but also the systematic biases that exist, there have been several attempts to improve or train metacognition in the classroom and in the laboratory. These attempts have yielded mixed results. Several interventions to improve metacognitive accuracy have used feedback. For example, studies show that providing repeated feedback to participants about the accuracy of their predictions appears to lead to modest improvements in metacognitive accuracy. In one study, participants viewed word pairs under various encoding tasks and then rated their confidence that their answers were correct on five successive days of sessions (62). Participants who received feedback after each session, showed moderate improvement in their confidence accuracy from session one to five while control condition participants did not improve. In another study, participants made retrospective confidence judgments 9

10 about the accuracy of their responses to general knowledge questions and received feedback about their accuracy (63). After 23 1-hour sessions, participants who were overconfident at the beginning of the training improved their calibration; although the majority of the improvement occurred between the first and second sessions. The authors wrote the training protocol for the study was both arduous and expensive, so much so that they completed a second experiment to determine if similar improvement would be seen with a shortened training program that was only 11 sessions long instead of 23 (pg. 166). The shortened training program did yield similar improvement. Another theme of intervention studies has been to give participants practice making predictions and practice taking memory tests. For example, when participants make one global judgment of learning, complete a practice test, and then are allowed to adjust their performance prediction, they become more accurate (64). Sometimes, these interventions improve predictions for certain participants and not others. For example, Kelemen and colleagues showed that participants predictions improved significantly after 5 sessions of making performance predictions, but it was only the high achieving students who were able to improve their metacognitive accuracy (65). This outcome when an intervention designed to benefit low achieving students has a greater benefit for high achieving students has become known as the Matthew effect (36, 65). There are other non-intervention approaches that have been used to improve metacognitive accuracy in the laboratory, too. For example, one way to improve accuracy is to increase the time between when the subject finishes studying and when the prediction is made (66). This improvement is known as the delayed- JOL effect and is thought to occur because participants are only able to use the contents of long term memory when the judgment is made (i.e., the monitoring dual memories hypothesis), which matches what the participant will use during the test. The advantage of delayed-jols over immediate JOLs has recently been confirmed through a meta-analysis involving more than 40 studies (67). Methods to improve metacognition in the classroom have used interventions that include practice (68), feedback (41), incentives (24), self-reflection (69) and combinations of these interventions. Results from these studies indicate that metacognitive monitoring ability is very resistant to intervention, and in instances when the intervention does improve metacognitive monitoring it sometimes only benefits the highest-performing students. Education and Metacognition At the same time modern metacognitive research was beginning, Ann Brown acknowledged the topic as having huge importance for educational practice (70). All varieties of study designs (i.e., correlational, cross-sectional, and longitudinal studies) have confirmed the importance of instruction using metacognitive theory beginning in very early grade-levels and continuing through secondary education to improve performance outcomes (71). On the other hand, metacognitive inaccuracy can lead to underachievement (72). Further, it has been argued that 10

11 flawed self-assessment has had the effect of higher rates of attrition from STEM fields (73). Not only should instructors be cognizant of and use instruction that fosters students metacognitive sophistication, instructors need to inform, and sometimes retrain students about effective study strategies (74). Fostering students understanding that monitoring processes are flawed is a key factor. For example, most students are not aware of the most effective study strategies, preferring less effective study strategies like re-reading or highlighting (75). At the extreme, some strategies used by students can even lead their monitoring accuracy and control to become more distorted. These strategies lead to so-called illusions of competence and can be detrimental to academic achievement (76, 77). With the publication of the current volume, enhancing learning and instruction with an understanding of metacognitive processes will continue to be an important applied research area. Some themes of this research include people s metacognitive capabilities, causes and consequences of metacognitive inaccuracy as well as benefits of accurate metacognition, interventions to improve monitoring and control, and selection of optimal study strategies. Understanding what people know about their own cognition and how they use that knowledge to guide their future behavior in order to develop more effective interventions and teaching techniques will have a significant impact on students lives in the high-stakes world of education. References 1. James, W. The Principles of Psychology; Henry Holt: New York, 1890; pp Flavell, J. H. Metacognition and cognitive monitoring. Amer. Psychol. 1979, 34, Kreutzer, M. A.; Leonard, C.; Flavell, J. H. An interview study of children s knowledge about memory. Monogr. Soc. Res. Child Dev. 1975, 40, Metcalfe, J. In Handbook of Metamemory and Memory; Dunlosky, J.; Bjork, R. A., Eds.; Psychology Press: New York, 2008; pp Ebbinghaus, H. Memory: A Contribution to Experimental Psychology (translated by H. A. Ruger); Teachers College Press: New York, 1913; pp Titchener, E. B. Systematic Psychology: Prolegomena; Macmillan: New York, 1929; pp Hart, J. T. Memory and the feeling-of-knowing experience. J. Educ. Psychol. 1965, 56, Brown, R.; McNeill, D. The tip of the tongue phenomenon. J. Verbal Learn. Verbal Behav. 1966, 5, Nelson, T. O.; Narens, L. In The Psychology of Learning and Motivation; Bower, G. H., Ed.; Academic Press: New York, 1990; Vol. 26, pp Tauber, S. K. & Dunlosky, J. In The Oxford Handbook of Metamemory; Dunlsoky, J.; Tauber, S. K., Eds.; Oxford University Press: New York, 2016; pp

12 11. Alicke, M. Global self-evaluation as determined by the desirability and controllability of trait adjectives. J. Pers. Soc. Psychol. 1985, 49, Kruger, J.; Dunning, D. Unskilled and unaware of it: How difficulties recognizing one s own incompetence lead to inflated self-assessments. J. Pers. Soc. Psychol. 1999, 77, Arbuckle, T Y.; Cuddy, L. L. Discrimination of item strength at time of presentation. J. Exper. Psychol. 1969, 81, Schwartz, B. L. Sources of information in metamemory: Judgments of learning and feelings of knowing. Psychon. Bull. Rev. 1994, 1, Koriat, A. How do we know that we know? The accessibility model of the feeling of knowing. Psychol. Rev. 1993, 100, Schwartz, B. L.; Metcalfe, J. Cue familiarity but not target retrievability enhances feeling-of-knowing judgments. J. Exp. Psychol.: Learn., Mem., Cognit. 1992, 18, Hunt, R. R.; Ellis, H. C. Fundamentals of Cognitive Psychology; McGraw Hill: New York, 2004; pp Schraw, G.; Kuch, F.; Gutierrez, A. P. Measure for measure: Calibrating ten commonly used calibration scores. Learn. Instr. 2013, 24, Higham, P. A.; Zawadzka, K.; Hanczakowski, M. In The Oxford Handbook of Metamemory; Dunlsoky, J.; Tauber, S. K., Eds.; Oxford University Press: New York, 2016; pp Underwood, B. J. Individual and group prediction of item difficulty for free learning. J. Exp. Psychol. 1966, 71, Strain, A. C.; Azevedo, R.; D Mello, S. K. Using a false biofeedback methodology to explore relationships between learners affect, metacognition, and performance. Contemp. Educ. Psychol. 2013, 38, Dunlosky, J.; Hertzog, C. In Metacognition in Educational Theory and Practice; Hacker, D. J.; Dunlosky, J.; Graesser, A. C., Eds.; Erlbaum: Mahwah, NJ, 1998; pp Metcalfe, J.; Kornell, N. A region of proximal learning model of study time allocation. J. Mem. Lang. 2005, 52, Miller, T. M.; Geraci, L. Training metacognition in the classroom: How incentives and feedback influence exam predictions. Metacognit. Learn. 2411, 6, Miller, T. M.; Geraci, L. Unskilled but aware: Reinterpreting overconfidence in low-performing students. J. Exp. Psychol.: Learn., Mem. Cognit. 2011, 37, Everson, H. T.; Tobias, S. The ability to estimate knowledge and performance in college: A metacognitive analysis. Instr. Sci. 1998, 26, Nelson, T. O.; Dunlosky, J.; Graf, A.; Narens, L. Utilization of metacognitive judgments in the allocation of study during multitrial learning. Psychol. Sci. 1994, 5, Nietfeld, J. L.; Cao, L.; Osborne, J. W. The effect of distributed monitoring exercises and feedback on performance, monitoring accuracy, and self-efficacy. Metacognit. Learn. 2006, 1,

13 29. Thiede, K. W. The importance of monitoring and self-regulation during multitrial learning. Psychon. Bull. Rev. 1999, 6, Thiede, K. W.; Anderson, M. C. M.; Therriault, D. Accuracy of metacognitive monitoring affects learning of texts. J. Educ. Psychol. 2003, 95, Rawson, K. A.; O Neil, R.; Dunlosky, J. Accurate monitoring leads to effective control and greater learning of patient education materials. J. Exp. Psychol.: Appl. 2011, 17, Bjork, R. A.; Dunlosky, J.; Kornell, N. Self-regulated learning: Beliefs, techniques, and illusions. Annu. Rev. Psychol. 2013, 64, Preuss, G. S.; Alicke, M. D. Everybody loves me: Self-evaluations and metaperceptions of dating popularity. Pers. Soc. Psychol. Bull. 2009, 35, Knouse, L. E.; Bagwell, C. L.; Barkley, R. A.; Murphy, K. R. Accuracy of self-evaluation in adults with ADHD: Evidence from a driving study. J. Atten. Disord. 2005, 8, Williams, A. F. Views of U.S. drivers about driving safety. J. Saf. Res. 2003, 34, Hacker, D. J.; Bol, L.; Horgan, D.; Rakow, E. A. Test prediction and performance in a classroom context. J. Educ. Psychol. 2000, 92, Ehrlinger, J.; Johnson, K.; Banner, M.; Dunning, D.; Kruger, J. Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent. Organ. Behav. Hum. Decis. Processes 2008, 105, Dunning, D. Self-insight: Roadblocks and Detours on the Path to Knowing Thyself; Taylor & Francis Books: New York, 2005; pp Dunlosky, J.; Serra, M. J.; Matvey, G.; Rawson, K. A. Second-order judgments about judgments of learning. J. Gen. Psychol. 2005, 132, Gramzow, R. H.; Willard, G.; Mendes, W. B. Big tales and cool heads: Academic exaggeration is related to cardiac vagal reactivity. Emotion 2008, 8, Hacker, D. J.; Bol, L.; Bahbahani, K. Explaining calibration accuracy in classroom contexts: The effects of incentives, reflection and explanatory style. Metacognit. Learn. 2008, 3, Krueger, J.; Mueller, R. A. Unskilled, unaware, or both? The better-thanaverage heuristic and statistical regression predict errors in estimates of own performance. J. Pers. Soc. Psychol. 2002, 82, Serra, M. J.; DeMarree, K. G. Unskilled and unaware in the classroom: College students desired grades predict their biased grade predictions. Mem. Cognit. 2016, 44, Saenz, G. D.; Geraci, L.; Miller, T. M.; Tirso, R. Metacognition in the classroom: The association between students exam predictions and their desired grades. Conscious. Cognit. 2017, 51, Shin, H.; Bjorklund, D. F.; Beck, E. F. The adaptive nature of children s overestimation in a strategic memory task. Cognit. Dev. 2007, 22,

14 46. Lichtenstein, S.; Fischoff, B.; Phillips, L. D. In Judgment Under Uncertainty: Heuristics and Biases; Kahneman, D.; Slovic, P.; Tversky, A., Eds.; Cambridge University Press: New York, 1982; pp Koriat, A.; Sheffer, L.; Ma ayan, H. Comparing objective and subjective learning curves: Judgments of learning exhibit increased underconfidence with practice. J. Exp. Psychol.: Gen. 2002, 131, Kornell, N.; Bjork, R. A. A stability bias in human memory: Overestimating remembering and underestimating learning. J. Exp. Psychol.: Gen. 2009, 138, Grimes, P. W. The overconfident principles of economics student: An examination of a metacognitive skill. J. Econ. Educ. 2002, 33, Hertzog, C.; Sinclair, S. M.; Dunlosky, J. Age differences in the monitoring of learning: Cross-sectional evidence of spared resolution across the adult life span. Dev. Psychol. 2010, 46, Hertzog, C.; Kidder, D. P.; Powell-Moman, A.; Dunlosky, J. Aging and monitoring associative learning: Is monitoring accuracy spared or impaired? Psychol. Aging 2002, 17, Dunlosky, J.; Kubat-Silman, A.; Hertzog, C. Training monitoring skills improves older adults associative learning. Psychol. Aging 2003, 18, Hertzog, C.; Dunlosky, J. Metacognition in later adulthood: Spared monitoring can benefit older adults self-regulation. Curr. Dir. Psychol. Sci. 2011, 20, Mendez, M. F.; Cummings, J. L. Dementia: A Clinical Approach. Butterworth Heinemann: Philadelphia, PA, 2003; pp Salat, D. H.; Kaye, J. H.; Janowsky, J. S. Selective preservation and degeneration in the prefrontal cortex in aging and Alzheimer disease. Arch. Neurol. 2001, 58, Vilkki, J.; Servo, A.; Outi, S. Word list learning and prediction of recall after frontal lobe lesions. Neuropsychology 1998, 12, Pannu, J. K.; Kaszniak, A. W. Metamemory experiments in neurological populations: A review. Neuropsychol. Rev. 2005, 15, Schwartz, B. L.; Bacon, E. In Handbook of Metamemory and Memory; Dunlosky, J.; Bjork, R. A., Eds.; Psychology Press: New York, 2008; pp Fleming, S. M.; Weil, R. S.; Nagy, Z.; Dolan, R. J.; Rees, G. Relating introspective accuracy to individual differences in brain structure. Sci. 2010, 329, Kao, Y. C.; Davis, E. S.; Gabrieli, J. D. E. Neural correlates of actual and predicted memory formation. Nat. Neurosci. 2005, 8, Rounis, E.; Maniscalco, B.; Rothwell, J. C.; Passingham, R. E.; Lau, H. Theta-burst transcranial magnetic stimulation to the prefrontal cortex impairs metacognitive visual awareness. Cognit. Neurosci. 2010, 1, Adams, P. A.; Adams, J. K. Training in confidence-judgments. Amer. J. Psychol. 1958, 71, Lichtenstein, S.; Fischoff, B. Training for calibration. Organ. Beh. Hum. Perform. 1980, 26,

15 64. Miller, T. M.; Geraci, L. Improving metacognitive accuracy: How failing to retrieve practice items reduces overconfidence. Conscious. Cognit. 2014, 29, Kelemen, W. L.; Winningham, R. G.; Weaver, C. A., III Repeated testing sessions and scholastic aptitude in college students metacognitive accuracy. Eur. J. Cognit. Psychol. 2007, 19, Nelson, T. O.; Dunlosky, J. When people s judgments of learning are extremely accurate at predicting subsequent recall: The delayed-jol effect. Psychol. Sci. 1991, 2, Rhodes, M. G.; Tauber, S. K. The influence of delaying judgments of learning on metacognitive accuracy: A meta-analytic review. Psychol. Bull. 2011, 137, Bol, L.; Hacker, D. J.; O Shea, P.; Allen, D. The influence of overt practice, achievement level, and explanatory style on calibration accuracy and performance. J. Exp. Educ. 2005, 73, Nietfeld, J. L.; Cao, L.; Osborne, J. W. Metacognitive monitoring accuracy and student performance in the postsecondary classroom. J. Exp. Educ. 2005, 74, Brown, A. L. In Advances in Child Development and Behavior; Reese H. W., Ed.; Academic Press: New York, 1975; Vol. 10, pp Kron-Sperl, V.; Schneider, W.; Hasselhorn, M. The development and effectiveness of memory strategies in kindergarten and elementary school: Findings from the Würzburg and Göttingen longitudinal studies. Cognit. Dev. 2008, 23, Dunlosky, J.; Rawson, K. A. Overconfidence produces underachievement: Inaccurate self evaluations undermine students learning and retention. Learn. Instr. 2012, 22, Stinebrickner, R.; Stinebrickner, T. R. A major in science? Initial beliefs and final outcomes for college major and dropout. Rev. Econ. Stud. 2016, 81, Dunlosky, J.; Rawson, K. A.; Marsh, E. J.; Nathan, M. J.; Willingham, D. T. Improving students learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychol. Sci. Public Interest 2013, 14, Karpicke, J. D.; Butler, A. C.; Roediger, H. L., III Metacognitive Strategies in student learning: Do students practice retrieval when they study on their own? Mem. 2009, 17, Benjamin, A. S.; Bjork, R. A.; Schwartz, B. L. The mismeasure of memory: When retrieval fluency is misleading as a metamnemonic index. J. Exp. Psychol.: Gen. 1998, 127, Rhodes, M. G.; Castel, A. D. Memory predictions are influenced by perceptual information: Evidence for metacognitive illusions. J. Exp. Psychol.: Gen. 2008, 137,