EXPLORATION FLOW 4/18/10
|
|
- Justina McBride
- 5 years ago
- Views:
Transcription
1 EXPLORATION Peter Bossaerts CNS 102b FLOW Canonical exploration problem: bandits Bayesian optimal exploration: The Gittins index Undirected exploration: e-greedy and softmax (logit) The economists and psychologists view on logit: unobserved heterogeneity or random utility Directed exploration: exploration bonuses Contrast with ambiguity aversion and exploration malus The numerical analyst s view: simulated annealing, tabu search etc. Neurobiological foundations: human imaging The role of the neurotransmitter dopamine and norepinephrine 2 1
2 CANONICAL EXPLORATION PROBLEM: BANDITS Which slot machine ( arm ) to choose? Can try only one at a time Don t see payoff of arms that are not chosen Assume arms are independent And stationary, i.e., payoff distribution remains the same (opposite: restless ) Multi-armed bandit 3 BAYESIAN OPTIMAL EXPLORATION: THE GITTINS INDEX Prior, likelihood, posterior Maximize expected gain (minimize expected loss) General property of optimal policy: eventually play a single arm that is not necessarily the truly best one Trade-off exploitation-exploration Complicated dynamic programming problem Solution is of the index type: each arm is tracked using an index, and at any point in time, pick the arm with the highest (Gittins) index 4 2
3 THE GITTINS INDEX At t, for each arm k (which is in state s) compute the stopping time that would maximize the per- (discounted) period expected (discounted) payoff: Then pick the arm with the highest Gittins index Problem: in all but a few specific cases, Gittins indices are hard to compute and don t apply to more realistic problems like restless bandits Armsthat are not visited are not updated, so one may get stuck with a (truly) suboptimal arm 5 UNDIRECTED EXPLORATION: E-GREEDY AND SOFTMAX (LOGIT) Heuristic ways of exploring Epsilon-greedy: follow the currently deemed optimal arm (option) with probability 1-e, and try any other option with probability e Problem: exploration does not take into account estimated value of sub-optimal options Improvement: softmax: explore sub-optimal options with a frequency that decreases with the estimated value 6 3
4 SOFTMAX E.g., for 6 options with (estimated) values Q(l,T) at T: 1/beta is interpreted as the exploration intensity (temperature see later for physics interpretation) 7 SOFTMAX IS OPTIMAL Find best (mixed) strategy that trades off values of exploitation (first term) and of exploration (entropy of mixing; second term) Value of option l at T (6 options) Mixing This is undirected exploration: choices do not depend on how uncertain estimated values are, but on the entropy of the choice policy (Specifically, one would go for one option ONLY if that option is far superior to all others, irrespective of how sure one is about this!) 8 4
5 SOFTMAX (LOGIT) FUNCTION Choice Probability Difference in estimated value between choices 9 THE ECONOMISTS AND PSYCHOLOGISTS VIEW ON LOGIT: UNOBSERVED HETEROGENEITY OR RANDOM UTILITY Humans make choices that look erratic The economist interprets this as reflecting that she (the economist) has insufficient information to model all aspects of preferences/utility (McFadden) (Remember, for economists: choice==preference!) This is referred to as unobserved heterogeneity The psychologist (and some decision theorists like Luce) interprets this as reflecting random utility 10 5
6 RANDOM UTILITY/UNOBSERVED HETEROGENEITY With binary choice: These errors are either unobserved factors affecting preferences over time (random utility) or unobserved heterogeneity (in cross-section) 11 LOGIT Details: 12 6
7 IMPORTANT REMARK Economists do not believe that exploration is something that is added to valuation/preferences Because: CHOICE==PREFERENCES Exploration is just the difference between picking the option that is best under myopia (shortsightedness) and the one that is overall the best Only in rare instances is myopia optimal 13 LOGIT/SOFTMAX MAY NOT BE THE ENTIRE STORY IT IS A MAINTAINED, UNTESTED ASSUMPTION IN MUCH EMPIRICAL WORK $!!"#,!"# +!"# BAD FIT!"#$%&'()*+)#+,--%$)+./012%+ *!"# )!"# (!"# '!"# &!"# %!"# $!"#./0/-1#./0/-2#!"# -)# -'# -%#!# %# '# )# 3%)+45$%-)%6+782()*+#9+./012%+ 14 7
8 DIRECTED EXPLORATION: EXPLORATION BONUSES Valuation is based on: Value v at t from continuing to choose an option PLUS the novelty of the option at t (Kakade-Dayan, Neural Networks 2002) Novelty=how often has this option been visited? How often has it been seen? Is it salient? So, the bonus for an option is related to the uncertainty about its estimated value, the estimation uncertainty This is DIRECTED exploration: you visit options about which you want to know more 15 THE ECONOMISTS PERSPECTIVE: AMBIGUITY AVERSION AND EXPLORATION MALUS Ambiguity = not knowing probabilities Measure of ambiguity = estimation uncertainty (for Bayesians: posterior variance of estimated probabilities) Most humans are ambiguity averse (see later) They PENALIZE choices for which they do not know the probabilities (Hansen and Sargent): Utility = Estimated Value ß *Estimation Uncertainty This criterion is also used in robust control in engineering (estimation uncertainty is referred to there as model uncertainty) 16 8
9 HUMAN BEHAVIOR An exploration bonus or malus? We don t know Recent evidence from one of my PhD students: Undirected exploration a la softmax An exploration malus like in economics (ambiguity aversion) 17 THE NUMERICAL ANALYST S VIEW: SIMULATED ANNEALING, TABU SEARCH ETC. Numerical analysts have had to deal a lot with difficult optimization problems They have developed a toolkit of well-operating exploration procedures Originally inspired by the problem of avoiding local maxima Most primitive: re-start your search at new initial conditions, to be sure 18 9
10 RE-STARTING OPTIMIZATION 19 SIMULATED ANNEALING Annealing: Heating metal so that random movement of atoms would change the structure SLOW cooling - need enough local exploration for these atoms End result is better structure Numerical version: Randomly try something else Stay with probability P, even if value at trial is lower than original one! Try out the neighborhood 20 10
11 SIMULATED ANNEALING FLOWCHART As time passes, lower the temperature, i.e., the size of the random move 21 SIMULATED ANNEALING AND SOFTMAX Random moves in simulated annealing are often disciplined: use the softmax rule This means that random moves to sub-optimal choices are less likely as these choices become (estimated to be) more inferior The picture on slide 19 is NOT simulated annealing! 22 11
12 ANOTHER EXPLORATION RULE: TABU SEARCH Proven to be useful in very difficult combinatorial optimization problems, such as traveling salesman problem The idea is, e.g., to NOT take routes that you think are close to optimal Very effective where it takes time (several trials) to figure out whether an option is worth it 23 NOTE DIFFERENCE BETWEEN SOFTMAX AND TABU SEARCH Softmax: your modal choice is the (currently deemed) optimal one Tabu search: your are not allowed to re-visit the optimal move (for a while) Need to INHIBIT urge to stick to what is temporarily optimal We (work with K Preuschoff of U Zurich) have recently noticed that tabu search works much better if you are in an environment where payoff probabilities change abruptly With tabu search, you quickly pick up new optimal learning rates (at the expense of temporary exploration when it is not needed) 24 12
13 NEUROBIOLOGICAL FOUNDATIONS: HUMAN IMAGING 25 EXPLORATION-RELATED ACTIVATION 26 13
14 CONTRAST THIS WITH REWARD-RELATED ACTIVATIONS 27 IMPORT OF THESE FINDINGS Separation of valuation and exploration Unlike economists view that value of an option should include both Exploitation: value of continuing with the same option forever ( immediate exercise value ) Exploration: value of trying the option and thereby learning that it may become a better one ( optional value ) (In financial economics, however, immediate exercise and optional values are often analyzed separately) 28 14
15 THE ROLE OF THE NEUROTRANSMITTER DOPAMINE Dopamine neurons signal novelty (Schultz 1998) SN/VTA activation correlated with novelty (oddball) (Bunzeck ea Neuron 2006) At genetic level, polymorhisms of dopamine D4 receptors are associated with novelty seeking (mice; humans) (Ebstein, ) D4 receptor agonist induces time spent with novel objects without affectin overall locomotor activity in mice (Powell ea, 2003) Link with ADH disorder and substance abuse 29 AND NOREPINEPHRINE Enhancing NE levels induces rats to abandon old hypotheses and find the newly optimal paths in a navigation task (Or are they more sensitive to signals of UNEXPECTED UNCERTAINTY, i.e., that something changed? Yu-Dayan, Neuron 2005) 30 15
16 FIRING MODE OF NE CHANGES Note the explicit separation of value of exploitation and value of exploration Also, implicitly, there is the idea that exploration applies to cases where it is not clear that anything has to be learned (boredom) 31 16
Neurobiological Foundations of Reward and Risk
Neurobiological Foundations of Reward and Risk... and corresponding risk prediction errors Peter Bossaerts 1 Contents 1. Reward Encoding And The Dopaminergic System 2. Reward Prediction Errors And TD (Temporal
More informationReinforcement Learning. With help from
Reinforcement Learning With help from A Taxonomoy of Learning L. of representations, models, behaviors, facts, Unsupervised L. Self-supervised L. Reinforcement L. Imitation L. Instruction-based L. Supervised
More informationReinforcement Learning : Theory and Practice - Programming Assignment 1
Reinforcement Learning : Theory and Practice - Programming Assignment 1 August 2016 Background It is well known in Game Theory that the game of Rock, Paper, Scissors has one and only one Nash Equilibrium.
More informationLecture 2: Learning and Equilibrium Extensive-Form Games
Lecture 2: Learning and Equilibrium Extensive-Form Games III. Nash Equilibrium in Extensive Form Games IV. Self-Confirming Equilibrium and Passive Learning V. Learning Off-path Play D. Fudenberg Marshall
More informationExploration and Exploitation in Reinforcement Learning
Exploration and Exploitation in Reinforcement Learning Melanie Coggan Research supervised by Prof. Doina Precup CRA-W DMP Project at McGill University (2004) 1/18 Introduction A common problem in reinforcement
More informationExploiting Similarity to Optimize Recommendations from User Feedback
1 Exploiting Similarity to Optimize Recommendations from User Feedback Hasta Vanchinathan Andreas Krause (Learning and Adaptive Systems Group, D-INF,ETHZ ) Collaborators: Isidor Nikolic (Microsoft, Zurich),
More informationK Armed Bandit. Goal. Introduction. Methodology. Approach. and optimal action rest of the time. Fig 1 shows the pseudo code
K Armed Bandit Goal Introduction Methodology Approach Epsilon Greedy Method: Greedy Method: Optimistic Initial Value Method: Implementation Experiments & Results E1: Comparison of all techniques for Stationary
More informationPractical Bayesian Optimization of Machine Learning Algorithms. Jasper Snoek, Ryan Adams, Hugo LaRochelle NIPS 2012
Practical Bayesian Optimization of Machine Learning Algorithms Jasper Snoek, Ryan Adams, Hugo LaRochelle NIPS 2012 ... (Gaussian Processes) are inadequate for doing speech and vision. I still think they're
More informationArtificial Intelligence Lecture 7
Artificial Intelligence Lecture 7 Lecture plan AI in general (ch. 1) Search based AI (ch. 4) search, games, planning, optimization Agents (ch. 8) applied AI techniques in robots, software agents,... Knowledge
More informationTwo-sided Bandits and the Dating Market
Two-sided Bandits and the Dating Market Sanmay Das Center for Biological and Computational Learning Massachusetts Institute of Technology Cambridge, MA 02139 sanmay@mit.edu Emir Kamenica Department of
More informationToward a Mechanistic Understanding of Human Decision Making Contributions of Functional Neuroimaging
CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE Toward a Mechanistic Understanding of Human Decision Making Contributions of Functional Neuroimaging John P. O Doherty and Peter Bossaerts Computation and Neural
More informationOutline for the Course in Experimental and Neuro- Finance Elena Asparouhova and Peter Bossaerts
Outline for the Course in Experimental and Neuro- Finance Elena Asparouhova and Peter Bossaerts Week 1 Wednesday, July 25, Elena Asparouhova CAPM in the laboratory. A simple experiment. Intro do flexe-markets
More informationComputational approaches for understanding the human brain.
Computational approaches for understanding the human brain. John P. O Doherty Caltech Brain Imaging Center Approach to understanding the brain and behavior Behavior Psychology Economics Computation Molecules,
More informationAnthony Robbins' book on success
Anthony Robbins' book on success This is a motivational book that provides you with the inspiration and techniques with which you can achieve your goals. In this book you will be taught to not give up
More informationLearning from data when all models are wrong
Learning from data when all models are wrong Peter Grünwald CWI / Leiden Menu Two Pictures 1. Introduction 2. Learning when Models are Seriously Wrong Joint work with John Langford, Tim van Erven, Steven
More informationWhy so gloomy? A Bayesian explanation of human pessimism bias in the multi-armed bandit task
Why so gloomy? A Bayesian explanation of human pessimism bias in the multi-armed bandit tas Anonymous Author(s) Affiliation Address email Abstract 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
More informationBayesOpt: Extensions and applications
BayesOpt: Extensions and applications Javier González Masterclass, 7-February, 2107 @Lancaster University Agenda of the day 9:00-11:00, Introduction to Bayesian Optimization: What is BayesOpt and why it
More informationA critical look at the use of SEM in international business research
sdss A critical look at the use of SEM in international business research Nicole F. Richter University of Southern Denmark Rudolf R. Sinkovics The University of Manchester Christian M. Ringle Hamburg University
More informationThe Missing Link of Risk why Risk Attitude Matters
The Missing Link of Risk why Risk Attitude Matters Workbook to support the conversation between Penny Pullan and Ruth Murray-Webster 7th July 2009 My colleagues and I have spent so much of our time over
More informationNatural Scene Statistics and Perception. W.S. Geisler
Natural Scene Statistics and Perception W.S. Geisler Some Important Visual Tasks Identification of objects and materials Navigation through the environment Estimation of motion trajectories and speeds
More informationA Model of Dopamine and Uncertainty Using Temporal Difference
A Model of Dopamine and Uncertainty Using Temporal Difference Angela J. Thurnham* (a.j.thurnham@herts.ac.uk), D. John Done** (d.j.done@herts.ac.uk), Neil Davey* (n.davey@herts.ac.uk), ay J. Frank* (r.j.frank@herts.ac.uk)
More informationPractices for Demonstrating Empathy in the Workplace
Practices for Demonstrating Empathy in the Workplace These practices have been developed to help leaders at all levels to develop and demonstrate empathy. These practices, when employed in combination,
More information2Lesson. Outline 3.3. Lesson Plan. The OVERVIEW. Lesson 3.3 Why does applying pressure relieve pain? LESSON. Unit1.2
Outline 2Lesson Unit1.2 OVERVIEW Rationale: This lesson introduces students to inhibitory synapses. To review synaptic transmission, the class involves a student model pathway to act out synaptic transmission.
More informationChoice adaptation to increasing and decreasing event probabilities
Choice adaptation to increasing and decreasing event probabilities Samuel Cheyette (sjcheyette@gmail.com) Dynamic Decision Making Laboratory, Carnegie Mellon University Pittsburgh, PA. 15213 Emmanouil
More informationLiving a Healthy Balanced Life Emotional Balance By Ellen Missah
This devotional was given during Women s Awareness Week 2007 at the General Conference Morning Worships in Silver Spring, MD. The devotional may have some portions specific to the writer. If you use the
More informationNature Neuroscience: doi: /nn Supplementary Figure 1. Task timeline for Solo and Info trials.
Supplementary Figure 1 Task timeline for Solo and Info trials. Each trial started with a New Round screen. Participants made a series of choices between two gambles, one of which was objectively riskier
More informationIntroduction to Computational Neuroscience
Introduction to Computational Neuroscience Lecture 5: Data analysis II Lesson Title 1 Introduction 2 Structure and Function of the NS 3 Windows to the Brain 4 Data analysis 5 Data analysis II 6 Single
More informationA Guide to Help You Reduce and Stop Using Tobacco
Let s Talk Tobacco A Guide to Help You Reduce and Stop Using Tobacco Congratulations for taking this first step towards a healthier you! 1-866-710-QUIT (7848) albertaquits.ca It can be hard to stop using
More informationThe Recovery Journey after a PICU admission
The Recovery Journey after a PICU admission A guide for families Introduction This booklet has been written for parents and young people who have experienced a Paediatric Intensive Care Unit (PICU) admission.
More informationMaking Your Treatment Work Long-Term
Making Your Treatment Work Long-Term How to keep your treatment working... and why you don t want it to fail Regardless of the particular drugs you re taking, your drugs will only work when you take them.
More informationExpert System Profile
Expert System Profile GENERAL Domain: Medical Main General Function: Diagnosis System Name: INTERNIST-I/ CADUCEUS (or INTERNIST-II) Dates: 1970 s 1980 s Researchers: Ph.D. Harry Pople, M.D. Jack D. Myers
More informationI. Introduction. Armin Falk IZA and University of Bonn April Falk: Behavioral Labor Economics: Psychology of Incentives 1/18
I. Introduction Armin Falk IZA and University of Bonn April 2004 1/18 This course Study behavioral effects for labor related outcomes Empirical studies Overview Introduction Psychology of incentives Reciprocity
More informationIntroduction to Behavioral Economics Like the subject matter of behavioral economics, this course is divided into two parts:
Economics 142: Behavioral Economics Spring 2008 Vincent Crawford (with very large debts to Colin Camerer of Caltech, David Laibson of Harvard, and especially Botond Koszegi and Matthew Rabin of UC Berkeley)
More informationBayesian Reinforcement Learning
Bayesian Reinforcement Learning Rowan McAllister and Karolina Dziugaite MLG RCC 21 March 2013 Rowan McAllister and Karolina Dziugaite (MLG RCC) Bayesian Reinforcement Learning 21 March 2013 1 / 34 Outline
More informationIII. WHAT ANSWERS DO YOU EXPECT?
III. WHAT ANSWERS DO YOU EXPECT? IN THIS CHAPTER: Theories and Hypotheses: Definitions Similarities and Differences Why Theories Cannot be Verified The Importance of Theories Types of Hypotheses Hypotheses
More informationPaper: Uncertainty bridging troubled waters by Inger Lise Johansen & Marvin Rausand Comments by Jørn
Paper: Uncertainty bridging troubled waters by Inger Lise Johansen & Marvin Rausand Comments by Jørn : I like the discussions and presentation of different positions : I am not that happy with all the
More informationFEELING AS INTRINSIC REWARD
FEELING AS INTRINSIC REWARD Bob Marinier John Laird 27 th Soar Workshop May 24, 2007 OVERVIEW What are feelings good for? Intuitively, feelings should serve as a reward signal that can be used by reinforcement
More informationThe Neural Basis of Financial Decision Making
The Neural Basis of Financial Decision Making Camelia M. Kuhnen Kellogg School of Management Northwestern University University of Michigan - August 22, 2009 Dopamine predicts rewards Tobler et al. (2005)
More informationWhat to Do When a Loved One Is Severely Depressed
What to Do When a Loved One Is Severely Depressed There are no easy answers for helping someone struggling with depression, especially if you ve already tried and tried. Here are some tips from experts.
More informationAUTISM AIMS: KS4 (England/Wales) S4-6(Scotland) Year (Northern Ireland)
lesson plan 1 AIMS: A window into our world To understand that autism is a spectrum condition which affects each person differently. To understand the barriers that people can face in achieving their ambitions.
More informationTHEORIES OF PERSONALITY II
THEORIES OF PERSONALITY II THEORIES OF PERSONALITY II Learning Theory SESSION 8 2014 [Type the abstract of the document here. The abstract is typically a short summary of the contents of the document.
More informationWhy so gloomy? A Bayesian explanation of human pessimism bias in the multi-armed bandit task
Why so gloomy? A Bayesian explanation of human pessimism bias in the multi-armed bandit tas Anonymous Author(s) Affiliation Address email Abstract 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
More informationDopamine enables dynamic regulation of exploration
Dopamine enables dynamic regulation of exploration François Cinotti Université Pierre et Marie Curie, CNRS 4 place Jussieu, 75005, Paris, FRANCE francois.cinotti@isir.upmc.fr Nassim Aklil nassim.aklil@isir.upmc.fr
More informationChoosing Life: empowerment, Action, Results! CLEAR Menu Sessions. Adherence 1: Understanding My Medications and Adherence
Choosing Life: empowerment, Action, Results! CLEAR Menu Sessions Adherence 1: Understanding My Medications and Adherence This page intentionally left blank. Understanding My Medications and Adherence Session
More informationMindful Learning When Practice Makes Imperfect
Unleashing the Power of Us: Mindful Learning When Practice Makes Imperfect Micah Fierstein micahfierstein@earthlink.net 503 799 9003 In2:InThinking Network 2007 Forum April 13, 2007 Micah Fierstein 1 Seven
More informationCognitive modeling versus game theory: Why cognition matters
Cognitive modeling versus game theory: Why cognition matters Matthew F. Rutledge-Taylor (mrtaylo2@connect.carleton.ca) Institute of Cognitive Science, Carleton University, 1125 Colonel By Drive Ottawa,
More informationRisk attitude in decision making: A clash of three approaches
Risk attitude in decision making: A clash of three approaches Eldad Yechiam (yeldad@tx.technion.ac.il) Faculty of Industrial Engineering and Management, Technion Israel Institute of Technology Haifa, 32000
More informationarxiv: v1 [stat.ap] 2 Feb 2016
Better safe than sorry: Risky function exploitation through safe optimization Eric Schulz 1, Quentin J.M. Huys 2, Dominik R. Bach 3, Maarten Speekenbrink 1 & Andreas Krause 4 1 Department of Experimental
More informationModule 28 - Estimating a Population Mean (1 of 3)
Module 28 - Estimating a Population Mean (1 of 3) In "Estimating a Population Mean," we focus on how to use a sample mean to estimate a population mean. This is the type of thinking we did in Modules 7
More informationExploring Complexity in Decisions from Experience: Same Minds, Same Strategy
Exploring Complexity in Decisions from Experience: Same Minds, Same Strategy Emmanouil Konstantinidis (em.konstantinidis@gmail.com), Nathaniel J. S. Ashby (nathaniel.js.ashby@gmail.com), & Cleotilde Gonzalez
More informationAudio: In this lecture we are going to address psychology as a science. Slide #2
Psychology 312: Lecture 2 Psychology as a Science Slide #1 Psychology As A Science In this lecture we are going to address psychology as a science. Slide #2 Outline Psychology is an empirical science.
More informationBelief Formation in a Signalling Game without Common Prior: An Experiment
Belief Formation in a Signalling Game without Common Prior: An Experiment Alex Possajennikov University of Nottingham February 2012 Abstract Using belief elicitation, the paper investigates the formation
More informationCourse summary, final remarks
Course "Empirical Evaluation in Informatics" Prof. Dr. Lutz Prechelt Freie Universität Berlin, Institut für Informatik http://www.inf.fu-berlin.de/inst/ag-se/ Role of empiricism Generic method Concrete
More informationAgents. This course is about designing intelligent agents Agents and environments. Rationality. The vacuum-cleaner world
This course is about designing intelligent agents and environments Rationality The vacuum-cleaner world The concept of rational behavior. Environment types Agent types 1 An agent is an entity that perceives
More informationAn Introduction to Mindfulness
An Introduction to Mindfulness The intention behind mindfulness - To bring your awareness back to the present moment without judgment whenever we become aware of projecting our thoughts to the future or
More informationBoredom, Information-Seeking and Exploration
Boredom, Information-Seeking and Exploration Andra Geana (ageana@princeton.edu) *+ Robert C. Wilson (bob@email.arizona.edu) Nathaniel Daw (ndaw@princeton.edu) *+ Jonathan D. Cohen (jdc@princeton.edu) *+
More informationFault Detection and Localisation in Reduced Test Suites
UNIVERSITY OF SZEGED Fault Detection and Localisation in Reduced Test Suites Árpád Beszédes University of Szeged, Hungary The 29 th CREST Open Workshop, London November 2013 Overview University of Szeged,
More informationUtility Maximization and Bounds on Human Information Processing
Topics in Cognitive Science (2014) 1 6 Copyright 2014 Cognitive Science Society, Inc. All rights reserved. ISSN:1756-8757 print / 1756-8765 online DOI: 10.1111/tops.12089 Utility Maximization and Bounds
More informationProviding for Your Companion Animal s Future Without You
Providing for Your Companion Animal s Future Without You The Humane Society Legislative Fund Since companion animals usually have shorter life spans than their human caregivers, you may have planned for
More informationRunning head: Cognitive decision models in cocaine abusers. Cognitive modeling analysis of the decision-making processes used by cocaine abusers
Cognitive Decision Models 1 Running head: Cognitive decision models in cocaine abusers Cognitive modeling analysis of the decision-making processes used by cocaine abusers Julie C. Stout, Jerome R. Busemeyer
More informationDiscover Simple Neuroscience Body Hacks that Easily Increase Personal Performance, Accelerate Physical Results and Relieve Pain
Discover Simple Neuroscience Body Hacks that Easily Increase Personal Performance, Accelerate Physical Results and Relieve Pain Welcome to the Z-Health Neuroscience Body Hacks Webinar! Because our webinar
More informationAnalysis of complex patterns of evidence in legal cases: Wigmore charts vs. Bayesian networks
Analysis of complex patterns of evidence in legal cases: Wigmore charts vs. Bayesian networks V. Leucari October 2005 Typical features of the evidence arising from legal cases are its complex structure
More informationNARM Study-Practice Group
NARM Study-Practice Group Highlighting the R in NARM: Working with the Relational Process Brad Kammer, MFT, LPCC & Laurence Heller, PhD NARM Relational Process We recognize that the therapeutic process
More informationUsing Experimental Methods to Inform Public Policy Debates. Jim Murphy Presentation to ISER July 13, 2006
Using Experimental Methods to Inform Public Policy Debates Jim Murphy Presentation to ISER July 13, 2006 Experiments are just one more tool for the applied economist Econometric analysis of nonexperimental
More informationReview: Conditional Probability. Using tests to improve decisions: Cutting scores & base rates
Review: Conditional Probability Using tests to improve decisions: & base rates Conditional probabilities arise when the probability of one thing [A] depends on the probability of something else [B] In
More informationSubstance Prevention
First Name Last Name Period Substance Prevention POINTS ASSIGNMENT /65 pts Worksheet Total /10 pts Book Activity Page /10 pts Group Discussion on Substance Use in Teenagers /10 pts Teenage Drinking Brain
More informationRepresentativeness Heuristic and Conjunction Errors. Risk Attitude and Framing Effects
1st: Representativeness Heuristic and Conjunction Errors 2nd: Risk Attitude and Framing Effects Psychology 355: Cognitive Psychology Instructor: John Miyamoto 05/30/2018: Lecture 10-3 Note: This Powerpoint
More informationEconomics 2010a. Fall Lecture 11. Edward L. Glaeser
Economics 2010a Fall 2003 Lecture 11 Edward L. Glaeser Final notes how to write a theory paper: (1) A highbrow theory paper go talk to Jerry or Drew don t listen to me. (2) A lowbrow or applied theory
More informationIntroduction to Machine Learning. Katherine Heller Deep Learning Summer School 2018
Introduction to Machine Learning Katherine Heller Deep Learning Summer School 2018 Outline Kinds of machine learning Linear regression Regularization Bayesian methods Logistic Regression Why we do this
More informationOptimization and Experimentation. The rest of the story
Quality Digest Daily, May 2, 2016 Manuscript 294 Optimization and Experimentation The rest of the story Experimental designs that result in orthogonal data structures allow us to get the most out of both
More informationSearch e Fall /18/15
Sample Efficient Policy Click to edit Master title style Search Click to edit Emma Master Brunskill subtitle style 15-889e Fall 2015 11 Sample Efficient RL Objectives Probably Approximately Correct Minimizing
More informationStudy Designs. Lecture 3. Kevin Frick
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationDepression Care. Patient Education Script
Everybody has the blues from time to time, or reacts to stressful life events with feelings of anxiety, sadness, or anger. Normally these feelings go away with time but when they persist, and are present
More informationLecture 2: Foundations of Concept Learning
Lecture 2: Foundations of Concept Learning Cognitive Systems - Machine Learning Part I: Basic Approaches to Concept Learning Version Space, Candidate Elimination, Inductive Bias last change October 18,
More informationWho will benefit from using this app?
INSTRUCTIONS for THE MINDFUL EATING COACH APP (available at Apple Store) (These instructions provide more detail than what is currently in the app under the Coaching tab) Who will benefit from using this
More informationExperimental Design Notes
Experimental Design Notes 1 Introduction We have previously discussed how microeconomic systems can be implemented as economic experiments. I have also provided some slides as a sample of how a particular
More information(WG Whitfield Growden, MD; DR Diane Redington, CRNP)
2795 Estates Drive Park City, UT 84060 TRANSCRIPT FOR VIDEO #6: HOW TO FIND A CLINICAL TRIAL WITH DR. WHITFIELD GROWDEN Interview, Massachusetts General Hospital January 5, 2017 Produced by (WG Whitfield
More informationUnderstanding Addiction and Dugs Of Abuse
Understanding Addiction and Dugs Of Abuse Wilkie A. Wilson, Ph.D. DukeLEARN wawilson@duke.edu There is a lot of epidemiological evidence that addiction begins before brain maturity, and lately some biological
More informationArtificial Intelligence Programming Probability
Artificial Intelligence Programming Probability Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science University of San Francisco p.1/25 17-0: Uncertainty
More informationYou probably don t spend a lot of time here, but if you do, you are reacting to the most basic needs a human has survival and protection.
Emotional Eating Food Diary An emotional eating food diary will take some work on your part. You can dismiss it because you don t feel like doing it or you don t think it will help. However, if you choose
More informationClearing the Air: What You Need to Know and Do to Prepare to Quit Smoking. Getting Ready to Quit Course
Clearing the Air: What You Need to Know and Do to Prepare to Quit Smoking Getting Ready to Quit Course Sponsored by: American Lung Association of Maryland, Baltimore County Health Department & MDQuit Can
More informationNeurons and neural networks II. Hopfield network
Neurons and neural networks II. Hopfield network 1 Perceptron recap key ingredient: adaptivity of the system unsupervised vs supervised learning architecture for discrimination: single neuron perceptron
More informationObstacle- something that obstructs or hinders progress or action.
Obstacle- something that obstructs or hinders progress or action. Notice that there are two main ways that an obstacle gets in the way of progress. The first is that an obstacle may obstruct progress.
More informationStudy Guide for Why We Overeat and How to Stop Copyright 2017, Elizabeth Babcock, LCSW
Study Guide for Why We Overeat and How to Stop Copyright 2017, Elizabeth Babcock, LCSW This book can be discussed in many different ways. Whatever feels productive and enlightening for you and/or your
More informationReinforcement Learning
Reinforcement Learning Michèle Sebag ; TP : Herilalaina Rakotoarison TAO, CNRS INRIA Université Paris-Sud Nov. 9h, 28 Credit for slides: Richard Sutton, Freek Stulp, Olivier Pietquin / 44 Introduction
More informationAchievement: Approach versus Avoidance Motivation
LP 11E Achievement motivation 1 Achievement: Approach versus Avoidance Motivation Approach motivation: A motivation to experience positive outcomes Avoidance motivation: A motivation not to experience
More informationHuman and Optimal Exploration and Exploitation in Bandit Problems
Human and Optimal Exploration and ation in Bandit Problems Shunan Zhang (szhang@uci.edu) Michael D. Lee (mdlee@uci.edu) Miles Munro (mmunro@uci.edu) Department of Cognitive Sciences, 35 Social Sciences
More informationDeveloping Intellectual Character
Developing Intellectual Character 2016-2017 One of our main aims at King Henry VIII School is to allow young people to be the best that they can be in whatever activity they choose. We believe that children
More informationBehavioral Finance 1-1. Chapter 5 Heuristics and Biases
Behavioral Finance 1-1 Chapter 5 Heuristics and Biases 1 Introduction 1-2 This chapter focuses on how people make decisions with limited time and information in a world of uncertainty. Perception and memory
More informationThe Fallacy of Taking Random Supplements
The Fallacy of Taking Random Supplements Healthview interview with Dr. Paul Eck Healthview: We can see from our conversations that you are totally against people taking random supplements even if people
More informationRising Scholars Academy 8 th Grade English I Summer Reading Project The Alchemist By Paulo Coelho
Rising Scholars Academy 8 th Grade English I Summer Reading Project The Alchemist By Paulo Coelho Welcome to 8th grade English I! Summer is a time where you can relax and have fun, but did you know you
More informationTowards Learning to Ignore Irrelevant State Variables
Towards Learning to Ignore Irrelevant State Variables Nicholas K. Jong and Peter Stone Department of Computer Sciences University of Texas at Austin Austin, Texas 78712 {nkj,pstone}@cs.utexas.edu Abstract
More informationAxiomatic Methods, Dopamine and Reward Prediction. Error
Axiomatic Methods, Dopamine and Reward Prediction Error Andrew Caplin and Mark Dean Center for Experimental Social Science, Department of Economics New York University, 19 West 4th Street, New York, 10032
More informationEvolutionary Programming
Evolutionary Programming Searching Problem Spaces William Power April 24, 2016 1 Evolutionary Programming Can we solve problems by mi:micing the evolutionary process? Evolutionary programming is a methodology
More informationChapter 02 Developing and Evaluating Theories of Behavior
Chapter 02 Developing and Evaluating Theories of Behavior Multiple Choice Questions 1. A theory is a(n): A. plausible or scientifically acceptable, well-substantiated explanation of some aspect of the
More informationRecognizing Ambiguity
Recognizing Ambiguity How Lack of Information Scares Us Mark Clements Columbia University I. Abstract In this paper, I will examine two different approaches to an experimental decision problem posed by
More informationHow to Choose a Counsellor
How to Choose a Counsellor Many survivors of sexual assault, sexual abuse, or other forms of violence find counselling to be a helpful part of their recovery process. Counselling can accelerate the relief
More informationThe Psychology of Inductive Inference
The Psychology of Inductive Inference Psychology 355: Cognitive Psychology Instructor: John Miyamoto 05/24/2018: Lecture 09-4 Note: This Powerpoint presentation may contain macros that I wrote to help
More informationLecture 7 Part 2 Crossroads of economics and cognitive science. Xavier Gabaix. March 18, 2004
14.127 Lecture 7 Part 2 Crossroads of economics and cognitive science. Xavier Gabaix March 18, 2004 Outline 1. Introduction 2. Definitions (two system model) 3. Methods 4. Current research 5. Questions
More informationLearning Styles Questionnaire
This questionnaire is designed to find out your preferred learning style(s) Over the years you have probably developed learning habits that help you benefit from some experiences than from others Since
More information