Using judgement analysis to research experts' clinical decision making and develop effective training for novices.

Transcription

1 Using judgement analysis to research experts' clinical decision making and develop effective training for novices. DECISION MAKING IN NEUROLOGICAL REHABILITATION Friday 15 th November 2013, 8.30 am 5.15 pm Lecture Theatre, 33 Queen Square, London Dr Priscilla Harries, Brunel University, London.

2 Why research clinical decision making? To identify thinking behind best practice in order to maximise effectiveness of clinical practice.

3 This is done in multiple fields of research Fire fighting Aviation Sports Farming Clinical

4 Novice (Sinclair 2003) Slow, deliberative no experience, so dependent on theory to guide practice not able to prioritize distracted by irrelevant information

5 Expert (Sinclair 2003) Focused inquiry. Clear understanding of issues. Quick and intuitive with solutions even with ill-structured problems.

6 Daniel Kahneman is an internationally renowned psychologist. In recognition of his groundbreaking work on human judgment and decision-making, Kahneman received the 2002 Nobel Prize. In this lecture he explores the idea of intuition. you tube Kaheman lecture on Intutition Start 7.30 minutes for examples

7 So how does the expert think Perception and search strategy differently? Pattern recognition

8 Perception and search strategy attention test

9 Typical Eye Movements in Reading a Mammogram Micro-calcification cluster Expert Novice Images: Dr Krupinski

10 Pattern recognition

11 PARIS IN THE THE SPRING

12 Elstein, A. S., Shulman, L. S. and Sprafka, S. A. (1978). Medical problem solving: an analysis of clinical reasoning. Cambridge, MA, Harvard University Press. Hypothetico-deductive reasoning Cue acquisition Hypothesis generation Cue interpretation Hypothesis evaluation

13 So what do judgement analysis researchers study?

14 AHP studies using judgement analysis Dalgleish, L.I. (1988) Decision making in child abuse cases: Applications of Social Judgment Theory and Signal Detection Theory. In B. Brehmer & C.R.B. Joyce (Eds) Human judgment: The SJT view. North Holland Elsevier. Rassafiani et al (2008) Occupational therapists Decision Making in the Management of Clients with Upper Limb Hypertonicity. Scandanavian Journal of Occupational Therapy, 15 (2):

15 AHP studies using judgement analysis Thompson C, Foster A, Cole I, Dowding D. (2005) Using social judgement theory to model nurses use of clinical information in critical care education. Nurse Education Today, 25 (1): Harries, P., Tomlinson, C., Notley, E., Davies, M. and Gilhooly, K., (2012) Effectiveness of a decision-training aid on referral prioritisation capacity: A randomized controlled trial, Journal of Medical Decision Making 32 (6) :

16 Harries et al. research using judgement analysis Researched decision making on the detection and prevention of elder financial abuse of professionals working in social care, health and banking in the United Kingdom. Results have been used to developed training resources which are effective in improving professionals' ability to identify elder financial abuse, and know what sort of action should be taken. Harries et al. ( ) NDA funding 300K. ESRC follow-on grant 100K. Developing decision training tools to enhance the ability of professionals to detect and prevent financial elder abuse.

17 Current studies using judgement analysis Harries P.A & Unsworth C. Development of a web-based decision aid to assist occupational therapists to make optimal fitness-to-drive decisions for disabled and older drivers. UKOTRF ( ). Harries P.A. & Hickson M. Prioritisation of referrals for dietetic acute adult services. Imperial College Healthcare Charity. ( ). White N, Stone P, Reid F, Harries P. Information use in predicting accurate estimates of survival (Oct )

18 Judgement analysis study example

19 Referral prioritisation: UK, Sample from UK Australia, NZ 40 experienced occupational therapists prioritised 90 referral scenarios by marking a cross along a visual analogue scale (see sample referral). 30 repeat referrals were added to test consistency. 19

20 Results Occupational therapists prioritisation policies were statistically modelled (R 2 =82.4%). Cluster analysis to identify subgroups 20

21 mean weightings Experts' Results weightings given to referral information when prioritising referrals reason for referral diagnosis violence referral information 21

22 mean weightings Priority given to differing diagnoses schiz. dep. anx & dep. OCN anxiety types of diagnosis 22

23 Educating the students 1. Prioritise set of referrals 2. Train students in expert policy 3. Prioritise new set of referrals 4. Analyse students prioritisation policies before and after training 23

24 Analysis Ratings (mark on line between low and high priority) Policy : cue weights (e.g. diagnosis v violence) Cue content (e.g. depression v schizophrenia) G, Rs and C Student consensus Consistency (intra rater reliability) 24

25 Mean correlation of students ratings and expert ratings (achievement) subject pre-training post-training 37 Students r = 0.23 r = 0.70 p <

26 weightings 0.6 student 3: judgement policy before training with expert policy violence support history living sit. referral information 26

27 mean weightings Experts' weightings given to referral information when prioritising referrals reason for referral diagnosis violence referral information 27

28 weightings student 3: policy after training reason for referral diagnosis referral information violence 28

29 mean ratings given to referrals Comparison of ratings given by expert and pre-trained students for each of the levels of the 'diagnosis' cue experts students pre-training schizophrenia depression anxiety & depression OCN anxiety levels of the 'diagnosis' cue

30 mean ratings given to referrals. Comparison of ratings given by experts and post trained students for each of the levels of the 'diagnosis' cue experts students' post training schizophrenia depression anxiety & depression levels of the 'diagnosis' cue OCN anxiety

31 Cochran Weiss Shanteau (CWS) CWS = discrimination inconsistency Higher score-more expert Lower score -less expert 31

32 Expert 37, CWS =

33 Student 11 Pre-training, CWS =

34 Student 11 Post-training, CWS =

35 Questions? References can be found at