Decision Support. HAP 752 Advanced Health Informa6on Systems. Janusz Wojtusiak, PhD George Mason University Spring 2014

Transcription

1 Decision Support HAP 752 Advanced Health Informa6on Systems Janusz Wojtusiak, PhD George Mason University Spring 2014

2 Evidence does not make decisions, people do. - Haynes, Devereaux, GuyaQ (BMJ 2002)

3 A Decision Support System Daniel Kroening, Ofer Strichman, Decision Procedures, Springer, 2008

4 What is a Decision Support System? A decision support system (DSS) is any soyware that supports decision makers in making decisions

5 Decision Support Systems Usually include several elements User interface Model for reasoning (Op6onal) knowledge base Input DSS Output

6 Decision Support Systems Simula6ons Calcula6ons Knowledge- based Alerts Diagnoses Predic6ons Reminders

7 Tradi6onal Compu6ng Input Algorithm Output Input is processed by hard- coded algorithms Algorithms implemented in C++, Java, Perl, Mumps, PHP, or any other programming language

8 Knowledge- based Decision Support Domain Knowledge Input Algorithm Output Algorithms apply reasoning methods based on provided domain knowledge Domain knowledge represents evidence that is applied to cases described in input.

9 Clinical Decision Support Systems Clinical decision support is a process for enhancing health- related decisions and ac6ons, ( ); informa6on delivered can include general clinical knowledge and guidance, intelligently processed pa6ent data, ( ) - HIMSS

10 Clinical Decision Support Systems CDSS are DSS applied in clinical secngs Computer applica6ons that assist with the clinical decision making process: Match pa6ent informa6on with clinical knowledge Communicate results

11 Brief History of CDS in Healthcare Late 1950s ini6al work, not much done Late 1960s Applica6on of Bayes theorem to abdominal pain diagnosis (Unv. Of Leeds), CASNET 1970s first rule- based systems MYCIN, HELP, INTERNIST- I, PIP 1980s golden age of expert systems 1990s inclusion of machine learning 2000s standardiza6on 2010s regula6on, back to simpler systems

12 CDSS vs. Expert Systems Similar in terms of technologies used Different purpose Decision support Modeling expert

13 CDS Five Rights Right Informa6on Right Person Right CDS Interven6on Format Right Channel Right Point of Workflow HIMSS.org

14 CDS Five Rights Source: HIMSS.org

15 How CDSS work? Knowledge base or model Inference engine Interface Standalone system Connected to EMR Integrated Third party tools

16 Inference Engine A set of algorithms that apply knowledge/ models from knowledge base to provided input Includes reasoning methods Logic (strict or fuzzy) Probability Mixed taicarmen.wordpress.com

17 Inference x=3 y=2 Database x=? y=? Database z=? z=10 z = 2x + y Knowledge base z = 2x + y Knowledge base Deduc0on Abduc0on x=3 x1=3 y=2 Database z =?f(x,y) Knowledge base z=10 y1=2 x2=4 y2=3 Database Knowledge base z1=10 z2=? Induc0on Analogy Based on Coiera, 2003

18 Types of Inference Source: Michalski, 2003

19 Deduc6on Truth- preserving inference Given true premises, arrive at true conclusions The most used inference method in CDSS x=3 y=2 Database z=? z = 2x + y Knowledge base Deduc0on

20 Inference in Proposi6onal Logic A form of deduc6ve reasoning. Nota6on (β is derived from α by inference) α Ⱶ β α β

21 Rules of Inference Modus ponens (a.k.a. implica6on- elimina6on) α β, α β Modus tollens (denying the consequent) α β, ~β ~ α

22 Rules of Inference cont. And- elimina6on α 1 α 2 α n α i And- introduc6on α 1, α 2, α n α 1 α 2 α n

23 Rules of Inference cont. Or- introduc6on α i α 1 α 2 α n Unit resolu6on α β, ~ β α There are more inference rules.

24 How Decision Support Systems Do It? What happens if there are many rules? Forward chaining Backward chaining Different queuing strategies

25 Forward Chaining 1. Start with all available data/facts 2. Use all knowledge in knowledge base and inference engine to arrive at more data 3. Repeat 2 un6l goal is reached or no new data can be inferred The method is a direct mul6ple applica6on of modus ponens

26 Forward Chaining Source:

27 Example Facts (data): a, b, c Rules: b & f à g a & d à e a v c à f g & e à z Ques6on: is z true?

28 Forward Chaining Example Facts (data): a, b, c b & f à g a & d à e a v c à f g & e à z Facts: a, b, c, f b & f à g a & d à e a v c à f g & e à z Facts: a, b, c, f, g Not sa6sfied Not sa6sfied Sa6sfied f added to facts Not sa6sfied Sa6sfied g added to facts Not sa6sfied Sa6sfied nothing happens Not sa6sfied

29 Forward Chaining Example Facts (data): a, b, c, f, g b & f à g a & d à e a v c à f g & e à z Sa6sfied nothing happens Not sa6sfied Sa6sfied nothing happens Not sa6sfied Nothing happened stop algorithm Result: z cannot be inferred

30 Backward Chaining 1. Start with goal 2. Iden6fy rules that imply the goal 3. Check if the goal is sa6sfied Yes: stop algorithm No: add all facts from premise of rules to list of goals 4. Repeat 2 and 3 un6l no change

31 Backward Chaining

32 Example Facts (data): a, b, c Rules: b & f à g a & d à e a v c à f g & e à z Ques6on: is z true?

33 Backward Chaining Example Facts (data): a, b, c Goal: z g & e à z Goals: e, g, z b & f à g a & d à e Goals: d, e, f, g, z Rules: b & f à g a & d à e a v c à f g & e à z

34 Backward Chaining Example Facts (data): a, b, c Goals: d, e, f, g, z a v c à f Facts: a, b, c, f Goals: d, e, g, z b & f à g Facts: a, b, c, f, g Goals: d, e, z Rules: b & f à g a & d à e a v c à f g & e à z

35 Backward Chaining Example Facts: a, b, c, f, g Goals: d, e, z a & d à e g & e à z Rules: b & f à g a & d à e a v c à f g & e à z STOP algorithm no changes can be made! Could the algorithm be stopped sooner?

36 Queuing, Priori6zing Tricks Rules wai6ng for execu6on are put into a queue In order for inference algorithms to work more efficiently not all rules are executed equally Priori6es are assigned to rules

37 Abduc6on

38 Abduc6on (Abduc6ve Reasoning)

39 Abduc6on Abduc6on is oyen mistaken with deduc6on These forms of inference are very different The goal of abduc6on is to find the most plausible explana6on Abduc6on is a form of con6ngent inference Falsity preserving Example: finding diagnoses based on symptoms

40 Induc6on Falsity preserving inference Most data mining and machine learning methods do some forms of induc6ve inference Most typical example Scien6fic hypotheses formula6on Learning from data x=3 y=2 Database z =?f(x,y) Knowledge base Induc0on z=10

41 Learning from Data The role of induc6on is to infer plausible hypothesis that explains phenomenon described by data For example, given a set of pa6ents, an algorithm may induce a hypothesis why readmission rate is higher for some pa6ents than others AYer tes6ng, the induced hypothesis can serve as a model for predic6ng readmission for future pa6ents

42 Analogy Combina6on of Induc6on and Deduc6on Example: Pa6ent A has diabetes Pa6ent B is similar to pa6ent A Perhaps pa6ent B also has diabetes x1=3 y1=2 x2=4 y2=3 Database Knowledge base z1=10 z2=? Analogy

43 Uncertainty Again Randomness Probability is a good way to model i.e., out of 100 cases if a condi6on is given, the conclusion follows 80 6mes Vagueness Models imprecision, i.e., high temperature Adequacy Weigh6ng importance of rules to approximate expert behavior

44 Probabilis6c Reasoning The most common way of probabilis6c reasoning in healthcare is through Bayes formula and its variants The formula is counter- intui6ve, so one needs to be careful when plugging in numbers

45 Bayes Formula (odds form) Likelihood Ra6o Posterior Odds Prior Odds

46 Bayes Formula (odds form) If we have condi6onal independence of C1,, Cn in predic6ng H, the formula becomes

47 Bayes Formula Example Predic6ng re- hospitaliza6on risk p(r A,I,S,B)/p(not R A,I,S,B) = p(a R)/p(A not R) p(i R)/p(I not R) * p(s,b R)/p(S,B not R) * pr)/p(not R) A- age, I- insurance, S- surgery, B- hospitalized before Assump6on that age, and insurance are condi6onally independent for predic6ng re- hospitaliza6on

48 Bayesian Networks

49 Problem with Probability Using only probabilis6c models may not be adequate For example, if a certain infec6on is unlikely but fatal it must be adequately considered Probabilis6c models give preference to more common outcomes

50 Problem with Probability Clinicians make big decisions with small data, computers make small decisions with big data

51 Probability and Rules IF Fever AND Spots THEN Measles (with 0.93) Suppose that p(fever)=0.8 and p(spots)=0.65 p(measles) = 0.65 * 0.93 = (we use the lowest probability from condi6on and mul6ply by rule probability)

52 Reasoning with Clinical Knowledge Early example of CDSS (or expert system) is MYCIN (Shortliffe and Buchanan, 1975) Assist physicians who are not experts in an6bio6cs with treatments of blood infec6ons Detect if the pa6ent has significant infec6on Determine possible organisms involved Select set of drugs that may be appropriate Select the most appropriate drug or combina6on of drugs

53 MYCIN Physician User Consulta6on Program Dynamic Pa6ent Data Explana6on Program Sta6c Knowledge Base Knowledge Acquisi6on Program Infec6ous Disease Expert Source: Buchanan and Shortliffe, 1984

54 MYCIN Knowledge base used includes rules IF condi3on 1 holds with certainty x 1 AND condi3on 2 holds with certainty x 2 AND condi3on m holds with certainty x m THEN draw conclusion 1 with certainty y 1 AND draw conclusion 2 with certainty y 2 AND draw conclusion n with certainty y n Certainty factors: CF(ac6on)=CF(premise) x CF(rule) CF(premise)=min(CF(condi6on i )) for all condi6ons i

55 MICIN Rule Example IF 1) the stain of the organism is gramneg, and 2) the morphology of the organism is rod, and 3) the aerobicity of the organism is aerobic THEN there is strongly sugges6ve evidence (0.8) that the class of the organism is enterobacteriaceae Assuming certainty for condi6ons 1.0, 0.8, and 0.6, respec6vely, the certainty of conclusion is?

56 MYCIN Combining Evidence If more than one rule draws conclusion about a parameter, the following formula is used (X and Y are confidence factors from two rules)

57 Popular CDSS Alerts Reminders InfobuQons

58 Alerts Linked to CPOS or EMR systems Used to inform clinicians when something goes wrong Life threatening situa6ons, i.e. drug- drug interac6ons Cost- saving, i.e., duplicate tests Other, i.e., providing guidance, sugges6ng addi6onal tests

59 Alerts Usually implemented using rules Clinicians can override or accept Many studies on how clinicians react to alerts False posi6ves, alert fa6gue, acceptance, frequency Passive vs. ac6ve Knowledge quality Display Level severity of alert

60 Reminders Non- cri6cal situa6ons Immuniza6ons, tests, etc. Cri6cal situa6ons Tests due for ICU pa6ents Need to be integrated into workflow

61 InfobuQons HL7 standard for Context- Aware Knowledge Retrieval Passive form of providing evidence (in contrast to alerts) Many studies analyze use of infobuqons by clinicians

62 InfobuQons Source: Del Fiol et al., 2012

63 Non- Clinical Decision Support Revenue cycle management Medical coding & documenta6on Cost control (i.e., avoiding duplicate tests) Fraud detec6on Management

64 Other Issues

65 Legal Aspects Negligence law product or ac6vity must meet reasonable expecta6on of safety. Liability law product must not be harmful. It is unreasonable to assume that CDSS always make correct assessments physicians don t! Some problems: What happens if CDSS is wrong? What happens if CDSS is right but physician is wrong?

66 Regulatory Aspects Meaningful use Data privacy and security Regula6ons on medical soyware IOM recommends crea6on of a federal agency similar to NTSB (IOM, 2012)

67 Some Barriers and Challenges Alert fa6gue Over- relying on decision support Impact on care process and outcomes Match CDS to user inten6ons Integra6on with work processes & other systems Resources needed Acquisi6on and valida6on of pa6ent data Modeling/upda6ng medical knowledge Valida6on of systems performance

68 Current Direc6ons & Research Sharing Standardiza6on Regula6on Some research areas Where does the knowledge comes from? Natural language processing

69 HAP 752 Janusz Wojtusiak