The Conference That Counts! March, 2018
Statistics, Definitions, & Theories The Audit Process Getting it Wrong Practice & Application
Some Numbers You Should Know
Objectivity Analysis Interpretation Reflection Evaluation Inference Exposition Problem Solving Decision Making Predicting Transforming Logical Reasoning Skepticism Open-mindedness Respects Evidence Good Judgement Creativity Job Knowledge Inquisitiveness Transferring Knowledge Rational Thinking Willing to Change
the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. Criticalthinking.org
You either have critical thinking skills or you don t How do you get to Carnegie Hall? Understand the skill Learn the techniques Practice Get feedback
1. We think for a purpose 2. Within a point of view 3. Based on assumptions 4. Leading to implications and consequences 5. We use data, facts, and experiences 6. To make inferences and judgements 7. Based on concepts and theories 8. To answer a question or solve a problem Adapted from Analytic Thinking The Foundation for Critical Thinking
1. What is my fundamental purpose? 2. What is my point of view with respect to the issue? 3. What assumptions am I using in my reasoning? 4. What are the implications of my reasoning? 5. What information do I need to answer my question? 6. What are my most fundamental inferences or conclusions 7. What is the most basic concept in the question? 8. What is the key question I am trying to answer? Adapted from Analytic Thinking The Foundation for Critical Thinking
Clarity Accuracy Precision Relevance Depth Breadth Logic Significance Fairness Adapted from Analytic Thinking The Foundation for Critical Thinking
Statistics, Definitions, & Theories The Audit Process Getting it Wrong Practice & Application
Risk Assessment Process Documentation Interviewing Testing Reporting
The 5 C s Condition Criteria Cause Consequence Corrective Action
The 5 C s Condition Gathering relevant information Reporting relevant information The focus of interviews and tests
The 5 C s Condition Criteria How is success defined How is success measured SMART?
The 5 C s Condition Criteria Cause Mind the Gap PA 2320-2 Root Cause Analysis The Five Whys
The 5 C s Condition Criteria Cause Consequence Part of the initial risk assessment Risks to objectives
The 5 C s Condition Criteria Cause Consequence Corrective Action Does it fill the gap? Negotiations Did the client really agree?
Performance Standard 2310 Identifying Information Sufficient Reliable Relevant Useful
Business Knowledge The Big Short Business Objectives Business, department, process Objectives to Risks to Controls
Put down the camera Know what is needed Know what is expected Designed/Operating as designed/operating effectively and efficiently Steps which add value Control and risk Risk vs. Cost
What am I trying to find out? How do my questions support that purpose? Flexible listening Active listening When the answer comes Wrap up Meetings are the same thing
Making a conscious effort to hear not only the words being said, but the complete message being delivered Attending Following Reflecting
Did the answer match the question that was asked? Does the answer address the issue? How do I follow-up? What is the next question? Do I have everything I need?
(Yes, you just heard this) What am I trying to find out? How do my tests support that purpose? Trying to find too many dots
What is the report for? Who is the report for? What do they care about? What are you trying to say? (Tweet the report)
Statistics, Definitions, & Theories The Audit Process Getting it Wrong Practice & Application
The Mistakes We Make Psychological/Cognitive Biases The Mistakes They Make Logical Fallacies
Psychological/Cognitive Bias Logical Fallacies
Looking for information that confirms your belief Ensure controls are effective vs Identify opportunities to improve controls vs Find where controls are ineffective Avoiding: Challenge what you think you see Six Hats Thinking Talk to others Find a devil s advocate
First impression bias Basing final judgement on information gained early Prove the agent committed fraud Past controls Avoiding: No quick decisions Reflect on the decision-making history Ladder of Influence
Placing too much faith in your own knowledge and opinions Acting on hunches I assumed Related to Availability Heuristic Avoiding: Review the data What sources have you used? Was information gathered systematically? Was all the data used?
Expecting past events to influence the future The Coin Toss and the Monty Hall problem 200% of Nothing - Dewdney Testing the past to predict the future Related: Clustering Illusion Avoiding: Look at trends from a number of angles Avoid looking at information sequentially Root cause analysis
Blaming others and external events Stereotyping or perceived flaws The marketing department Avoiding: Look at events non-judgmentally Root cause Understand why people behave the way they do Use input from others
Ostrich Effect Ignoring dangerous or negative information Information Bias Seeking information when it does not affect the action Outcome Bias Judging based on outcome rather than how the decision was made Conservatism Bias Believing prior evidence more than newly emerging evidence
Blind-Spot Bias Failing to recognize your cognitive bias is a bias in itself
Psychological/Cognitive Bias Logical Fallacies
1. It s true if I believe it 2. It s true if we believe it 3. It s true if I want to believe it 4. It s true if it serves my vested interest to believe it The Thinker s Guide to Fallacies Paul & Elder
Don t argue about belief, argue about arguments The key to making a rational argument is that your assertions must be falsifiable --Seth Godin
Ad Hominem: Attack the Person You ve always been out to get us. You don t know what you re talking about. Argument from Authority Trust me. We know how to handle this. Begging the Question: Assume the Point Do you want controls or profits?
Slippery Slope If we do not enforce this control, all controls will begin to fail. Straw Man: Caricaturing a Position We ll make sure that is approved, and get that approval approved, and get that approval of the approval approved. Post Hoc/Ergo Propter Hoc The last time we implemented your suggestions, our production dropped.
Pointing to Another Wrong You say I don t have controls, what controls do you have? Appeal to Authority The CEO has said this is approved. Appeal to Experience You don t understand the business the way I do.
Appeal to Pity You don t understand how hard it is to get this done. Hard-Cruel-World Argument We have to do this to succeed. Appeal to Tradition It s always been done this way.
Posture of Righteousness Of course we made a mistake, but our intentions are good. Red Herring Have you seen the problems in the other department? Flattery I ve always respected audit and the positive benefits it provides to the organization.
Demonize Those bean counters are always bayonetting the wounded. We do everything possible to support this company. Deny or Defend Inconsistencies I never said that. Shifting the Burden of Proof How do we know that test is correct?
Question Conclusions Make Much of Inconsistencies Attack the Evidence Your testing is incorrect. Look at this one mistake. And just because you saw this one item doesn t mean the entire thing is wrong.
Ignore the Main Point Raises Nothing but Objections Evade Questions Hedge What you Say Ignore the Evidence
Statistics, Definitions, & Theories The Audit Process Getting it Wrong Practice & Application
Use Wasted Time A Problem a Day Internalize Intellectual Standards Keep an Intellectual Journal Reshape Your Character Deal with Your Ego Redefine the Way You See Things Get In Touch With Your Emotions Analyze Group Influences on Your Life
QUESTIONS?