MSc Software Testing MSc Prófun hugbúnaðar
|
|
- Joleen Bridges
- 5 years ago
- Views:
Transcription
1 MSc Software Testing MSc Prófun hugbúnaðar Fyrirlestrar 43 & 44 Evaluating Test Driven Development 15/11/2007 Dr Andy Brooks 1
2 Case Study Dæmisaga Reference Evaluating Advantages of Test Driven Development: a Controlled Experiment with Professionals, Gerardo Canfora et. Al., Proceedings of the 2006 ACM/IEEE international symposium on empirical software engineering (ISESE 06), pp , ACM 15/11/2007 Dr Andy Brooks 2
3 1. INTRODUCTION Test Driven Development (TDD) First the developer defines the classes and their interfaces. Then the developer writes a test suite for each class which includes assertions required to verify method behaviour. Then the developer writes method bodies and executes tests. If a test fails, the developer changes the code to remove the bug. The process ends when all the tests pass. 15/11/2007 Dr Andy Brooks 3
4 A quick and dirty guide to JUnit public class Math { static public int add(int a, int b) { return a + b; } } import junit.framework.*; public class TestMath extends TestCase { public void testadd() { int num1 = 3; int num2 = 2; int total = 5; int sum = 0; sum = Math.add(num1, num2); assertequals(sum, total); } } from 15/11/2007 Dr Andy Brooks 4
5 1. INTRODUCTION TDD advantages Test documentation is within the code. developers do not need to search for it The tests provide an unambiguous quality indicator for the code. a test either passes or fails We believe that: (i) TDD is more time consuming than TAC; but (ii) TDD improves the quality of unit testing. TAC testing after coding 15/11/2007 Dr Andy Brooks 5
6 1. INTRODUCTION Two research questions Is TDD more or less productive than TAC? Does TDD improve the quality of unit testing? accuracy and precision high accuracy but low precision high precision, but low accuracy 15/11/2007 Dr Andy Brooks 6
7 2. RELATED WORK A structured experiment of test-driven development. Boby George Laurie Williams. Information and Software Technology, Volume 46, Issue 5, 15 April 2004, Pages Elsevier B.V. Abstract Test Driven Development (TDD) is a software development practice in which unit test cases are incrementally written prior to code implementation. We ran a set of structured experiments with 24 professional pair programmers. One group developed a small Java program using TDD while the other (control group), used a waterfall-like approach. Experimental results, subject to external validity concerns, tend to indicate that TDD programmers produce higher quality code because they passed 18% more functional black-box test cases. However, the TDD programmers took 16% more time. Statistical analysis of the results showed that a moderate statistical correlation existed between time spent and the resulting quality. Lastly, the programmers in the control group often did not write the required automated test cases after completing their code. Hence it could be perceived that waterfall-like approaches do not encourage adequate testing. This intuitive observation supports the perception that TDD has the potential for increasing the level of unit testing in the software industry 15/11/2007 Dr Andy Brooks 7
8 2. RELATED WORK Test-driven development as a defect-reduction practice. Williams, L., Maximilien, E.M., and Vouk, M. 14th International Symposium on Software Reliability Engineering (ISSRE 03), pp 34-45, IEEE Abstract Test-driven development is a software development practice that has been used sporadically for decades. With this practice, test cases (preferably automated) are incrementally written before production code is implemented. Test-driven development has recently re-emerged as a critical enabling practice of the extreme programming software development methodology. We ran a case study of this practice at IBM. In the process, a thorough suite of automated test cases was produced after UML design. In this case study, we found that the code developed using a test-driven development practice showed, during functional verification and regression tests, approximately 40% fewer defects than a baseline prior product developed in a more traditional fashion. The productivity of the team was not impacted by the additional focus on producing automated test cases. This test suite aids in future enhancements and maintenance of this code. The case study and the results are discussed in detail. 15/11/2007 Dr Andy Brooks 8
9 2. RELATED WORK Experiment about test-first programming. Muller, M.M. and Hagner, O. Software, IEE Proceedings, 2002, Vol149(5), pp IEEE Abstract Test-first programming is one of the central techniques of extreme programming. Programming test-first means (i) write down a test-case before coding and (ii) make all the tests executable for regression testing. Thus far, knowledge about test-first programming is limited to experience reports. Nothing is known about the benefits of test-first compared to traditional programming (design, implementation, test). This paper reports an experiment comparing. test-first to traditional programming. It turns out that test-first does not accelerate the implementation, and the resulting programs are not more reliable, but test-first seems to support better program understanding. 15/11/2007 Dr Andy Brooks 9
10 2. RELATED WORK A prototype empirical evaluation of test driven development. Geras, A., Smith, M. and Miller, J. 10th International Symposium on Software Metrics (METRICS 04), pp IEEE Abstract Test driven development (TDD) is a relatively new software development process. On the strength of anecdotal evidence and a number of empirical evaluations, TDD is starting to gain momentum as the primary means of developing software in organizations worldwide. In traditional development, tests are for verification and validation purposes and are built after the target product feature exists. In test-driven development, tests are used for specification purposes in addition to verification and validation. An experiment was devised to investigate the distinction between test-driven development and traditional, test-last development from the perspective of developer productivity and software quality. The results of the experiment indicate that while there is little or no difference in developer productivity in the two processes, there are differences in the frequency of unplanned test failures. This may lead to less debugging and more time spent on forward progress within a development project. As with many new software development technologies however, this requires further study, in particular to determine if the positive results translate into lower total costs of ownership. 15/11/2007 Dr Andy Brooks 10
11 2. RELATED WORK Towards empirical evaluation of test-driven development in a university environment. Pancur, M., Ciglaric, M.,Trampus, M. and Vidmar, T. EUROCON Computer as a Tool. Vol 2, pp IEEE Abstract Test driven development (TDD) is an agile software development technique and it is one of the core development practices of Extreme programming (XP). In TDD, developers write automatically executable tests prior to writing the code they test. We ran a set of experiments to empirically assess different parameters of the TDD. We compared TDD to a more "traditionally" oriented iterative test-last development process (ITL). Our preliminary results show that TDD is not substantially different from ITL and our qualitative findings about a development process are different from results obtained from other researches. 15/11/2007 Dr Andy Brooks 11
12 3. THE EXPERIMENT The two experimental hypotheses H 01 : there is no difference in the productivity between TDD and TAC. H 02 : there is no difference in quality of unit tests between TDD and TAC. quality 15/11/2007 Dr Andy Brooks 12
13 3. THE EXPERIMENT Subjects 28 company employees Soluziona Software Factory all had at least one year with the company All with a BSc Computer Science. All with 5 years of Java experience. All with experience of several software engineering projects. All with a wide knowledge of programming and databases. But no previous experience of TDD. 3 hours of TDD training was performed before the experiment. 15/11/2007 Dr Andy Brooks 13
14 3. THE EXPERIMENT Experimental platform Java Eclipse IDE JUnit Andy notes: no version numbers? 15/11/2007 Dr Andy Brooks 14
15 3. THE EXPERIMENT Experimental task The subjects were required to write a program to act as a TextAnalyzer for a supplied piece of text. The first requirement was to calculate the frequency of the words in the text and the position of their first occurrences. The second requirement was to calculate the maximum and minimum distance between two words indicated by the user. see the article s appendix for detailed descriptions 15/11/2007 Dr Andy Brooks 15
16 3. THE EXPERIMENT Experimental forms, examples Subjects completed two forms, one for each experimental run. The End Time was recorded by the subjects when the tests succeeded. 15/11/2007 Dr Andy Brooks 16
17 3. THE EXPERIMENT Variables MeanTPA mean time per assertion MeanTime mean time taken by subjects for testing TotalTime total time taken by a subject MeanAPM mean assertions per method AssertTot total number of assertions in a project H 02 So quality is being assessed by simply counting assertions... 15/11/2007 Dr Andy Brooks 17
18 3. THE EXPERIMENT Table 1. The Experimental Design - within subjects and counter balanced - Two runs, each lasting five hours. Each subject implemented both requirements. Each subject used TDD then TAC or TAC then TDD. The training session on TDD included a seminar and lab exercises. 15/11/2007 Dr Andy Brooks 18
19 4. ANALYSIS OF DATA Figure Descriptive Statistics - requirements 1 and 2 considered together - Employees = 28 Experimental runs = 2 Time Per Assertion Assertions Per Method TDD takes more time. TDD seems to result in only a few more assertions. 1,75 more assertions per project Andy says: MeanTPA seems ill-defined here. 15/11/2007 Dr Andy Brooks 19
20 4. ANALYSIS OF DATA Figure Descriptive Statistics - requirements 1 and 2 considered together - TDD is said by the authors to foster greater testing precision by increasing the number of assertions that are written. 1,75 more assertions per project TDD is said by the authors to foster greater testing accuracy because more time was taken identifying completely equivalence classes. 15/11/2007 Dr Andy Brooks 20
21 4. ANALYSIS OF DATA Figure Descriptive Statistics - requirements 1 and 2 considered together - TDD is said to be more predictable but the standard deviations for TDD variables are larger than for TAC variables! (See also Table 7.) Andy asks: have outlying data points not been removed? 15/11/2007 Dr Andy Brooks 21
22 The TDD box for AssertTot is much bigger than TAC. The TDD box for TimeTot is a little smaller than TAC but TDD has 4 outlying values as opposed to the one for TAC. 15/11/2007 Dr Andy Brooks 22
23 See Table 3. 4,2 Hypotheses testing Mann-Whitney tests were used because the data was not normal. The null hypothesis H 01 was rejected. TDD takes more time than TAC. TDD on average was 50 minutes longer per project. The null hypothesis H 02 was not rejected. The differences in the number of assertions could have arisen by chance. 1,75 more assertions per project was not statistically significant 15/11/2007 Dr Andy Brooks 23
24 4.3 Lessons learned to improve the experimental design. The authors suggest it would be useful to examine code quality as they believed code quality was improved using TDD. The authors suggest also using larger time windows as applying TDD properly can be time consuming. 15/11/2007 Dr Andy Brooks 24
25 5. Internal validity issues A within-subjects design helps reduce differences caused by subject variability. Subject variability was also controlled by using subjects who all had a similar professional background and who all received training in TDD and JUnit at the same seminar. Requirements 1 and 2 were designed to be as independent as possible to reduce learning effects between the two experimental runs. Mann-Whitney tests found no evidence of learning effects between the two experimental runs. See Table 4. 15/11/2007 Dr Andy Brooks 25
26 5. Internal validity issues Fatigue effects were controlled by holding training and the two experimental runs on three separate but consecutive days. Fatigue effects were not detected. some subjects asked for a longer time Subjects were motivated to take part since learning about TDD and JUnit could benefit them in their daily work. Both experimental runs were supervised to prevent subjects working together or otherwise sharing solutions. Mann-Whitney tests found no statistically significant differences between the data for requirement 1 and the data for requirement 2. See Table 5. 15/11/2007 Dr Andy Brooks 26
27 5. External validity issues The subjects were all professionals. The use of Java, Eclipse, and JUnit is representative of industrial working environments in software development. The two requirements, however, are not comparable to real industrial projects. 15/11/2007 Dr Andy Brooks 27
28 6. Conclusions TDD requires more time. No statistical significant evidence was found to suggest that TDD improves the accuracy and precision of unit testing. We are convinced that TDD increases such quality aspects and that evidence might be obtained in a longer experiment... 15/11/2007 Dr Andy Brooks 28
29 6. Conclusions TDD is more predictable than TAC. Andy says: their data does not support this! The authors are planning to: replicate the experiment enlarge the period of observation time to 6 or even 12 months analyze code quality with regard to software maintenance 15/11/2007 Dr Andy Brooks 29
30 Critical commentary by Andy Why did the authors not debrief subjects to find out possible reasons for the outlying data points? A subject using TAC wrote the most assertions. A TDD subject took almost 3x the average time for one project. The claim that TDD is more predictable is simply wrong. A plot of individuals TotalTime against AssertTot might have exposed a time-accuracy trade-off. Why did the authors not compare the quality of assertion writing between TDD and TAC? 15/11/2007 Dr Andy Brooks 30
Test-Driven Development
On the Influence of Test-Driven Development on Software Design by SzeChernTan School of Informatics University of Edinburgh 12 February 2009 Agenda Introduction Overview of paper Experimental design Results
More informationMost Common Mistakes in Test-Driven Development Practice: Results from an Online Survey with Developers
Most Common Mistakes in Test-Driven Development Practice: Results from an Online Survey with Developers Mauricio Finavaro Aniche, Marco Aurélio Gerosa Department of Computer Science - University of São
More informationCausal Factors, Benefits and Challenges of Test-Driven Development: Practitioner Perceptions
Full citation: Buchan, J., Li, L., & MacDonell, S.G. (2011) Causal Factors, Benefits and Challenges of Test-Driven Development: Practitioner Perceptions, in Proceedings of the 18th Asia-Pacific Software
More informationFoundations of software engineering
Foundations of software engineering Test Driven Development Dr. Julie Greensmith G51 Overview Test Driven Development in Context A recap of XP and Agile development Test First Design What it is and what
More informationOutline. TDD = Too Dumb Developers? Implications of Test-Driven Development on maintainability and comprehension of software
TDD = Too Dumb Developers? Implications of Test-Driven Development on maintainability and comprehension of software Marco Torchiano: marco.torchiano@polito.it Alberto Sillitti: alberto.sillitti@unibz.it
More informationTest-Driven Development
Test-Driven Development Course of Software Engineering II A.A. 2009/2010 Valerio Maggio, Ph.D. Student Prof. Sergio Di Martino Contents at Glance What is TDD? TDD and XP TDD Mantra TDD Principles and Patterns
More informationTest Driven Development. Course of Software Engineering II A.A. 2011/2012 Valerio Maggio, PhD Student Prof. Marco Faella
Test Driven Development Course of Software Engineering II A.A. 2011/2012 Valerio Maggio, PhD Student Prof. Marco Faella Development process Let's think about the development process of this example: 3
More informationTest-Driven Development Exposed Growing Complex Software One Test at a Time
Test-Driven Development Exposed Growing Complex Software One Test at a Time Introduction Software engineering has seen its share of controversies, fads, and techniques recently One technique that has stirred
More informationQuality Metrics of Test Suites in Test-Driven Designed Applications
Quality Metrics of Test Suites in Test-Driven Designed Applications Alkaoud, Hessah Hassan Department of Computer Science University of Colorado at Colorado Springs, USA halkaoud@gmail.com Kristen Walcott-Justice
More informationControlled experiments in Software Engineering: an introduction
Controlled experiments in Software Engineering: an introduction Filippo Ricca Unità CINI at DISI, Genova, Italy Filippo.ricca@disi.unige.it 1 Controlled experiments......are like chocolate once you try
More informationA Survey on Code Coverage as a Stopping Criterion for Unit Testing
A Survey on Code Coverage as a Stopping Criterion for Unit Testing Ben Smith and Laurie Williams North Carolina State University [bhsmith3, lawilli3]@ncsu.edu Abstract The evidence regarding code coverage
More informationName of the paper: Effective Development and Testing using TDD. Name of Project Teams: Conversion Team and Patient Access Team.
1 Name of the paper: Effective Development and Testing using TDD Name of the author: Payal Sen E-Mail Id: payal.sen@siemens.com Contact Number: 9748492800 Name of Project Teams: Conversion Team and Patient
More informationOn the Effectiveness of Unit Tests in Test-driven Development
On the Effectiveness of Unit Tests in Test-driven Development ABSTRACT Ayse Tosun Istanbul Technical University Istanbul, Turkey tosunay@itu.edu.tr Burak Turhan Brunel University London Middlesex, UK burak.turhan@brunel.ac.uk
More informationSOFTWARE development organizations face the difficult
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 38, NO. 3, MAY/JUNE 2012 Comparing the Defect Reduction Benefits of Code Inspection and Test-Driven Development Jerod W. Wilkerson, Jay F. Nunamaker Jr.,
More informationState coverage: an empirical analysis based on a user study
State coverage: an empirical analysis based on a user study Dries Vanoverberghe 1, Emma Eyckmans 1, and Frank Piessens 1 Katholieke Universiteit Leuven, Leuven, Belgium {dries.vanoverberghe,frank.piessens}@cs.kuleuven.be
More informationTDD HQ : Achieving Higher Quality Testing in Test Driven Development
TDD HQ : Achieving Higher Quality Testing in Test Driven Development Adnan Čaušević, Sasikumar Punnekkat and Daniel Sundmark School of Innovation, Design and Engineering Mälardalen University, Västerås,
More informationRegression Cycle, Debug & Tracking
Regression Cycle, Debug & Tracking Balasubramanian Gopalakrishnan, Daniel Rigby, Mark Daniel - restricted - The Regression Cycle DUT TB Regression Failures/ Cover Holes Daily regression cycle Fixed version
More informationTest Driven Development (TDD)
Test Driven Development (TDD) Outline TDD Overview Test First vs. Test Last Summary Quotes Kent Beck said Test-first code tends to be more cohesive and less coupled than code in which testing isn t a part
More informationSelecting a research method
Selecting a research method Tomi Männistö 13.10.2005 Overview Theme Maturity of research (on a particular topic) and its reflection on appropriate method Validity level of research evidence Part I Story
More informationIndependent Variables Variables (factors) that are manipulated to measure their effect Typically select specific levels of each variable to test
Controlled Experiments experimental investigation of a testable hypothesis, in which conditions are set up to isolate the variables of interest ("independent variables") and test how they affect certain
More informationFactors Limiting Industrial Adoption of Test Driven Development: A Systematic Review
Factors Limiting Industrial Adoption of Test Driven Development: A Systematic Review Adnan Causevic, Daniel Sundmark, Sasikumar Punnekkat Mälardalen University, School of Innovation, Design and Engineering,
More informationSUPPLEMENTARY INFORMATION
Supplementary Statistics and Results This file contains supplementary statistical information and a discussion of the interpretation of the belief effect on the basis of additional data. We also present
More information2017 Edument AB. All Rights Reserved. Stepping Back: The Big Picture
2017 Edument AB. All Rights Reserved. Stepping Back: The Big Picture Traditional views of testing In the (oft-ridiculed) waterfall model, we distinguish the following phases: Developers hand off a finished
More informationSAP Hybris Academy. Public. February March 2017
SAP Hybris Academy Public February March 2017 Agenda Introduction SAP Hybris Academy Overview Java Knowledge Needed for SAP Hybris Development HY200 SAP Hybris Commerce Functional Analyst: Course Content
More informationEMPIRICAL RESEARCH METHODS IN VISUALIZATION
EMPIRICAL RESEARCH METHODS IN VISUALIZATION and some thoughts on their role in Master, PHD and postdoctoral projects Talk at University of Sydney, 11. August 2014 Stephan Diehl University of Trier / Universität
More informationFault Detection and Localisation in Reduced Test Suites
UNIVERSITY OF SZEGED Fault Detection and Localisation in Reduced Test Suites Árpád Beszédes University of Szeged, Hungary The 29 th CREST Open Workshop, London November 2013 Overview University of Szeged,
More informationUnderstanding the Dynamics of Test-driven Development
Understanding the Dynamics of Test-driven Development Davide Fucci, supervised by Burak Turhan and Markku Oivo Department of Information Processing Science Faculty of Information Technology and Electrical
More informationAre Students Representatives of Professionals in Software Engineering Experiments?
Are Students Representatives of Professionals in Software Engineering Experiments? Iflaah Salman, Ayse Tosun Misirli e Natalia Juristo 37th IEEE International Conference on Software Engineering - ICSE
More informationA Critique of How We Measure and Interpret the Accuracy of Software Development Effort Estimation
A Critique of How We Measure and Interpret the Accuracy of Software Development Effort Estimation Magne Jørgensen Simula Research Laboratory magnej@simula.no Abstract This paper criticizes current practice
More informationA Qualitative Survey of Regression Testing Practices
A Qualitative Survey of Regression Testing Practices Emelie Engström and Per Runeson Department of Computer Science, Lund University, SE-221 00 LUND, Sweden {Emelie.Engstrom,Per.Runeson}@cs.lth.se Abstract.
More informationEvaluation: Scientific Studies. Title Text
Evaluation: Scientific Studies Title Text 1 Evaluation Beyond Usability Tests 2 Usability Evaluation (last week) Expert tests / walkthroughs Usability Tests with users Main goal: formative identify usability
More informationStatistical analysis DIANA SAPLACAN 2017 * SLIDES ADAPTED BASED ON LECTURE NOTES BY ALMA LEORA CULEN
Statistical analysis DIANA SAPLACAN 2017 * SLIDES ADAPTED BASED ON LECTURE NOTES BY ALMA LEORA CULEN Vs. 2 Background 3 There are different types of research methods to study behaviour: Descriptive: observations,
More informationFigure 1. The Test-Driven Development process, emphasizing writing unit tests prior to solution code
Student Adherence to Test-Driven Development and Evaluating Methods in Computer Science Kevin Buffardi Virginia Tech 114 McBryde Hall (0106) Blacksburg, Virginia +1 540-231-5723 1. PROBLEM AND MOTIVATION
More informationAn Empirical Study of Process Conformance
An Empirical Study of Process Conformance Sivert Sørumgård 1 Norwegian University of Science and Technology 3 December 1996 Abstract An experiment was carried out at the Norwegian University of Science
More informationAcceptanceTest Driven. Developmen t Julian Goddard 14/04/2015. (c) 2014 Plaxion Limited. All rights reserved.
AcceptanceTest Driven Developmen t Julian Goddard 14/04/2015 1 Interactive seminar Ask questions as they arise Discuss points as they arise Presentation will be available on the BCS website Some slides
More informationBiostatistics for Med Students. Lecture 1
Biostatistics for Med Students Lecture 1 John J. Chen, Ph.D. Professor & Director of Biostatistics Core UH JABSOM JABSOM MD7 February 14, 2018 Lecture note: http://biostat.jabsom.hawaii.edu/education/training.html
More informationDefect Removal. RIT Software Engineering
Defect Removal Agenda Setting defect removal targets Cost effectiveness of defect removal Matching to customer & business needs and preferences Performing defect removal Techniques/approaches/practices
More informationApplying the Experimental Paradigm to Software Engineering
Applying the Experimental Paradigm to Software Engineering Natalia Juristo Universidad Politécnica de Madrid Spain 8 th European Computer Science Summit Current situation 16.3% of software projects are
More informationSteps to writing a lab report on: factors affecting enzyme activity
Steps to writing a lab report on: factors affecting enzyme activity This guide is designed to help you write a simple, straightforward lab report. Each section of the report has a number of steps. By completing
More informationIntroduction Journal of Applied Information Science and Technology, 7:1, (2014)
Problem Solving and Decision Making: Consideration of Individual Differences in Computer Programming Skills Using Myers Briggs Type Indicator (MBTI) and Chidamber and Kemerer Java Metrics (CKJM) Ezekiel
More informationLecture 3. Previous lecture. Learning outcomes of lecture 3. Today. Trustworthiness in Fixed Design Research. Class question - Causality
Lecture 3 Empirical Research Methods IN4304 design (fixed design) Previous lecture A research plan Consider: purpose, theory, research question, mode inquiry, sampling method Research question Descriptive/
More informationCSC2130: Empirical Research Methods for Software Engineering
CSC2130: Empirical Research Methods for Software Engineering Steve Easterbrook sme@cs.toronto.edu www.cs.toronto.edu/~sme/csc2130/ 2004-5 Steve Easterbrook. This presentation is available free for non-commercial
More informationEmpirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 32, NO. 10, OCTOBER 2006 771 Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults Yuming Zhou and Hareton Leung,
More informationCRITICAL EVALUATION OF BIOMEDICAL LITERATURE
Chapter 9 CRITICAL EVALUATION OF BIOMEDICAL LITERATURE M.G.Rajanandh, Department of Pharmacy Practice, SRM College of Pharmacy, SRM University. INTRODUCTION Reviewing the Biomedical Literature poses a
More informationEXPERIMENTAL RESEARCH DESIGNS
ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 14A LECTURE NOTES [02/28/14] EXPERIMENTAL RESEARCH DESIGNS PAGE 1 Topic #5 EXPERIMENTAL RESEARCH DESIGNS As a strict technical definition, an experiment is a study
More informationExperimental evaluation of an object-oriented function point measurement procedure
Information and Software Technology 49 (2007) 366 380 www.elsevier.com/locate/infsof Experimental evaluation of an object-oriented function point measurement procedure Silvia Abrahão a, *, Geert Poels
More informationEHR Usability Test Report
EHR Usability Test Report Product: Procentive Version: 2015 Date of Usability Test: November 3, 6, 7, 9 and 10 2017 Date of Report: November 2017 Report Prepared By: Procentive Michael Rundle, Customization
More information04/12/2014. Research Methods in Psychology. Chapter 6: Independent Groups Designs. What is your ideas? Testing
Research Methods in Psychology Chapter 6: Independent Groups Designs 1 Why Psychologists Conduct Experiments? What is your ideas? 2 Why Psychologists Conduct Experiments? Testing Hypotheses derived from
More informationConfirmation Bias in Software Development and Testing: An Analysis of the Effects of Company Size, Experience and Reasoning Skills
Confirmation Bias in Software Development and Testing: An Analysis of the Effects of Company Size, Experience and Reasoning Skills Gul Calikli Department of Computer Engineering, Software Research Laboratory,
More informationReliability of Ordination Analyses
Reliability of Ordination Analyses Objectives: Discuss Reliability Define Consistency and Accuracy Discuss Validation Methods Opening Thoughts Inference Space: What is it? Inference space can be defined
More informationA NEW, ADVANCED HIGH- THROUGHPUT SYSTEM FOR AUTOMATED INHALER TESTING
A NEW, ADVANCED HIGH- THROUGHPUT SYSTEM FOR AUTOMATED INHALER TESTING Two years ago, Novi Systems Ltd set out to shake up the inhaler automation market. On December 8th, 2015, at the Drug Delivery to the
More informationEvaluation: Controlled Experiments. Title Text
Evaluation: Controlled Experiments Title Text 1 Outline Evaluation beyond usability tests Controlled Experiments Other Evaluation Methods 2 Evaluation Beyond Usability Tests 3 Usability Evaluation (last
More informationPsychology Research Process
Psychology Research Process Logical Processes Induction Observation/Association/Using Correlation Trying to assess, through observation of a large group/sample, what is associated with what? Examples:
More informationChapter 23. Inference About Means. Copyright 2010 Pearson Education, Inc.
Chapter 23 Inference About Means Copyright 2010 Pearson Education, Inc. Getting Started Now that we know how to create confidence intervals and test hypotheses about proportions, it d be nice to be able
More informationThe University of Michigan Radiation Oncology Clinic Analysis of Waste in the Radiation Oncology Clinic Patient Flow Process.
The University of Michigan Radiation Oncology Clinic Analysis of Waste in the Radiation Oncology Clinic Patient Flow Process Final Report Prepared for: Kathy Lash, Director of Operations University of
More informationDEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING. CP 7026-Software Quality Assurance Unit-I. Part-A
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CP 7026-Software Quality Assurance Unit-I 1. What is quality? 2. What are the Views of Quality? 3. What is Quality Cost? 4. What is Quality Audit? 5. What
More informationDesigning Experiments... Or how many times and ways can I screw that up?!?
www.geo.uzh.ch/microsite/icacogvis/ Designing Experiments... Or how many times and ways can I screw that up?!? Amy L. Griffin AutoCarto 2012, Columbus, OH Outline When do I need to run an experiment and
More informationModeling the Use of Space for Pointing in American Sign Language Animation
Modeling the Use of Space for Pointing in American Sign Language Animation Jigar Gohel, Sedeeq Al-khazraji, Matt Huenerfauth Rochester Institute of Technology, Golisano College of Computing and Information
More informationThe Effect of Code Coverage on Fault Detection under Different Testing Profiles
The Effect of Code Coverage on Fault Detection under Different Testing Profiles ABSTRACT Xia Cai Dept. of Computer Science and Engineering The Chinese University of Hong Kong xcai@cse.cuhk.edu.hk Software
More informationTHE EFFECT OF EXPECTATIONS ON VISUAL INSPECTION PERFORMANCE
THE EFFECT OF EXPECTATIONS ON VISUAL INSPECTION PERFORMANCE John Kane Derek Moore Saeed Ghanbartehrani Oregon State University INTRODUCTION The process of visual inspection is used widely in many industries
More informationModeling Terrorist Beliefs and Motivations
CREATE Research Archive Research Project Summaries 2010 Modeling Terrorist Beliefs and Motivations Richard John University of Southern California, richardj@usc.edu Heather Rosoff University of Southern
More informationLesson 9: Two Factor ANOVAS
Published on Agron 513 (https://courses.agron.iastate.edu/agron513) Home > Lesson 9 Lesson 9: Two Factor ANOVAS Developed by: Ron Mowers, Marin Harbur, and Ken Moore Completion Time: 1 week Introduction
More informationHypothesis generation and experimental design. Nick Hunt Bosch Institute School of Medical Sciences
Hypothesis generation and experimental design Nick Hunt Bosch Institute School of Medical Sciences Nicholas.hunt@sydney.edu.au Today s talk Hypothesis generation Hypothesis testing Elements of good experimental
More informationPredicting Task Difficulty for Different Task Types
Predicting Task Difficulty for Different Task Types Jingjing Liu, Jacek Gwizdka, Chang Liu, Nicholas J. Belkin School of Communication and Information, Rutgers University 4 Huntington Street, New Brunswick,
More informationModels of Information Retrieval
Models of Information Retrieval Introduction By information behaviour is meant those activities a person may engage in when identifying their own needs for information, searching for such information in
More informationDefect Removal Metrics. SE 350 Software Process & Product Quality 1
Defect Removal Metrics 1 Objectives Understand some basic defect metrics and the concepts behind them Defect density metrics Defect detection and removal effectiveness etc. Look at the uses and limitations
More informationHuman-Computer Interaction IS4300. I6 Swing Layout Managers due now
Human-Computer Interaction IS4300 1 I6 Swing Layout Managers due now You have two choices for requirements: 1) try to duplicate the functionality of an existing applet; or, 2) create your own (ideally
More informationEvaluating Quality in Creative Systems. Graeme Ritchie University of Aberdeen
Evaluating Quality in Creative Systems Graeme Ritchie University of Aberdeen Graeme Ritchie {2007} Some Empirical Criteria for Attributing Creativity to a Computer Program. Minds and Machines 17 {1}, pp.67-99.
More informationOutline. Experimental Evaluation in Computer Science: A Quantitative Study. Related Work. Introduction. Select CS Papers.
Experimental Evaluation in Computer Science: A Quantitative Study Paul Lukowicz, Ernst A. Heinz, Lutz Prechelt and Walter F. Tichy Journal of Systems and Software January 1995 Introduction Large part of
More informationAre Test Cases Needed? Replicated Comparison between Exploratory and Test-Case-Based Software Testing
Pre-print article accepted for publication in Empirical Software Engineering Journal, June 2013. Copyright Springer 2013 DOI: 10.1007/s10664-013-9266-8 The final publication is available at link.springer.com
More informationSimultaneous Real-Time Detection of Motor Imagery and Error-Related Potentials for Improved BCI Accuracy
Simultaneous Real-Time Detection of Motor Imagery and Error-Related Potentials for Improved BCI Accuracy P. W. Ferrez 1,2 and J. del R. Millán 1,2 1 IDIAP Research Institute, Martigny, Switzerland 2 Ecole
More informationInferential Statistics
Inferential Statistics and t - tests ScWk 242 Session 9 Slides Inferential Statistics Ø Inferential statistics are used to test hypotheses about the relationship between the independent and the dependent
More informationCross-over trials. Martin Bland. Cross-over trials. Cross-over trials. Professor of Health Statistics University of York
Cross-over trials Martin Bland Professor of Health Statistics University of York http://martinbland.co.uk Cross-over trials Use the participant as their own control. Each participant gets more than one
More informationSimple Linear Regression the model, estimation and testing
Simple Linear Regression the model, estimation and testing Lecture No. 05 Example 1 A production manager has compared the dexterity test scores of five assembly-line employees with their hourly productivity.
More informationImplementation Guide for the DoseControl Dosimetry System
1.0 SCOPE The GEX DoseControl System is intended for use as a single dosimetry system solution, specifically developed to satisfy the generic dosimetry needs of radiation sterilization application users
More informationCHAPTER LEARNING OUTCOMES
EXPERIIMENTAL METHODOLOGY CHAPTER LEARNING OUTCOMES When you have completed reading this article you will be able to: Define what is an experiment Explain the role of theory in educational research Justify
More informationSystematic Mapping Studies
Systematic Mapping Studies Marcel Heinz 23. Juli 2014 Marcel Heinz Systematic Mapping Studies 23. Juli 2014 1 / 44 Presentation Overview 1 Motivation 2 Systematic Mapping Studies 3 Comparison to Systematic
More informationChapter 02 Developing and Evaluating Theories of Behavior
Chapter 02 Developing and Evaluating Theories of Behavior Multiple Choice Questions 1. A theory is a(n): A. plausible or scientifically acceptable, well-substantiated explanation of some aspect of the
More informationMammogram Analysis: Tumor Classification
Mammogram Analysis: Tumor Classification Term Project Report Geethapriya Raghavan geeragh@mail.utexas.edu EE 381K - Multidimensional Digital Signal Processing Spring 2005 Abstract Breast cancer is the
More informationDolly or Shaun? A Survey to Verify Code Clones Detected using Similar Sequences of Method Calls
Dolly or Shaun? A Survey to Verify Code Clones Detected using Similar Sequences of Method Calls Alexandre Paiva, Johnatan Oliveira, Eduardo Figueiredo Department of Computer Science Federal University
More informationChapter 5: Field experimental designs in agriculture
Chapter 5: Field experimental designs in agriculture Jose Crossa Biometrics and Statistics Unit Crop Research Informatics Lab (CRIL) CIMMYT. Int. Apdo. Postal 6-641, 06600 Mexico, DF, Mexico Introduction
More informationRunning head: How large denominators are leading to large errors 1
Running head: How large denominators are leading to large errors 1 How large denominators are leading to large errors Nathan Thomas Kent State University How large denominators are leading to large errors
More informationTowards natural human computer interaction in BCI
Towards natural human computer interaction in BCI Ian Daly 1 (Student) and Slawomir J Nasuto 1 and Kevin Warwick 1 Abstract. BCI systems require correct classification of signals interpreted from the brain
More informationHSMR Dashboard. User Guide Document Control: Document Control Version 1.2 Date Issued 12 th August 2016
HSMR Dashboard User Guide 2016 Document Control: Document Control Version 1.2 Date Issued 12 th August 2016 Author(s) Robyn Munro Other Related Documents Comments to NSS.isdQualityIndicators@nhs.net Introduction...
More informationExperimentally Investigating the Effects of Defects in UML Models
Experimentally Investigating the Effects of Defects in UML Models Christian Lange C.F.J.Lange@TUE.nl www.win.tue.nl/~clange www.win.tue.nl/empanada SAN meeting February 11 th, 25 Overview Introduction
More informationPsychology Research Process
Psychology Research Process Logical Processes Induction Observation/Association/Using Correlation Trying to assess, through observation of a large group/sample, what is associated with what? Examples:
More informationHow do we identify a good healthcare provider? - Patient Characteristics - Clinical Expertise - Current best research evidence
BSC206: INTRODUCTION TO RESEARCH METHODOLOGY AND EVIDENCE BASED PRACTICE LECTURE 1: INTRODUCTION TO EVIDENCE- BASED MEDICINE List 5 critical thinking skills. - Reasoning - Evaluating - Problem solving
More informationJournal of Operations Management
Journal of Operations Management 30 (2012) 24 43 Contents lists available at ScienceDirect Journal of Operations Management jo ur n al homep a ge: www.elsevier.com/locate/jom Impact factor as a metric
More informationCohen and the First Computer Virus. From: Wolfgang Apolinarski
Seminar Malware Daniel Loebenberger B-IT Bonn-Aachen International Center WS 2007/08 Cohen and the First Computer Virus From: What do we want to discuss today? Short biography of Fred Cohen Virus The theoretical
More informationWhen Passion Obscures the Facts: The Case for Evidence-Based Testing
When Passion Obscures the Facts: The Case for Evidence-Based Testing Stuart Reid, 2010 Stuart Reid Testing Solutions Group 117 Houndsditch London EC3 UK Tel: 0207 469 1500 Fax: 0207 623 8459 www.testing-solutions.com
More informationAugmented Cognition to enhance human sensory awareness, cognitive functioning and psychic functioning: a research proposal in two phases
Augmented Cognition to enhance human sensory awareness, cognitive functioning and psychic functioning: a research proposal in two phases James Lake MD (egret4@sbcglobal.net) Overview and objectives Augmented
More informationCommunication Research Practice Questions
Communication Research Practice Questions For each of the following questions, select the best answer from the given alternative choices. Additional instructions are given as necessary. Read each question
More informationRELIABILITY OF THE DENISON ORGANISATIONAL CULTURE SURVEY (DOCS) FOR USE IN A FINANCIAL INSTITUTION IN SOUTH AFRICA CHRISSTOFFEL JACOBUS FRANCK
RELIABILITY OF THE DENISON ORGANISATIONAL CULTURE SURVEY (DOCS) FOR USE IN A FINANCIAL INSTITUTION IN SOUTH AFRICA by CHRISSTOFFEL JACOBUS FRANCK submitted in part fulfilment of the requirements for the
More informationDesign the Flexibility, Maintain the Stability of Conceptual Schemas
Design the Flexibility, Maintain the Stability of Conceptual Schemas Lex Wedemeijer 1 ABP Netherlands, Department of Information Management, P.O.Box 4476, NL-6401 CZ, Heerlen, The Netherlands L.Wedemeijer@ABP.NL
More informationStatistical Methods and Reasoning for the Clinical Sciences
Statistical Methods and Reasoning for the Clinical Sciences Evidence-Based Practice Eiki B. Satake, PhD Contents Preface Introduction to Evidence-Based Statistics: Philosophical Foundation and Preliminaries
More informationWhat is the Difference Between the New BAM-1020 PM2.5 FEM and Older BAM-1020 Monitors?
Met One Technical Bulletin Revised: April 21, 2009 By: Dennis Hart Met One Instruments, Inc. 1600 N.W. Washington Blvd. Grants Pass, OR 97526 (541) 471-7111 www.metone.com What is the Difference Between
More informationMaterialism and the Mind and Body Problem:
Materialism and the Mind and Body Problem: A Critical Analysis SeanAgius The philosophy of mind has become dominated by the mind and body problem: the relation between the physical (the body) and the mental
More informationMultimodal Driver Displays: Potential and Limitations. Ioannis Politis
Multimodal Driver Displays: Potential and Limitations Ioannis Politis About me (http://yannispolitis.info/hci/) Background: B.Sc. Informatics & Telecommunications University of Athens M.Sc. Advanced Information
More informationPaper presentation: Preliminary Guidelines for Empirical Research in Software Engineering (Kitchenham et al. 2002)
Paper presentation: Preliminary Guidelines for Empirical Research in Software Engineering (Kitchenham et al. 2002) Who? Eduardo Moreira Fernandes From? Federal University of Minas Gerais Department of
More information