A Dependability Case Language for a Radiation Therapy System

Transcription

1 A Dependability Case Language for a Radiation Therapy System Michael Ernst, Dan Grossman, Jon Jacky, Calvin Loncaric, Stuart Pernsteiner, Zachary Tatlock, Emina Torlak, Xi Wang University of Washington D C L

2 vision end-to-end verification for safety critical systems

3 IronClad Memory Model Quark sel4 SUPPORTED BY

4 IronClad Quark sel4 Formal

5

6 Formal

7 Formal Dependability Cases End-to-end

8 Dependability cases Integrate diverse sources of evidence check interfaces of design, testing, proof, review Argue end-to-end claim based on evidence show claim holds across all layers of a system Focus on physical system properties eases validation and focuses verification effort

9 Dependability case engineering Many large software systems display fragility or a lack Many of dependability large software caused systems by inattention display fragility or a to details lack at various stages of development (e.g., Many of dependability large software caused systems by inattention display fragility or a missing to details data, undocumented assumptions, lack at various stages of development (e.g., lack of testing), Many of dependability resulting large software caused in a failure systems by inattention to catch display fragility or a missing to details data, undocumented assumptions, errors. This lack at various stages of development (e.g., lack of testing), technical Many of dependability resulting large note software explains caused a failure how systems by to inattention to catch display fragility or a missing create a dependability to details data, undocumented assumptions, errors. This lack at case various for a stages system of that development helps (e.g., lack of testing), technical Many of dependability resulting large note software explains caused a failure how systems by to inattention to catch display fragility or a identify and missing create a keep dependability to track details data, undocumented such details. A assumptions, errors. This lack at case various for a stages system of that development helps (e.g., dependability lack identify and case of testing), technical Many of dependability keep defined resulting large note software explains caused track here such as details. a structured a failure how systems by to inattention to catch display fragility or a missing create a dependability to details data, undocumented A assumptions, argument dependability providing errors. This lack at case various for a stages system of that development helps (e.g., lack case of evidence testing), technical Many of dependability defined that resulting large note here a system software explains caused as a structured a failure how systems by to inattention to catch display fragility or a identify and missing meets argument its specified create a keep providing dependability to track details data, undocumented such details. A assumptions, errors. This lack at evidence requirements. case various for a stages system of that development helps (e.g., dependability lack case of testing), technical Many of dependability defined that resulting large note here a system software explains caused as a structured a failure how systems by to inattention to catch display fragility or a The meets technical identify its specified note describes and missing create a keep dependability to track how details data, to undocumented requirements. structure such details. A assumptions, argument providing errors. This lack at case various for a stages system of that development helps (e.g., evidence that a system the The argument dependability lack technical and note present case of testing), technical Many of dependability describes evidence defined resulting large note software explains caused how to to support here as structure it. a structured a failure how systems by to inattention to catch display fragility or a identify and missing meets its specified create a keep dependability to track details data, undocumented such details. A assumptions, requirements. A sample the argument problem argument and is presented, providing errors. This lack at case various for a stages system of that development helps (e.g., dependability lack evidence case of evidence testing), technical Many of dependability well defined as that resulting large note a system software explains caused The technical note describes how to to support here as structure it. a structured a failure how systems by to inattention to catch display fragility or a identify and missing issues A sample raised meets by problem that its problem specified create a keep is presented, and dependability to track details data, undocumented such details. A assumptions, argument providing errors. future This lack evidence technical of at well goals. requirements. dependability case various for a stages the argument and evidence as that note a system explains system caused of that development how by to helps inattention (e.g., dependability lack The technical identify note describes and case missing of testing), defined resulting how to to support here as structure it. a structured a failure to catch issues A sample raised meets by problem that its problem specified create a keep is presented, and dependability to track details data, of undocumented at such future well goals. requirements. case various details. for a stages A assumptions, system of that development helps (e.g., argument providing errors. as Many large the software The argument dependability lack This evidence technical that note a system explains how to technical systems and identify display note describes and case fragility evidence missing of testing), keep defined track data, resulting how or to a to of support undocumented here as structure it. a structured a failure such details. A assumptions, to catch issues raised meets by that its problem specified create and future goals. requirements. lack Many of dependability A sample large the software argument caused problem argument systems by and is inattention presented, providing errors. a dependability case for a system that helps dependability lack This well as The technical display note case of evidence testing), technical describes fragility evidence is defined that resulting note a system explains how or to a to support here as structure it. a structured a failure how to to catch identify to details lack of at issues dependability various A sample raised meets stages by caused problem of that its development problem specified create and keep track of such details. A argument and by is inattention presented, providing errors. a dependability (e.g., future This evidence technical well goals. requirements. case as that note for a system explains a system how that to helps Many large the software argument dependability The technical systems and identify display case fragility evidence is defined or to a support here as note describes how to structure it. a structured missing to details data, undocumented at issues various raised meets stages by assumptions, lack of dependability A sample caused problem of that its development problem specified create and a keep and dependability track of such by is inattention presented, (e.g., future well goals. requirements. case details. for a system A that helps argument as lack missing of testing), Many data, resulting large the software argument dependability providing evidence that a system undocumented in The a failure technical systems and identify assumptions, to catch display note describes and case fragility evidence keep is defined track how or to a to of support here structure such as details. it. a structured A to details at issues various raised meets stages by of that development problem and (e.g., future goals. errors. lack This lack of testing), technical Many of dependability A sample resulting large note the software explains argument caused problem argument its specified dependability requirements. dependability a failure how systems by is to to and inattention presented, providing well as missing data, undocumented assumptions, catch display case evidence fragility evidence is defined that or to a support here a system as it. a structured The to details at issues various raised meets technical note describes how to structure stages by of that development problem and (e.g., future goals. errors. lack This lack of testing), technical Many of dependability A sample resulting large note software explains caused problem argument its specified a failure how systems by is to to inattention presented, providing dependability evidence requirements. as well as that a system the The argument and missing data, undocumented assumptions, catch display fragility evidence or to a support it. to details at issues various raised meets technical stages by of that its development problem specified note describes and dependability how to (e.g., future goals. requirements. structure errors. lack This lack of testing), technical Many of dependability A sample resulting large note the software explains caused problem a failure how systems by is to to inattention presented, as well as The argument technical and missing data, undocumented assumptions, catch display note describes fragility evidence how or to a to support structure it. to details at issues various raised stages by of that development problem and (e.g., future goals. errors. lack This lack of testing), technical Many of dependability A sample resulting large note the software explains argument caused problem a failure how systems by to to and is inattention presented, missing data, undocumented assumptions, catch display fragility evidence as well as or to a support it. to details at issues various stages of development (e.g., errors. lack This lack of testing), technical Many of dependability A sample raised by resulting large note software explains caused problem that problem a failure how systems by is and future goals. to to inattention presented, as well as missing data, undocumented assumptions, catch display fragility or a to details at issues various raised stages by of that development problem and (e.g., future goals. errors. lack This lack of testing), technical Many of dependability resulting large note software explains caused a failure how systems by to to inattention missing data, undocumented assumptions, catch display fragility or a to details at various stages of development (e.g., errors. lack This lack of testing), technical Many of dependability resulting large note software explains caused a failure how systems by to to inattention missing data, undocumented assumptions, catch display fragility or a to details at various stages of development (e.g., errors. lack This lack of testing), technical of dependability resulting note explains caused a failure how by to to inattention missing data, undocumented assumptions, catch to details at various stages of development (e.g., errors. lack This of testing), technical resulting note explains a failure how to to missing data, undocumented assumptions, catch errors. lack This of testing), technical resulting note explains a failure how to tocatch errors. This technical note explains how to Difficult to develop Difficult to check Difficult to maintain SUPPORTED BY?

10 Formal Checkable Dep. Cases Dependability Cases End-to-end

11 Developing a Dependability Case Language 1. Target specific system Move from specific to general avoid attempt to design silver bullet

12 Developing a Dependability Case Language Claims 1. Target specific system 2. Develop dep. claims

13 Developing a Dependability Case Language Claims Design Application Platform Env 1. Target specific system 2. Develop dep. claims 3. Gather evidence Alloy Rosette Coq Manual Review

14 Developing a Dependability Case Language DCL Claims 1. Target specific system 2. Develop dep. claims Design Application Platform Env 3. Gather evidence 4. Design + build DCL Find general tradeoffs and patterns make simple easy and hard possible Impact real-world projects bring current PL tech to the trenches D C L

15 results an end-to-end dependability case for CNTS

16 Checking safety of CNTS Clinical Neutron Therapy System (CNTS) at UW 30 years of incident-free service. Controlled Prescription by custom software, Sensors built by CNTS engineering staff. Third generation of Therapy Control software now being built. Therapy Control Software Beam, motors, etc. 16

17 Checking safety of CNTS EPICS documentation / semantics The Maximize Severity attribute is one of NMS (Non-Maximize Severity), MS (Maximize Severity), An end-to-end MSS (Maximize property Status that and spans Severity) the or MSI (Maximize entire system, Severity not if just Invalid). software. It determines whether alarm severity is propagated across links. If the attribute is MSI only a severity of INVALID_ALARM is propagated; settings of MS or MSS Experimental propagate all alarms Physics that and are more severe Industrial than the record's Control current System severity. For input CNTS links (EPICS) the Couch alarm Dataflow Safety severity Language Property: of the record referred to by the link is propagated to the record containing the link. For output links the alarm The severity beam of will the record turn off containing if the the link is propagated couch rotation to the record angle referred moves to out by the link. of If the tolerances severity is during changed treatment the associated alarm and status the is operator set LINK_ALARM, has not issued except if the the attribute manual is MSS override when the command. alarm status will be copied along with the severity. Prescription Sensors Therapy Control Software Beam, motors, etc. 17

18 An informal dependability case for couch safety couch rotates out of tolerances and no manual override => beam shuts off couch rotates OOT => TMC measures OOT rotation TMC measures OOT rotation => TC receives OOT rotation TC receives OOT rotation and no manual override => TC sets Therapy Sum interlock TC sets Therapy Sum interlock => PLC disables Therapy Sum relay PLC disables Therapy Sum relay => beam shuts off Couch Treatment Motion Controller Therapy Control Software Programmable Logic Controller Hardwired Safety Interlock System Ethernet Network 18

19 A formal dependability case for couch safety couch rotates out of tolerances and no manual override => beam shuts off all r: Couch.rotation (properties and r.angle not in Prescription.tolerance and no Event.GantryCouch_Turntable_Override) => some Beam.state & BeamOff PLC disables Therapy Sum relay => beam shuts off evidence[ 63c8d380", PLC_Analysis,, Proof] => all relaystate: plc.relay2754 & RelayOpen one coilstate: plc.sentmsgs & relaystate.^next coilstate.coilnumber = Coil1623 coilstate.coilvalue = False 19

20 Generating evidence for couch safety Couch Treatment Motion Controller Therapy Control Software Programmable Logic Controller Hardwired Safety Interlock System Ethernet Network Expert Review Validator EPICS Linter PLC Checker A solver-aided verifier for the subset of EPICS used in CNTS. EPICS Verifier EPICS- PLC Signal Tracer 20

21 Checking couch safety Dependability case Dependability Case Complier (DCC) Expert Review Validator EPICS Linter EPICS Verifier PLC Analyzer EPICS- PLC Signal Tracer Alloy Analyzer Counterexample or bounded proof 21

22 Deep analysis with <2000 LOC of tool code Therapy Control Software TC receives OOT rotation and no manual override => TC sets Therapy Sum interlock EPICS Verifier Found a bug in the Therapy Control software (preventing beam shut off), masked by a bug in the EPICS runtime! concrete counterexample 22

23 Thanks! Recent Verification Successes C Formal D L Dependability Cases End-to-end