Medical Device Software Development Management: Following FDA Guidelines for Software Validation

Transcription

1 Medical Device Sftware Develpment Management: Fllwing FDA Guidelines fr Sftware Validatin

2 On June 7, 1997, the FDA issued the General Principles f Sftware Validatin, which utlines validatin principles that the FDA cnsiders applicable t the validatin f medical device sftware r the validatin f sftware used t design, develp, r manufacture medical devices. Devices categrized as class II and III, as well as sme class I devices are subject t design cntrls; f these class the fllwing types f sftware must be validated fr FDA apprval: Sftware used as a cmpnent, part, r accessry f a medical device; Sftware that is itself a medical device (e.g., bld establishment sftware); Sftware used in the prductin f a device (e.g., prgrammable lgic cntrllers in manufacturing equipment); and Sftware used in implementatin f the device manufacturer's quality system (e.g., sftware that recrds and maintains the device histry recrd). As an effective means t gain apprval, the FDA recmmends that medical device sftware develpment teams take a sftware develpment lifecycle (SDLC) apprach that integrates risk management strategies with principles fr sftware validatin. An integrated SDLC merges validatin and verificatin activities, including defect preventin practices such as unit testing, peer cde reviews, static analysis, manual testing, and regressin testing, thrughut the SDLC. The result f such an apprach is an emphasis n planning, verificatin, testing, traceability, and cnfiguratin management. Develping sftware fr medical devices that cmplies with the FDA's Quality System regulatin is a challenging endeavr that's as much a business issue as it is an engineering feat. In this paper, we identify sftware develpment challenges that medical device makers face when attempting t integrate the principles utlined by the FDA. Furthermre, we describe hw Parasft's autmated defect preventin slutins help rganizatins vercme the challenges f an integrated SDLC apprach. Lastly, we prvide a pint-t-pint index f FDA principles and the Parasft capabilities that supprt them. Burdens f the Least Burdensme Apprach The FDA guidance des nt prescribe specific practices, tls, cding methds r any ther technical activity. The FDA, instead, prescribes the seemingly inncuus cncept f the Least Burdensme Apprach. In this apprach, rganizatins determine, and strictly adhere t their self-defined validatin and verificatin prcesses. Develpment activities and utcmes must be clearly defined, dcumented, verified, and validated against the rganizatin's prcess. The gal f this apprach is t give medical device makers enugh rpe t determine hw t best ensure public safety. But in practice, the effect has been that rganizatins have enugh rpe t hang themselves. This is because the requirements, expressed in FDA 21 CFR, represent extensive planning and testing, which require validatin. The fllwing examples are just a fractin f the ttal challenges sftware engineers must vercme: The sftware validatin prcess cannt be cmpleted withut an established sftware requirements specificatin, which specifies the intended use. Results must nt nly verify that the specificatins are met, but they must be reprduced cnsistently (validatin). Testing methds, such as regressin testing, can be implemented t meet the requirement. Validatin must be established and re-established fr even small changes. This means that validatin activities, including static analysis, unit testing, cde review, etc., must be repeated if the cde has changed. Furthermre, as sftware cntinues t becme mre and mre cmplex, tests that validate the changes shuld be cnducted in scale with the applicatin t ensure that n ther part f the system is affected. 1

3 Changes t the requirements deemed significantly different enugh frm the riginally registered design may require the prduct t be re-registered per FDA Sectin 501(K). There are n "FDA certified" tls r methds. N persn, rganizatin r tl can claim any frm f sme suppsed FDA certificatin. Hwever, any sftware used t autmate any part f the device prcess r any part f the quality system must als be validated. Yu must be able t run any tls used t assist in the verificatin and validatin effrts n a cntrl cde base and cnfirm that the results are cnsistent, which may affect yur time-t-market. The FDA has established grunds fr apprval in a way that effectively amunts t punting the respnsibility f ensuring quality and public safety back t the device makers. The true bstacles hampering sftware develpment, thugh, are the breakpints between what the sftware engineers believe t be the gals f their develpment effrts and the business expectatins, which are rarely cmmunicated in a way that serves all parts f the rganizatin. Lack f Sftware Develpment Plicy The current sftware develpment prcess in mst rganizatins is mdeled n a culture that fails t bridge the gap between business gals and the develpment prcess. Sftware engineers either dn't knw what's expected r d nt understand the business bjective behind the guidelines driving their prducts. They are expected t write cde that meets the requirements, but they are nt necessarily required t understand why requirements have been established in the first place. We believe that vercming the business gals and sftware develpment gap, as well as driving the develpment prcess n a platfrm based n plicy-driven develpment is the best way t satisfy the FDA's requirements fr medical device sftware develpment. Plicy-driven develpment invlves 1) clearly defining expectatins and dcumenting them in understandable plices, 2) training the engineers n the business bjectives driving thse plicies, and 3) mnitring plicy adherence in an autmated, unbtrusive way. Integrating these principles int the develpment prcess gives businesses the ability t accurately and bjectively measure prductivity and applicatin quality. The result is lwer cst ver the ttal sftware develpment lifecycle frm build t supprt and reduced risk. Adpting a plicy-driven develpment prcess is key fr achieving the fllwing gals: Ensuring that engineers dn t make tradeffs that ptentially cmprmise reliability and perfrmance. Ensuring that engineers build security int the applicatin, safeguarding it frm ptential attacks. Preventing defects that culd result in cstly recalls, litigatin, r a damaged market psitin. Accurately and cnsistently applying quality prcesses. Gaining the traceability and auditability required t ensure cntinued plicy cmpliance. Sftware engineers make business decisins with every line f cde, every test cnducted (r nt cnducted), and every guideline r standard fllwed (r nt fllwed). With public safety, ptential litigatin, market psitin and ther cnsequences n the line, it behves sftware develpment teams and peple in the traditinal business management psitins t cme tgether n plicy and implement the strategy int their sftware develpment lifecycle. Visit fr mre infrmatin abut plicy-driven develpment. Parasft Supprt fr FDA Principles f Sftware Validatin Parasft supprts the FDA s visin f an integrated SDLC fr C, C++, Java, and.net with Parasft Cncert fr Medical Device Sftware Develpment, a sftware develpment management platfrm that is pre-cnfigured with prcesses and best practices described in FDA guidelines and medical device industry standards. 2

4 Parasft's sftware develpment management platfrm enables rganizatins t integrate prject and task management with Autmated Defect Preventin and end-t-end sftware verificatin and validatin. Leveraging plicy-driven develpment, it creates an envirnment that drives prductivity and sftware quality. Parasft slutins fr medical device sftware develpment features: Cnfigurable templates fr FDA, IEC 62304, IEC, SIL and mre Prcess, prject, and task management Cmprehensive requirements traceability Integrated defect preventin, validatin and verificatin A cntinuus plicy-driven cmpliance prcess with real-time visibility Crrelatin f all key artifacts, frm tests, t requirements, t cde, t builds, t prject tasks Parasft has ver 25 years f experience helping the majrity f the Frtune 500 cmpanies incrprate these practices thrughut the SDLC and knws what it takes t rapidly establish an integrated quality prcess fr medical device develpment, as well as ensure that the prcess is repeatable and sustainable. Parasft is the industry leader in defect preventin in fact, we wrte the bk n it (Autmated Defect Preventin, Wiley-IEEE, 2007). Backgrund: The General Principles f Sftware Validatin Sectins 1, 2, and 3 set the purpse, scpe, and cntext fr sftware validatin fr medical device sftware. Since these sectins fcus n identifying terms rather than utlining expectatins, we will use Sectin 4 (Principles f Sftware Validatin) and Sectin 5 (Activities and Tasks) t highlight hw Parasft delivers end-t-end slutins fr the medical device sftware industry. Sftware testing is ne f many verificatin activities intended t cnfirm that sftware develpment utput meets its input requirements. Hwever, quality sftware cannt be delivered by testing alne. Quality sftware is delivered cnsistently via a slid, repeatable prcess, which requires an integrated system that assists with defining requirements, ensuring gd cding practices, and testing effectively. This prcess needs t be visible, measurable, and mst imprtantly repeatable. Parasft brings all these elements tgether. It supprts: SDLC Integratin and Prcess Definitin Quality Plicy Management Requirements Management Iteratin / Release Planning Task Management Static Cde Analysis Pattern-Based Flw-Based Metrics-Based Autmated Cde Review Unit Testing Framewrk Cde Cverage Analysis Runtime Errr Detectin Memry Errr Detectin Message/Prtcl Testing Penetratin Testing Service Virtualizatin Functinal Testing Business Prcess Testing Lad Testing Prcess Visibility & Cntrl Traceability 3

5 4.1 Requirements A dcumented sftware requirements specificatin prvides a baseline fr bth validatin and verificatin. The sftware validatin prcess cannt be cmpleted withut an established sftware requirements specificatin. Parasft Supprt A system fr mapping requirements t develpment tasks and mnitring the implementatin and validatin f each requirement. An pen API and ut-f-the-bx cnfiguratins fr the mst ppular resurce management and bug management systems and tls like Excel, Wrd and MS Prject. Requirements testing--highlights which requirements need t be tested. Requirements traceability crrelates requirements t iteratins, tasks, cde, tests, builds, and artifacts. Graphical reprting f requirement status as indicated by develpers. 4.2 Defect Preventin Sftware quality assurance needs t fcus n preventing the intrductin f defects int the sftware develpment prcess rather than trying t "test quality int" the sftware cde after it is written. Sftware testing is limited in its ability t surface all latent defects in cde. Sftware testing by itself is nt sufficient t establish cnfidence that the sftware is fit fr its intended use. The industry's mst cmprehensive autmated defect preventin system. A prven autmated defect preventin system that can be implemented int any sftware develpment envirnment Technlgies that autmate defect preventin practices t ensure their cnsistent and cmprehensive applicatin. An autmated infrastructure that drives the defect preventin prcess t ensure that it remains n track and des nt disrupt the team s wrkflw. A system that mnitrs adherence t defect preventin plicies. Capabilities include: Quality Plicy Management Static Cde Analysis Pattern-Based Flw-Based Metrics-Based Autmated Peer Cde Review Cntextual Peer Cde Review Unit Testing Framewrk Cde Cverage Analysis 4

6 4.3 Time and Effrt Preparatin f sftware validatin shuld begin early; i.e., during design and develpment planning and design input. Parasft Supprt Precnfigured FDA templates. A central system that dcuments and defines requirements, expected tasks, timelines and utcmes as well as manages by exceptin t ensure that the prject is meeting expectatins. A cntinuus, end-t-end quality prcess that ensures defect preventin and detectin tasks are nt nly deplyed acrss every stage f the SDLC, but als ingrained int the team s wrkflw. A system that answers in real-time: Will I be n time? Will I be n budget? Will I have the expected functinality? Will it wrk? 4.4 Sftware Life Cycle Sftware validatin takes place within the envirnment f an established sftware life cycle. The sftware life cycle cntains sftware engineering tasks and dcumentatin necessary t supprt the sftware validatin effrt. In additin, the sftware life cycle cntains specific verificatin and validatin tasks that are apprpriate fr the intended use f the sftware. Sftware develpment management platfrm integrates SDLC int the brader develpment infrastructure; flexible prcess/wrkflw definitin tl that allws fr a visible and repeatable SDLC. Prcess-based implementatin drives manual and autmated validatin tasks acrss the SDLC, ensuring cnsistency and traceability. Services that integrate and autmate the SDLC t ensure that quality sftware can be prduced cnsistently and efficiently. Services that imprve develpment prductivity and frm the fundatin fr a repeatable, sustainable quality prcess Plans The sftware validatin prcess is defined and cntrlled thrugh the use f a plan. The sftware validatin plan defines "what" is t be accmplished thrugh the sftware validatin effrt. Sftware validatin plans are a significant quality system tl. Sftware validatin plans specify areas such as scpe, apprach, resurces, schedules and the types and extent f activities, tasks, and wrk items. Plans are expressed as custmizable templates that define cmmn sftware develpment and validatin plans. A system fr mapping quality plan requirements t develpment tasks and mnitring the implementatin and validatin f each requirement. Services that ensure the validatin plan is clearly defined and enfrceable. Centralized definitin and management f rganizatinlevel and team-level plicies fr implementing the validatin plan. 5

7 4.6 Prcedures The sftware validatin prcess is executed thrugh the use f prcedures. These prcedures establish "hw" t cnduct the sftware validatin effrt. The prcedures shuld identify the specific actins r sequence f actins that must be taken t cmplete individual validatin activities, tasks, and wrk items. 4.7 Sftware Validatin after a Change Due t the cmplexity f sftware, a seemingly small lcal change may have a significant glbal system impact. Whenever sftware is changed, a validatin analysis shuld be cnducted nt just fr validatin f the individual change, but als t determine the extent and impact f that change n the entire sftware system. Parasft Supprt Plicy defines prcedures and the Parasft sftware develpment management system autmatically rchestrates the all tasks in the apprpriate sequence with cmplete traceability. In this way, checklist items are cnverted int an executable prcess. Autmated applicatin f quality plicies acrss the SDLC. Mnitrable quality gates and threshlds thrughut the SDLC. Wrkflw ptimizatin t ensure that tasks t supprt quality plicies can becme a sustainable part f the team's existing wrkflw. Precnfigured FDA templates. Cntinuus regressin testing, which applies a brad range f validatin methds t immediately alert the team when mdificatins impact applicatin behavir. Change-based testing, which helps teams identify and execute nly the test cases directly related t the mst recent surce cde mdificatins. Requirements traceability crrelates requirements t iteratins, tasks, cde, tests, builds, and artifacts. 4.8 Validatin Cverage Validatin cverage shuld be based n the sftware's cmplexity and safety risk - nt n firm size r resurce cnstraints. The selectin f validatin activities, tasks, and wrk items shuld be cmmensurate with the cmplexity f the sftware design and the risk assciated with the use f the sftware fr the specified intended use. Validatin dcumentatin shuld be sufficient t demnstrate that all sftware validatin plans and prcedures have been cmpleted successfully. Autmated assessment f high-risk cde using industrystandard metrics. Identificatin f specific pieces f cde that exceed industry-standard r custmized cmplexity metrics threshlds. Cverage analyzer, including statement, branch, path, and MC/DC cverage, helps users gauge test suite efficacy and cmpleteness. Archived reprts and trend graphs dcument validatin effrts and quality imprvements. 4.9 Independence f Review Self-validatin is extremely difficult. When pssible, an independent evaluatin is always better, especially fr higher risk applicatins. Objective, autmated validatin based n the rganizatin s predefined quality gals and/r the industry s mst cmprehensive library f prven sftware develpment best practices. Executable prcesses ensure that required review tasks are perfrmed at the apprpriate time and recrd sign-ffs. 6

8 4.10 Flexibility and Respnsibility Sftware is designed, develped, validated, and regulated in a wide spectrum f envirnments, and fr a wide variety f devices with varying levels f risk. Sftware validatin activities and tasks may be dispersed, ccurring at different lcatins and being cnducted by different rganizatins. Hwever, regardless f the distributin f tasks, cntractual relatins, surce f cmpnents, r the develpment envirnment, the device manufacturer r specificatin develper retains ultimate respnsibility fr ensuring that the sftware is validated. Parasft Supprt A plicy-driven, flexible, repeatable, and traceable validatin prcess that can span distributed envirnments and include bth autmated and manual tasks. The ability t define a test suite that starts verifying sftware n the hst develpment envirnment then reuse that same test suite t validate sftware functinality in ther envirnments n simulatrs, target devices, and ther platfrms. The visibility and cnsistency needed t reduce the risks f utsurcing and gegraphically-distributed develpment. An autmated framewrk that manages sftware verificatin methds t ensure that all sftware develpment activities meet expectatins. Supprt fr defect reslutin, nt just defect preventin and detectin. Each issue detected is priritized, autmatically crrelated t the develper wh intrduced it, then distributed t his r her IDE with direct links t the prblematic cde. Eventually, develpers start writing cmpliant cde as a matter f habit. Mrever, thrugh integratin with the develpment infrastructure, results are crrelated with requirements, bugs, and surce cde changes cnverting data int actinable infrmatin. 5.1 Sftware Life Cycle Activities Activities in a typical sftware life cycle mdel include the fllwing: Quality Planning System Requirements Definitin Detailed Sftware Requirements Specificatin Sftware Design Specificatin Cnstructin r Cding Testing Installatin Operatin and Supprt Maintenance Retirement Verificatin, testing, and ther tasks that supprt sftware validatin ccur during each f these activities. A life cycle mdel rganizes these sftware develpment activities in varius ways and prvides a framewrk fr mnitring and cntrlling the sftware develpment prject. A plicy-based apprach that defines the rganizatin s expectatins fr quality acrss each f these SDLC phases, ingrains practices fr measuring plicy cmpliance int the team s wrkflw acrss the SDLC, and autmatically mnitrs plicy cmpliance fr visibility and traceability. A centralized and enfrceable plicy that nt nly establishes the rganizatin s expectatins, but als keeps the team n track twards achieving thse expectatins prviding a framewrk fr prducing predictable utcmes. The ability t define a truly cmprehensive plicy that nt nly enfrces cding requirements thrugh static analysis, but als addresses dynamic testing requirements regarding unit, integratin, and system-level testing, cverage analysis, and regressin testing. Precnfigured FDA templates. 7

9 5.2.1 Quality Planning Design and develpment planning shuld culminate in a plan that identifies necessary tasks, prcedures fr anmaly reprting and reslutin, necessary resurces, and management review requirements, including frmal design reviews. A sftware life cycle mdel and assciated activities shuld be identified, as well as thse tasks necessary fr each sftware life cycle activity. Parasft Supprt Plans are expressed as an interperable business prcess. Precnfigured, custmizable templates define cmmn sftware quality plans. A system fr mapping quality plan requirements t develpment tasks and mnitring the implementatin and validatin f each requirement. Services that ensure the validatin plan is clearly defined and enfrceable. Centralized definitin and management f rganizatinlevel and team-level plicies fr implementing the quality plan Requirements The sftware requirements specificatin dcument shuld cntain a written definitin f the sftware functins. A sftware requirements traceability analysis shuld be cnducted t trace sftware requirements t (and frm) system requirements and t risk analysis results. In additin t any ther analyses and dcumentatin used t verify sftware requirements, a frmal design review is recmmended t cnfirm that requirements are fully specified and apprpriate befre extensive sftware design effrts begin Design In the design prcess, the sftware requirements specificatin is translated int a lgical and physical representatin f the sftware t be implemented. The sftware design specificatin is a descriptin f what the sftware shuld d and hw it shuld d it. At the end f the sftware design activity, a Frmal Design Review shuld be cnducted t verify that the design is crrect, cnsistent, cmplete, accurate, and testable, befre mving t implement the design. A system fr mapping quality plan requirements t develpment tasks and mnitring the implementatin and validatin f each requirement. Traceability thrugh requirements-based testing, which links test cases, the requirements defined in the specificatin, and the related surce cde prviding realtime visibility int which requirements are actually wrking as expected, and which still require testing. Wrkflw autmatin fr design dcument reviews. Autmated rchestratin f apprval/sign-ff tasks in the apprpriate sequence, and with cmplete traceability. Plicies specify design best practices that prevent cmmn design pitfalls; ensure that the design is crrect, cnsistent, cmplete, accurate, and testable; and help teams satisfy critical design attributes such as usability, perfrmance, efficiency, scalability, r mdularity. Wrkflw autmatin fr design dcument reviews. Autmated rchestratin f apprval/sign-ff tasks in the apprpriate sequence, and with cmplete traceability. 8

10 Cnstructin r Cding Surce cde shuld be evaluated t verify its cmpliance with specified cding guidelines. Such guidelines shuld include cding cnventins regarding clarity, style, cmplexity management, and cmmenting. Surce cde evaluatins are ften implemented as cde inspectins and cde walkthrughs. Such static analyses prvide a very effective means t detect errrs befre executin f the cde. A surce cde traceability analysis is an imprtant tl t verify that all cde is linked t established specificatins and established test prcedures. A surce cde traceability analysis shuld be cnducted and dcumented t verify that: Each element f the sftware design specificatin has been implemented in cde; Mdules and functins implemented in cde can be traced back t an element in the sftware design specificatin and t the risk analysis; Tests fr mdules and functins can be traced back t an element in the sftware design specificatin and t the risk analysis; and Tests fr mdules and functins can be traced t surce cde fr the same mdules and functins. Parasft Supprt Pattern-based static analysis ensures that the cde meets unifrm expectatins arund reliability, perfrmance, security, and maintainability. Includes precnfigured templates fr FDA. Data flw static analysis detects cmplex runtime errrs withut requiring test cases r applicatin executin. Metrics analysis nt nly calculates metrics but als identifies specific pieces f cde that exceed industrystandard r custmized metrics threshlds. Peer cde inspectin prcess autmatin autmates and manages the peer cde review wrkflw including preparatin, ntificatin, and tracking and reduces verhead by enabling cde review n the desktp. Traceability thrugh requirements-based testing, which links test cases, the requirements defined in the specificatin, and the related surce cde prviding realtime visibility int which requirements are actually wrking as expected, and which still require testing Testing by the Sftware Develper Test plans and test cases shuld be created as early in the sftware develpment prcess as feasible. Once the prerequisite tasks (e.g., cde inspectin) have been successfully cmpleted, sftware testing begins. It starts with unit level testing and cncludes with system level testing. Cde-based testing is als knwn as structural testing r "white-bx" testing. It identifies test cases based n knwledge btained frm the surce cde, detailed design specificatin, and ther develpment dcuments. Structural testing can identify dead cde that is never executed when the prgram is run. The level f structural testing can be evaluated using metrics that are designed t shw what percentage f the sftware structure has been evaluated during structural testing. These metrics are typically referred t as "cverage" and are a measure f cmpleteness with respect t test selectin criteria. A framewrk that allws develpers t start testing each unit as sn as it is cmpleted. After examining the surce cde t determine hw t test it, a wide variety f white-bx test cases are autmatically generated t check cde rbustness, expsing ptential reliability prblems. A framewrk that supprts the rapid additin f userdefined tests that verify sftware crrectness and functinality. Autmated identificatin and refactring f unused cde, duplicate cde, and dead cde. Cverage analyzer, including statement, branch, path, and MC/DC cverage, helps users gauge test suite efficacy and cmpleteness. Parasft fllws the industry standard in defining cverage as cde cverage btained by actually executing cde with test cases nt simulated cverage. Autmated integratin-level and system-level testing. Runtime errr detectin efficiently identifies defects nly manifested at runtime. Memry errr detectin identifies difficult-t-track prgramming and memry-access errrs, as well as ptential defects and memry usage inefficiencies. 9

11 User Site Testing User site testing shuld fllw a pre-defined written plan with a frmal summary f testing and a recrd f frmal acceptance. Dcumented evidence f all testing prcedures, test input data, and test results shuld be retained Maintenance and Sftware Changes When changes are made t a sftware system, either during initial develpment r during pst release maintenance, sufficient regressin analysis and testing shuld be cnducted t demnstrate that prtins f the sftware nt invlved in the change were nt adversely impacted. This is in additin t testing that evaluates the crrectness f the implemented change(s). Parasft Supprt Step-by-step capture f user acceptance test prcesses. Each manual step is captured s the cmplete manual sequence can be easily retrieved, reviewed, and repeated adding bjective traceability t the prcess. Autmated generatin f a regressin test suite that captures the cde's current behavir as a baseline. Daily executin f this test suite ensures that the team is immediately alerted if cde mdificatins impact r break existing functinality. A cntinuus regressin testing prcess which ensures that the impacts f cde mdificatins are identified and addressed daily, and the regressin test suite stays in synch with the evlving applicatin. A framewrk that supprts the rapid additin f new tests that verify the crrectness f the implemented change(s). Abut Parasft Fr 25 years, Parasft has researched and develped sftware slutins that help rganizatins deliver defect-free sftware efficiently. By integrating end-t-end testing, dev/test envirnment management, and sftware develpment management, we reduce the time, effrt, and cst f delivering secure, reliable, and cmpliant sftware. Parasft's enterprise and embedded develpment slutins are the industry's mst cmprehensive including static analysis, functinal testing with requirements traceability, service virtualizatin, and mre. The majrity f Frtune 500 cmpanies rely n Parasft in rder t prduce tpquality sftware cnsistently and efficiently. T learn mre, visit Cntacting Parasft USA 101 E. Huntingtn Drive, 2nd Flr Mnrvia, CA Tll Free: (888) Tel: (626) inf@parasft.cm URL: Eurpe France: Tel: +33 (1) UK: Tel: + 44 (0) Germany: Tel: inf-eurpe@parasft.cm Other Lcatins See Parasft Crpratin All rights reserved. Parasft and all Parasft prducts and services listed within are trademarks r registered trademarks f Parasft Crpratin. All ther prducts, services, and cmpanies are trademarks, registered trademarks, r servicemarks f their respective hlders in the US and/r 10