Software Engineering Institute

Transcription

1 Tchnical Rport CMU/SEI-89-TR-1 ESD-TR-89-' ooi Carngi-Mllon Univrsity Softwar Enginring Institut

2 Tchnical Rport CMU/SEI-89-TP--«ESD-TR Fbruary 1989 Th Stat of Softwar Enginring Practic: A Prliminary Rport Watts S. Humphry David H. Kitson Tim C. Kass Softwar Procss Assssmnt Projct Approvd for public rlas. Distribution unlimitd. Softwar Enginring Institut Carngi Mllon Univrsity Pittsburgh, Pnnsylvania 15213

3 This tchnical rport was prpard for th SEI Joint Program Offic ESD/AVS HanscomAFB, MA Th idas and findings in this rport should not b construd as an official DoD position. It is publishd in th intrst of scintific and tchnical information xchang. Rviw and Approval This rport has bn rviwd and is approvd for publication. FOR THE COMMANDER Karl H. Shinglr ^ SEI Joint Program Offic This work is sponsord by th U.S. Dpartmnt of Dfns. Copyright 1989 by Carngi Mlton Univrsity. This documnt is availabl through th Dfns Tchnical Information Cntr. OTIC provids accss to and transfr ol scintific and tchnical information for DoD prsonnl, DoD contractors and potntial contractors, and othr U.S. Govrnmnt agncy prsonnl and thir contractors. To obtain a copy, plas contact DTIC dirctly: Dfns Tchnical Information Cntr, Attn: FDRA, Camron Station. Alxandria, VA Copis of this documnt ar also availabl through th National Tchnical Information Srvic. For information on ordring, plas contact NTIS dirctly National Tchnical Information Srvic, U.S. Dpartmnt of Commrc, Springfild, VA Us of any tradmarks in this rport is not intndd in any way to infring on th rights of th tradmark holdr.

4 Tabl of Contnts Excutiv Summary 1 1. Introduction Softwar Procss Focus Softwar Procss Maturity Mod! Softwar Procss Assssmnt Instrumnt Examining Softwar Procsss SEI-Assistd Assssmnts Slf-Assssmnts Capability Evaluations Workshop Assssmnts Data Collction and Analysis Mthodology Basis for th Rport Data Usag Considrations Data Analysis Softwar Procss Maturity Lvl Distribution Profils of Ngativ Rsponss to Ky Qustions Implications and Rcommndations Lvl 1 Organizations Softwar Supplirs Acquisition Authoritis Lvl 2 Organizations Softwar Supplirs Acquisition Authoritis 20 Rfrncs 2 2 Appndix A. Ky Qustions and Rspons Profils 23 CMU/SEI-TR-89-1

5 CMU/SEI-TR-89-1

6 List of Tabls Tabl A.1: Ky Qustions for Lvl 2 26 Tabl A.2: Ky Qustions for Lvl 3 29 CMU/SEI-TR-89-1

7 iv CMU/SEI-TR-89-1

8 List of Figurs Figur 1.2.1: SEI Softwar Procss Maturity Modl 6 Figur : Figur : Softwar Procss Maturity Lvl Distribution (113 Data Points) 14 Softwar Procss Maturity Lvl Distribution (55 Data Points) 15 Figur A.1: Prcnt Ngativ Rspons of Lvl 1 Projcts to Lvl 2 Ky Qustions - Workshop Assssmnt Data (96 Data Points) 24 Figur A.2: Prcnt Ngativ Rspons of Lvl 1 Projcts to Lvl 2 Ky Qustions - SEI-Assistd Assssmnt Data (41 Data Points) 25 Figur A.3: Prcnt Ngativ Rspons of Lvl 2 Projcts to Lvl 3 Ky Qustions - Workshop Assssmnt Data (16 Data Points) 27 Figur A.4: Prcnt Ngativ Rspons of Lvl 2 Projcts to Lvl 3 Ky Qustions - SEI-Assistd Assssmnt Data (12 Data Points) 28 CMU/SEI-TR-89-1

9 Vl CMU/SEI-TR-89-1

10 Acknowldgmnts Th authors acknowldg th fforts of thos individuals (both within th SEI and from organizations which support th SEI) who contributd to th work dscribd in this rport. In particular, w acknowldg th contributions of Kn Dymond, who prformd various analyss of th assssmnt data and providd th authors with rlvant information and graphics. Also, w acknowldg Linda Psant, who significantly improvd th radability of th rport through hr tchnical diting skills. W xtnd a spcial thanks to all th individuals and sponsoring organizations that wr involvd in th various assssmnts, and to our collagus at th SEI who rviwd th various drafts lading to this rport. W particularly apprciat th fforts of th mmbrs of th final tchnical rport rviw tam: Kn Dymond, Ron Higura, Mark Paulk, Rich Pthia, and Bill Swt. CMU/SEI-TR-89-1 vii

11 I Th Stat of Softwar Enginring Practic: A Prliminary Rport Abstract: This is th first in a sris of SEI rports to provid priodic updats on th stat of softwar nginring practic in th DoD softwar community. Th SEI has dvlopd, and is rfining, a procss framwork and assssmnt mthodology for charactrizing th procsss usd by softwar organizations to dvlop and volv softwar products. This rport provids a brif ovrviw of th procss framwork and assssmnt approach, dscribs assssmnt rsults obtaind to dat, and discusss implications of th currnt stat of th practic for both customrs and supplirs of DoD softwar. Excutiv Summary Th Softwar Enginring Institut (SEI) was stablishd by th U.S. Dpartmnt of Dfns (DoD) to transition improvd softwar mthods into gnral practic. As part of this mission, work is undr way to charactriz and rport on th stat of th practic of softwar nginring in th DoD softwar community. 1 Prliminary rsults of this work indicat that th majority of th softwar organizations in this community ar oprating at an immatur lvl of softwar procss maturity. 2 In a matur softwar procss, popl, mthods, tchniqus, and tchnology ar ffctivly and fficintly coupld to consistntly produc quality softwar within th constraints of cost and schdul rquirmnts. In an immatur softwar procss, costs and schduls ar largly unprdictabl, quality is gnrally marginal, and tchnology is oftn usd inffctivly. Spcifically, organizations with immatur procsss ar oftn dficint in on or mor of th following aras: projct planning projct managmnt configuration managmnt softwar quality assuranc 1 By DoD softwar community w man DoD agncis and DoD contractors ngagd in th acquisition, production, or maintnanc of softwar. ^Bcaus of th way organizations wr slctd for inclusion in this study, th data upon which th rport is basd dos not ncssarily constitut a statistically valid masur of th stat of DoD softwar community practic. CMU/SEI-TR-89-1

12 Softwar profssionals gnrally nd most hlp in controlling rquirmnts, coordinating changs, managing (and making) plans, managing intrdpndncs, and gtting hlp on systms dsign issus. Sinc ths and similar problms gnrally consum much of vry practitionr's tim, this is whr managmnt can provid th most immdiat hlp. For lowmaturity organizations, tchnical issus almost nvr appar at th top of ky priority issu lists. This is not bcaus tchnical issus ar not important but simply bcaus so many managmnt problms must b handld first. A matur softwar procss will not liminat th ongoing nd to undrstand th application, to dal with changing rquirmnts, and to manag systm dsign issus. Howvr, organizations with mor matur procsss will b bttr positiond to addrss ths issus ffctivly and avoid th unncssary xacrbation of ths and othr problms. According to th SEI fiv-lvl procss maturity modl, 3 currnt softwar nginring practic is largly at th initial lvl (lvl 1, or lowst lvl) of procss maturity. Thr is a small numbr of lvl 2 organizations and a fw lvl 3 projcts in som organizations. No projcts hav bn rportd or assssd at lvl 4 or 5. Narly all lvl 1 softwar organizations urgntly nd to improv thir managmnt systm for controlling thir softwar procss. Many managrs nd guidanc on how to conduct projct rviws, what ky indicators to xamin, and how to us basic managmnt mthods and tools. For projct managrs, this training should includ th mthods and procdurs for stimating softwar siz, stimating rsourc nds, and dvloping schduls. Th scond ara rquiring immdiat attntion for lvl 1 softwar organizations is Softwar Quality Assuranc (SQA). SQA, whil gnrally availabl, is not ffctivly prforming its rol bcaus of inadquat rsourcs, inadquat task dfinition, or inadquat managmnt support. As organizations start to improv thir softwar procss, thy should also bgin gathring data on thir cod and tst rrors. Sinc lvl 1 organizations ar typically high-risk supplirs, w suggst that acquisition authoritis that dal with lvl 1 organizations rquir aggrssiv action by ths organizations to improv to lvl 2. Th rlativly fw lvl 2 organizations ar currntly among th most capabl softwar groups in th DoD softwar community. But vn though thy hav advancd substantially byond lvl 1, thy still hav considrabl room for improvmnt. Many of ths organizations ar quippd to 3 Th maturity modl is discussd in Sction 1.2. CMU/SEI-TR-89-1

13 advanc rapidly; a full 69% hav a Softwar Enginring Procss Group (SEPG). 4 Organizations without a SEPG should promptly stablish on. Lvl 2 organizations typically do not adquatly train thir softwar popl. A furthr dficincy in lvl 2 organizations is th lack of mchanisms to assur that SQA is valuating rprsntativ sampls of th softwar procss. Th lack of adquat rgrssion tsting is also a common problm. This gnrally lads to lat discovry of problms, last-minut tsting criss, and poor product quality. Whil lvl 2 organizations typically hav thir costs and schduls undr rasonabl control, thy gnrally do not hav ordrly mthods for tracking, controlling, and improving th quality of thir softwar or of thir softwar procss. Furthr, fw of ths organizations hav adquat rsourcs or action plans dirctd at long-trm softwar procss improvmnt. With th incrasing rlianc of critical dfns systms on complx softwar, th ncssary improvmnts rquir aggrssiv action. Thus, w suggst that acquisition authoritis rquir lvl 2 organizations to ddicat rsourcs to procss improvmnt and to stablish and rport on th actions ndd to progrss to maturity lvl 3. An appropriat vhicl for doing so might b to provid for such improvmnt fforts as an allowabl cost to th contract, or in th statmnt of work. W also suggst that acquisition authoritis rquir that contractor SQA organizations b adquatly staffd and ffctivly usd. Bcaus thr ar only a fw lvl 3 projcts in organizations, our sampl siz is too small to draw maningful conclusions about thir improvmnt nds. Whil SQA will continu to play an important and significant rol in achiving th objctivs of high-quality softwar dlivrd on tim and within budgt, w bliv that SQA alon is not sufficint to mt ths objctivs. Ultimatly, w nd capabl and motivatd profssionals, matur softwar procsss, and commitmnt to build quality into vry product; in short, w nd th rcognition that quality is vryon's job. 4 An SEPG is a group of softwar profssionals spcifically chartrd to focus on softwar procss improvmnt. CMU/SEI-TR-89-1

14 1. Introduction This rport dscribs th initial rsults of a continuing Softwar Enginring Institut ffort to charactriz and rport on th currnt stat of softwar nginring practic. Charactrizing, undrstanding, and facilitating improvmnt in th practic of softwar nginring is important to th SEI, th Dpartmnt of Dfns, and th nation. Th SEI Softwar Procss Program has th goal of improving th procss of dvloping and volving softwar. Our approach mphasizs th following: 1. Dvloping and validating a softwar procss framwork and valuation mthodology for idntifying capabl contractors. 2. Transitioning th valuation mthodology to DoD softwar acquisition agncis and thir prim contractors. 3. Dvloping and rfining an associatd assssmnt mthodology for us by th DoD softwar community for intrnally assssing softwar nginring capability and dtrmining improvmnt nds. 4. Charactrizing and rporting on th stat of softwar nginring practic in th DoD softwar community. 5. Facilitating softwar procss improvmnt in th DoD softwar community. Th focus of this rport is th currnt stat of softwar nginring practic from a softwar procss prspctiv; that is, th rport will charactriz th softwar procsss currntly usd by softwar managrs and practitionrs in organizations doing DoD softwar work. Th SEI has considrd and our rsults ar gnrally consistnt with th rsults of a numbr of prior studis [BAS84, DRU82, REI88, THA82]. This rport is organizd in thr parts. Th first sction provids th background and framwork for collcting th data upon which th rport is basd. Sction 2 dscribs th data collctd and th analyss prformd on this data. Sction 3 discusss implications and rcommndations for customrs and supplirs of DoD softwar Softwar Procss Focus Sinc arly 1987, th SEI Softwar Procss Program has focusd on softwar procss as a mans of improving th ability of softwar CMU/SEI-TR-89-1

15 organizations to produc softwar products according to plan whil simultanously improving th organization's ability to produc bttr products. This focus on softwar procss is basd on th prmiss that 1) th procss of producing and volving softwar products can b dfind, managd, masurd, and progrssivly improvd and 2) th quality of a softwar product is largly govrnd by th quality of th procss usd to crat and maintain it. Th softwar procss is th st of activitis, mthods, and practics which guid popl (with thir softwar tools) in th production of softwar. An ffctiv procss must considr th rlationships of th rquird tasks, th tools and mthods, and th skills, training, and motivation of th popl involvd. Softwar procss managmnt is th application of procss nginring concpts, tchniqus, and practics to xplicitly monitor, control, and improv th softwar procss. It is only on of svral activitis which must b ffctivly prformd for softwar-producing organizations to b consistntly succssful. Capabl and motivatd tchnical popl ar also ndd. Knowldg of th ultimat application nvironmnt is critical also, as is dtaild undrstanding of th nd usr's nds [CUR88]. Evn with all ths capabilitis, howvr, inattntion to th softwar managmnt problms dscribd in Sction 2 will likly rsult in disappointing organizational prformanc. [KIT89] provids a mor comprhnsiv discussion of th rol and significanc of softwar procss and th disciplin of softwar procss managmnt. This viw of procss and procss managmnt has ld to th cration of a procss maturity modl and a rlatd softwar procss assssmnt instrumnt, which ar important lmnts of SEI mthods for xamining softwar procsss. Th rmaindr of Sction 1 brifly discusss ths lmnts and som mthods of applying thm to th softwar procsss of organizations. CMU^EI-TR-SQ-I

16 1.2. Softwar Procss Maturity Modl Th softwar nginring capability of an organization can b charactrizd with th aid of th softwar procss maturity modl shown in Figur This modl provids fiv maturity lvls, idntifis th ky improvmnts rquird at ach lvl, and stablishs a priority ordr for moving to highr lvls of procss maturity. Figur 1.2.1: SEI Softwar Procss Maturity Modl At th initial lvl (lvl 1), an organization can b charactrizd as having an ad hoc, or possibly chaotic, procss. Typically, th organization oprats without formalizd procdurs, cost stimats, and projct plans. Evn if formal projct control procdurs xist, thr ar no managmnt mchanisms to nsur that thy ar followd. Tools ar not wll intgratd with th procss, nor ar thy uniformly applid. In addition, chang control is lax, and snior managmnt is not xposd to or dos not undrstand th ky softwar problms and issus. Whn projcts do succd, it is gnrally bcaus of th hroic fforts of a ddicatd tam rathr than th capability of th organization. An organization at th rpatabl lvl (lvl 2) has stablishd basic projct controls: projct managmnt, managmnt ovrsight, product assuranc, and chang control. Th strngth of th organization stms from its xprinc at doing similar work, but it facs major risks whn prsntd with nw challngs. Th organization has frqunt quality problms and lacks an ordrly framwork for improvmnt. CMU/SEI-TR-89 1

17 ' At th dfind lvl (lvl 3), th organization has laid th foundation for xamining th procss and dciding how to improv it. Th ky actions ndd to mov from th rpatabl lvl to th dfind lvl ar to stablish an SEPG within th organization, to stablish a softwar procss architctur that dscribs th tchnical and managmnt activitis rquird for propr xcution of th procss, and to introduc a family of softwar nginring mthods and tchnologis. Th managd lvl (lvl 4) builds on th foundation stablishd at th dfind lvl. Whn th procss is dfind, it can b xamind and improvd but thr is littl data to indicat ffctivnss. Thus, to advanc to th managd lvl, an organization should stablish a minimum st of masurmnts for th quality and productivity paramtrs of ach ky task. Th organization should also stablish a procss databas with rsourcs to manag and maintain it, to analyz th data, and to advis projct mmbrs on its maning and us. Two rquirmnts ar fundamntal to advanc from th managd to th optimizing lvl (lvl 5). Data gathring should b automatd, and managmnt should rdirct its focus from th product to procss analysis and improvmnt. At th optimizing lvl, th organization has th mans to idntify th wakst procss lmnts and strngthn thm, data ar availabl to justify applying tchnology to various critical tasks, and numrical vidnc is availabl on th ffctivnss with which th procss has bn applid. Th ky additional activity at th optimizing lvl is rigorous dfct caus analysis and dfct prvntion. Ths maturity lvls hav bn slctd bcaus thy do th following: Rasonably rprsnt th historical phass of volutionary improvmnt of actual softwar organizations. Rprsnt a masur of improvmnt that is rasonabl to achiv from th prior lvl. Suggst intrim improvmnt goals and progrss masurs. Mak obvious a st of immdiat improvmnt prioritis, onc an organization's status in this framwork is known. Whil thr ar many aspcts to th transition from on maturity lvl to anothr, th basic objctiv is to achiv a controlld and masurd procss as th scintific foundation for continuous improvmnt. It has bn our xprinc (basd on tn SEI-assistd assssmnts conductd sinc Fbruary 1987) that whn softwar organizations ar assssd against this maturity framwork, th assssmnt mthod has nabld us to accuratly plac thm on th maturity scal and idntify ky CMU/SEI-TR-89-1

18 improvmnt nds. W bliv softwar procss maturity is a usful indicator of an organization's softwar nginring capability,.g., its ability to produc quality softwar products on tim and within budgt. W also bliv that whil th us of tools and tchnology can nhanc softwar nginring capability, thir contribution is oftn of limitd valu for organizations with low-maturity softwar procsss. [HUM88] and [KIT89] provid mor comprhnsiv dscriptions of softwar procss managmnt and th maturity modl Softwar Procss Assssmnt Instrumnt Th assssmnt instrumnt is a structurd st of ys-no qustions which hlps to facilitat th conduct of rasonably objctiv and consistnt assssmnts of softwar organizations [HUM87]. It has also bn dsignd to assist DoD acquisition organizations in idntifying softwar contractors with accptabl softwar nginring capabilitis. Sinc th instrumnt and mthod for applying it ar publicly availabl, softwar contractors can us thm to idntify aras for improvmnt. Th SEI provids training on how to conduct ffctiv assssmnts for organizations intrstd in conducting thir own. Th qustions in th assssmnt instrumnt covr thr aras: 1. Organization and rsourc managmnt. This sction dals with functional rsponsibilitis, prsonnl, and othr rsourcs and facilitis. 2. Softwar nginring procss and its managmnt. This sction concrns th scop, dpth, and compltnss of th softwar nginring procss and th way in which th procss is masurd, managd, and improvd. 3. Tools and tchnology. This sction dals with th tools and tchnologis usd in th softwar nginring procss. It hlps dtrmin th ffctivnss with which th organization mploys basic tools and mthodologis. Som sampl qustions from th assssmnt instrumnt ar: Is thr a softwar nginring procss group or function? Is a formal procdur usd to mak stimats of softwar siz? Ar cod and tst rrors projctd and compard to actuals? "8~ CMU/SEI-TR-89-1

19 1.4. Examining Softwar Procsss Thr ar a numbr of ways th softwar procss framwork (softwar procss concpts and principls + maturity modl + assssmnt instrumnt) can b applid; th SEI has dvlopd, and has xprinc with, th following: SEI-assistd assssmnts Slf-assssmnts Capability valuations Workshop assssmnts Th paragraphs blow brifly discuss ach typ of application. A mor comprhnsiv discussion of how assssmnts ar conductd and th rol of assssmnt in improving softwar nginring capability is containd in [KIT89] SEI-Assistd Assssmnts An SEI-assistd assssmnt is an appraisal, by a traind tam of xprincd softwar profssionals, of an organization's currnt d facto softwar procss. Typically, a tam is composd of four or fiv SEI profssionals and on to thr sit profssionals. A mthodology for conducting assssmnts has bn dvlopd by th SEI [OLS89]. Th assssmnt tam rcivs training in th mthodology prior to conducting th actual assssmnt. Th goal for this typ of assssmnt is to facilitat improvmnt of th organization's softwar procss. Th assssmnt tam idntifis th most important softwar procss issus currntly facing th organization and dvlops rcommndations to dal with ths issus. Sinc th objctiv is improvmnt within a givn organization, validation of qustionnair rsponss (.g., rqusting substantiating documnts) is limitd to thos having a dirct baring on transition to th nxt highr lvl of procss maturity (contrast this with contractor capability valuation as discussd in Sction 1.4.3). SEI-assistd assssmnts ar conductd in accordanc with an assssmnt agrmnt signd by th SEI and th organization bing assssd. This writtn agrmnt contains provisions for snior managmnt involvmnt, organizational rprsntation on th assssmnt tam, confidntiality of rsults, and follow-up actions. Th SEI has bn conducting this typ of assssmnt sinc Fbruary 1987 and is using th knowldg and information acquird to rfin an mrging pictur of th stat of th practic of softwar nginring in th DoD softwar community. CMU/SEI-TR-89-1

20 Slf-Assssmnts Slf-assssmnts ar similar to SEI-assistd assssmnts, with th primary diffrnc bing assssmnt tam composition. Slf-assssmnt tams ar composd primarily of softwar profssionals from th organization bing assssd, with on or two SEI softwar profssionals optionally prsnt. Th contxt, objctiv, and dgr of validation ar th sam as for SEI-assistd assssmnts. Th SEI offrs slf-assssmnt training on a limitd basis for organizations committd to improving thir softwar nginring capability. Organizations that participat in th SEI-providd training xcut a writtn agrmnt with th SEI which provids for sharing of assssmnt rsults, intgrity of th assssmnt mthodology, and optional participation of SEI assssmnt tam mmbrs Capability Evaluations Capability valuations, lik SEI-assistd assssmnts and slf-assssmnts, ar appraisals of an organization's currnt softwar procss; howvr, th contxt, purpos, and assssmnt tam composition ar diffrnt. Th contxt of capability valuation is th DoD acquisition procss, and th purpos is to provid information concrning th organization's softwar nginring capabilitis for th acquisition agncy. This information is thn considrd, along with othr rlvant information, in th sourc slction dcision. Hnc, validation of assssmnt instrumnt rsponss is a gratr considration hr than it is in assssmnts. Capability valuations ar conductd by traind tams of valuators from th acquisition agncy. Th SEI provids th ncssary training for th valuation tams using our mthodology, but w do not participat in valuations. Th rsults of capability valuations ar supplid by th valuation tam to th acquisition agncy. Non-attributd, "sanitizd" rsults ar providd to th SEI to hlp us rfin th assssmnt instrumnt and valuation mthodology; thy also contribut to our mrging pictur of th status of DoD softwar nginring capability Workshop Assssmnts At workshop assssmnts, profssionals from various organizations larn about procss managmnt concpts, assssmnt tchniqus, and th SEI assssmnt mthodology. Thy also complt an assssmnt instrumnt and supply dmographic data basd on a projct with which thy ar familiar. This format is dsignd for popl who wish to larn mor about th SEI assssmnt mthodology with minimal invstmnt. 10 CMU/SEI-TR-89-1

21 (ftk Th data collctd at workshop assssmnts is addd to th SEI assssmnt databas and is usd for various analyss. Workshop assssmnts ar typically conductd at confrncs and symposia attndd by DoD and DoD contractor softwar profssionals (.g., National Scurity Industrial Association, Elctronic Industris Association, and th annual SEI affiliats symposium). CMU/SEI-TR

22 2. Data Collction and Analysis Mthodology This chaptr provids a charactrization of th data usd in this rport, idntifis som of th considrations in using this data, and dscribs th analyss which wr prformd to driv th rsults prsntd Basis for th Rport This rport is basd on information of two typs: Rsponss to th assssmnt instrumnt (th qustions in th instrumnt ar ys-no qustions). Th rsponss wr collctd from workshop assssmnts and SEI-assistd assssmnts. Th collctiv knowldg and xprinc which th SEI has acquird as a rsult of our involvmnt in th dvlopmnt and application of th various assssmnt mthods discussd in Sction 1.4. Assssmnt participants includ softwar and hardwar/softwar dvloprs from DoD organizations, DoD contractors, and commrcial ntrpriss. Tn organizations participatd in SEI-assistd assssmnts (with 4 to 6 projcts involvd in ach assssmnt), and ovr 70 organizations wr rprsntd in th workshop assssmnts, rprsnting 168 data points 5 from assssmnts across th Unitd Stats. In vry assssmnt, th SEI signs an agrmnt that thr will b no attribution of th rsults to a spcific company. Th implications and rcommndations prsntd in Chaptr 3 of this rport, thrfor, rprsnt an aggrgat viw Data Usag Considrations Th rsults dscribd in this rport rflct th stat of th softwar nginring practic basd on th data, xprinc, and knowldg acquird by th SEI sinc Fbruary This sction dscribs som mthodological considrations which w fl ar grman to radrs of this rport. First, th sampl population was not statistically slctd. Most of th rspondnts cam from organizations that ar affiliatd with th SEI. Ths rspondnts varid in th typ and dgr of involvmnt with th projcts thy rportd on. 5 A data point is on st of ys-no rsponss to th softwar procss assssmnt instrumnt; th scop of ths rsponss is a spcific softwar projct. 12 CMU/SEI-TR-89-1

23 Anothr considration is th dgr of validation of th rsponss; th xtnt to which corroboration of rsponss was rqustd dpndd on th typ of assssmnt bing conductd. At this tim, w hav no way of dtrmining th ffct of this factor on th rsponss. In comparing th qustion rsponss rcivd from workshop assssmnts and from SEI-assistd assssmnts, svral points should b notd: 1. Th SEI-assistd assssmnts wr conductd on-sit by a traind tam, with participation from knowldgabl projct managrs and tchnical profssionals. 2. Th workshop assssmnt rspondnts containd a mix of managmnt and non-managmnt profssionals, som of whom likly had dtaild knowldg of th tchnical points. 3. For SEI-assistd assssmnts, many thrshold rsponss wr vrifid; howvr, no workshop assssmnt rsponss wr vrifid Data Analysis Two viws of th data wr prpard and analyzd: (1) softwar procss maturity lvl distribution and (2) prcnt ngativ rspons to ky qustions. 6 For th purposs of this rport, w sparatd rspons data from SEI-assistd assssmnts and that from workshop assssmnts, trating thm as two distinct data populations. Bcaus of th considrations mntiond in Sction 2.2, w do not bliv that gratr dpth of analysis than that prsntd in this rport is justifid Softwar Procss Maturity Lvl Distribution Th distribution of softwar procss maturity lvl across th sampl population provids a high-lvl viw of th stat of th practic; Figurs and show th softwar procss maturity distributions for workshop assssmnts and SEI-assistd assssmnts, rspctivly. For both figurs, th vrtical axis rprsnts th prcntag of data points in th population; th horizontal axis rprsnts th softwar procss maturity scal-lvls 1 through 5. In ordr to show additional fin structur, th maturity scal has bn furthr dividd into quartils-four quartils for ach maturity lvl (for a total of 20 quartils, or 20 vrtical bars). Th quartils ar idntifid in th charts using th notation x.y, whr x is th maturity lvl (1- "Ky qustions ar thos for which a high prcntag of affirmativ rsponss is rquird to qualify for a particular maturity lvl. S Sction CMU/SEI-TR

24 5), and y is th quartil (1-4). In Figur , for xampl, 2.4 rfrs to th fourth (and last) quartil for lvl 2 and contains approximatly 13% of th sampl population. Not that sinc no data points hav bn obsrvd to dat at lvl 4 or abov, that portion of th graph has not bn shown. Each data point was placd in th maturity lvl distribution basd upon a dtrmination of how many additional affirmativ rsponss would hav bn ndd to rat th projct at th nxt highr lvl of procss maturity. Th rang of ths valus was thn qually dividd into four "buckts" or quartils. Thus, th highr th quartil numbr, th closr th projct is to bing ratd at th nxt highr maturity lvl. 50 -r This chart rflcts corrctions mad Jun i 1 r 1 1 r-""h Maturity Lvl Quartil 3.4 Figur : Softwar Procss Maturity Lvl Distribution - Workshop Assssmnt Data (113 Data Points) 7 'Not that th prcntags may not total 100 du to rounding rrors. 14 CMU/SEI-TR-89-1

25 This chart rflcts corrctions mad Jun Maturity Lvl Quartll Figur : Softwar Procss Maturity Lvl Distribution - SEI- Assistd Assssmnt Data (55 Data Points) 8 Th workshop assssmnt rsults, shown in Figur , indicat that th majority of th rspondnts rportd projcts at th initial lvl of maturity. Figur shows a larg prcntag of th rspondnts to b in th fourth quartil of lvl 1 (quartil 1.4); with minimal improvmnt, ths projcts could b classifid as lvl 2. Fourtn prcnt of all th workshop rspondnts rportd projcts at th rpatabl lvl, and only 1% of thos rspondnts dscribd projcts at lvl 3, th dfind lvl. No workshop rspondnts rportd projcts at ithr th managd or th optimizing lvl of softwar procss maturity. 8 Not that th prcntags may not total 100 du to rounding rrors. CMU/SEI-TR

26 Th maturity lvl distribution for projcts rviwd by SEI-assistd assssmnts, shown in Figur , is vry similar to that for th workshop data. Although workshop participants wr largly mid- to uppr-lvl managrs not currntly managing a projct (as opposd to th projct managrs who providd data for SEI-assistd assssmnts), th profils of procss maturity ar surprisingly similar. Som ky diffrncs ar apparnt, howvr. First, Figur shows that th sampl population is skwd slightly towards highr lvls of procss maturity. Scondly, largr numbrs of projcts ar in quartil 4 of maturity lvls 1, 2, and 3, poisd for moving to th nxt highr lvl of softwar procss maturity Profils of Ngativ Rsponss to Ky Qustions For th purposs of this rport, two attributs of th assssmnt instrumnt qustions ar grman. First, ach qustion is associatd with a particular maturity lvl; for xampl, th qustion "Is a formal procdur usd to mak stimats of softwar siz?" is a lvl 2 qustion. This mans that an organization that has all of th attributs of a lvl 2 softwar organization (with rspct to th SEI procss maturity modl) would rspond affirmativly to this qustion. Scond, crtain qustions ar dsignatd as bing ky In ordr to qualify at a givn lvl of procss maturity, an organization must rspond affirmativly to 90% of th ky qustions for that lvl. To analyz th rsponss to ky qustions, w dtrmind th prcntag of th population rsponding ngativly to ach ky qustion for lvls 2 and 3 and displayd th rsults in dcrasing ordr. Ths computations wr prformd for both th workshop assssmnt data and for SEI-assistd assssmnt data. Th rsults ar providd in Appndix A and ar rfrncd in appropriat parts of Sction 3. An xamination of th rsults in Appndix A shows a clos, though not xact, corrlation btwn th two data sampls; for xampl, for both lvl 2 and lvl 3 profils, four out of th top fiv qustions for SEI-assistd assssmnts wr among th top fiv qustions for th workshop assssmnt profils. 16 CMU/SEI-TR-89-1

27 w.-.lv> " I 3. Implications and Rcommndations This sction discusss th implications of th currnt stat of softwar nginring practic and suggsts improvmnt actions. W discuss implications and rcommndations, first for lvl 1 organizations, thn for lvl 2 organizations. Our viws ar offrd for two audincs: softwar supplirs and acquisition authoritis Lvl 1 Organizations Softwar Supplirs Narly all lvl 1 softwar organizations urgntly nd to improv thir projct managmnt mthods (CN24, CN42, CN43,CN44, CN46, CN77, CN84). 9 Many managrs nd guidanc on conducting projct rviws, slcting ky indicators to xamin, and using basic managmnt mthods and tools. For projct managrs, training should includ th mthods and procdurs for stimating softwar siz, stimating rsourc nds, and dvloping schduls. Whil organizations in th highst quartil of lvl 1 typically hav th ability to mak rsourc and schdul projctions, siz stimating is a problm for fully 66% (CN42) of lvl 1 projcts. 10 Softwar siz tracking is also a problm for 64% (CN46) of this group. As a rsult, projcts gnrally undrstimat rsourcs and rly on ovrly optimistic schduls. Th introduction of mor formal procdurs for stimating and tracking softwar siz will thus substantially contribut to improvd projct cost and schdul prformanc. On of th first stps organizations must tak whn thy start to sriously addrss softwar quality is to gathr data on th rrors found in th product. This ara should rciv arly focus in any procss improvmnt program as it is a prrquisit to significant improvmnts in ovrall procss quality and productivity. Sinc th final cod and tst stags ar gnrally th asist to masur, this is whr data gathring should start. Of th workshop assssmnt rspondnts, narly 60% indicat that such data was not gathrd (CN48). Anothr ara rquiring immdiat attntion for lvl 1 softwar organizations concrns th rol of Softwar Quality Assuranc (SQA). Whil 70% of lvl 1 9 CN» control numbr. Th control numbr uniquly idntifis a particular qustion and is invariant across vrsions of th assssmnt instrumnt. Whr conclusions ar dirctly supportd by qustion rsponss, th rlvant assssmnt instrumnt qustion control numbr is citd. Th qustion txt can b found in Appndix A. 10 This, and subsqunt, prcnt ngativ rspons valus ar takn from th workshop assssmnt charts providd in Appndix A. CMU/SEI-TR

28 organizations hav rporting channls sparat from dvlopmnt for thir SQA groups (CN6), 56% of th organizations rport that thy do not hav indpndnt audits of ach stp in thir softwar dvlopmnt procss. 11 As a rsult, SQA, whil gnrally availabl, is not ffctivly prforming its rol. Th rasons may b a lack of adquat rsourcs, a lack of adquat task dfinition, or inadquat managmnt support. In any vnt, ffctiv SQA is rquird to assur managmnt that its stablishd mthods, standards, and procdurs ar bing applid. SQA can only b ffctiv, howvr, whn it addrsss clarly idntifid and statd objctivs. Whrvr SQA is stablishd mrly to mt a contractual provision, it is not likly to contribut significantly to ovrall prformanc and may, in fact, dtract Acquisition Authoritis Sinc lvl 1 organizations ar typically high-risk supplirs, w suggst that acquisition authoritis who dal with lvl 1 organizations rquir aggrssiv action by ths organizations to improv to lvl 2. Th ky itms to xamin in dtrmining whthr an organization is at lvl 1 or lvl 2 ar dfind in th SEI softwar maturity modl and th softwar procss assssmnt instrumnt [HUM88, HUM87]. If a dtaild rviw is impractical, howvr, a critical xamination of currnt practics for siz and rsourc stimating and schduling should idntify th most critical xposurs. Whn ths ar not adquat, improvmnt commitmnts should includ th stablishmnt of a formal planning and rviw systm as wll as comprhnsiv managmnt training in softwar projct planning. Acquisition agncis should also b particularly intrstd in contractor procdurs for gathring cod and tst rror statistics sinc this data provids a good indication of product quality. Whn th data is availabl and can b rviwd during th projct, it provids arly warning of quality problms. Without this data, quality problms ar gnrally first dtctd in final tst, whn it is too lat to rcovr without srious schdul and cost consquncs. Thus, w suggst that th acquisition agncy rqust cod and tst rror statistics as part of its normal projct rviw procss. W furthr suggst that softwar acquisition agncis rquir th contractor to stablish and maintain an ffctiv SQA organization with adquat rsourcs to rviw th ky stps in th procss. This SQA rol should b clarly dfind and documntd. SQA rsponsibilitis should b focusd on th policis, mthods, procdurs, and standards for making plans and tracking progrss against thm. Onc ths ar in plac and consistntly 11 Th 56% figur is basd on rsponss to qustion CN30 - a non-ky qustion ("For ach projct, ar indpndnt audits conductd for ach stp of th softwar dvlopmnt procss?"). 18 CMU/SEI-TR-89-1

29 T followd, th SQA rol should b xpandd to includ pr rviws 12 and tst. If th masurs abov ar coupld with a sparat managmnt rporting chain to assur that SQA nonconcurrncs and issus ar rsolvd, SQA is likly to quickly bcom ffctiv Lvl 2 Organizations Softwar Supplirs Though organizations at lvl 2 hav advancd substantially byond lvl 1, thy still hav considrabl room for improvmnt. Across all organizations at all maturity lvls, training was found to b th ara most nding improvmnt. Fully 88% of th lvl 2 organizations in th workshop assssmnts did not hav adquat training for rviw ladrs (CN20) and ovr half th organizations did not hav a rquird training cours for softwar dvloprs (CN19). Although a lack of training may b accptabl for simpl or noncritical applications, training is crucial in organizations rsponsibl for dvloping advancd softwar systms. Softwar practitionrs nd to b knowldgabl and skilld in th us of languags and organizational procdurs, undrstand th projct rquirmnts and th application ara, and hav a common undrstanding of th systm protocols and architctural dsign. Without adquat training, projcts oftn hav srious schdul and cost problms; and thy hav difficulty nsuring that th rquirmnts and th systm architctur ar consistntly implmntd. A full 69% of th lvl 2 organizations do hav Softwar Enginring Procss Groups (CN15) and, thus, ar quippd to advanc rapidly to a mor matur status. Convrsly, 31% of th lvl 2 organizations hav not stablishd SEPGs and, thus, ar hindrd in planning and implmnting significant softwar procss improvmnt actions. Th lack of a procss focus is dmonstratd by th fact that 50% of lvl 2 organizations do not track softwar dsign rrors (CN47). Rgrssion tsting hlps to nsur that cod changs in a product baslin which rndr prviously implmntd functions inoprabl ar idntifid. Whn rgrssion tsting is not adquatly prformd, such damag is gnrally not found until latr in th procss, whn it is mor xpnsiv and tim consuming to fix. Rgrssion tsting is a problm for narly 80% (CN99) of th organizations in th workshop assssmnt population, 12jjy pr rviw w man a rviw of a softwar product (spcification, dsign, cod, tst plan, tc.) by prs of th producr(s) of th product for th purpos of idntifying dfcts and improvmnts. Pr rviws rang from walk throughs to formal inspctions, as dscribd in IEEE standard 1028, "Standard for Softwar Rviws and Audits." CMU/SEI-TR

30 indicating that lat problm discovry is a common problm. This situation can b substantially rducd with rlativly simpl rgrssion tst procdurs. A furthr srious nd for lvl 2 organizations carris ovr from lvl 1. Almost on-third of ths organizations do not hav mchanisms in plac to assur that SQA is valuating rprsntativ sampls of th softwar procss (CN98). It has also bn found that many SQA organizations ar undrstaffd, or thir rol is ill-dfind, or thy ar not adquatly supportd by managmnt. Th continuing lack of adquat SQA gnrally rsults in inconsistnt us of stablishd mthods and procdurs. Without ffctiv SQA, organizations will find it difficult, if not impossibl, to improv to lvl 3. Until th basic mthods and procdurs of lvl 2 ar consistntly and ffctivly applid, furthr procss improvmnt fforts ar likly to b inffctiv Acquisition Authoritis Basd on th SEI data and xprinc to dat, th rlativly fw lvl 2 organizations ar currntly among th most capabl softwar groups in th DoD softwar community. Thy typically hav thir costs and schduls undr rasonabl control; howvr, thy gnrally do not hav ordrly mthods for tracking, controlling, and improving th quality of ithr thir softwar or thir softwar procss. Furthr, fw of ths organizations hav adquat rsourcs or action plans dirctd at long-trm softwar procss improvmnt. Lvl 2 organizations should concntrat on stablishing SEPGs as a focal point for procss improvmnt. W suggst that acquisition authoritis rquir organizations to ddicat rsourcs to procss improvmnt, including initiating and monitoring th actions ndd to progrss to maturity lvl 3. An appropriat vhicl for doing so might b to provid for such improvmnt fforts as an allowabl cost to th contract, or in th statmnt of work. Th ky nds ar for procss standardization; improvd mthods for dsign, implmntation, and tst; and th idntification and application of improvd tools and tchnologis. Typically, th lack of an SEPG mans that no on is rsponsibl for dfining mtrics, installing an rror tracking systm, rtaining and analyzing th rsulting data, or rporting on progrss in quality or procss improvmnt. Whil th spcific improvmnt prioritis vary across organizations, th common nd is for rsourcs ddicatd to procss improvmnt. Training is a particularly snsitiv problm. Unlss th contractor has an xprincd tam which is alrady familiar with th systm and its application and is fully familiar with th languags and tools thy ar to us, som training programs ar ssntial. Evn with such an xprincd tam, som training is valuabl. Though spcific cours nds vary among 20 CMU/SEI-TR-89-1

31 - I organizations and training involvs som xpns, th costs ar invariably lss than th hiddn costs of trial-and-rror mthods. Training is xpnsiv, but not narly as xpnsiv as not training. Rgrssion tsting is ssntial for any wll-run softwar projct. Without slctiv rtsting of th systm or componnt to vrify that modifications hav not causd unintndd ffcts, thr is no assuranc that prviously intgratd functions still prform and that th systm or componnts still comply with th spcifid rquirmnts. Unlss adquat rgrssion tsting is routinly prformd as changs occur, larg numbrs of problms ar likly to b found whn th complt tst suit is run at accptanc tsting. Th tim usd to fix dfcts and rrun th tsts can b substantial whn ths activitis occur during th final phas of tsting. W suggst that acquisition agncis closly xamin th rgrssion tst plans of thir lvl 2 contractors. W also suggst that acquisition authoritis rquir thir contractors to adquatly staff SQA organizations and ffctivly us thm. Although a high prcntag of lvl 2 organizations hav SQA organizations in plac, only 31% of thm hav stablishd mthods for nsuring that SQA sampls ar appropriatly slctd. Thus, it is likly that many SQA groups rprsnt a substantial xpns but do not produc masurabl bnfits for th organization. If th contractor has an SQA group, th acquisition agncy should rquir clar vidnc that it is bing usd ffctivly. Such vidnc should includ: an SQA chartr signd by a snior xcutiv; approvd standards against which SQA conducts audits; and a rcord of SQA nonconcurrncs and th corrctiv actions takn. If an SQA group is not in plac, its ffctiv stablishmnt should b a rquirmnt in th contract. CMU/SEI-TR

32 Rfrncs 1. [BAS84] Basili, V.R., Gannon, J.D., Hamlt, R.G., Yh, R.T., Zlkowitz, M.V., "Softwar Enginring Practics in th US and Japan," IEEE Computr, [CUR88] Curtis, B., Krasnr, H., Isco, N., "A Fild Study of th Softwar Dsign Procss for Larg Systms," Communications of th ACM, Novmbr [DRU82] Druffl, L.E., Lt. Col. USAF, t al., Rport of th DoD Joint Srvic Task Forc on Softwar Problms, Dpartmnt of Dfns, July [HUM87] Humphry, W.S., Swt, W., t al., A Mthod for Assssing th Softwar Enginring Capability of Contractors, Softwar Enginring Institut, (CMU/SEI-87-TR-23, ADA ), Sptmbr [HUM88] Humphry, W.S., "Charactrizing th Softwar Procss: A Maturity Framwork," IEEE Softwar, March [KIT89] Kitson, D.H., Humphry, W.S., Th Rol of Assssmnt in Softwar Procss Improvmnt, Softwar Enginring Institut, (CMU/SEI-89-TR-3), March [OLS89] Olson, T.G., Humphry, W.S., Kitson, D.H., Conducting SEI- Assistd Softwar Procss Assssmnts, Softwar Enginring Institut, (CMU/SEI-89-TR-7), Fbruary [REI88] Rifr, DJ., Final Rport: Softwar Quality Survy, Amrican Socity for Quality Control, [THA82] Thayr, R.H., Pystr, A., Wood, R.C., "Validating Solutions to Major Problms in Softwar Enginring Projct Managmnt," IEEE Computr, August CMU/SEI-TR-89-1

33 Appndix A. Ky Qustions and Rspons Profils This sction of th rport provids a viw of slctd portions of th rspons data from workshop assssmnts and SEI-assistd assssmnts. Figurs A.1 and A.2 show ngativ rspons profils (with rspct to thos projcts ratd ovrall to b at lvl 1) for lvl 2 ky assssmnt instrumnt qustions. Tabl A.1 provids th txt of th sam ky qustions indxd by control numbr (CN). For xampl, Figur A.1 shows that of th workshop assssmnt projcts rportd to b at lvl 1 (96 out of a total of 113 projcts), 64% rspondd ngativly to qustion CN46 ("Ar profils of softwar siz maintaind for ach softwar configuration itm, ovr tim?"). Qustion CN46 is a ky qustion for advancing to lvl 2. Figurs A.3 and A.4 show ngativ rspons profils (with rspct to thos projcts ratd ovrall to b at lvl 2) for lvl 3 ky assssmnt instrumnt qustions. Tabl A.2 provids th txt of th sam ky qustions indxd by control numbr. CMU/SEI-TR

34 Figur A.1 Prcnt Ngativ Rspons of Lvl 1 Projcts to Lvl 2 Ky Qustions - Workshop Assssmnt Data (96 Data Points) 100 -r p r c n t N p g a t I V R s P 0 n s CN42 CN46 CN48 CN43 CN44 CN24 CN84 CN6 CN87 CN77 CN14 CN96 Ky Qustion Control Numbrs 24 CMU/SEI-TR-89-1

35 Figur A.2: Prcnt Ngativ Rspons of Lvl 1 Projcts to Lvl 2 Ky Qustions - SEI-Assistd Assssmnt Data (41 Data Points) 100 -r p r c n t N g a t i v R s P o n s CN42 CN46 CN48 CN43 CN44 CN24 CN84 CN6 CN87 CN77 CN14 CN96 Ky Qustion Control Numbrs CMU/SEI-TR

36 Tabl A.1: Ky Qustions for Lvl 2 N 13 Qustion 6 Dos th Softwar Quality Assuranc (SQA) function hav a managmnt rporting channl sparat from th softwar dvlopmnt projct managmnt? 14 Is thr a softwar configuration control function for ach projct that involvs softwar dvlopmnt? 24 Is a formal procdur usd in th managmnt rviw of ach softwar dvlopmnt prior to making contractual commitmnts? 42 Is a formal procdur usd to mak stimats of softwar siz? 43 Is a formal procdur usd to produc softwar dvlopmnt schduls? 44 Ar formal procdurs applid to stimating softwar dvlopmnt cost? 46 Ar profils of softwar siz maintaind for ach softwar configuration itm, ovr tim? 48 Ar statistics on softwar cod and tst rrors gathrd? 77 Dos snior managmnt hav a mchanism for th rgular rviw of th status of softwar dvlopmnt projcts? 84 Do softwar dvlopmnt first-lin managrs sign off on thir schduls and cost stimats? 87 Is a mchanism usd for controlling changs to th softwar rquirmnts? 96 Is a mchanism usd for controlling changs to th cod? (Who can mak changs and undr which circumstancs?) 13 CN control numbr - th control numbr uniquly idntifis a particular qustion and is invariant across vrsions of th assssmnt instrumnt. Ths qustions ar prsntd hr in control numbr ordr. 26 CMU/SEI-TR-89-1

37 Figur A.3: Prcnt Ngativ Rspons of Lvl 2 Projcts to Lvl 3 Ky Qustions - Workshop Assssmnt Data (16 Data Points) 100 T p r c n t N 9 a t I v R s P o n s CN20 CN99 CN19 CN47 CN15 CN98 CN23 CN83 CN95 CN59 CN61 CN91 CN92 Ky Qustion Control Numbrs CMU/SEI-TR

38 Figur A.4: Prcnt Ngativ Rspons of Lvl 2 Projcts to Lvl 3 Ky Qustions - SEI-Assistd Assssmnt Data (12 Data Points) p r c n t N g a t I v R s P o n s 0 H h 0 1 h CN20 CN99 CN19 CN47 CN15 CN98 CN23 CN83 CN95 CN59 CN61 CN91 CN92 Ky Qustion Control Numbrs 28 CMU/SEI-TR-89-1

39 Tabl A.2: Ky Qustions for Lvl 3 CN Qustion 15 Is thr a softwar nginring procss group function? 19 Is thr a rquird softwar nginring training program for softwar dvloprs? 20 Is a formal training program rquird for dsign and cod rviw ladrs? 23 Dos th softwar organization us a standardizd softwar dvlopmnt procss? 23 Dos th softwar organization us a standardizd and documntd softwar dvlopmnt procss on ach projct? 47 Ar statistics on softwar dsign rrors gathrd? 59 Ar th action itms rsulting from dsign rviws trackd to closur? 61 Ar th action itms rsulting from cod rviws trackd to closur? 83 Is a mchanism usd for nsuring complianc with th softwar nginring standards? 91 Ar intrnal softwar dsign rviws conductd? 92 Is a mchanism usd for controlling changs to th softwar dsign? 95 Ar softwar cod rviws conductd? 98 Is a mchanism usd for vrifying that th sampls xamind by Softwar Quality Assuranc ar truly rprsntativ of th work prformd? 99 Is thr a mchanism for assuring th adquacy of rgrssion tsting? CMU/SEI-TR

40 UNLIMITFn, imr.t AQQTFTFn»CCURITY CLASSIFICATION OF THIS PAGE 1*. REPORT SECURITY CLASSIFICATION UNCLASSIFIED 2«. SECURITY CLASSIFICATION AUTHORITY N/A 2b. OECLASSIFICATION/OOWNGRAOING SCHEDULE N/A 4. PERFORMING ORGANIZATION REPORT NUMBER(S) REPORT DOCUMENTATION PAGE lb. RESTRICTIVE MARKINGS NONE 3. DISTRIBUTION/AVAILABILITY OF REPORT APPROVED FOR PUBLIC RELEASE DISTRIBUTION UNLIMITED 5. MONITORING ORGANIZATION REPORT NUMBER(S) CMU/SEI-89-TR-1 6* NAME OF PERFORMING ORGANIZATION SOFTWARE ENGINEERING INSTITUTEl SEI Sc ADDRESS (City. Statt and ZIP Cod*) CARNEGIE MELLON UNIVERSITY PITTSBURGH, PA Sb. OFFICE SYMBOL (If applicabl) ESD-TR «. NAME OF MONITORING ORGANIZATION SEI JOINT PROGRAM OFFICE 7b. ADDRESS (City, Stat and ZIP Cod) ESD/XRS1 HANSCOM AIR FORCE BASE, MA a. NAME OF FUNDING/SPONSORING ORGANIZATION SEI JOINT PROGRAM OFFICE 8b. OFFICE SYMBOL (If applicabl) SEI JPO 9. PROCUREMENT INSTRUMENT IDENTIFICATION NUMBER F C0003 8c ADDRESS (City. Stat and ZIP Cod) CARNEGIE MELLON UNIVERSITY SOFTWARE ENGINEERING INSTITUTE JPO PITTSBURGH. PA TITLE (Includ Scurity Clauification) THE STATE OF SOFTWARE ENGINEERING PRACTICE: 12. PERSONAL AUTHOR(S) Watts S. Humphry, David H. Kitson, Tim C. Kass 13«. TYPE OF REPORT FINAL 16. SUPPLEMENTARY NOTATION 13b. TIME COVEREO FROM TO 10. SOURCE OF FUNDING NOS. PROGRAM ELEMENT NO. PROJECT NO. N/A PRELIMINARY REPORT 14. DATE OF REPORT (Yr.. Mo.. Day) Fbruary 1989 TASK NO. N/A 15. PAGE COUNT 31. WORK UNIT NO. N/A 17. COSATI CODES 18. SUBJECT TERMS (Continu on rvri if ncuary and idntify by block numbr) FIELD GROUP softwar assssmnt capability valuation SUB. GR. softwar procss maturity modl assssmnt instrumnt softwar procss maturity slf-assssmnt softwar nginring capability 19. ABSTRACT (Continu on rvrt if nctsary and idntify by block numbr) This is th first in a sris of SEI rports to provid priodic updats on th stat of softwar nginring practic in th DoD softwar community. Th SEI has dvlopd, and is rfining, a procss framwork and assssmnt mthodology for charactrizing th procsss usd by softwar organizations to dvlop and volv softwar products. This rport provids a brif ovrviw of th procss framwork and assssmnt approach, dscribs assssmnt rsults obtaind to dat, and discusss implications of th currnt stat of th practic for both customrs and supplirs of DoD softwar. 20. OISTRIBUTION/AVAILABILITY OF ABSTRACT UNCLASSIFIED/UNLIMITEO XX SAME AS RPT. O OTIC USERS 21. ABSTRACT SECURITY CLASSIFICATION UNCLASSIFIED, UNLIMITED 22». NAME OF RESPONSIBLE INDIVIDUAL KARL SHINGLER 22b TELEPHONE NUMBER {includ Ara Cod) (U2) c OFFICE SYMBOL SEI JPO I C CT Tf r*n

41 ^ZZT Softwar Enginring Institut Our distribution rcords show that you rcivd a copy of th Softwar Enginring institut tchnical rport ntitld Th Stat of Softwar Enginring Practic: A Prliminary Rport and baring th rport numbrs CMU/SEI-89-TR-1 and ESD- TR-89-Äfc &Of W rcntly discovrd svral data-ntry rrors in that tchnical rport. Enclosd ar corrctd vrsions of th pags containing th rrors. Plas insrt ths chang pags into your copy of th rport. Sincrly, Purvis M. Jackson information Managmnt Softwar Enginring Institut Carngi Mllon Univrsity Pittsburgh, Pnnsylvania (412)