An Empirical Analysis of Software Productivity

Transcription

1 An Empirical Analysis of Softwar Productivity Martin Shpprd, Carolyn Mair, Pkka Forslius Abstract Th aim of our rsarch is to discovr what factors impact softwar projct productivity (masurd as function points pr hour) using ral world data. Within this ovrall goal w also compar productivity btwn diffrnt businss sctors and projct typs. W analysd a data st of mor than 661 projcts that hav bn collctd by STTF from a numbr of Finnish companis sinc Ths projcts ar quit divrs typ (nw and maintnanc projcts), in trms of siz (6 to ovr 5000 function points), ffort (55 to ovr prson hours), application domain and implmntation tchnology. Thr ar thr main findings. First productivity varis normously btwn projcts. Scond, projct typ has limitd influnc on productivity. Third, application domain or businss ara has a major impact upon productivity. Bcaus this data st is not a random sampl gnralisation is somwhat problmatic, w hop that it contributs to an ovrall body of knowldg about softwar productivity and thrby facilitats th construction of a biggr pictur. Kywords Projct managmnt, projcts, softwar productivity, mpirical analysis. 1. Introduction Dfining productivity is problmatic for a numbr of rasons [1]. Howvr, it rmains an important aspct of softwar projct managmnt and a prcursor to ffctiv cost prdiction. For th purposs of this rsarch w dfin productivity using Function Points (FP) as a masur of output and prson hours of ffort as th input. Our goal is to invstigat som of th mor important factors which impact softwar projct productivity using ral world data from Finnish softwar projcts. This work is motivatd by prvious work of Maxwll and Forslius [2]) who found that th particular company and th businss sctor wr th most important variabls in xplaining productivity. Mor rcntly, Prmraj and Shpprd [1] found that th most significant factors in xplaining productivity, wr in dcrasing ordr of importanc, Company, Businss Sctor, Yar, and lastly Hardwar. Th rmaindr of th papr is organisd as follows. Sction 2 dscribs th data and outlins brifly ruls for rmoving and classifying th data prior to inclusion in th databas. In Sction 3 w analys th data st in trms of dlivry rat (FP/hour). W conclud in Sction 4 with a discussion of th analysis and commnt on th factors which impact softwar projct productivity. 2. Th Finnish data st Th data st usd for analysis in this papr is drivd from th 2006 rlas of th Exprinc data st, also known as th Finnish data st. This data st is a commrcial initiativ by Softwar Tchnology Transfr Finland (STTF) to provid support for softwar dvlopmnt organisations for both projct cost stimation and productivity analyss. An annual subscription allows organizations to accss th data by mans of Exprinc Pro16. This tool is also usd to submit data and nsurs standardization of variabls. 16 Th Exprinc Pro tool is availabl at 283

2 Bfor bing addd to th databas, th projct data ar carfully assssd by xprts at STTF. Th databas compriss nw or dvlopmnt projcts drivd from a wid rang of businss sctors, platforms and dvlopmnt tchnologis. Projct data includ siz (FPs), ffort and a rang of factors that charactriz th typ of projct, th dvlopmnt circumstancs, dvlopmnt and targt tchnology. Th possibility xists to submit data on 102 variabls, although som variabls ar difficult to analys bcaus of th significant proportion of missing valus. A fullr dscription of th data st may b found in Maxwll and Forslius [2, 3] and furthr information about Exprinc Points. Th data st comprisd 737 projcts compltd btwn 1978 and Th organisation rsponsibl for collcting th data thn rmovd 54 projcts on th grounds of data quality concrns. W thn rmovd a furthr 22 projcts on th grounds of implausibl dlivry rat. This is discussd furthr in th nxt sction. 3. Analysis Tabl 1: Data claning of th Finnish data st Data st Count Prcntag Complt data st % Low quality data rmovd % Implausibl data rmovd % 3.1. Dlivry Rat stats Whilst th main thm of th papr is productivity w chos to us its invrs i.. Dlivry rat (hour / Function Points) as th indicator of productivity sinc w bliv this is asir to visualis. Essntially productivity gnrally lads to small fractions of an hour whras dlivry rat yilds valus gratr than unity. Howvr, on can b convrtd to th othr mrly by taking th rciprocal As prviously discussd bcaus of th larg rang (mor than thr ordrs of magnitud) in dlivry rats w rmovd outlirs, as suggstd in Prmraj and Shpprd [1]. To do this w, in consultation with STFF staff, considrd valus with xtrmly short or long dlivry rat to b implausibl as thy rprsnt unrportd or misrportd factors. Th valus which ar considrd xtrm ar <1 FP/hour and >30 FP/hour. Thr wr 22 cass with implausibl data for dlivry rat, ths wr rmovd. Summary statistics for th rmaining 661 cass ar givn in Tabl 2. Tabl 2: Summary statistics for 661 projcts (projcts with outlirs data hav bn rmovd) Variabl Man Mdian Min Max Dlivry rat (FP/hr)

3 DlivryRat Figur 1: Dlivry rat for 661 projcts (projcts with outlirs data hav bn rmovd) 3.2. Naïv modl of siz and ffort Th first stp was to construct a simplistic rgrssion modl using just siz in function points as th indpndnt variabl (Siz). From th scattr plot (Figur 2) it can b sn that thr is a positiv rlationship such that as siz incrass thr is a tndncy for ffort to also incras. Howvr, it can also b obsrvd that thr is a substantial amount of scattr about th rgrssion lin. Th cofficint of dtrmination (R2) is 63.5% (s Tabl 3) which indicats that th naïv modl can prdict lss than two thirds of th ovrall variation in ffort. Th othr intrsting fatur of this modl is that th intrcpt is not significantly diffrnt from zro, i.. th 95% standard rror (s..) whn combind with th cofficint ncompasss zro. Tabl 3:Rgrssion modl statistics Cofficint s.. of Coff Probability Intrcpt Siz

4 60000 E f f o r t ( h ) Siz Figur 2: Scattr plot of siz and ffort Sinc th naïv modl only has a modrat fit to th data w nxt considr th impact of projct typ upon dlivry rat Analysis by projct typ Originally th projcts in th data st wr classifid into 8 typs (Tabl 4). Howvr, for as of analysis, ths wr rgroupd into 4 typs: Enhancmnt, Maintnanc, Nw Dvlopmnt (incorporating both product and tailord) and Othr (incorporating th rmaining typs) (rfr to Fig. 3 for a pi chart dscribing th rlativ proportions of ach catgory). Combining th projcts into 4 typs rsultd in nw dvlopmnt projcts accounting for 76% of th data st, maintnanc projcts 13%, nhancmnt projcts 9% and othr 2%. Th group Othr projct typ is composd of projcts which do not fit into mainstram catgoris. 286

5 Tabl 4: Projct typ catgoris and frquncis Projct typ Count Mrgd projct Rvisd count % typs Annual maintnanc 1 Othr 2.6 Intgration 2 Othr Convrsion 6 Othr Othr 8 Othr 17 Nw dvlopmnt product 12 Nw dvlopmnt Enhancmnt 70 Enhancmnt Maintnanc 96 Maintnanc Nw dvlopmnt tailord 466 Nw dvlopmnt Projct typ Enhancmnt Maintnanc Nw dvlopmnt Othr Figur 3: Projct typ (combind catgoris) Tabl 5: Summary statistics for dlivry rat for ach catgory of projct typ Projct typ Count Man Mdian Lowr quartil Uppr quartil Enhancmnt Maintnanc Nw dvlopmnt Othr

6 D l i v r y R a t Ent Mnt Nd Othr Projct typ Figur 4: Sid by sid box plots for Projct typ dlivry rats [4] Nxt w conductd an analysis of varianc for th diffrnt Dlivry Rats for ach projct typ to s if any of th diffrncs rvald by th box plots (Figur 5) wr actually significant. Th computd F-ratio is with a tabld probability of This suggsts that Projct typ has a significant impact upon dlivry rat. Howvr, w nd to dtrmin mor spcifically whn this th cas so w conduct pair wis Mann-Whitny tsts to compar ach typ. From this w find that th only significant diffrncs involv th group typ Othr (Nw dvlopmnt p=0.004; Maintnanc p=0.006; Enhancmnt p=0.002) Thr is no significant diffrnc btwn th dlivry rat for nhancmnt and nw dvlopmnt projcts. Howvr, thr is a significant diffrnc btwn projcts groupd into Othr projcts and th othr thr typs. On might xpct som diffrnc in dlivry rat to b prsnt among th projct5 typs. Howvr, this was dtctd only btwn Othr projcts and th rmaining thr projct typ groups. Th implications ar that it is vry difficult to hav an ffctiv univrsal productivity modl and that th full divrsity of projct typs must b takn into account Analysis by businss typ Having stablishd that thr is no significant diffrnc btwn projct typs, w now considr th impact of businss typ on dlivry rat. Othr studis hav rportd this factor to b highly influntial [4]. Th numbr and proportion for ach businss typ is shown in Figur 6. Businss typ Othr now includs Construction. 288

7 Businss Banking CT Insuranc Manufacturing Othr Public admin Rtail Figur 5: Pi chart of Businss Typ proportions Tabl 6: Businss typ catgoris and frquncis Businss typ Count % Banking CT Insuranc Manufacturing Othr Public admin Rtail Tabl 7: Summary statistics for dlivry rat for ach catgory of Businss Typ Businss typ Count Man Mdian Lowr quartil Uppr quartil Banking CT Insuranc Manufacturing Othr Public admin Rtail

8 30.0 D l i v r y R a t Bng CT Ins Mng Othr Pa Rtail Businss Figur 7: Sid by sid box plots for Projct typ dlivry rats Nxt w conductd an analysis of varianc for th diffrnt Dlivry Rats for ach businss typ to s if any of th diffrncs rvald by th boxplots (Figur 4) wr actually significant. Th computd F-ratio is with a tabld probability of < This suggsts that businss typ has a highly significant impact upon dlivry rat. Again, w nd to dtrmin mor spcifically whn this th cas so w conduct pairwis Mann- Whitny tsts to compar ach typ. From this w find that th businss typs sm to fall into two groups (Banking, CT and Insuranc) which significantly mor productiv than th rmaining businss typs (Manufacturing, Othr, Public Admin and Rtail). Th implication hr is that it is bttr to tailor a spcific cost modl to th individual businss typ rathr than adopt a on siz fits all stratgy. 4. Discussion In this papr w hav dscribd a prliminary analysis of softwar productivity (dlivry rat). Th principl findings ar that vn aftr considrabl fforts to rmov rogu data thr is still a grat dal of variability in productivity. Consquntly, naïv modls that sk to find a linar rlationship btwn siz (masurd in function points) and ffort only xhibit modrat (although highly significant) fit with th data. Nxt w obsrvd that whilst to our surpris thr wr limitd diffrncs btwn maintnanc/nhancmnt projcts and nw dvlopmnt thr wr a numbr of Othr projcts that did not fit into this classification that displayd vry diffrnt productivity bhaviour. On th othr hand w found that businss typ or application domain had a major impact upon productivity. For this rason w bliv som caution must b xrcisd whn using on siz fits all styl of cost modl. 290

9 Ths rsults show som similarity with othr studis, particularly th larg variability in productivity and th impact of diffrnt businss typs (s for xampl Maxwll and Forslius [2]). Clarly it would b intrsting to mak a mor systmatic comparison particularly with othr wll-known rpositoris such as th ISBSG data st. Lastly, is th qustion so what or how can w gnralis from ths rsults? A clar difficulty is that this is a non-random sampl from a rathr ill dfind population of softwar projcts. It is likly that th projcts com from companis who ar intrstd in masurmnt and procss improvmnt. This would likly mak thm bttr than lvl on in trms of th CMM modl [5]. In othr words w ar probably daling with th top quartil of projcts from non-safty critical, non mbddd systm typ applications. Howvr, w hop that as mor studis such as this ar publishd it nabls a largr pictur to b stablishd and gratr confidnc to b placd in th findings. 5. Rfrncs [1] R. Prmraj, M. Shpprd, B. Kitchnham, and P. Forslius, "An Empirical Analysis of Softwar Productivity Ovr Tim," prsntd at 11th IEEE Intl. Softw. Mtrics Symp. (Mtrics05), Como, Italy, [2] K. D. Maxwll and P. Forslius, "Bnchmarking softwar dvlopmnt productivity," IEEE Softwar, vol. 17, pp , [3] K. Maxwll, L. Van Wassnhov, and S. Dutta, "Prformanc valuation of gnral and company spcific modls in softwar dvlopmnt ffort stimation," Managmnt Scinc, vol. 45, pp , [4] K. Maxwll, L. Van Wassnhov, and S. Dutta, "Softwar dvlopmnt productivity of Europan spac, military and industrial applications," IEEE TSE, vol. 22, pp , [5] M. C. Paulk, B. Curtis, M. B. Chrissis, and C. V. Wbbr, "Capability maturity modl, vrsion 1.1," IEEE Softwar, vol. 10, pp ,