IBM Research Report. A Method of Calculating the Cost of Reducing the Risk Exposure of Non-compliant Process Instances

Transcription

1 RC24930 (W ) January 8, 2010 Computr Scinc IBM Rsarch Rport A Mthod of Calculating th Cost of Rducing th Risk Exposur of Non-compliant Procss Instancs Yurdar N. Doganata, Francisco Curbra IBM Rsarch Division Thomas J. Watson Rsarch Cntr P.O. Box 704 Yorktown Hights, NY Rsarch Division Almadn - Austin - Bijing - Cambridg - Haifa - India - T. J. Watson - Tokyo - Zurich LIMITED DISTRIBUTION NOTICE: This rport has bn submittd for publication outsid of IBM and will probably b copyrightd if accptd for publication. It has bn issud as a Rsarch Rport for arly dissmination of its contnts. In viw of th transfr of copyright to th outsid publishr, its distribution outsid of IBM prior to publication should b limitd to pr communications and spcific rqusts. Aftr outsid publication, rqusts should b filld only by rprints or lgally obtaind copis of th articl (.g., paymnt of royaltis). Copis may b rqustd from IBM T. J. Watson Rsarch Cntr, P. O. Box 218, Yorktown Hights, NY USA (mail: rports@us.ibm.com). Som rports ar availabl on th intrnt at

2 A Mthod of Calculating th Cost of Rducing th Risk Exposur of Non-compliant Procss Instancs Yurdar N. Doganata IBM T. J. Watson Rsarch Cntr 19 Skylin Driv Hawthorn, NY Francisco Curbra IBM T. J. Watson Rsarch Cntr 19 Skylin Driv Hawthorn, NY ABSTRACT A mthod is introducd to masur th risk of bing noncompliant and th cost of rducing th risk by prforming intrnal audits with th hlp of automatd audit tools. Risk xposur of a businss procss is dfind in trms of th prvalnc of noncompliant procss instancs that ar subjct to pnalty. Th risk xposur can b rducd by dtcting th non-compliant procss instancs in advanc with th hlp of manual audits and automatd auditing tools. Th cost of this hybrid approach, howvr, should b kpt lss than th rduction amount of risk xposur. Catgoris and Subjct Dscriptors C.4 [Prformanc of Systms]: Computr Systms Organization, Prformanc of Systms, Subjct: Modling tchniqus Gnral Trms Managmnt, Prformanc, Dsign, Lgal Aspcts, Vrification Kywords Audit, Automatd Audit Tools, Complianc, Risk Exposur 1. INTRODUCTION Th cost of rducing th risk of bing non-compliant could run into millions of dollars for many organizations [1]. A businss that has not takn adquat stps to achiv complianc, on th othr hand, may b subjct to srious financial pnalty. On a broadr and mor practical lvl, complianc hlps organizations control oprations bttr and rmain comptitiv. Amount of invstmnt companis nd to mak to stay compliant, howvr, nd som analysis which is th subjct of this articl. Th main factor that dtrmins how much a company should invst to rmain compliant is th risk xposur factor. In this papr w limit th dfinition of risk xposur as th cost of bing non-compliant for all procss instancs that ar subjct to auditing. Th impact of procss instancs that ar not subjct to auditing is xcludd from th scop of this papr. Hnc, our dfinition of risk xposur is proportional to th numbr of procss instancs, prcntag of instancs covrd by audit and th pnalty paid for vry non-compliant cas. In ordr for th invstmnt to mak financial sns, th rturn of invstmnt must b at last positiv. In this cas, rturn is th amount that risk xposur is rducd. A company is xpctd to rduc th risk xposur at last as much as it spnds for complianc assuranc. In th absnc of procss automation softwar that can control and rcord rsourc and organizational accss (who did what and whn), complianc chck is a costly and tim consuming task prformd manually by auditors [2]. Automatd continuous auditing systms, on th othr hand, provid for an almost costfr auditing opportunity if th initial cost of building such a systm is xcludd. Such a systm can run continuously and prforms valuation for all procss instancs without adding to th cost of auditing. Whil continuous auditing systms liminat or rduc th dpndncy on audit profssionals, thy ar not infallibl. Th tools that ar built to raliz automatd continuous auditing rly on information xtraction from procss vnts and information, including -mail transactions btwn th popl within th organizations. Th xtractd information about th procsss may contain rrors and du to ths rrors th dcision on th complianc may b faulty. Morovr, th tsting of a complianc condition may rquir a lvl of txt analysis that is not yt availabl in automatd systms. Hnc, th automatd systms can prform fast and xtnsiv auditing of th intrnal control points at th cost of making mistaks. As a rsult, som complianc failurs may b missd whil som othr cass that ar compliant may b dclard non-compliant. In this articl, w introduc a cost modl for prforming intrnal auditing to rduc th risk xposur for bing non-compliant as wll as a modl to comput th associatd risk in a businss procss. Th auditing mthodology w mployd in this articl is basd on using both xprt opinion on a limitd st of procss instancs and fallibl automatd audit machins on all procss instancs. 2. AUTOMATED AUDITING TOOL An automatd auditing tool is a softwar systm that capturs information rlvant to th intrnal control points of a businss procss, puts thm into contxt and computs th complianc status for ach control point. Auditing tools rly on corrlating th data xtractd from th undrlying IT systm to th rlvant aspcts of businss control points ffctivly. Hnc, rlating th businss goals to IT lvl data constituts th cor of this

3 tchnology as dscribd in [3]. Figur 1 outlins th stp of building such a systm which starts with convrting businss ruls and rgulations into complianc goals (Stp 1). Complianc goals ar idntifid by xamining th businss ruls and dciding what action stps ar ndd. In othr words, from th businss ruls xprssd in th languag of businss popl, complianc goals ar idntifid (Stp 2). This lays th ground work for stting up IT ruls for complianc. Onc th complianc goals ar idntifid; tasks, activitis, rsourcs, artifacts and thir rlations that ar rlvant to th idntifid goal ar dtrmind and mappd onto a data modl (Stp 3). Rcording probs collct businss artifacts from th undrlying information systm and maps thm onto provnanc data (Stp 4 and 6). A provnanc graph is thn formd with th data objcts constituting th nods and th rlations among th data objcts th dgs. Th data objcts ar corrlatd by using th complianc goals and th undrlying data modl (Stp 5). Businss control points ar thn xprssd in trms of data ntitis xtractd from th procss xcution trac as graph pattrns (Stp 7). Hnc, control points provid a bridg btwn various componnts of th businss oprations and th actual data that could b consumd th IT systm. A businss control point that can b xprssd in trms of th data producd and consumd by th IT systm can b computd to chck complianc in stp 8. Root caus analysis of complianc failurs can b don by qurying th provnanc graph in stp 9. Th dottd lin shows th systm componnts involv in root caus analysis. Provnanc graph is formd by running corrlation ruls ovr th provnanc data objct. Root caus analysis is prformd by qurying th provnanc data. Spcification Businss Ruls and Rgulations.g. Sarbans-Oxly Act rquirmnts on accuracy of financial statmnts and disclosurs Dsign Dploymnt Excution Prsntation Provnanc Data Modl rlvant businss artifacts and, thir rlations Complianc Goals oprational constraints and tst procdurs Rcording Probs slctiv vnt adaptors to runtim componnts Corrlation Ruls invstigation of objct contnt and contxt Figur 1 Stps for complianc chcking Provnanc Data instancs of modld businss artifacts Control Point Pattrn instancs of complianc goals Root Caus Analysis provnanc data navigation & fraud pattrn dtction Complianc Chck automatd control point invstigation 3. SAMPLE BASED UNMANAGED BUSINESS PROCESS In ordr to masur th ffctivnss an automatd auditing tool in dtcting complianc failurs ovr an unmanagd businss procss, th following scnario is prsntd. An -hosting company manags th customr machins ovr th intrnt protctd by a firwall and rsponsibl for scuring th information assts against unauthorizd ntry. Th company scurity policy dictats that a firwall managr dfins th firwall scurity controls and nsurs on an ongoing basis that firwall policis ar implmntd using an auditabl procss. Th procss is calld Firwall Rul Rvalidation procss and it involvs in crating firwall rulsts and communicating ths rulsts to th customr and rcivs approval. Th objctivs of th procss ar to nsur both th -hosting account rprsntativs and th customrs undrstand what ruls xist in th customr nvironmnt and nsur customr is awar of xisting dviations from bst practics dfind by th -hosting scurity policy. If such a procss is not implmntd, th customr may b at risk du to no longr ndd protocols bing availabl for transit traffic, or not bing mad awar of what protocols ar in plac and rquird for support of thir nvironmnt. Th -hosting company may b hld liabl for inscur activitis, if th customrs is not informd of and signs off on th risk involvd. Figur 2 dpicts th Firwall Rul Rvalidation procss whr thr ar thr actors of th procss, information scurity advisor, account tam and customr. Figur 2 Firwall Rul Rvalidation procss flow Th rsponsibilitis of ths actors ar dfind as blow: Information Scurity Advisor (ISA): Prpars firwall rulsts according to th bst scurity practics, modifis thm as ndd, snds thm to th account tam and copy to -mail archiv databas. Account Tam Mmbr: Rcivs firwall rulsts from ISA and snds thm to customr, rcords customr rspons into th - mail archiv Customr: Rcivs firwall rulsts from th account tam, rviws thm and rplis with accptanc or chang rqusts Firwall rul rvalidation is don onc a yar. Bfor th rvalidation cycl complts, ISA asks for th firwall ruls from th ntwork administrator and chcks if th ruls ar consistnt with customr rqusts and scurity policis and mak modifications if ncssary. Onc th nw rulsts is cratd, ISA attachs th rulst to an -mail and snd th -mail to to th account tam. Onc th rulst is rady for customr rviw th procss starts. ISA attachs th -mail to th rulst and snds to th account tam, rminding that it is tim for yarly rvalidation and ask account tam 3.1 Ky-Control points In ordr to assur propr rvalidation vry yar, svral intrnal control points ar dfind. Th complianc of ky control points assurs that firwall rul rvalidation is compltd succssfully. Th dscription of ths intrnal ky control points (KCPs) ar dscribd blow: KCP1: A procss rcord xists with a copy of th mail from ISA to account tam with firwall rulst is attachd. KCP2: Th nw rulst is prpard bfor th rvalidation priod nds.

4 KCP3: A rvalidation mail must b snt by th account tam to th customr within 5 days aftr mail from ISA was rcivd KCP4: An accptanc mail rspons from customr must b within 10 days aftr th first mail snt by th account tam to th customr KCP5: Th rvalidation procss complts within 30 days of bing startd KCP6: Th rvalidation procss was compltd within th rviw intrval aftr th prior rvalidation compltion Ky control points within th businss procss in gnral hlp idntifying risks throughout th organization bfor thy caus intgrity lapss. KCP1-KCP6 ar drivn from th ruls and rgulations in businss documnts and spcifications writtn in natural languag. KCPs ar dirctly xprssd in trms of th businss provnanc ntitis which ar th nods and dgs of th provnanc graph formd during th xcution of businss oprations as xplaind in [3]. Whil our approach dos not rquir logical analysis of th businss ruls and is implmntd by mploying a simpl SQL/XPATH basd qury intrfac to th provnanc stor, th rul dvloprs ar xpctd to awar of th runtim oprational nvironmnt and th businss contxt. Hnc, it lacks th rusability and th flxibility of formal rprsntation systms. A formal rprsntation of ths ky control points ar not within th scop of this papr. Rgardlss, thr has bn a numbr of works in th litratur focusing on formal rprsntation of intrnal control points by using various rul languags [4]-[9]. Automatd complianc chcking procss is basd on analyzing all th mails in th -mail archiv and classifying thm basd who snds th -mail and for what purpos. A data modl is built that captur th rlvant aspct of th procss and according to th dscription givn in [3]. Rlationships among data itms ar xtractd by using ky control point dfinitions. As an xampl, in ordr to valuat th status of KCPs, all th -mail snt by ISAs, Account Tam mmbrs and customrs ar xamind. Txt analysis is usd to xamin th unstructurd parts of th -mails such as body and subjct; th -mail addrsss ar xtractd from to and from filds. Basd on th xtractd rlations, ach - mail is scord and labld as ithr from ISA to Account Tam or from Account Tam to Customr or from Customr to Account Tam. Th rlations btwn th -mails, thir contxt, rcivrs and sndrs ar stablishd. Th dats of th labld - mail ar xtractd to chck th complianc status of ach control point. As an xampl, th complianc of KCP1 is chckd as follows: All th -mails that ar snt by ISAs to th account tam ar procssd. Basd on th analysis of th subjct and th body of th -mails and thir attachmnts, it is dtrmind if th firwall rul rvalidation procss has startd proprly or not. 3.2 Effctivnss of an automatd audit tool In ordr to masur th ffctivnss of th automatd audit tool, w considr a mthodology that nabls dtcting th largst numbr of non-compliant instancs within a budgt constraint. Th mthodology is basd on valuating all procss instancs by using th automatd audit machin and asking xprts randomly r-valuat som of th instancs markd as compliant and noncompliant by th automatd machin. W assum that th budgt prmits th xprt valuation of only M =M 1 +M 2 cass. Hr M 1 is th numbr of cass markd as non-compliant and M 2 is th numbr of cass markd as compliant by th audit machin. This way th sampl spac that th xprts oprat is rducd. Th ffctivnss of th proposd mthodology can b masurd by comparing th xpctd numbr of non-compliant procss instancs dtctd. If th numbr is highr than what xprts would hav dtrmind undr budgt constraint without using th mthodology, thn w can conclud that th mthodology improvs th auditing procss in gnral. Th analysis of this mthodology is dtaild in [10] and it is shown that using automatd auditing tools and th mthodology dscribd abov improvs th dtction of non-complianc instancs by a factor of I as blow: 1 = ψ ) +ψ I (1) whr ψ = (1- θ )/η, θ is th spcificity and η is th snsitivity of th tool and p is th prvalnc of non-complianc instanc in th population. Equation (1) rvals that if th sum of th snsitivity and th spcificity of th tool is 1, thn thr is no improvmnt. Automatd tools ar fallibl for using xtractd information from th IT systm and thr is always th possibility of making rronous dtction. A tool with ψ = 1 may not hlp dtcting all non-compliant procss instancs, howvr, as long as I is gratr than 1, th mthodology prforms bttr than dtcting non-compliant instancs by manual auditing only. 4. COST MODEL W assum that thr is an associatd pnalty for vry noncompliant procss instanc. Hnc, if no action is takn, th xpctd pnalty is proportional with th siz of th procss instancs as wll as th prvalnc of non-complianc and th prcntag of th procss instancs auditd. Th pnalty amount xpctd to b paid for a st of non-compliant procss xcution instancs is calld th risk xposur, R and it can b xprssd as th multiplication ofω, N, p and r R = ωnpr (2) whr ω is th ratio of th procss instancs xtrnally auditd, N is th total siz of th procss instancs, p is th prvalnc of non-complianc for th population and r is th pnalty to b paid pr non-compliant instanc. As an xampl, lt s assum that %1 of 10,000 procss instancs ar non-compliant, hnc th prvalnc of non-complianc, p, is 0.01 and N is 10,000. If only %10 of th procss instancs ar auditd, i.. w=0.1, and th pnalty pr non-compliant instanc is $10,000, thn th risk xposur is found as R =$100,000. Risk xposur can b rducd by auditing th procss instancs intrnally, dtcting and liminating th causs of noncomplianc. Whil risk xposur can b compltly liminatd by auditing vry procss instanc, this may not b a cost ffctiv solution. If th cost of auditing a procss instanc is A, thn th cost of liminating th risk xposur compltly will b N tims A, sinc w do not know which procss instancs ar noncompliant and hnc w hav to chck vry on of thm. If N tims A is largr than R, thn th cost of liminating th risk xposur compltly bcoms largr than th risk itslf. Hnc,

5 budgt for rducing th xposur should b limitd and it may not b possibl to audit all procss instancs. Lt 0 < λ < 1 is th ratio of th procss instancs w can audit manually within th budgt constraint. Th cost of this partial auditing is found as λ NA. Givn this budgt constraint, th avrag numbr of non-compliant instancs that can b dtctd is found as λ Np. This is basd on th assumption that no auditing tool is usd. If an automatd tool is usd thn avrag numbr of non-compliant instancs dtctd is found as λ INp whr I is th improvmnt factor givn in quation (2) By dtcting and fixing som of non-compliant cass within th st of all procss instancs, th prvalnc of non-complianc is improvd sinc thr is lss numbr of non-compliant instancs aftr th dtctd non-compliant instancs is fixd. As a rsult, th nw prvalnc of non-complianc is found as follows: λ p' = I) = ) p 0 ψ ) + ψ λ (3) ' p is always gratr than zro. If all th non-compliant instancs ar dtctd thn prvalnc bcoms zro. Consquntly, th nw risk xposur is thn calculatd as λ R ' ω. N. p'. r = ω. N. p.(1 ). r ψ ) + ψ = (4) Th nw rducd risk xposur is obtaind by using th automatd audit tool and by mploying manual auditing on a limitd numbr of cass du to budgt constraints. Hnc, th total cost of rducing th risk xposur C, is th sum of th cost of acquiring th tool, T, and hiring auditors to audit cass. C T + λna λ. N = (5) whr A is th cost of using auditors to audit a procss instanc. Th mthodology and th tool proposd in [Rf] is usful providd that th total invstmnt mad to rduc th risk xposur, C is lss than th amount of risk xposur rducd. Hnc th cost ffctivnss of using th audit tool can b assssd by comparing C to th risk xposur rduction amount R R. ' R R = ωnλpr = IωNλpr > T + λna ψ ) + ψ whr ω is th ratio of procss instancs xtrnally auditd, N is th total siz of th procss instancs, p is th prvalnc of non-complianc for th population r is th pnalty to b paid pr non-compliant instanc. λ is th ratio of th procss instancs that can b auditd manually within th budgt constraint T is th cost of acquiring th tool A is th cost of auditing a procss instanc manually I is th improvmnt factor (6) 5. COST ANALYSIS Figur 3 shows th rduction in risk xposur as a function of th auditing tools improvmnt factor. Th graph is obtaind by utilizing quation (6) and (2) for diffrnt valus of ω, th ratio of procss instancs xtrnally auditd. For highr ω valus and highr improvmnt factors, risk xposur is rducd significantly. This is xpctd sinc for highr ω valu, risk xposur is highr. As th audit ratio incrass th possibility of dtcting mor non-complianc instancs incrass, hnc th pnalty incrass as wll. Th risk xposur is howvr rducd significantly with high prforming audit tools that dtct noncompliant instancs in advancd. On th othr hand, th cost of rducing th xposur is found from quation (5) as 45K whn p = 0.1, T= $5K, λ =0.2, N=1000 and A=$200. Figur 3 indicats that cost of rducing th xposur is lss than th rducd amount of risk as long as improvmnt factor is mor than 2.2 whn ω =0.1. Figur 3 shows how much rduction is possibl with th givn improvmnt factor of th tool and also if th cost of rduction is justifiabl Figur 3 Rduction in risk xposur as a function of improvmnt factor W dvlopd an automatd auditing tool to valuat th complianc status of th ky control points of th firwall rvalidation procss that ar dfind in th prvious sction. For th firwall rul rvalidation procss, first row of Tabl 1 shows th stimatd prvalnc of non-complianc for ach ky control point. Th scond and th third rows ar th snsitivity η and spcificity θ masurs. Finally, th last raw shows th improvmnt in dtcting non-compliant instancs by using th automatd tool obtaind by using quation (1). KCP1 KCP2 KCP3 KCP4 KCP 5 KCP 6 E(p) E(η )

6 E(θ ) I % Tabl 1. Prvalnc of non-complianc, xpctd snsitivity and spcificity and associatd improvmnt factors for th six ky control points. In ordr to dtrmin th cost ffctivnss of our tool in rducing th risk xposur, w mad th following assumptions displayd in Tabl 2 and plottd th rduction in risk xposur for ach ky control point. Risk rduction is calculatd by mploying quation (5) and (6) as wll as th p valus and improvmnt factors spcific to ach ky control point displayd in Tabl 1. ω is th ratio of procss instancs xtrnally auditd = 0.2 N is th total siz of th procss instancs = 1000 r is th pnalty to b paid pr non-compliant instanc = 10K λ is th ratio of th procss instancs that can b auditd manually within th budgt constraint = 0.2 T is th cost of acquiring th tool = 10K A is th cost of auditing a procss instanc manually = 250 Cost of rducing th risk xposur = 60K Tabl 2. Assumd paramtr valus for Figur. 4 Figur 4 shows that although th improvmnt factor is th highst for KCP1, th rduction of risk is th most of KCP6 undr th assumptions of tabl 2. This is du to having diffrnt prvalnc factors for diffrnt control points. Rduction in risk xposur ($M) KCP1 KCP2 KCP3 KCP4 KCP5 KCP6 Ky Control Points Figur 4 Rduction in risk xposur for ach ky control point 6. CONCLUSION Nw automatd auditing systms allow organizations to monitor and continuously improv complianc by providing liv tracking of businss control xcution, which in turn supports arly intrvntion and rmdiation. Automatd auditing has to b dployd as part of an intgratd auditing stratgy which ncssarily involvs th us of manual as wll as automatd auditing rsourcs. This papr proposs a cost modl for valuating th ffctivnss of thos smi-automatd auditing stratgis. W illustratd this approach in an application hosting complianc scnario. Th cost modl introducd hr hlps to dtrmin th amount of invstmnt ncssary to rduc th risk of xposur for bing non-compliant. Without a cost modl or a tool, organizations cannot calculat th rturn of thir invstmnts for complianc. 7. REFERENCES 1. Grngard, S Complianc Softwar s Bonus Bnfits. Businss Financ Magazin 2. Gartnr Simplifying Complianc: Bst Practics and Tchnology, Frnch Caldwll, Businss Procss Managmnt Summit. 3. Curbra, F., Doganata, Y., Martns, A., Mukhi, M., Slominski, A Businss Provnanc - A Tchnology to Incras Tracability of End-to-End Oprations. OTM Confrncs (1) Christophr, G., Müllr, S., Pfitzmann, B.: From Rgulatory Policis to Evnt Monitoring Ruls Towards Modl-Drivn Complianc Automation. IBM Rsarch Rport RZ 3662, IBM Zurich Rsarch Laboratory. 5. Milosvic, Z., Gibson, S., Linington, J.C and Kulkarni, S On Dsign and implmntation of a contract monitoring facility. In B. Bnatallah, ditor, First IEEE Intrnational Workshop on Elctronic Contracts, pags IEEE, Prss. 6. Govrnatori, G., Milosvic, Z A Formal Analysis of a Businss Contract Languag. Intrnational Journal of Cooprativ Information Systms 15(4), Govrnatori, G., Milosvic, Z., Sadiq, S Complianc chcking btwn businss procsss and businss contracts. In: Procdings of th 10th IEEE Confrnc on Entrpris Distributd Objct Computing. 8. Govrnatori, G Rprsnting Businss Contracts in RulML. Intrnational Journal of Cooprativ Information Systms 14(2 3), (2005) 9. L, J. K. and Sohn, My M Th Xtnsibl Rul Markup Languag. Communications of ACM, 46(5): Doganata, Y., Curbra, F Effct of using automatd auditing tools on dtcting complianc failurs in unmanagd procsss. Businss Procss Managmnt Confrnc.