Paper SC-005. Maintaining code is a big time-consumer and time-waster. This paper provides tips for avoiding this trap.

Size: px

Start display at page:

Download "Paper SC-005. Maintaining code is a big time-consumer and time-waster. This paper provides tips for avoiding this trap."

Ilene Conley
5 years ago
Views:

1 ABSTRACT We have all dne it. Paper SC-005 THINK Befre Yu Type Best Practices Learned the Hard Way Marje Fecht Prwerk Cnsulting Ltd, Mississauga, Ontari, Canada and Prwerk Cnsulting LLC, Cape Cral FL, USA Yu leave a meeting with a new prject n yur plate yu sit dwn at yur cmputer and start prgramming a day later and lts f cde n the screen, yu try t remember the real purpse f yur latest task. Or, wrse, yu finish yur task nly t find that it is nt what was requested. Discipline and planning aren t always easy but they pay ff in the lng run! OR Yu deliver the quick adhc and mve nt the next task n the list. A mnth later, smene wants the adhc again with just a wee variatin. WHAT??? Yu thught yu wuld never lk at that cde again? Generalizatin was the last thing n yur mind and nw yu have t dig thrugh all that cde Discipline and planning aren t always easy but they pay ff in the lng run! This presentatin fcuses n best practices t help yu minimize effrt and maximize results. Althugh the cncepts are mre prject-fcused than SAS -specific, example SAS cde will be shared s that yu can apply the cncepts mre quickly. INTRODUCTION After 30 years f reprting, analytics, and SAS usage, it seems an apprpriate time t reflect n the lessns learned. There are s many examples I encuntered (and created) including the gd, the bad, and the ugly. Each example becmes a lessn learned if yu reflect and cnsider hw yu can imprve n what yu have just dne r seen r inherited. This paper presents a series f examples t demnstrate sme best practices mst f them were learned the hard way! Fr example, we all enjy the thrill f quickly delivering results, s there is ne less task n the list. But, the quickly part can cme back t haunt us if we dn t think beynd the task t the bigger picture. Example 1: Have yu ever QUICKLY written cde assuming it will never be used again?? Is it nw 5 years later and the quickly develped SPAGHETTI CODE is still in prductin? Wrse still des the cde require YOUR time t tweak every time yu run it? And, even wrse, have thers COPIED the cde and put it int their prductin prcesses (and even given yu credit fr the cde)? Maintaining cde is a big time-cnsumer and time-waster. This paper prvides tips fr aviding this trap. Example 2: Have yu inherited a time-cnsuming prcess r cde that takes frever t get ready t use?? Des that prcess nt nly need t cntinue but it als has t be replicated? This paper explres a real-life campaign reprting prcess that cnsumed abut 5 days f time t prduce reprting n a single campaign. Multiply that by ver 5 campaigns a mnth and that desn t leave much time fr ther wrk. The revised prcess nw takes abut ½ hur per campaign. 1

2 Example 3: D yu wrk n large prjects that take s lng t cmplete, that n ne can remember why they were requested in the first place? Wrse still, d the results prvide n value when they are finally delivered? This paper prpses an iterative apprach that gets results in frnt f the requesters quickly, s that yur effrts add value early in the prject lifecycle. BENEFITS What are sme f the benefits yu might enjy after reading this paper? Reusable / repeatable prcesses Mdularity and generalizatin f cde Speed t market Cnsistent results (via cde sharing) Satisfactin that yu delivered value t the riginal team (befre they all change jbs)! THE QUICK ADHOC?! Lessn Learned: There is n such thing as the quick adhc! Yu may think that the latest request will never be expanded upn r requested again but trust me, it will. Befre yu prcess the request and crank ut the results, Write up a brief explanatin f the request. In fact, write it up as a COMMENT blck fr yur cde! Include the specifics like: What is the ppulatin f interest (ie: what subset f data is being requested? ) Is there a particular time frame? If this is a reprt, what might it lk like in terms f Rws Clumns Frmat (dynamic vs static) Metrics / measurements Quantities / statistics desired. Send the abve infrmatin back t the requester t cnfirm that yu captured their needs. Then, befre yu start writing a prgram, think abut what culd change abut this request in the future? If the request is time-specific r if it fcuses n just a subset f the available ppulatin, chances are gd that yu can generalize the cde s that it can be easily adapted t future requests. (see examples f cde generalizatin frm the suggested reading sectin including (1) and (2)) The end result? While the abve steps may add minutes t the task, yu have accmplished a few key benefits: Yu cnfirmed that yu understd the request, rather than delivering smething that is ff target. Yu built a prgram that can be adapted t future requests. Dn t Stp Here Yu DID deliver the results, but cnsider a nice next step that will nly take 5 minutes, and save time later. Build a histry f this request (and thse that will cme) that enables yu t easily lcate and rerun the results r similar requests. Fr example Utilize a numbering system fr all requests; it will help in cmmunicatins with the requester(s) and will help yu identify all cmpnents f the request including Specificatins Prgrams Data files Lgs Results. 2

3 Keep a Request Lg (inventry) fr easy referral. This might be smething as simple as a dcument r Excel spreadsheet. Or, it culd be a web-based applicatin that tracks status and prgress n all requests. When results are delivered, lg the request as Clsed and nte the date delivered. The Key: Make sure the lg is searchable by number, and by keywrd. I like t search fr similar requests t gain valuable backgrund and als t re-use existing cde mdules. File Naming Utilize a file naming cnventin that includes at least the fllwing cmpnents Request Number Versin Date in yyyymmdd frmat fr easy srting Request infrmatin such as requester r fcus f request Example: VINT07_AB_IAV_ sas (which fllws the frmat f RequestID_Requester_Prduct_yyyymmdd.sas) Extend the same naming cnventin t SAS prgrams Lgs Results Validatin files. REPEATABLE PROCESSES Lessn Learned: A repeatable prcess is nly truly repeatable if it is designed t be repeatable! I have inherited A LOT f nt-s-repeatable prcesses. The instructins have cpy and paste thrughut. And, the instructins als rely n a lt f manual searches t find the right files, cde, phrases, etc.. Suppse yu prduce reprts that use the same lgic, extracts, and reprting frmats n a regular basis fr each new prduct / campaign / regin. The prgrams and results are similar except fr changes t: Dates Inclusin / exclusin criteria Title and ftnte text Search lgic Output lcatins Reprt cntents etc. Yur current apprach is t hunt (everywhere!) fr the mst recent versin(s) yu can find f the query / reprting prgram and then after yu FINALLY find the file, yu cpy the cde and change the name. Next yu have t: Step thrugh the cde t lcate all f the required changes Start typing t input the new infrmatin Run the cde and OOPS fix the semiclns and qutes yu inadvertently vertyped Try again A better apprach is t design the prcess as truly repeatable. Fr example, similar t the adhc example, identify cmpnents f yur current cde that are reusable. If yu islate and generalize thse cmpnents and identify them as includable mdules, then yu can design yur prcesses t reduce the pain. APPROACH Identify extensible cde segments and stre them as a single dated cpy f each mdule (segment f surce cde) Dcument the input that each mdule requires Create a driver prgram that prvides macr variable input fr the current scenari and then calls the apprpriate mdules Unlike the adhc example, a repeatable prcess requires additinal and extensive testing t cnfirm that changes accmmdate all drivers that call the mdules. 3

4 EXAMPLE Assuming that yu have generalized the extractin, summarizatin, and reprting prcesses, then when yu are ready t start reprting fr a new initiative, just prvide the parameter values as input t yur standard prgrams. The belw DRIVER prgram assumes that yu have tw generalized surce prgrams that accept input and prvide the extract, summarizatin, and reprting that is required. CampaignReprtingExtract_cheque_2009_01_10.sas CampaignReprtingOutput_cheque_2009_01_13.sas Cntents f driver prgram *** Driver Prgram specify apprpriate values fr parameters; %let type = cheque; *** Campaign Type (cheque, prdchg, usage, etc.); %let prestart = 01OCT2008; *** Pre campaign perid fr cmparisn; %let prestp = 31DEC2008; *** End f Pre campaign perid; %let pststart = 01JAN2009; *** start campaign tracking; %let pststp = 30JUN2009; *** expire date - stp tracking; %let title = "January 2009 Intrductin f Lwer Interest Rates"; %let prducts = ('VRS', 'GQL', 'BGO', 'DLA'); %let cdes = ('015', '119', '214'); *** run standard extract and reprting prgrams; %include 'CampaignReprtingExtract_&type._2009_01_10.sas'; %include 'CampaignReprtingOutput_&type._2009_01_13.sas'; IMPROVED EXAMPLE The abve apprach wrks great, until yur surce cde changes. Then, yu need t find all drivers that are calling the surce, and yu need t change the date stamp in the %include. Nt fun! T avid changing the versin in 100 s f drivers, use a placehlder reference in yur drivers s that the mst current surce prgram is called. Create a prgram that references the latest surce versin: Cntents f CampaignReprtingExtract_cheque_CurrentPgm.sas *** call latest versin f surce prgram; %include 'CampaignReprtingExtract_cheque_2009_01_10.sas'; Revised cntents f driver prgram *** Driver Prgram specify apprpriate values fr parameters; %let type = cheque; *** Campaign Type (cheque, prdchg, usage, etc.); %let prestart = 01OCT2008; *** Pre campaign perid fr cmparisn; %let prestp = 31DEC2008; *** End f Pre campaign perid; %let pststart = 01JAN2009; *** start campaign tracking; %let pststp = 30JUN2009; *** expire date - stp tracking; %let title = "January 2009 Intrductin f Lwer Interest Rates"; %let prducts = ('VRS', 'GQL', 'BGO', 'DLA'); %let cdes = ('015', '119', '214'); *** run standard extract and reprting prgrams; %include 'CampaignReprtingExtract_&type._CurrentPgm.sas'; %include 'CampaignReprtingOutput_&type._CurrentPgm.sas'; 4

5 The abve apprach really wrks! A current hands-ff prductin prcess publishes ver 75 daily and 200 mnthly reprts. When reprting upgrades ccur (new metrics / revised frmats / new SAS features), they immediately rll ut t all reprts that are still running. Obviusly, this requires end-t-end testing. NEW PROJECTS Lessn Learned: Yu are NOT just a prgrammer Yu can add value and decrease time-t-market by prviding structure and cntent t the prcess. We all dread the prject-launch meeting request that has ver 20 invitees and n backgrund infrmatin. Or wrse withut any invlvement in the prcess - the that says here is what the prject team wants yu t prduce. Many flks in analyst r reprting rles frget that they have a say! D nt sit back and play rder taker. Take an active rle in new prjects by prviding structure and encuraging: Well designed specificatins Phasing s that key results are delivered quickly t get buy-in and maintain prject fcus. Cnsider sme f the prcesses detailed in the next sectin, t help with yur invlvement in prjects. KEY LEARNINGS The examples in this paper have many cmmn themes, including fllwing a prcess t maintain fcus! In general, the prcess t aim fr is: Specificatins (develp / signff) Research/planning Cde plan Cde Develpment fr a Phase Debugging Testing / Validatin Deliver Phase f results Back t cde plan fr next phase Specificatins Gathering Infrmatin by meeting with the intended business users f yur results What decisins will be made (questins answered) with what yu deliver? Requirements what d they think they want? Priritizatin f needs (t help with phasing) What d they perceive t be the mst imprtant pieces? This will help yu develp phases (fr releasing results quickly) t get early cnfirmatin and buy-in. This is als a key fr shwing early successes! Frmat and delivery channel fr results: Hw d they want t receive the infrmatin? 5

6 Develp dcumentatin f what yu perceive was requested (frmality depends n scpe f prject). Include: Phasing (s that yu can deliver results quickly, get buy-in, and keep prject spnsrs happy) Cntent f each phase Expected results frm each phase Apprximate timing fr each phase ( based n specificatins as stated) Cst estimates f each phase, if apprpriate. Exclusins / inclusins / Ppulatin f interest Metrics / Dimensins / hierarchies including perceived definitins Usability cnsideratins. Deliver the Specificatins and request Sign-Off / Apprval. Research Understand the data Indicate that develpment des nt begin until the specs are agreed t. Fr the metrics, determine ranges / expected answers befre yu get caught up in cranking ut results Identify data experts fr help with validatin, t make sure the numbers really make sense What are the definitins and calculatins required? Fr example, if # Active Accunts is a metric, hw is an active accunt defined? What is timeframe (last 30 days? Last quarter?)? What activity (transactins, payments, etc) are included? Are there data anmalies t be dcumented / cnsidered? Are there ther reprts / prjects that are similar? If yes, what is different abut the current prject? What can be brrwed? AVOID prducing differing versins f the same infrmatin! If yur results are different, explain in advance why and make sure yu have buy-in. Cde Plan Build prgram cnstruct by starting with cmments that detail yur apprach, including the steps needed t reach yur end results. This will serve as the utline fr yur prgram (much like an utline yu use when preparing a paper r presentatin) Be granular, since with a phased apprach many f the steps will cme later in the prcess, and thus the cmments will help yu remember what is next. Include the Phase numbers, s that yu avid building cde befre yu shuld. Include yur plans fr: Lcatins fr lgs, utput, prgrams in bth test and prductin envirnments 6

7 Data surces and validatin ntes (frm Research step) Details f calculatins Plan fr EXPANSION... Very few requests remain static, s acknwledge that and build yur apprach as generally as pssible. Refer t Suggested Reading (1) and (2) fr tips n generalizatin f cde. Cde develpment fr a Phase Fcus n GENERALIZING whenever yu can Fcus n MODULARIZATION. If yu are brrwing cde, d NOT cpy / paste. Instead, cnsider including the step via a macr call r %include. Refer t Suggested Reading (1) and (2) fr tips n mdularizatin f cde. Include dcumentatin f validatin effrts, s yu dn t frget what yu learned Be sure t save and review Lgs. Utilize prc printt and include date-time stamps in the file names. Refer t Suggested Reading (2) fr tips n file naming. Use Versin cntrl fr prgrams and cde segments. Als incrprate prgram versin infrmatin in the reprt ftntes. Yu need t be able t link results t the apprpriate cde and als be able t rll-back t smething that wrked Testing / Validatin Assuming that yu identified valid values, ranges, and expected results in the Research phase, then that is the first thing t review! Test yur prgrams t cnfirm that the lgic is prperly handling all pssibilities. Refer t Suggested Reading (3) fr sme testing tips. View samples f the intermediate datasets, t cnfirm that values, sequences, etc make sense. Utilize prc univariate, r similar tls t validate metric ranges and values. When changing cde, utilize prc cmpare t cnfirm that yur changes have nt negatively impacted results. Cnfirm apprpriateness f Variable frmats, lengths, and labels Dataset attributes. Deliver Phase f Results As each phase f results becmes available Freeze the cde versin(s) and d nt tuch!! Publish the results, and annunce t the requesters. Request feedback n Any data cncerns Usability f reprt frmat and cntents. While yu await feedback, yu can begin back at Cde Plan fr the next phase. 7

8 Mving t Prductin Fr a request that will run regularly (daily, mnthly, quarterly), be sure t carefully dcument the prductin prcess including When and hw des it run? (autmatically vs manual prcess) What data / infrmatin (e.g., dependencies) needs t be available befre it can run? Hw d users knw when infrmatin is ready? Cnsider autmated alerts when infrmatin is ready Cnsider Web publishing f infrmatin. Always maintain versin cntrl! Make sure that all versins f cde (and mdules) are prperly named with date f last update, s that yu can lcate and trublesht as needed. Als, maintain the mdificatins sectin f yur cmments, s that yu always knw what changes were intrduced (and by whm n what date). MY TOP 5 SUGGESTIONS Always write yur cde as generically as pssible Use a phased apprach, which includes delivering imprtant results quickly (fr buy-in) Participate in meetings d nt becme an rder taker - wrk directly with the infrmatin cnsumer Use cde cmments liberally, including starting with a cde plan (in the frm f cmments) SMILE it really is fun t deliver knwledge! SUGGESTED READING 1) Fecht, Marje and Stewart, Larry. Are Yur SAS Prgrams Running Yu?, Prceedings f SAS Glbal Frum ) Drgendyk, Harry and Fecht, Marje. Demystifying the SAS Macr Facility, Prceedings f SUGI ) Huse, Laura. Presentatin at TASS (September 21, 2007). Testing Traps: Creating Gd Test Data 101, ACKNOWLEGMENTS The authr wuld like t thank the fllwing peple Lisa Eckler fr including this presentatin at SAS Glbal Frum 2009 Laura Huse and Thmas Rthschild fr their review and valued feedback Rbert Fecht fr his wisdm and insight, especially abut the macr language cmpnents Harry Drgendyk fr being Harry CONTACT INFORMATION Yur cmments and questins are valued and encuraged. Cntact the authr via at: marje.fecht@prwerk.cm SAS and all ther SAS Institute Inc. prduct r service names are registered trademarks r trademarks f SAS Institute Inc. in the USA and ther cuntries. indicates USA registratin. Other brand and prduct names are trademarks f their respective cmpanies. 8

Assessment Field Activity Collaborative Assessment, Planning, and Support: Safety and Risk in Teams

Assessment Field Activity Collaborative Assessment, Planning, and Support: Safety and Risk in Teams Assessment Field Activity Cllabrative Assessment, Planning, and Supprt: Safety and Risk in Teams OBSERVATION Identify a case fr which a team meeting t discuss safety and/r safety planning is needed r scheduled.