Application of Analytical Hierarchy Process and Bayesian Belief Networks for Risk Analysis

Transcription

1 Volume 2 Application of Analytical Hierarchy Process and Bayesian Belief Networks for Risk Analysis A. Ahmed, R. Kusumo 2, S Savci 3, B. Kayis, M. Zhou 2 and Y.B. Khoo 2 School of Mechanical and Manufacturing Engineering, University of New South Wales, Sydney, NSW 2052 AUSTRALIA {ammar, b.kayis}@unsw.edu.au 2 Manufacturing and Infrastructure Technology, CSIRO, Melbourne, VIC 3072 AUSTRALIA {Raden.Kusumo, Mingwei.Zhou, Yongbing.Khoo}@csiro.au 3 Boeing-Hawker de Havilland, PO Box 30, Bankstown, NSW 2200 AUSTRALIA Sule.Savci@boeing.com Abstract Concurrent engineering or integrated product development is complex and challenging to manage while aiming reduction of the number of design iterations between different steps of the new product development (NPD) process. The efficiency and reliability of this concurrent engineering approach can be enhanced through risk management principles applied to project management. This paper presents a framework developed for an intelligent risk management system based on the Australia/ New Zealand Risk Management Standard (AS/NZS 4360) ane conceptualisation of a risk management tool consisting of several modules addressing establishment of the risk assessment context, risk identification, risk assessment, risk evaluation and risk mitigation planning. These modules are linked to each other through a database providing risk analysis through quantitative and multi-criteria decision tools using the Analytical Hiererchy Process (AHP) and Bayesian Belief Networks(BBN). This paper presents the conceptualisation of AHP and BBN based decision support modules in the Intelligent Risk Mapping and Assessment Systems (IRMAS TM ) for risk analysis.. Introduction The risk management process as applied to project management for new product development has been adopted from the Australian Risk Management Standard and is presented in figure (AS/NZS 4360, 999). It is as much about identifying opportunities as Copyright 2005

2 s avoiding or mitigating losses (Ahmed, Kayis, et.al., 2003). According to the standard, the governing body must develop and implement systems which: Are fully supported by management and backed up by an organisational policy and framework, which are fully communicated to all affected parties; Result in an effective program for the management of risk; Ensure that the risk management process is adopted by those in charge of projects; and Ensure that the risk management activities remain effective and relevant by way of regular monitoring and review. criteria Establish the context structure Identify risks Analyse risks leve l of risk Assess risks risk acc eptable? No Treat risks r esidual risk acceptable? Y Y accept accept Establish the context: Decide the structure of the process Develop criteria against which risk will be assessed Identify risks: Generate a list of events which can happen Consider possible causes and scenarios Analyse risks: Estimate Likelihood and consequences Combine these elements to a level of risk Assess risks: Compare level of risk against criteria Prioritise risks and identify management priorities Treat risks: Identify and evaluate treatment options Prepare and implement treatment plans No Figure. The Risk Management Process and Tasks (AS/NZS 4360, 999) The development of a risk management framework such as the one employed in the development of Intelligent Risk Mapping and Assessment System (IRMAS TM ) presents several pre-requisites, i.e. an understanding of the risk management process, the concurrent engineering (CE) product development process, the environment in which organisations conduct business and manage projects, an understanding of the overall mix of products and processes and benchmarks for such activities (Ahmed, Amornsawadwatana, et.al., 2003). 2. Risk Analysis The risk management process within a risk management tool such as IRMAS TM can be summarised as follows. Provide an interrogratory interface for determining risks (in a new product development project). Maintain a current database of risk items and organise interrogratory content based on pre-defined risk models. Determine true consequence and likelihood of risk items by modifying user input to questionnaires through an internal weighing scheme and consideration inter-relations. Inherit risk quantities from previous entities in the risk model. Highlight risk items after analysis through decision support modules. 2 Copyright 2005

3 Provide facilities to describe risk items in detail and tag mitigation actions. Provide facilities for enabling selection of mitigation options through analysis tools and a repository of lessons learnt, case studeis and industry best practices. Generate an action plan which is a list of all risk items that require mitigation and have resources allocated. The context for risk management was developed mainly through case studies in design projects implemented in a local organisation, in addition to an extensive literature review. The organisational and project environments were viewed through processes, procedures, design requirements and standard practices within the organisation. The nature of the project environment was found to be dynamic in nature due to the new product development activity as well as due different client organisations in any given project (Ahmed, et.al., 2003). Figure 2 presents the framework developed for the Intelligent Risk Mapping and Assessment System (IRMAS TM ) based on AS/NZS To maintain a universal applicability for the risk management context and to facilitate elicitation of risks, a questionnaire-based approach was adopted. The primary aim of such an elicitation process is to define particular risks in detail through identification of risk areas within project management. The input from questionnaires is tagged to additional information features such as technical, financial, schedule, organisational, etc. aspects of the project environment to provide relevance for risk assessment. A risk query mechanism was then formulated through causal diagram representation of risk items and imposed on a phase based process model to collate risk interactions and evaluate quantitative risk data and imposition of qualitative criteria on data through analysis techniques. The risk evaluation consists of decision support systems using the Analytical Hierarchy Process (AHP) and Bayesian Belief Networks (BBN) to utilize previous knowledge to evaluate the real consequence and likelihood of risk items respectively. Risks worth investigation are highlighted through analysis due to their high chance of occurring or high potential impacts, based on previous knowledge incorporated into the decision support modules. This analysis then provides a possibility for being pursued further in detailing risk items in detail, leading to the risk mitigation function. 3. The AHP Methodology The AHP concept stems from the following three principals for explicit logical analysis (Saaty, 200): Hierarchy representation and decomposition: Breaking down the problem into separate elements; Priority discrimination and synthesis: Ranking the elements by relative importance; and Logical consistency: Ensuring that elements are grouped logically and ranked consistently according to a logical criterion. The first principle of AHP concept involves construction of a functional hierarchy to decompose complex systems into their constituent parts according to their essential relationships. The elements in the hierarchy compose of clusters of system objectives, decision criteria, the attributes of criteria and alternative solutions. Each set of elements in a functional hierarchy occupies a level of the hierarchy. The top level of the hierarchy is the focus, consists of only one element i.e., the broad, overall system objective. Subsequent levels may each have several elements and because the elements in one level are to be compared 3 Copyright 2005

4 with one another against a criterion in the next higher level, the elements in each level must be of the same order of magnitude. Thus, when the elements of a level cannot be readily compared, a new level with finer distinction must be created. The second level of the hierarchy usually hosts the decision criteria, which are linked to their attributes in Level three of the hierarchy (Saaty, 200). To further examine the decision attributes, various levels of sub-attributes need to be inserted in the hierarchy, thus expending the level of hierarchy. Forming the last level of the hierarchy are the alternative solutions. The alternative solutions are linked to the decision attributes, based on which the alternative will be judged (Russell and Taylor III, 2000). A typical functional hierarchy used in AHP concept is presented in Figure3. Context Establishment Risk Model & Query Mechanism Interactive & Collaborative Interface Risk Identification Risk Analysis Risk Evaluation Prior Risk Knowledge (Repository) Qualitative & Quantitative Measures Decision Support Systems Risk Focussed Project Team Treat Risks Risk Mitigation Planning Figure 2. The framework for risk management developed for IRMAS TM based on the risk management standard AS/NZS 4360: 999 Having constructed the system hierarchy, the AHP concepts are used to analyse the priorities of elements in the hierarchy in terms of their contribution to the focus of the hierarchy (system objective). The element priority analysis begins by making pair wise comparison that is to compare the elements in pairs against a given criterion in a matrix format. The pair wise comparison process starts at the top of the hierarchy (system objective) that will be used for making the first comparison. Then from the level immediately below, take the elements to be compared (criteria to criteria i ). To populate pair wise comparison matrix, the process of judgment of the relative importance (RI) of one element over another 4 Copyright 2005

5 with respect to the property is utilized. The judgments are presented on a - 9 scale as follows (Saaty, 200): (a) Equally important ; (b) Moderately more important 3; (c) Essentially more important 5; (d) Strongly more important 7; (e) Extremely more important 9; (f) Intermediate values between two adjacent judgments are 2,4,6,8. A typical pair wise comparison matrix is presented in Table. Decision Objective Criteria Criteria 2 Criteria Criteria J Attributes Interdependency Attribute Attribute 2 Attribute Attribute J Alternative fulfillment Alternative Alternative 2 Alternative Alternative i Figure 3. The AHP functional hierarchy With the pair wise comparison matrix established, the process of elements priority analysis proceeds to integrate the pair wise comparison judgments, to determine the overall estimate of relative priorities with respect to system objective. The integration process involves the evaluation of the Vector of Priorities (VP) that designate the relative ranking of the dependent decision attributes for the objective in consideration. To translate the pair wise comparison matrix into VP of attributes, the following steps are prescribed:. Raise the pair-wise comparison matrix to powers that are successively squared each time; The rows sums are then calculated and normalized; Table. Pair wise comparison of Decision Criteria (Where RI is relative importance). Objective Criteria Criteria 2 Criteria 3 Criteria i Criteria RI 2 RI 3 RI i Criteria 2 / RI 2 RI 23 RI 2i Criteria 3 /RI 3 /RI 23 RI 3i Criteria i /RI i /RI 2i /RI 3i The steps described above are repeated until the differences between these sums in two consecutive calculations is smaller than Copyright 2005

6 The results of all the levels of the hierarchy are combined to obtain the Overall Vector of Priority (OVP). For illustration, the computation process of OVP is presented in Table 2. AVP in Eq. is the attribute s vector of priorities with respect to particular criteria, AOVP in Wq.2 is the attribute s overall vector of priorities and VP is the vector of priority. Table 2.Overall Vector of Priority computation process Overall Criteria Criteria 2 Criteria 3 Criteria i Vector of Priorities Attribute - AVP 2 AVP 3 AVP i AOVP Attribute 2 AVP 2 - VP 23 AVP 2i AOVP 2 Attribute 3 AVP 3 AVP 32 - AVP 3i AOVP 3 Attribute i AVP i AVP i2 AVP i3 - AOVP i Where, AVP = AttributeVP CriteriaVP () ij i j AOVP i = AVP in i n= (2) The element prioritisation procedure is repeated for the decision alternatives identified (Figure 3), to evaluate the OVP of decision alternatives that meets the decision attributes for the slated objective. If the alternatives do not depend on how many and what kind of other alternatives there are, the vector of priorities of the alternatives need to be set in the ideal mode, by dividing the intensity weight with the largest intensity s for each alternative. The intensity s weight is calculated by multiplying the vector of priorities of the intensity with the overall vector of priorities of the corresponding risk item. 3. Appication of AHP in risk analysis The requirement for the AHP based decision support module arose in IRMAS TM due to the need to determine the true consequence of a risk item from user input in the questionnaire based expert interview facility. This meant that the user input for consequence of a risk item needed adjustment based on an internal weighing scale taking into account all previous experiences and best practices. The pairwise comparison scheme in AHP is ideally suited to work out the relative importance of a risk item and providing an account for multiple criteria in evaluating important consequences. AHP takes into account the perceived importance of multiple criteria as pairwise comparisons by domain experts on a scale of to 9 and then mathematically determines the stable points in the domain that can justify the perception. In engineering applications AHP is most commonly used in analysing feasible design options during the conceptual product development process when several design options need to be considered on a rational basis. 6 Copyright 2005

7 4. The BBN Methodology The Bayesian theorem is founded on subjective interpretation, where a probability A) is understood as a measure of the decision maker s lack of knowledge or uncertainty about the occourrance or non occourrance of an event A. This notion of probability is therefore a personal belief and reflects the degree of belief a person has at a particular point of time. Hence, unlike the relative frequency interpretation (i.e. frequentist and Fisherian), Bayesian theorem does not prescribe a general set of procedures by which to determine the number A). This is because Bayesian theorem interpretation of probability is derived from a set of knowledge on the cause of event A. Having derived the probability from knowledge or degree of belief, Bayesian theorem allows A) to be computed using incomplete information (i.e. uncertanities), which is apparent in most decision making problems (Das, 999). Bayesian theorem stems from conditional probability theory, where the basic expression of an event A given event B is presented in the form, A, B) P ( A B) = B) (3) When the joint probability of A,B) is considered, then from the cumulative property of logic, it follows that the following two propositions are the same: (A,B) = A and B are both true (B,A) = B and A are both true Thus, they must have a truth value and same probability not matter what the state of knowledge is. Hence A,B) = B,A). Applying this relationship to the conditional probabilities results in the Bayes formula, Posteriorα Likelihood Prior or A B) B) B A) = A) (4) Where, B) is the initial belief that event B will occur, also called the prior probability. A B) is the belief that some event A will be found once the event B has actually occurred. A) is the belief that event A will be found to be true under general circumstances and B A) is the belief that event B will occur after some evidence of event A is known to support or deny event B (posterior probability). Consider a simple BBN network as represented in Figure 4, the network structure provides the conditional probabilities Y X), E Y) and F Y). The propagation of evidence can be analysed in two modes, i.e. propagaion along the links and propagation against the links. In progatation along the links mode, if evidence arrives such that X has assumed the value x, the probability distribution over the states of Y are given by Y x ), which can directly be calculated from the conditional probabilities. Similarly, E x ) is calculated by the network conditional probabilities using the following mathematical expression, P ( E x) = E Y ) Y x) y (5) Thus, in propation alon the links, when the prior distribution is fixed over the node X, depending upon the prior knowledge, this distribution propogates along the links to fix the prior probability distribution over all other nodes. 7 Copyright 2005

8 Figure 4. Representation of a simple BBN. Alternatively, in propagation against the link, if evidence arrives such that E has assumed a value e and F has assumed a value of f, then these evidences update the probability distribution over the states of X according to the dictates of the Bayesian theorem as presened below, X e, f ) = y yx f Y ) e Y ) Y X ) X ) f Y ) e Y ) Y X ) X ) In Eq. 6, the probability X e,f ) is the posterior probability and X) is the prior probability. Thus, in propagation against the links, BBN are constructed such that the root nodes (such as X in Figure 4) represent variables that are not directly observable. When observable evidence arrives, these are channelled through the intermediate nodes to the root nodes. Probability distribution is then updated through the dictates of the Bayes theorem. (6) 4.2 Application of Bayesian belief networks in risk analysis The requirement for the BBN based decision support module arose in IRMAS TM for the need to determine the true likelihood of a risk item from user input into the questionnaire and adjust the likelihood based on other connected risk events and prior probabilities of these events. BBN theory by its characteristics was found to be well suited for dealing with relationships between risk items that are not explicit and uncertainities exist in relationships because they are not definite. Typically all available information, like observation of symptoms are inputted into the BBN and then it is interrogated to find the cumulative effects of the observations and/or other uncertain ententies in the system that have no logical basis for measurement. BBN provide a methodology for uncertainty analysis where probabilistic models represent provide a simplistic expression of the real world problem or where the human intuitive approach is too broad to be feasible as a framework for decision analysis. BBN is a confluence of the two approaches, where the Bayes s theorem provides an adjustment for our belief in a prior probability and a belief probability is determined based on observations made to the Bayesian network model. Bayesian belief networks are widely applied in uncertainty analysis where Monte Carlo simulation techniques and other statistical methods are either not applicable or are too cumbersome to represent as models. 5. The AHP and BBN decision support modules For the facilitation of risk analysis, eight categories of risks were defined in IRMAS TM, also called risk factors and represent broad areas of risk in a particular phase of a new product development project. Risk items were then defined within each risk factor, narrowing down the scope of risk events. Interactive relationships were then expressed between risk items 8 Copyright 2005

9 through causal diagrams. This also facilitate the design of questionnaires, ensuring that the general risk information gathered is sufficient for the purpose and not repetitive in nature. Once the risk quantities are inputted by the user, the challenge then is to modify these inputs are biased towards risk items that need a higher priority as determined by the use of the concurrent engineering design philosophy, prior organisational experiences, best practices and company standards. This in turn, lead to highlighting of high risk items and make up the first pass of risk analysis in a particular phase. Detailed risk analysis follows where trivial risk items are demoted in the order of priorities and the important ones are detailed for mitigation actions. The AHP and BBN decision support modules facilitate risk analysis leading to a risk items list. 5. Application of AHP in IRMAS TM Definition of risk items in particular phases based on the concurrent engineering design process, the specific domain of application to a local organisation and the clumping of risk items under risk factors meant that a two level hiererchy is appropriate to accommodate the AHP analysis technique in IRMAS TM. The determination of relative importance of risk items according to their consequence with respect to other risk items under the same risk factor is a reasonable construct to rank risk consequences in the second level of analysis. The first level of hierarchy is that of the risk factors themselves, which vary from project phase to project phase. For example, in the detailed design phase technical risks, schedule risks and network risks dominate, because this phase determines the number of design iterations that are sufficient to satisfy design specifications. Ideally, a sound understanding of the design specifications is promoted through networking amongst the multi-functional design team members and the detailed design task schedule is maintained. Common understanding of technical specifications and ample team interaction ensures that desing iterations minimal. Similarly, the same logic can be applied to the testing phase where all risks are usually significantly low, unless unqualified materials have been used, testing facilities are not available or the product was under designed. Hence, this phase represents limited technical, financial and resource risks. The pariwise comparison in AHP, then elucidates how much more important one risk factor is in comparison to the others in a particular phase. 5.2 Application of BBN in IRMAS TM The physical form of the Bayenesian belief networks is the same as the causal diagram for that phase with the addition of entities for user input in a backward chaining mechanism. Some risk events are inputs to risk factors while others are outputs from that phase. Entities that inherit risk from previous phases are also added. These relationships are defined in BBN and also their prior probabilities are are attached. The prior probabilites for relationships are not always explicit and usually is the expert knowledge that humans use intuitively for making decisions. On the whole, the cumulative effects of relationships among risk events are very complex and not possible for the human mind to comprehend the flow-on effects of risk events due to likelihood of occourrance on one another. BBN also allows definition of speculative relationships between risk events in the form of probabilities of a risk item due to its relationship with one or more risk items. The Bayes theorem then determines posterior probabilities based on observations that are made to the network such that flow-on effects are taken into account. 9 Copyright 2005

10 6. Conclusion This paper presents conceptualisation of risk analysis decision support modules in Intelligent Risk Mapping and Assessment System (IRMAS TM ). The techniques of Analytical Hierarch Process and Bayesian Belief Networks were used in a novel way to account for prior organisational knowledge in comparison to other knowledge based techniques such as expert systems and statistical analysis tools. The applications described in this paper are at a confluence of rule based systems and numerical techniques. The methodologies found in Commercial Off The Shelf (COTS) software for risk identification and analysis typically fell into the groups of interrogative or Monte Carlo Analysis. Interrogative tools deduced risks by finding whether a benchmarked practice is being followed and then analyse its magnitude as a score, while Monte Carlo based COTS tools provided graphical tools for modelling statistical distributions of numerical data pertaining to a risk event. In contrast to these techniques in IRMAS TM, the interrogative approach was adopted for risk analysis through a unique application of AHP and BBN techniques to account for consequence of risk items based on prior knowledge and accounting for influence on each other for determining likelihood of risk events. Acknowledgements The authors wish to thank the Cooporative Research Centre for Intelligent Manufacturing Systems and Technologies (CRCIMST) for providing support for the development of IRMAS TM under Sub-Program 4.2. References AS/NZS 4360 (999), Risk Management Standard, Standards Association of Australia: Sydney, NSW. Ahmed A., Kayis B, Zhou M., Khoo Y.B. and Kusumo, R. (2003), A Risk Management Approach for Concurrent Product/Process Design and Development, Proccedings of International Business Information Management Conference, December 6-8, 2003, Cairo, Egypt, p Ahmed A., Amornsawadwatana, S. and Kayis B. (2003) A Conceptual Framework for Risk Analysis in Concurrent Engineering, Proceedings of the 7th International Conference on Production Research Blacksburg, Virginia (R.6 Paper No. 86). Das B. (999), Representing Uncertainty Using Bayesian Networks, Publication DSTO-TR- 098, Department of Defense, Defence Science and Technology Organisation, Salisbury, Australia. Russell R.S. and Taylor III B.W. (2000), Operations Management, Upper Saddle River, NJ: Prentice Hall Inc, pp Saaty T.L. (200), Decision Making for Leaders The Analytic Hierarchy Process for Decisions in a Complex World, 3 rd Edition, RWS Publications, USA, Copyright 2005