1David S. Bowles, Ph.D., P.E., P.H., F.ASCE. Professor, Utah State University and Principal, RAC Engineers & Economists, Utah, U.S.A.
ABSTRACT. Portfolio2 risk assessment (PRA) can now be considered to be a standard of practice in Australia. In this paper various advances in the state-of-the-practice for performing PRA's are reviewed, including some pitfalls and limitations. The uses of PRA outcomes by owners are discussed, along with some ways to improve the value derived from PRAs. The challenges that are common in seeking to achieve an integration of the PRA process into the owner's dam safety management program and with broader business processes, and the importance of targeting PRA outcomes to an owner's specific business needs, are emphasised.
Soon after corporatisation of many Australian dam operating organisations, these new dam owners initiated assessments of the safety of their dams. The underlying approach was typically that recommended in ANCOLD's 1994 Dam Safety Management Guideline. In addition, some owners experimented with risk assessments for individual dams. This took place partly in response to the publication of ANCOLD's 1994 Risk Assessment Guidelines. If well conducted, these individual risk assessments typically provided valuable insights into the technical and business implications of safety issues for the study dam; but as a result, owners became concerned about what they did not understand about the other dams in their portfolios. This concern, combined with ANCOLD's shift to using risk-based approaches in its guidelines, and the need for these new corporations to develop a portfolio-wide perspective of the business risks of dam ownership, gave birth to the introduction of portfolio risk assessment in 1996 (Bowles 1996). Now, initial PRAs have been completed, or are on going, for most major Australian portfolios of dams. PRA is also being used in the USA and applications are planned in several other countries. This paper reviews the current state of the practice of PRA, including some recent advances, and the uses of PRA outcomes by owners and others. Various pitfalls and limitations are highlighted and ways to improve the value derived from PRAs are discussed.
The management of dam safety includes both: 1) on-going (or recurrent) activities, such as monitoring and surveillance, investigations, and emergency preparedness planning; and 2) the construction of remedial measures (fixes) and supporting investigations. In the past decade, many private and public dam owning organisations have shifted the point at which dam safety decisions are made from engineering departments, which relied heavily on engineering standards and judgments that were based on current practice and the personal values of individual engineers, to senior management and boards of directors or government bureaucrats who frequently have little or no dams engineering background. This shift has been succinctly captured by Brian Haisman (1998) in the phrase, "from engineers on top to engineers on tap". It presents engineers with the challenge of effectively advocating for dam safety investment in a competitive business environment.
In addition to engineering inputs, the new decision paradigm involves manifold business-related considerations, such as the following: customer service; business criticality and meeting contractual obligations; competition for business capital; effects on business value and return to the shareholder(s); legal concerns about due diligence, duty of care and possible negligence; emergency preparedness planning; business contingency planning; loss financing and insurance; and corporate image.
Through the systematic PRA approach, technical and business understanding of dam safety issues can be improved. PRA provides a risk profile for existing dams. It can also provide the basis for a dam safety improvement program, including remedial measures and supporting investigations. Other outcomes can be used to strengthen the owner's on-going dam safety management program, and to provide inputs to various business processes, with the goal of better integrating dam safety into the owner's business. As a result, the process often establishes a case for funding dam safety improvements, resulting in more rapid and cost effective risk reduction. A PRA should be a "living document" (Bowles et al 1997), which is periodically updated to incorporate new or improved inputs and revised evaluation or prioritisation criteria.
The PRA process builds on traditional standards-based engineering evaluations. Although it is risk based, PRA does not commit an owner to using risk assessment for selecting the extent of remedial measures. Variations in the PRA process have been developed for large portfolios using screening or sampling procedures (Watson et al 1997). Although generally less costly to implement than PRA, index prioritisation approaches are more limited in the scope and usefulness of their outcomes, especially in today's business decision environment. Many index approaches violate the risk metric when applied to low probability-high consequence dam safety risks, which can distort resulting priorities. Also, they are not designed to identify the most cost effective and rapid risk reduction strategies. Nevertheless, if carefully calibrated and applied, index approaches may serve a useful screening role for large portfolios and for more routine maintenance work, which is associated with higher frequency and lower consequence risks.
Overall Process
PRAs should be conducted through a close partnership between engineers, a PRA expert, the owner's dam safety manager and decision-makers, and other stakeholder representatives. The process comprises the following major parts (Figure 1): 1) identification of decision framework, 2) engineering assessment, 3) risk assessment, and 4) prioritisation. Each part is discussed in a separate subsection below.
It is recommended that, in a PRA, fixes be formulated as "separable construction upgrade packages" (SCUPs) rather than failure mode by failure mode fixes or by lumping all fixes for a single dam. Each SCUP should be formulated at a reconnaissance level of detail, and with the intent of meeting current engineering practice, including standards and guidelines. It is also useful to consider staged construction if it is a practical possibility. By so doing, each stage of a fix can be prioritised separately, a more rapid rate of portfolio risk reduction is likely to be achieved, and stronger justification to decision-makers will likely result. Non-structural measures and decommissioning should be considered, as appropriate. In at least one case, occupational health and safety (OH&S) upgrades and some O&M projects were also considered. Investigations needed to better understand potential dam safety deficiencies, or to support remedial design work, should be identified, and can be prioritised in a coordinated manner with fixes, as described in the subsection on "Prioritisation".
Figure 2 depicts the flow of information inputs into a PRA from activities that already exist in most dam safety programs (e.g. inspections, design reviews, etc.). It also shows the addition of specialised information, which is needed to complete a PRA (e.g. inundation modelling and consequences estimation). By identifying and establishing these information flows, maximum use can be made of available information to conduct and update an initial PRA. Engineering and risk assessments for initial PRAs are typically based primarily on available information and engineering judgement, with limited additional analyses. Hence, it is important that competent and highly experienced dams engineers provide engineering inputs in a framework that is overseen by an expert PRA facilitator. Supporting studies are usually performed at a reconnaissance level. They should yield consistently developed best estimates, although it is likely that uncertainty analyses will be introduced into PRAs in the foreseeable future.
It is important to recognize that for the level of analysis performed for a PRA, especially an initial PRA, evaluations against engineering and risk criteria can be considered to be only indicative. More detailed analyses will usually be necessary to achieve an adequate degree of defensibility to make "sign off" level engineering and risk evaluations.
Decision Framework
Understanding the decision framework is important for identifying PRA outcome "targets" that will benefit the owner's dam safety program and related business processes (represented by the right side of Figure 2), and other stakeholders. Some dam owners focus only on externally imposed requirements, such as those of a regulator or ANCOLD Guidelines, without giving adequate consideration to internal considerations, such as business criticality or alternatives for replacing project functionality, which might be less costly than dam safety rehabilitation (e.g. dam decommissioning). It is important that the outcome targeting process, whereby the PRA outcomes are designed to meet the owner's and other stakeholder's information needs, are begun at the outset of the PRA process. It is also important that the PRA process is adapted to meet the specific information needs associated with each portfolio of dams rather than producing a standard set of outcomes.
Engineering Assessment
The engineering standards perspective can provide a valuable input for decision-making. Engineering assessments indicate whether portfolio dams are expected to meet current engineering practice, including standards and guidelines, such as ANCOLDÕs. They serve to initially identify fixes that will be evaluated using risk assessment, and investigations that are needed to achieve adequate confidence in engineering assessments (see Figure 1).
A rating system is a useful means of summarising the results of engineering assessments and communicating them to decision-makers. The rating system developed and refined by RAC Engineers & Economists (SMEC/RAC 1995, Bowles et al 1998) is designed to minimise the conservative biases that are often associated with the limited information that is typically available for making engineering assessments during initial PRAs. "Pass" (P) and "No Pass" (NP) ratings are given when sufficient information is available to make assessments with the normal high level of confidence. When insufficient information is available, "Apparent Pass" (AP) and "Apparent No Pass" (ANP) ratings are given, based on available information and engineering judgment, to indicate the most likely outcome expected after sufficient investigations are completed, and to indicate the need for investigations (see Figure 1).
Risk Assessment
The risk assessment part of the PRA is an essential step for providing a picture of the risks associated with the existing portfolio, the need for risk reduction, the potential level of risk reduction for each SCUP, and the need for additional investigations. It provides information for decision-making using the "common currency" of risk, which is more readily related to a dam owner's overall business, than statements of "deficiencies" against engineering criteria or current practice.
Risk assessment includes the following steps for each dam: a) failure modes identification, b) risk analysis of the existing dam, c) risk evaluation of the existing dam, d) risk analysis of the SCUPs (for a range of inputs based on the potential range of investigation outcomes), and e) risk evaluation of the fixes (Figure 1).
Failure modes identification is the foundation upon which a risk assessment is built. It therefore should be carefully performed and documented for each dam. For a PRA, the risk analysis model is generally simplified through lumping of failure modes represented by event tree branches; a process that requires considerable care and skill to avoid significant distortions in risk estimates. Fell et al (2000) provide a presentation of the state of the art in the estimation of the probability of failure. Care should be taken in deciding the purpose for which monetary consequences are to be estimated; making distinctions between financial and economic losses; and making sure that the rules of loss estimation from economics are properly followed. It is recommended that a suitably experienced and qualified economist should guide this aspect of risk estimation. To avoid significant distortions in risk comparisons and prioritizations, it is important to minimise inconsistencies in the risk analysis of different dams.
The risk evaluation of the existing dams may lead to the identification of the need for fixes and investigations that were not identified during the engineering assessment (Figure 1). Life safety tolerable risk and financial/economic risk criteria (or guidelines) should be considered, including the ALARP (as-low-as-reasonably-practicable) principle. For PRAs conducted in Australia, life safety risk criteria should be the ANCOLD guidelines. It is also useful to use life safety criteria/guidelines from other organisations, such as the USBR (1997), for reference purposes. Evaluations against individual risk criteria are often omitted in initial PRAs, but consideration should be given to performing these evaluations on an indicative basis in future PRAs. If considered, OH&S risks should not be lumped with dam safety risks when performing life safety risk evaluations.
Loss financing criteria, business criticality, and other consequences evaluation considerations should be identified specifically for each dam owner and their associated stakeholders. Unlike life safety criteria, they should not be expected to be the same from one portfolio to another. Social and environmental impacts should be considered, at least at a broad level.
The results of the risk evaluation can be summarised using risk ratings and presented alongside engineering ratings for the existing dams and SCUPs. Such displays can be used to focus attention on dams that have the poorest ratings, without exposing decision makers to the full details of F-N charts, for example. They can be used to highlight changes in ratings that are indicated for SCUPs. Other information that is useful to decision makers can be included, such as, the strength of ALARP, de minimis risk, and benefit:cost or net present value justifications for fixes (Bowles 2000), although it should be recognised that dam safety fixes seldom perform well in traditional business models for capital projects evaluation.
Prioritisation
In of the first PRAs, it was the practice to give greater priority to fixes for which there was greater uncertainty associated with the outcome of future investigations. More recent PRAs are now including the inter-related prioritisation of remaining investigations. This approach, which is summarised in this paper, is now considered to be preferable to the original practice, and can be implemented with only modest additional effort in a well-organised PRA process.
SCUPs are typically prioritised to maximise the rate of annualised life safety risk reduction until a point of diminishing returns is reached (Bowles et al 1999b). This prioritisation should be based on unadjusted cost per statistical life saved (CPSLS), that is, without subtracting the economic benefits from the annualised cost of the fix3. The point of diminishing returns can be defined by a relatively large magnitude of the adjusted CPSLS (Bowles 2000). From that point, the remaining fixes are often prioritised by maximising the rate of reduction of annualised financial or economic risks (risk costs). The resulting prioritisation is usually considered to be an initial prioritisation, and is adjusted to account for factors, such as the following: very high-risks; needed implementation time (i.e. for planning, approvals, design and construction); linking the timing of dam safety fixes to the timing of capacity upgrades; completing all work at a dam under a single contract; and considerations of the effects on cascade dam failure risks (see discussion on cascade failure that follows in this subsection).
Investigations can include such activities as field investigations, materials testing, and engineering analyses. The prioritisation of investigations is based on the sensitivity of the prioritisation of SCUPs to the range of uncertainty in investigation outcomes. Clearly, investigations that are needed to complete a SCUP must be completed prior to its design, or in time to formulate upgrade program phases to obtain funding approvals. As investigations are completed, the prioritisation of SCUPs should be updated. Some organisations, in which funding for investigations is highly competitive, consider prioritisation of investigations to be of even greater short-term importance, than the prioritisation of fixes.
Prioritisation calculations require that estimates of risk reduction be made for each SCUP. Since the calculated risk reduction depends on the sequence of implementation of the SCUPs, and this sequence is in turn based on the cost effectiveness of risk reduction, these risk reduction calculations will necessarily be iterative. Typically, risk reductions are calculated for annualised life safety, annualised economic/financial losses, and probability of dam failure; but they can be calculated for different ranges of magnitude of life loss or economic/financial losses.
Total portfolio risk is usually calculated as the sum of annualised estimates of probability of failure and annualised life safety and economic/financial risks for all dams in the portfolio. This approach is only strictly correct if all failure modes at different dams are mutually independent. Failure modes at two or more dams may be correlated due to, for example, large area extreme storms, or earthquakes at faults that affect more than one dam simultaneously. Another example of correlation is for a cascade arrangement of dams. To avoid double counting of the risks for the cascade case (Bowles 1987), the consequences of downstream dam failure should be included in the upstream dam failure4. The probability of failure due to the upstream dam failure should not be included in the probability of failure for the downstream dam. An exception to this would be if a downstream reservoir were intended to store the flood wave from an upstream dam failure, without failing the downstream dam, as part of a river system approach to dam safety. In this case the upstream dam failure should be treated as part of the overall loading at the downstream dam and should add to the probability of failure and associated consequences at the downstream dam.
PRA Outcomes
PRA outcomes can be grouped into the following types (see the "Outcomes" box in Figure 1): 1) current risk profile; 2) improvements in on-going dam safety activities; 3) short-term risk reduction measures; 4) Dam Safety Improvement Program; and 5) inputs to business processes, such as capital budgeting, legal evaluations, loss financing, and contingency planning. It should be noted that, as indicated by the lines that connect boxes in Figure 1, information needed for developing these outcomes originate at many steps in the PRA process.
The scope and formatting of PRA outcomes should be adapted to meet the needs of the owner and other stakeholders through the outcome targeting process discussed in the "Decision Framework" subsection of this paper. In general, the level of detail of information that is communicated should decrease at higher levels of the owner's organisation. This may be pictured as a pyramid. At the base of the pyramid, much greater levels of detail are developed and used at a working level by the PRA team. At the apex of the pyramid, the Board should be provided with a clear but concise picture of the current risk profile, including summary comparisons with other portfolios, where available, and evaluations against various engineering and risk criteria. Bar charts that clearly show which dams pose higher risks from the probability of failure, life safety, and financial/economic perspectives can be effective presentation tools. Pie charts can be used to show the relative contributions to total risk from different types of failure modes, different dams, or dams located on different river systems. Pie charts can also be used to show the proportions of dams with different engineering or risk assessment ratings. Tabular presentations, with creative use of highlighting, can be effective tools for rapidly communicating information about the strength of justification for each SCUP, across the portfolio of dams, while making more detailed information available by taking a closer look at the table.
The Board should also be presented with a business case for the proposed Dam Safety Improvement Program. For management and budgeting purposes, it is useful to group fixes and investigations into phases. Earlier phases will include SCUPs with the highest levels of justification for implementation, while SCUPs with diminishing levels of justification will be assigned to later phases. Such as phased approach has resulted in obtaining early Board approval for the highest priority phases. As lower priority phases come up for Board consideration, it may be necessary to look carefully at benchmarking against similar dam owning organisations. This is further described in the "An Emerging Approach" subsection of this paper.
PRA Updating
Two tenets of the PRA approach are that 1) it is initially based primarily on available information, and 2) it should be periodically updated. PRA updates should incorporate the following:
The above list of "drivers" for updating a PRA is similar to those for performing periodic design reviews. Even for a mature dam safety program, after the completion of a comprehensive program of remedial measures, an up-to-date PRA can serve a useful role as a core activity in dam safety management program to facilitate an on-going link between dam safety issues and the rest of the business.
The ease with which a PRA can function as a "living document" depends to a large degree on both 1) the efficiency with which the necessary information updates can flow through the organisation (see Figure 2), and 2) the ease with which PRA revised calculations can be performed.
Frequently-asked Questions
Examples of questions that are often asked by managers, executives and directors include the following:
For some of these questions PRA can provide the basis for a response, or at least for a direction; but in other cases more detailed risk assessment would be needed; particularly to achieve adequate confidence for a decision that would affect the commitment of significant funds, and that would potentially have important liability implications for the owner, as well as significant public safety, economic, social, and environmental impacts.
An Emerging Approach
To date, the most significant users of PRA outcomes have been dam owners. Other users have included regulators [e.g. State of Victoria Ð see Watson and Perera (2000)], insurance companies with whom PRAs have been shared, and the public through community consultation efforts.
In this section, an emerging approach to dam safety risk reduction is summarised based on the author's experience working with dam owners, primarily in Australia, who have conducted initial PRAs. No attempt is made to compare specific organisations and only already published examples are cited to protect client confidentiality. Notwithstanding the description of an emerging approach, it should not be interpreted that all dam owners are following a uniform approach to dam safety risk reduction. In some cases the lack of uniformity appears to be explained by the differences in an individual owner's circumstances; but in other cases it is a result of differences in the relative importance that is placed on reducing dam safety risks. The approach adopted by a dam owner cannot be properly understood without taking into account differences in, for example, current risk profile, regulatory environment, the owner's financial capability, and other aspects of an owner's decision framework. Thus, caution must be exercised before applying any of the risk reduction strategies described below.
In general, the emerging approach to dam safety risk reduction appears to have the following characteristics:
Integration of Outcomes into the Owner's Business and Dam Safety Program
PRA outcomes should be used to strengthen on-going aspects of the owner's dam safety program, such as monitoring and surveillance, emergency preparedness planning, staff training, and operations and maintenance. If potentially high risks are identified during the PRA process, it is important that they are promptly investigated and that timely short- and long-term mitigation is implemented. This requires effective communication between the PRA team and the dam safety manager.
However, an effective PRA process should also yield information that can be of value to many different parts of a dam owning organisation. For example, non-dam failure risks with significant business risk implications might be identified, such as interrupting water supply or power generation. Other examples include information that will have important legal, due diligence, insurance, public consultation, capital budgeting, and contingency planning implications. One of the biggest challenges in the PRA process is to get the information generated by a PRA to the part of the owner's organisation where its benefits can be realised.
A key to the successful dissemination and utilisation of PRA outcomes is to develop "ownership" of the PRA process across a wide range of departments in the owner's organization. This is unlikely to be achieved if a PRA is viewed merely as a study that is performed by an external consultant, and especially if it is conducted merely to satisfy a regulatory requirement. A high level corporate risk management committee or other coordinating group might be an effective in house "owner" of the PRA process.
Another prerequisite for achieving a meaningful integration of PRA outcomes throughout the owner's business is an effective "outcome targeting" process. Through this process the information needs are identified for not only the corporate dam safety program, but for a wide range of other areas of the business (see section on "Decision Framework" in this paper and right side of Figure 2). "Outcome targeting" should begin at the outset of the initial PRA process, and should be reevaluated whenever the PRA is updated, and should be a determinant of the need for PRA updating.
Benefits
The PRA process is generally considered to be robust and defensible for corporate governance, and to justify its cost. The PRA approach was accepted in a recent audit of dam safety in the State of Victoria (2000). PRA relies primarily on the engineering understanding of dam safety issues, and provides a systematic means for identifying risks and prioritising investigations and fixes. For budgeting, board approval and management purposes, it is useful to group fixes and investigations into implementation phases, often with distinct risk reduction and project justification characteristics.
The PRA process typically provides many useful insights for strengthening dam safety management and for better integrating dam safety considerations into the owner's business (right side of Figure 2).
As illustrated in Figure 3, the PRA approach often leads to more rapid risk reduction than traditional prioritisation approaches, or index and FMECA methods, which do not preserve the risk metric. Thus, although risk assessment has sometimes been considered to be a tool for justifying less safety, the opposite should be true if PRA is properly applied. Bowles et al (1999a) state that the proper motivation for dam safety risk management should be to achieve:
Limitations
PRA outcomes should be used with full consideration given to their intended purposes and associated limitations. Engineering and risk-based evaluations should be considered to be only indicative, unless they are based on more detailed work performed outside the PRA process. It is unlikely that decisions on the selection of long-term risk reduction measures would be considered to be defensible if based solely on information from a PRA. Detailed engineering investigations, including the risk assessment of alternative risk reduction measures, are needed to provide adequate confidence and defensibility.
Bowles, D.S. 1987. Risk assessment for systems of dams. An invited presentation at the Engineering Foundation Conference on Risk Analysis and Management of Natural and Man-made Hazards, Santa Barbara, California. October.
Bowles, D.S. 1996. Potential Role for Portfolio Risk Assessment in Dam Safety Evaluation. Presentation to the Snowy Mountains Hydro-Electric Authority, Cooma, New South Wales, Australia. October.
Bowles, D.S., L.R. Anderson, and T.F. Glover. 1997. A Role for Risk Assessment in Dam Safety Management. Proceedings of the 3rd International Conference HYDROPOWER '97, Trondheim, Norway.
Bowles, D.S., L. R. Anderson, T. F. Glover and S.S. Chauhan. 1998. Portfolio Risk Assessment: A Tool for Dam Safety Risk Management. Proceedings of the 1998 USCOLD Annual Lecture, Buffalo, New York. August.
Bowles, D.S., L.R. Anderson, T.F. Glover, and S.S. Chauhan. 1999a. Understanding and Managing the Risks of Aging Dams: Principles and Case Studies. Proceedings of the 1999 USCOLD Annual Lecture, Atlanta, Georgia. May.
Bowles, D.S., A.M. Parsons, L.R. Anderson and T.F. Glover. 1999b. Portfolio Risk Assessment of SA Water's Large Dams. ANCOLD (Australian Committee on Large Dams) Bulletin 112:27-39. August.
Bowles, D.S. 2000. Dam Safety Risk Based Dam Safety Criteria and Guidelines. Proceedings of the Association of State Dam Safety Officials/Federal Emergency Management Agency Specialty Workshop on Risk Assessment for Dams, Institute for Dam Safety Risk Management, Utah State University, Logan, Utah. March.
Fell, R, D.S. Bowles, L.R. Anderson, and G Bell. 2000. The status of methods for estimation of the probability of failure of dams for use in quantitative risk assessment. Transactions of the Twentieth Congress on Large Dams, Beijing, China. Volume 1, Question 76. International Commission on Large Dams, Paris, France. pp. 213 - 235.
Haisman, B. 1998. In oral presentation of Dole, D. and B. Haisman. The Emerging River Murray Water Business: Developing Asset and Risk Management in an Inter-Government Context. ANCOLD Annual Meeting, Sydney, NSW, Australia. September.
SMEC/RAC. 1995. Review of Headworks. Final Report. Technical consulting report prepared for the Office of Water Reform, Department of Conservation and Natural Resources, Water Victoria, Victoria, Australia.
State of Victoria. 2000. Audit Report on Ministerial Portfolios, Part 3.5. pp. 229 - 240.
USBR. 1997. Guidelines for Achieving Public Protection in Dam Safety Decision Making. U.S. Bureau of Reclamation, Denver, Colorado.
Watson, D.J., D.S. Bowles, L.R. Anderson, T.F. Glover, C. Gratwick, P. Jacob, and G.S. Tarbox. 1997. State-wide Review of Headworks Dams: Status, Risks, and Future Business Focus, and Approach to Regulation. Transactions of the 19th International Commission on Large Dams (ICOLD) Congress, Florence, Italy. May.
Watson, D. and S. Perera. 2000. Risk assessment issues for dam safety management. Transactions of the Twentieth Congress on Large Dams, Beijing, China. Volume 1, Question 76. International Commission on Large Dams, Paris, France. pp. 313 - 324.