Software Engineering Risk Management

Contents

List of Figures

List of Tables

1 Introduction
1.1 Motivation and Background
1.2 Purpose and Structure of this Study

2 Risk in Software Engineering
2.1 Risk and Uncertainty
2.2 Delimitation of Software Engineering Risks
2.3 Software Development Risks and Their Sources

3 Heuristic Software Risk Analysis and Management Methodologies
3.1 Risk Management Objectives and General Strategies
3.2 Risk Management Planning
3.2.1 Planning and Implementation of the Risk Management Process .
3.2.2 Management Responsibilities
3.2.3 Team Management and Communication
3.3 Risk Identification
3.4 Risk Analysis and Evaluation
3.5 Planning and Implementation of Risk Handling and Controlling
3.6 Risk Tracking and Monitoring

4 Software Engineering Process Modeling
4.1 Meta-Models and Meso-Models for Software Development Processes .
4.1.1 Waterfall Process Models
4.1.2 Rapid and Evolutionary Prototyping and the Incremental Model
4.1.3 The Spiral Model
4.1.4 Unified Development Process and Rational Unified Process
4.1.5 Extreme Programming and ’Agile’ Development
4.1.6 Open Source Software Development Processes
4.1.7 Comparison and Evaluation
4.2 System Dynamics Models
4.3 Process Modeling Languages (PMLs)
4.3.1 Case Study Introduction: The Appache HTTP Server Project . .
4.3.2 Object-Oriented Process Modeling With the UML
4.3.3 Step-Based Process Modeling with JIL/Little-JIL
4.3.4 Petri Net Based Process Modeling With FUNSOFT
4.3.5 Case Study Evaluation

5 Verification, Validation and Testing
5.1 Static Analysis Techniques for Validating Software
5.2 Dynamic Analysis Techniques for Validating Software .

6 Risk Measurement and Quantification
6.1 Product Quality Measurement and Metrics
6.2 Process Measurement and Metrics
6.2.1 Capability Maturity Model (CMM)
6.2.2 ISO 9000 and Other Process Certification Models
6.3 Macro-Models for Cost and Schedule Estimation
6.3.1 The COCOMO I
6.3.2 From COCOMO I to COCOMO II
6.3.3 Applicability and Evaluation of COCOMO I/II .
6.4 A Quantitative Model on the Economies of Development Process Architectures
6.4.1 Model Introduction
6.4.2 The Optimization Model: Deterministic Part . .
6.4.3 The Optimization Model: Stochastic Part and Simulation
6.4.4 Model Conclusion, Limitations and Possible Extensions .

7 Conclusion and Further Research

References

Appendix A Tool and Model Documentation

A.1 Taxonomy-Based Risk Identification

A.2 Core Disciplines within the RUP

A.3 COCOMO Drivers and Values

Appendix B Figures

Appendix C Abbreviations

Appendix D Symbols

List of Figures

1 Definition of Risk

2 SEI’s Management Paradigm

3 Riskit Process Graph

4 Input/Output Diagram for Risk Management Planning

5 Input/Output Diagram for Risk Identification

6 Example of a Riskit Analysis Graph

7 Example of a Risk Exposure Matrix

8 Input/Output Diagram for the Risk Analysis Phase . .

9 Input/Output Diagram for the Risk Controlling Phase .

10 Input/Output Diagram for Risk Monitoring Phase

11 Notation in GERT Networks

12 Sequential “Waterfall” Process Model

13 Iterative “Waterfall” Process Model

14 Module-Based “Waterfall” Process Model

15 Incremental Time-Boxing Software Development Process Model .

16 The Spiral Model

17 The Rational Unified Process

18 Life-Cycle of the XP Method

19 UML Use Case Diagram of the Apache Process

20 UML Activity Representation of the Apache Process . .

21 UML Collaboration Representation of the Apache Process

22 Stereotype Symbols for UML Class Representation

23 UML Class Diagram of the Apache Process

24 Representation of Steps in Little-JIL

25 Apache Process in Little-JIL

26 Firing Behavior of Jobs in FUNSOFT Nets

27 Apache Process in a FUNSOFT

28 Software Quality Hierarchy, ISO 9126

29 COCOMO I effort functions for different project modes

30 Modeling the Development Progress

31 Influence of the Number of Iterations on the Total Development Time

32 Model Sensitivity Analysis (I)

33 Model Sensitivity Anlysis (II)

34 Impact of the Number of Iterations

35 Histogram of the Simulation Input Variables

36 Histograms of the Simulation Outputs

37 Results of the Simulation

38 UML Use Case Diagram of the Apache Process

39 UML Activity Representation of the Apache Process

40 UML Collaboration Representation of the Apache Process

41 Stereotype Symbols for UML Class Representation

42 UML Class Diagram of the Apache Process

43 Apache Process in Little-JIL

44 Apache Process in a FUNSOFT

List of Tables

1 Software Development Risk Sources

2 Team Risk Management Principles

3 Process Groups and Sub-Processes in Software Life-Cycles

4 Representation of Different Edge Types in FUNSOFT Nets

5 PML Comparison: UML, Little-JIL, FUNSOFT

6 Top Three Measures per Development Phase

7 COCOMO I calibration constants

8 Taxonomy of Software Development Risks

9 Disciplines and Workflows in the RUP

10 Cost-Drivers and Scale-Drivers in COCOMO I/II

11 Cost-Driver and Scale-Drivers Values in COCOMO II.1997

1 Introduction

1.1 Motivation and Background

While computer scientists have developed and provided several powerful computer languages and techniques in the last decades, facilitating the development of modular, maintainable and efficient code, software development itself has changed fundamentally. Software development today treats often with large-scale projects, immense development costs, and complex sys- tems which typically deploy multiple technologies and require multiple participants for their development. As with any large development exercise, the development of a complex system must be systematic and structured in order to manage this complexity, and in order to make possible the future maintenance and evolution of the system. Thus, while systematic and structured approaches are necessary for the development of such systems, software engineers have attempted to provide the structured methodologies and formalisms so often lacking in large software development projects. However, software development projects are still related with many different high risks. These risks cause software engineering projects to exceed bud- gets, miss deadlines, or deliver less than satisfactory products. As an example, U.S. companies alone spent an estimated $59 billion in cost overruns on IT projects and another $81 billion on cancelled software projects in 1995 (Johnson 1995). One reason for these high costs is that managers are not using adequate measures and executing efficient risk management assess and mitigate the risks involved in these projects.

Although risk taking is essential to progress, and failure is often a key part of learning, the inevitability of risks does not imply the inability to recognize and manage risks to minimize potential negative consequences while retaining the opportunities for creating new and better software. Obviously, this risk management process is particularly difficult for large-scale software projects and be handled in the same way as for small project, or just by providing more resources for all development factors.

1.2 Purpose and Structure of this Study

This paper tries to give an strategic and operational overview of how large software projects can be managed efficiently in order to avoid risks and unexpected outcomes as best as possible. The paper illustrates the complexities and of software development and discusses software engineering methods, techniques, notations and heuristics for addressing the identification, evaluation, and management of risks in software development projects in the form of a multiple- perspective approach.

During the past decades, different risk analysis and management approaches have been developed and introduced. Basically these software engineering risk management techniques can be categorized into

a) Qualitative Models and Heuristics: These techniques try to address, analyze and manage risk over the whole life-cycle of software projects. However, many of these approaches represent non-formalized approaches for risk management tools and describe only a practitioner’s point of view, e.g. that one of the IT manager, the project manager or the developer, of risk management techniques. Consequently, these methods are often relatively relatively simple but vague ambiguous in handling (e.g. Carr 1993, 1997; Charette 1989, 1991, 1996, 1997; Gluch 1994; Higuera 1994; Van Scoy 1992).
b) Quantitative Micro-Models: These models address particular software risks such as reliability or efficiency. Most of them are quality models that address the software product itself.
c) Quantitative Macro-Models: These empirical and quantitative models, the most- used is probably the COCOMO, try to address various known sources of risk in software development projects and to merge them into one consistent model (e.g. Boehm 1981, 1988, 1991, 1995, 1997, 1997a).

This study will systemize, structure and discuss strengths and weaknesses of all three differ- ent categories of published risk management methodologies. In addition to this, a quantitative optimization model is developed to model and evaluate the economies of process architectures of software development projects (Section 6.4). The rest of the paper is structured as following:

Section 2 introduces into fundamental concepts of risk (2.1), followed by a delimitation of risks this study is dealing with (2.2). Examples for typical risks in software engineering projects and their sources are explained (2.3). A first overview of heuristic but practical software engineering risk management strategies and their single steps and activities provides Section 3. Risks in software development projects are strongly influenced by the software development process. Section 4 deals with these processes. Different meta- and meso-models are introduced and described in their structure and functions (4.1) as well as specific process modeling languages (4.3). We perform a case study to analyze the contribution of different process modeling languages in software engineering risk management. After that, we analyze different methodologies for validation, verification and testing in Section 5. Section 6 treats different methods and approaches for quantification and measurement of risk within software engineering projects. Different software product and process metrics are evaluated (6.1, 6.2) before an empiric macro model for cost and schedule estimation is introduced (6.3). We present a comprehensive and new quantitative model on the economies of different development process architecture in Section 6.4, consisting of a deterministic model part (6.4.2) and a stochastic model part including a simulation (6.4.3). This study is ending with a conclusion and an overview of challenging future research issues within the software engineering risk management area in Section 7.

2. Risk in Software Engineering

2.1 Risk and Uncertainty

Human’s interest in risk exists now for several hundred years.¹ Today, the word “risk” is used (and misused) in science as well as in many everyday situations although the exact meaning of the word remains often unclear. Common for most risk understanding is that risk concerns future happenings and risk involves potential changes of some entities or their environment. However, these future events underly always uncertainty-otherwise they do not constitute any kind of risk but certain kinds of problems opportunities.

Three types of uncertainty are described in literature (Charette 1989): (a) Descriptive or structural uncertainty is connected with the lack of information concerning the value of a set of variables that define a future state of the system under study. When totally determined, this set of variables fully describes the system. (b) Measurement uncertainty describes the absence of information relating to the assignment of a value to the variables used to describe the system and is due to the fact that observations are limited and/or validation and calibration data are not available. (c) Event outcome uncertainty occurs when it is impossible to predict and identify future outcomes and their respective probabilities. This type of uncertainty becomes important in risk analysis when forecasts and estimations about future outcomes, e.g. of a software project using a new technology, have no prior history to draw from. Whereas traditional risk theories and the early insurance science consider risk as completely random and non-influenceable in their occurrence by external factors like humans, the understanding of risk is today much wider.² Risk which include a choice and which can be influenced by a person making decisions (without complete information of potential effects) are called speculative risks, whereas risks without any flexibility of choices and options in decision making for a person are treated as real risks.

Whereas some textbooks try to distinguish among risk (used in situations where the probabilities of alternative, possible outcomes are known) and uncertainty (used in situations where the frequency distributions of the possible outcomes are unknown), uncertainty will be understood as a characteristic of risk in this paper.

While a common understanding of risk includes often only the occurrence of potential prob- lems, injuries or losses³, modern definitions of risk in science are usually associated with

illustration not visible in this excerpt

Fig. 1: Definition of Risk as an UML Representation.

uncertainty that can both have positive and negative effects.⁴ In literature, these two different categories of risk are denoted as static risks and dynamic risks (Charette 1989). Since software risks represent in most of the cases, as the following chapters will show, deviations from expected outcomes that may contain aspects of both gain and loss and are valued in different ways by various stakeholders (see Fig.1), software risks will be considered to be of a dynamic and speculative nature in this study. Naturally, since the effect of unexpected problems is much worse than that of unexpected chances, the main focus will be on risks with negative consequences. For assessing software risks in general and potential losses in particular, it is important to keep in mind that a “potential loss” can mean either (a) a situation worse than the status quo or (b) a situation not as good as a feasible other situation might have been (“opportunity loss”).

2.2 Delimitation of Software Engineering Risks

Software development projects are embedded in all kinds of organizations in software industry and business world. Thus, software engineering projects face the same variety of risks as many other businesses do. However, some risks are not specific to software development and its processes but refer more to the corporation the software engineering project is embedded in. These risks, such as credit risks, liquidity risks, legal risks, social risks or market risks, are, seen from the project perspective, external or transferred risks which are not in a causal relation losses. Transferred to the French, it became the source of the word “risquer” (translated “to dare”, “to venture”, “to take a risk”) and covered also positive aspects of possible outcomes. with the software project itself. In this study, we will focus only on project internal or project related software engineering risks, for example cost risks which are directly influenced by the way the project is executed.

Generally, as a delimitation to our definition of risk, we will treat software risks as a devi- ation and form of unsatisfactory outcome from the expected software project objective, or as “situations or possible events that can cause a project to fail to meet its goals” (Boehm 200a). For customers and developers, budget overruns and schedule delays are unsatisfactory. For users, products with the wrong functionality, bad user-interfaces, reliability shortfalls and per- formance shortfalls are unsatisfying, whereas for maintainers, poor-quality software with bad documentation is unsatisfactory. Given that projects involve several classes of stakeholders (for example customers, developers, users, maintainers), each with different satisfaction criteria, it is clear that “unsatisfactory outcome is multidimensional” (Boehm 1991). This perspective of unsatisfactory outcome provide a first top-level checklist for the following risk identification and assessment Sections.

2.3 Software Development Risks and Their Sources

Basically, risks in software development projects become visible in three different dimensions of possible impacts:

1. Cost overruns: The overall development costs tend to be higher than estimated
2. Time overruns: The total development time until delivery to the client is longer than scheduled.
3. Quality shortfalls: Performance is lower than expected, the software has significant bugs or expected functionalities are missing.

Between the three factors is usually a functional relation, e.g. are costs influenced by the development time. Furthermore, all three categories of damages factors can lead to further (second-level) consequences as loss of revenue, tarnished image, loss of credibility, legal proceedings, demoralized staff and contract or project cancellation. Thus, software development risks can even lead to a failure of a complete business.

Other risk categorizations that are typically used⁵ distinguish among (a) the stage of the life cycle the risks occur in, or (b) the area the risk sources can be found in, for example customer mandate, scope and requirements, environment, execution.

Categorizing risks is an important prerequisite for further risk management activities. Dif- ferent approaches have been used to investigate software development risk factors (SDRFs), i.e. a particular characteristic or source that affect the probability of a negative event occurrence.

From them have emerged collections of risk factors or prioritized lists, questionnaires, tax- onomies and matrices for assessing software development risk. Major factors that contribute to success or failure of software projects can be categorized in seven groups (Procaccino 2002):

(1) management, (2) customers and users, (3) requirements, (4) estimation and scheduling, (5) the project manager, (6) the software development process, (7) development personnel. Unfortunately, many more detailed risk factor lists in literature do not precisely distinguish between risk source and risk impact and are often not minimal but contain still overlapping risk issues. The following list combines exemplary software development risk sources identieifed in different other studies (Charette 1989, Conrow et al. 1997, Houston 2000, Myerson 1996):

illustration not visible in this excerpt

Tab. 1: Software Development Risk Sources.

The above-mentioned risk sources show that efficient risk management must take into consideration technology and project aspects as well as the aspects concerning the business environment in which the project is operated since many projects fail because of larger organizational pressures that were ignored.

Each task of a software development process constitutes, very generally said, over the whole life-cycle (see Section 4.1) a potential risk. It may occur and become a problem as soon as it is not executed and performed with commitment and competence of all stakeholders. Thus, software risk management should be implemented throughout the life-cycle.

3 Heuristic Software Risk Analysis and Management Methodologies

3.1 Risk Management Objectives and General Strategies

Risk management is considered as the process associated with identifying, analyzing, moni- toring, and handling risks in a specific project (Myerson 1996). The risk management process includes up front planning, assessment (identification, analysis and prioritization), handling, and monitoring of risks in a continuous flow throughout the life cycle of the project or op- eration. It is a goal to pro-actively identify, analyze, control, and communicate risks, before they endanger the whole project or cause costs and failures. Parallel to this, risk management has to analyze interdependencies between risk and success and transform this into applicable principles.

To get started, we will assess a generic and heuristic approach for implementing risk management in software engineering projects as a sequential process which combines different other published approaches (e.g. Boehm 1991, Balzert 1998, Charette 1989):

1. Risk management planning: Development, planning of the basic structure and schedule of risk management activities and their implementation in the development life-cycle. As a basis of mutual consent and valid consideration, this includes the authorization from and a working agreement with the project owner and the definition of risk management responsibilities within the project management.
2. Risk identification: Creating a list of project-specific risk elements (excluding general project risks) that constitute potential threats. Typically used in this step a checklists, assump- tion analysis, and decomposition techniques in order to produce an exhaustive list.
3. Risk analysis and estimation: Qualitative or, where ever possible, quantitative risk assessment. Estimation of occurrence probabilities (use existing tables where able) and potential losses. Combined risks have to be broken up. Typical techniques include performance models, cost models, network analysis, statistical decision analysis and quality-factor analysis (for example reliability, availability, and security).
4. Risk prioritization: Ranking the risks according to estimated risk factors. It has to be decided which risks are considered acceptable risks and which cannot be accepted. In addition to a risk-exposure analysis Delphi or group-consensus techniques are typically used.
5. Risk controlling planning: Establishment of different risk control and reduction activities and integration of these activities into the project schedule. All risk management activi- ties should be communicated in advance with all stakeholders. Typical techniques include checklists for risk-resolution techniques, and cost-benefit analysis. The risk management plan provides the detailed planning required to incorporate the risk management process or a specific risk management activity into the respective organization or operation.
6. Risk controlling and resolution: Execution of all risk reducing and surmounting activi- ties, e.g. by prototype development, simulations, benchmarks, key-personal agreements, design-to-cost approaches, incremental development, mission analyses or requirement modification.
7. Risk monitoring: Surveillance of risk reduction progress by watching of all risky processes and checking for deviations from the specified objectives. Typical approaches include milestone tracking and a top-10 risk-item list that is highlighted at each regular project review.

We will systemize this framework structure with more detailed approaches and elaborate and extend each phase in the following Sections of this paper.

illustration not visible in this excerpt

Fig. 2: SEI’s Risk Management Paradigm (Carr 1993).

The Software Engineering Institute (SEI) at Carnegie Mellon University, Pittsburgh, has developed a risk management paradigm which is based on the above-described activities but does incorporate additionally different defined and formalized methods for implementing the risk management activities in a development process (Dorofee 1996). The risk management paradigm (Fig.2) represents a meta-model for risk management process implementation and depicts the basic functions of managing risks as identification, analysis, planning, tracking, controlling, and communication. The last aspect emphasizes that risk management is not an individual task but requires the involvement of several project stakeholders, including engi- neers, managers, customers, suppliers, and co-developers. In the SEI’s approach, risk manage- ment is intended to be conducted not only by the management of the development organization (government) but also on a less detailed level by the customer (contractor) himself. The development project is here seen as a cooperative work involving customer and supplier both with specific responsibilities and tasks.

The following Sections (3.2-3.6) will discuss different methodologies for heuristic-based risk management steps such as risk management planning, risk identification, analysis, controlling, and monitoring. The methodologies being described combine different heuristic process model frameworks based on “Riskit” (Kontio et al. 1997), “Taxonomy-based Risk Identification” (Carr et al. 1993), “Software Risk Evaluation” (Williams et al. 1999), and “Team Risk Management” (Higuera et al. 1994), representing different risk management approaches developed at the SEI and emphasizing very different key aspects although they have the same underlying meta- model.

The biggest disadvantages of these methodologies are at the same time their advantages: They represent all heuristic and qualitative methods. Since estimations are always difficult and have errors with probability one, estimation problems are reduced to the minimum by using these qualitative approaches and tools to analyze risk. At the same time, ambiguity of process definitions increases and it is much harder to identify sensitivities of particular risk factors. Nevertheless, “by emphasizing a qualitative understanding of risks, there is a better basis for understanding and communicating about risk,” as Kontio (1997) states. Even more, for practical risk management purposes it may be sufficient to identify the most dangerous risks whereas the exact values and probabilities do not matter for a practical handling strategy.

3.2 Risk Management Planning

3.2.1 Planning and Implementation of the Risk Management Process

Generally said, planning risk management processes can be done in the same manner and with the same tools and methods as planning any other software development process. Since process planning both for risk management activities and for development activities is an essentially important activity, we treat theories of process modeling separately in Section 4.

The Riskit method (Kontio et al. 1997) for software engineering risk management provides a defined process for conducting risk management and is supported by different techniques and guidelines. It provides particularly the following conceptual and graphical tools for modeling risk from different stakeholder perspectives qualitatively: (a) Riskit process definition, (b) Riskit analysis graph, (c) Riskit documentation templates.

The Riskit process as shown in the dataflow graph (Fig.3) represents a defined and therefore repeatable process framework for risk management activities in order to provide project and organization managers with precise information about project risks and to implement risk control and mitigation activities. The cyclical risk management process consists of seven steps that are briefly described and structured in the following.

Abbildung in dieser Leseprobe nicht enthalten

Fig. 3: Dataflow Diagram of the Riskit Process (Kontio 1997).

At the beginning of the risk management planning, the Riskit process starts with the risk management mandate definition. Purpose of this activity is to “define the scope and frequency of risk management” and tries to evoke consciousness at all relevant stakeholders. Particularly, project authorities and sponsors have to agree on the risk management objectives and on its scope. If necessary, for example after major project changes, the risk management mandate may be redefined. By this, the mandate definition operates as a “contract” for risk man- agement. This working agreement should-to ensure mutual consent and valid consideration- contain boundaries of planned activities, objectives, kinds of information sought, roles and responsibilities, and products the team will deliver. Depending on the total project and team size, the participating individuals and teams in the risk management process are the project manager, the risk management (RM) manager, a software risk evaluation (SRE) team, and eventually the software quality assurance (SQA) and the configuration management (CM) managers.

The second planning step, the goal review step, is part of the early requirement definition phase of the software development process. It ensures that stated project goals are correct and recognized and that implicit stated goals are formulated explicitly, so that all project stakeholders have the same understanding of them. Here, “goals” include defined objectives to be achieved, drivers of progress without well-defined achievement criteria, and constraints, i.e. limitations that have to be respected. Besides the goals included in the project contract, Riskit provides different goal definition templates.

Abbildung in dieser Leseprobe nicht enthalten

Fig. 4: Input/Output Flow Diagram for the Risk Management Planning Phase.

3.2.2 Management Responsibilities

In terms of risk management, the project management is typically responsible for an overall planning, identification and management of risks, development and selection of analysis of tools, methods, and metrics, and integrating the risk plan with the project management plan. A system coordinator provides risk management support to the project managers. Optionally, one or more specific risk management specialists works with each project manager to ensure that the established project planning documentation included the identification of risks, risk mitigation and risk monitoring. He/she would also ensure that the project manager has a common view of the project unique and shared identified risks and that a risk management plan is specifically tailored for each project.

In this context, management should according to Charette (1996) in particular

- replace individual worker’s rule-of-thumb methods with specialized, scientifically developed approaches;
- select, train, and develop workers using scientific methods, so each worker performs the right job;
- bring together scientifically selected workers and scientifically developed work to gain the optimal results; and
- ensure an equal division of work and responsibility between management and workers, and foster close cooperation between the two.

3.2.3 Team Management and Communication

SEI’s Team Risk Management (TRM) methodology (Higuera et al. 1994) defines an organi- zational structure and operational activities for managing risks throughout all phases of of a software development project such that all individuals within the organizations - groups, departments, and agencies directly involved in the program - are participating as active team members. TRM is based on nine principles (see Tab. 5) which combine the SEI risk man- agement paradigm (see Fig.2) and concepts of cooperative teams to form the foundation for a set of processes, methods, and tools for managing risks in software-dependent development programs.

Principle Description

1. Shared product vision A shared vision for success based upon commonality of purpose, shared ownership, and collective commitment.

2. Forward-looking search Thinking toward tomorrow, anticipating potential outcomes, identifying un- for uncertainties certainties, and managing program resources and activities while recognizing these uncertainties.

3. Open communications A free flow of information at and between all program levels through formal, informal, and impromptu communication and consensus-based processes.

4. Value of Individual per- The individual voice which can bring unique knowledge and insight to the ception identification and management of risk.

5. Systems perspective Software development being viewed within the larger systems-level definition, design, and development.

6. Integration into program Risk management being an integral and vital part of program management. management

7. Proactive strategies Proactive strategies that involve planning and executing program activities based upon anticipating future events.

8. Systematic and adaptable A systematic approach that is adaptable to the program’s infrastructure and methodology culture.

9. Routine and continuous A continuous vigilance characterized by routine risk identification and man- processes agement activities throughout all phases of the life cycle of the program.

illustration not visible in this excerpt

Tab. 2: Team Risk Management Principles (Higuera 1994).

Here, two principles are particularly important: A shared product vision and mission is not only important for reducing project risks by getting a common understanding of the product, but the vision also sets the goals and determines the direction of acting. Selected and communicated in the right way, a shared and “lived” vision can work as a strong long-term motivation provider.

Besides this, TRM identifies effective communication structures between all stakeholders (particularly within the development team) as one other key to success during all phases of the risk management and product development process. Communication underlies successful team risk management and is the foundation for team building between and within partner organizations. The challenge is to find ways to open the communication pathways to the decision makers and speed up the synthesis of concerns and issues into actionable risks. In contrast, risks, problems, and crises often occur when communication breaks down in an organization.

There are many reasons why risks are not communicated efficiently in a project. For ex- ample, those who understand technical details and implied software risks often may not have the expertise or communication path to recommend changes in the larger system (e.g., in the hardware or the system environment) that would mitigate risk most cost-effectively for the whole program. While management has the primary authority and power to make changes and decide the course of the program, it needs information on risk factor development from all of the members of the team. To establish efficient important risk communication paths, management can communicate to the entire program that it is vital that risks be identified and communicated upward for resolution and aggressively support those processes. In addition, management can encourage improved risk communication by fostering small inter-disciplinary groups to deal with risks as they are identified. Risk communication and public support of it should therefore be one of the points covered in the performance appraisals. Through these and similar efforts, management can provide leadership in achieving effective communication and strong, interpersonal, team cooperation. Additionally, program staff members must un- derstand and accept risk management practices as worthwhile and permanent and how it is embedded in the underlying risk management and system development process. However, the responsibility of realizing that shared vision and of creating a sense of collective ownership and responsibility requires all team members to contribute actively to the communications and team building efforts.

An organizational culture that supports open and transparent communication will have a positive influence on implementing risk management (Higuera et al. 1994). A limited time for open discussions in risk management meetings offers the opportunity to expose paradoxes, conflicts and dilemmas and can promote openness in communication. For this, personnel must feel emotionally safe and must not fear to be punished for mentioning any kind of concerns to superiors. Particularly in bigger organizations which often face the problem of anonymous individuals, community management and communication management can help establishing efficient ways of cross-functional cooperation and communication.

Besides aspects of organizational culture, several methods and tools can establish better communication paths. It is essential the risk reviews (if necessary also interviews) are held regularly both within the development team, and in cooperation with the customer. Modern intranet-based communication platforms can help making the exchange of information easier. Documentation of all planned and executed risk management activities is essential for effective communication paths.

3.3 Risk Identification

Risk identification is one of the most important parts within the risk management process, since only those risks can be controlled that are already identified as potential problems. For the identification process, the following questions have to be answered (Charette 1989): Which sources of disturbance and risk factors have influence on the risk domain? Where are potential problems the most probable? Which damages can result from a risk factor?

The risks in a software development project can be known, unknown, or unknowable (Carr 1993). As seen in Section 2.3, it is not easy to guarantee the exhaustive (and minimal) detection and description of all known (!) risk factors. Therefore, different tools have been developed for facilitating the systematic and repeatable identification of risks associated with the development of a software-dependent project. Besides different commonly used checklistbased approaches, taxonomy-based risk identification (TBRI) represents a method developed and actively tested by the SEI (Carr 1993).

Risk identifiers are ideally all people associated with the project. The Management has to ensure, e.g. respective incentives, that everybody feels responsible for providing pro-active risk identification and analysis. For this, there must be an open and retaliation-free environment where project team members feel comfortable in identifying risk. Since this ideal situation can only be found in few organizations, TBRI has specific risk identifiers in charge of conducting risk identification interviews.

The focus of the TBRI method is particularly on risks that are known whether or not they have yet been communicated to project management, and on unknown risks. The risk identification method achieves this focus through an taxonomy-based questionnaire designed as an interview instrument, and its application process.

Important underlying concept for the TBRI is the risk taxonomy developed by SEI (Carr 1993) which categorizes and organizes software development risks into 3 levels: classes, el- ements, and attributes. The taxonomy is organized into three major classes: (1) Product Engineering, i.e. the technical aspects of the work to be accomplished; (2) Development En- vironment, i.e. the methods, procedures, and tools used to produce the product; (3) Program Constraints, i.e. the contractual, organizational, and operational factors within which the software is developed but which are generally outside of the direct control of the local project management. The element level categorizes all attributes, i.e. the actual risk factors and sources, in a life-cycle-oriented way. The complete risk taxonomy is listed in the Appendix A.1 (Tab. 8).

illustration not visible in this excerpt

Fig. 5: Input/Output Flow Diagram for the Risk Identification Phase.

The taxonomy-based questionnaire consists of a list of unambiguous and non-judgmental questions to identify elicit issues and concerns and potential risks in each taxonomic class. The questionnaire is supposed to ensure that all risk areas are systematically addressed. A standardized application process has been designed to guarantee that the questions are asked of the right people and in the right manner to produce evident results.

Since project risks are likely to change over time in both characteristics (probability, impact, time frame) and content, the project should repeat the risk identification and follow-up pro- cesses periodically during the project life cycle. The risk of yesterday could be the problem of today or even not be a risk any longer, and new risks could arise. Particularly in early stages of the development process, this repetition was founded to be very effective (Carr 1993).

3.4 Risk Analysis and Evaluation

After having identified all risk factors, these have to be analyzed in order to assess to potential danger and the effects of the risk event in case its occurrence. In this context, measurement of risks becomes important. Section 6 will deal with different risk measurement methods in detail. Without anticipating the contents of that Section, we introduce already at this point the very generic concept of risk exposure, which can be used as a quantitative measure for the potential endangering of a system at a certain time. Here, the risk exposure R represents the expected loss/gain E (Q) and is the product of the occurrence probability p and the consequence Q, e.g. the expected damage:

Abbildung in dieser Leseprobe nicht enthalten

The risk exposure corresponds to the contingency that is needed to protect against the as- sumption error. However, as Kitchenham and Linkman (1997) remind, the probablistic nature of risk means that the allowed contingency cannot protect a project if the original assumption is invalid.

At the beginning of the risk analysis process, the Riskit methodology suggests that the identified risks are grouped, filtered and estimated. It follows a structured approach for decomposing risk and developing so-called risk scenarios. The Riskit analysis graph, which is accompanied by different textual clarification documents, represents a graphical modeling formalism as well as a conceptual template designed for this decomposition purpose and breaks up the risk analysis in six causally connected components (Kontio 1997):

1. Risk factor: A risk source, for example unstable requirements (see Section 2.3);
2. Risk event: An actual problem as an occurrence of a risk, for example a major requirement change;
3. Risk outcome: The consequences of the risk event before any action has been taken to reduce the effects of the risk/problem, for example additionally required work;
4. Risk reaction: Any type of (reactive) risk mitigation, for example a meeting with the cus- tomer;
5. Risk effect: The final impact(s) of a risk event to a specific project goal (after risk reaction), for example additional costs of $50.000;
6. Utility loss: Measures the overall impact of effects and the perceived harm by a stakeholder with respect to his utility function, for example the CEO’s harm.

Abbildung in dieser Leseprobe nicht enthalten

Fig. 6: Example of a Simplified Riskit Analysis Graph Without Utility loss Modeling (Kontio 1997).

Between these six sets of elements elements, deterministic and stochastic relationships can be graphically represented by solid and broken arrows, respectively in order to model the risk situation. For a quick assessment, each class of elements is drown in the Riskit analysis graph as a symbol in a special column as shown in the example (Fig.6). The SRE team analyzes each identified risk in terms of its consequence on cost, schedule, performance, and product quality. An individual risk may impact more than one of these categories. For example, frequently changing requirements will impact all four. Second, each SRE team estimates the probability that each risk will occur and its time frame. Probability can be established by a percentage figure or by selection of a probability class term (e.g., High, Medium, Low).

It follows typically a prioritization of all risks. Here, one approach is to determine a total risk level for each risk by mapping each risk onto a two dimensional risk matrix, taking into consideration the analyzed probability and severity of each risk scenario (Fig.7). Depending on the number of classes used for categorizing risk probability and impact, we get a risk exposure ranking on different levels. In Fig.7, we distinguish exemplarily five risk levels: (1) Tolerable Risk is a state where risk is identified as having little or no effect or consequence on project objectives; the probability of occurrence is low enough to cause little or no concern. (2) Low Risk is a state where risk is identified as having minor effects on project objectives; the proba- bility of occurrence is sufficiently low to cause only minor concern. (3) Medium Risk is a state where risk is identified as one that could possibly affect project objectives, cost, or schedule. The probability of occurrence is high enough to require close control of all contributing factors.

(4) High Risk is the state where risk is identified as having a high probability of occurrence and the consequence would affect project objectives, cost, and schedule. The probability of occurrence is high enough to require close control of all contributing factors, the establishment of risk actions, and an acceptable fallback position. (5) Intolerable Risk is the state where risk is identified as having a high probability of occurrence and the consequence would have significant impact on cost, schedule, and/or performance. These risks would constitute the risks with the highest priority for controlling and handling activities.

Abbildung in dieser Leseprobe nicht enthalten

Fig. 7: Example of a Risk Exposure Matrix.

Abbildung in dieser Leseprobe nicht enthalten

Fig. 8: Input/Output Flow Diagram for the Risk Analysis Phase.

Instead of ranking risks based on the on the probability and extent of losses (e.g. time, costs, quality), Riskit uses the concept of utility loss to compare and rank risks. By this, utility functions of different stakeholders can be included and modeled in order to measure the risk effect. The utility loss is estimated for each relevant stakeholder.

By this, instead of prioritizing risks according to the estimated risk exposure (which can also be based on utility losses) of the different scenarios, the scenarios can be efficiently ranked in a two-dimensional table according to their probability and the utility loss. Contrary to the risk exposure matrix method, this ranking technique results in a partial ranking based on the available information. By this, scenarios can be ranked along columns and lines, but when a scenario has a higher probability ranking and a lower utility loss ranking compared to another, they cannot be ranked in a total order. This partially ranking guarantees and significance and provides enough information for planning the action order of the risk controlling.

3.5 Planning and Implementation of Risk Handling and Controlling

In general, an identified risk becomes a problem when the value of quantitative metric crosses a predetermined threshold which depends on the grade of risk aversion of the management and the company’s assets and capabilities allowing to accept particular risks. After passing the threshold, certain pro-active activities should be started in order to either mitigate, reduce and avoid risks, to transfer risks and their potential negative outcomes, or - in the case of making the decision to take the risk and the related chance - being prepared for the occurring of the worst possible effects and being able to resist all potential damages. Whereas the first of these three fundamental risk handling strategies follows an approach that tries to influence the source of the risk (by influencing the occurrence probability and/or the extent), the second - more passive - strategy is based on a effect-based tactic.

Risk mitigation is an approach that deals with a specific risk by developing strategies and actions for reducing the impact and/or the likelihood of occurrence of the risk to some acceptable level. Possible actions are diverse and should address each risk factor specifically. The possibility of transferring risks to other organizational entities is particularly advantageous for small and unexperienced software development teams and individuals. The usage of risk transfer by the project management should be limited by project authorities to specific situation based on explicit contractual negotiation. Otherwise it can become a “free ticket” for the project developers/management not to worry about risks.

All risk controlling activities have to be planned carefully in order to be efficient. A risk mitigation plan usually outlines what should be done with a specific risk. It provides an action plan for individual or sets of related risks. Here, action planning addresses particularly risks that can be mitigated by immediate response. In contrast to this, contingency planning addresses risks that require monitoring for some future respond should the need arise. Thus, contingency planning is a strictly proactive strategy and is directly related to risk monitoring and measurement (see for a discussion in a detailed way Sections 3.6, 6).

Possibilities for both risk mitigation and risk transfer strategies will be discussed in the next paragraphs. However, risk mitigation and transfer strategies are never magic formulas that can allow to increase easily total profits (or to decrease losses) of a particular project. Similar to taking and avoiding risks on financial markets, software engineering risk mitigation strategies are usually not free of some kind of charge.

(a) Risk Mitigation and Avoidance:

- Resource and Capacity Reservation:

Limits of resources and capacities can include human, technical, financial, and other assets. To address the risk of insufficient experience with a new hardware architecture, for example, the action plan could provide hiring experienced personal, or finding a consultant to work with the project team. It may also involve shifting the time frame for the project.

- Resource Quality Improvement:

This activity addresses a quality improvement of all assets and includes e.g. additional training the development team or a better hardware/software environment for the de- velopment.

- Preparatory Work and Recovery Options:

This set of actions refers to all work that is done in order to prepare for a risk occurrence and is similar to contingency planning. Normally ”preparatory work cannot reduce the probability of a risk event occurring but only mitigate the effects. If for example safety and reliability are product-specific risks, redundancies in a system or database architecture are a way of preparing for the risk occurrence.

- Changing of Project Goals:

Some risks directly related to cost and schedule questions may only by mitigated with additional negotiations with the stakeholder community-here particularly with the cus- tomers. Particularly after requirements refinement, original product goals as well as cost and schedule goals might become unrealistic and have to be redefined, too. While it is tempting to use changing goals as an easy way of risk management, most stakeholder do not want to see it too often.

- Simulation, Testing, and Prototyping:

Although simulation has a long tradition in many different engineering and natural sci- ences, it has not found general use in software industry-partly be due to the fact that software engineering is a very human-intensive activity (Christie 1998). Simulation in software projects is not only a way of risk mitigation in early project stages but does also support risk measurement and provides alternative tools for activity-based cost and schedule planning compared to more traditional techniques. In contrast, prototyping fol- lows usually after simulation activities but is typically still mostly used in early project stages (see Section 4.1.2). Since verification and validation activities like testing are a central point in software engineering risk management, we will discuss this issue in a more detailed way in Section 5.

- Process Improvement:

A well-structured and defined process has been found to be an important prerequisite for minimizing the occurrence unknown risk events during the software development activities (Charette 1989). A good process model can help achieving a repeatable and predictable process outcome without budget and schedule overruns. It is the basis for cooperative and goal-oriented team work (see in detail Section 4).

- Component-Based Reuse:

Not only in terms of efficiency but also in terms of risk mitigation, the carefully reuse of software components provides advantages and can reduce verification and testing efforts.

- Over-Engineering:

This principle means the implementation of features in the software product or design so that there are alternative ways of action if the risk event occurs. For example, extra effort could be made during the design and coding phases of a software development project to ensure that alternative system architecture or compilers can be used.

- Creating of Compensating Benefits:

Compensating benefits for a risk event can reduce the utility loss caused by the effects of the risk event. If for example a risk scenario implies a two-month schedule delay, this utility loss could be compensated by reduced expenses for salaries, non-cash benefits, or free training.

(b) Risk Transfer Possibilities:

- Risk Sharing:

This strategy includes the involvement of other organizational entities in the project. For instance, subcontracting of defined development components, cooperation with other development teams within or without the same firm, or getting customer approval for specific risk consequences can allow to reduce responsibility and liability for different risk factors.

- Management Approval:

Similar to risk sharing, which transfers risk to other stakeholders, management approval transfers risk to the organizational management. Although this is an easy way for developers to transfer risks, the organizational risk exposure is not improved.

- Insurances:

The usage of external or corporate resources to hedge and insure against particular risks is still very limited. Real options are rare in software development. Different kinds of liability insurances (e.g. for the management) are traded by insurance companies but usually the costs for this strategy are very high.

Once the potential risk controlling actions have been identified, the most effective ones have to be selected in order to be implemented. Kontio et al. (1997) suggest four criteria for selecting risk controlling actions: (1) Risk reduction effectiveness, (2) resource availability, (3) stakeholder importance, (4) urgency of implementing the risk controlling action. The risk reduction leverage proposed by Boehm (1989), with

Abbildung in dieser Leseprobe nicht enthalten

takes the effectiveness of proposed risk controlling actions into account. However, the problem with this formula is that ordinal scale rankings of utility losses do not allow such a formula to be used. Even when distance or ratio scale estimates for utility loss were available, the accuracy of these estimates and those of cost estimations for controlling actions may make the formula impractical to use for utility losses. Here, subjective judgement is the only (rather bad) possibility to estimate the risk reduction effectiveness with respect to the utility loss.

The second selection criterion represents resource constraints. This can refer to the risk management budget if it has been defined or to the amount of available resources and skills. These constraints may rule out some otherwise effective risk controlling actions.

illustration not visible in this excerpt

Fig. 9: Input/Output Flow Diagram for the Risk Controlling Phase.

Finally, the urgency of implementing risk controlling actions influences the decision on what risks to mitigate in which way. The risk controlling action urgency depends both on the time of the risk event occurrence and the time delay in implementing the risk controlling action.

Risk controlling can be started as soon as the first risk controlling activity as been selected and planned. Although the risk monitoring process is ideally continuous, risk monitoring metrics are usually applied in weekly or biweekly review intervals.

3.6 Risk Tracking and Monitoring

Monitoring and evaluation of the implemented risk management activities should occur after the decisions concerning controlling strategy and tactics have been implemented in order to (a) check if the consequences of the decision were the same as envisioned; (b) identify opportunities for refinement of the risk controlling plan; (c) help provide feedback for future decisions about how to control new or current risks that are not responding to risk aversion, or whose nature has changed over time.

Risk tracking and monitoring processes involve establishing and maintaining risk status information, redefining action plans, and taking action based upon the status information (Higuera et al. 1994). For this, an objective, timely and accurate set of risk metrics-measures (qualitative and quantitative) for particular aspects of risks-has to be identified and critical threshold-values have to be determined (see Section 6). Metric values and triggers, i.e. the values of risk metrics that reflect a significant change or event occurring within a program, have to be monitored regularly. The essential elements of risk tracking and control are very similar to the equivalent processes in traditional program or project management and can be readily integrated into a program’s established tracking and control processes and methods.

illustration not visible in this excerpt

Fig. 10: Input/Output Flow Diagram for the Risk Monitoring Phase.

Each risk being tracked has an update cycle and associated schedule for these updates. Typically, status is updated at least monthly, but particular risks may require a more frequent update, particularly when the program is entering a critical phase. These updates ensure that the data reported is current and accurate.

A summary presentation of metrics, status indicators, and triggers consolidates information from many sources throughout the program into a manageable number of indications. Since each program has its own set of needs for risk-tracking metrics, adding risk tracking involves selecting from a set of recommended methods, those that best fit a program’s needs and that most effectively integrate into the program’s existing tracking measures and processes (Higuera et al. 1994).

4 Software Engineering Process Modeling

For organizations involved in designing and developing complex products like software, the efficiency and predictability of the processes they are engaged with is essentially important and can represent a competence that provides competitive advantage (Browning & Eppinger 2002). Therefore, we think, for pro-active risk management in software engineering projects process management is a crucial issue. But what do we exactly mean with the term “process”? According to IEEE 1074, a process is “a set of activities that is performed towards a specific purpose” (for example requirements, management and delivery). The IEEE standard lists a total of 17 processes (see Tab. 3). The processes are grouped into higher levels of abstractions called process groups. The single processes are again composed of grouped activities that are assigned to a person or team. During the life cycle modeling (see Section 4.1), the project manager customizes these activities for a specific project.

illustration not visible in this excerpt

Tab. 3: Process Groups and Sub-Processes in Software Life-Cycles (IEEE 1074).

There are various motivations for implementing standardized and formalized development processes with unified methodologies and easy-to-understand visual languages for specifying, constructing, and documenting software-intensive systems in organizations. We will discuss three primary benefits below:

A good, standardized, and shared process can establish a common “language” in organizations which are performing their business in multiple countries and cultures or with different subsidiaries as a result of of either natural growth or business mergers. The lack of a mutual language and a common way of understanding within one organization causes often friction and inefficiency in the business processes (Charette 1989).

[...]

---

¹ First elements of a theory of risk can already be found over 340 years ago at Antoine Arnauld. The theologist and jurist from Paris came up in 1662 with a relatively scientific understanding of risk, that is very similar to modern definitions: “Fear of harm ought to be proportional not merely to the gravity of the harm, but also to the probability of the event.”

² The most common example for types of insurances that cover also dynamic risks are liability insurances (e.g. management liability, car liability) and constitute a relatively new insurance instrument. However, taken into consideration that a health insurant can also influence his own health in different ways, dynamic insurances can be seen as a much older instrument.

³ The etymological origin of the word “risk” goes back to the Greek word “riza” (translated “cliff”, “crag”). Adapted to the Latin language, the related verb “risicare” got the meaning of “sailing around cliffs” and did represent a dangerous adventure. Thus, the original meaning of risk is indeed only connected to potential

⁴ In contrast to the meaning of “risk”, the word “hazard” covers only the negative aspects of risk. However, in the insurance economics and the whole financial world risks and chances are never treated separately but have to be assessed and quantified as one phenomenon.

⁵ See e.g. Charette (1989), Keil et al. (1998).