...implement processes.

We studied SPI in its organizational context through collaborative practice research (CPR), particular form of action research. The CPR program involved close collaboration between practitioners and researchers over a three-year period to understand and improve SPI initiatives in four Danish software organizations. The problem of understanding and managing risks in SPI teams emerged in one of the participating organizations and led to this research. We draw upon insights from the literature on SPI and software risk management as well as practical lessons learned from managing SPI risks in the participating software organizations.

Our research offers two contributions. First, we contribute to knowledge on SPI by proposing an approach to understand and manage risks in SPI teams. This risk management approach consists of a framework for understanding risk areas and risk resolution strategies within SPI and a related process for managing SPI risks. Second, we contribute to knowledge on risk management within the information systems and software engineering disciplines. We propose an approach to tailor risk management to specific contexts. This approach consists of a framework for understanding and choosing between different forms of risk management and a process to tailor risk management to specific contexts.

Keywords: Risk management, software process improvement, action research, collaborative practice research

**********

Introduction

Software process improvement (SPI) is a continuous and evolutionary approach to improve a software organization's capability to develop quality software in response to customer requirements (McFeeley 1996). The approach emphasizes stepwise improvement of software processes, systematic assessment of an organization's current operation, and application of normative models for organizing a software operation. These models describe different levels of software process maturity. They also serve as a basis for assessing current practices and as a guide for directing improvement initiatives. The best-known of these maturity models are the Capability Maturity Model (CMM) (Paulk et al. 1993), Bootstrap (Kuvaja and Bicego 1994), and Software Process Improvement and Capability dEtermination (SPICE) (Rout 1995).

Anecdotal evidence suggests that SPI initiatives have led to dramatic improvements of productivity, cycle time, and quality (Diaz and Sligo 1997; Haley 1996; Humphrey et al. 1991). Recent data from the Software Engineering Institute (SEI) at Carnegie-Mellon University (SEMA 2002) on firms that engage in SPI initiatives suggest, however, that there is a high number of failures. Out of 1,638 organizations self-reporting initial assessments, only 34 percent had proceeded to a second assessment. Of those that proceeded, 13 percent did not improve their capability to develop quality software and 3.1 percent moved to a lower level of capability. The time frame to move up one level (out of five) varied from 16 to 32 months. These numbers are not surprising. SPI efforts are complex change processes in which software organizations seek to change the conditions for and the actual behavior of the professionals involved in the software operation (Aaen et al. 2001). SPI initiatives, as well as other change initiatives, are faced with a number of risks (e.g., lack of management support, inability to learn from experiences, an overly strong belief in technical solutions, and resistance to change) that make it difficult to successfully improve the software operation (Grady 1997; McFeeley 1996; Zahran 1998).

This research was initiated at the IT department of Danske Bank, one of the largest financial institutions in Scandinavia. The IT department was one of four organizations involved in a large-scale Danish research program from 1997 to 2000 (Mathiassen et al. 2002). The aim of the program was to improve the software operation in the participating organizations and to contribute to the body of knowledge on how to design, conduct, and manage SPI efforts. The SPI teams at Danske Bank's IT department found it difficult to set up, organize, and manage their efforts in ways that would lead to satisfactory results. Some team members had positive experiences using risk management in software projects so they decided to address the problems they faced by adopting an approach to analyze risks in SPI teams (Grady 1997; Humphrey 1989; McFeeley 1996; Statz et al. 1997). This approach left many questions unanswered, it did not provide the SPI team with a good overview of their project, and it was difficult for the team to reach a shared understanding of the situation. There were no other approaches available for managing SPI risks so the team members asked us to help them address risks as an integral part of their efforts.

Our research was based on two questions: At the specific level, how can SPI teams within Danske Bank's IT department understand and manage risks to help achieve satisfactory results? In general, how can risk management help SPI teams understand and manage their efforts? We wanted to solve a specific problem in Danske Bank in collaboration with the SPI team members, and in doing so we wanted to make progress toward improved knowledge about SPI and risk management. Our research was therefore carried out as action research to help facilitate change within Danske Bank, and at the same time to pursue our research interests (Avison et al. 1999; Baskerville and Wood-Harper 1996; Checkland 1981; Hult and Lennung 1980).

Action research, as originally proposed by Lewin (1951) and influenced by work at the Tavistock Institute (Rapoport 1970; Trist 1976), uses intervention into problematic social situations as a means to develop scientific knowledge. Different action research approaches have been developed, one of the best known being Susman and Evered's action research cycle consisting of diagnosing, action planning, action taking, evaluating, and specifying learning (Davison et al. 2004; Susman and Evered 1978). Also, action research has been adopted and developed successfully as an approach to information systems research (Avison et al. 1999; Baskerville and Wood-Harper 1996; Checkland 1981; Hult and Lennung 1980). Our research followed a particular form of action research called collaborative practice research (CPR) (Mathiassen et al. 2002). CPR was developed as part of a Scandinavian information systems research tradition during the 1980s and 1990s and has the following characteristics (Checkland and Scholes 1990; Mathiassen 1998): First, the aim is to understand, to develop support for, and to improve specific professional practices within the participating organizations. Second, the activities are carried out in close collaboration between researchers and the involved practitioners. Third, the research process is guided by a pluralist methodology (Mingers 2001), with action research as the dominant approach and other conventional methods (e.g., case studies or field experiments) as supplementary approaches. Finally, each CPR effort can lead to a portfolio of focused research projects based on the ongoing and emerging problem-solving efforts in the participating organizations (see Mathiassen 2002; Mathiassen et al. 2002). The Danish SPI research program was organized according to CPR; one of the focused action research projects within the program is reported here.

Our research combines two streams of theory. First, it draws upon the literature on SPI. A number of comprehensive texts are available (e.g., Grady 1997; Humphrey 1989; Zahran 1998), a number of surveys of the SPI literature have been developed (Aaen et al. 2001; Fuggetta and Picco 1994; Paulk 2002), and there is an ongoing, critical debate about the feasibility and practicability of SPI initiatives (Bach 1995; Bollinger and McGowan 1991; Brodman and Johnson 1995; Curtis 1994; Fayad and Laitinen 1997; Herbsleb et al. 1997; Humphrey et al. 1991; Ngwenyama and Nielsen 2003). Second, we found inspiration in the software risk management literature. Many approaches have been developed to cope with software risks (Alter and Ginzberg 1978; Boehm 1991; Charette 1989; Davis 1982; Fairley 1994; McFarlan 1981). These approaches help practitioners question critical assumptions underlying specific projects and identify and handle critical incidents that threaten the success of their projects (Lyytinen et al. 1998).

The research offers two contributions. First, we propose an approach to understand and manage risks in SPI teams. The approach consists of a framework for understanding risk areas and resolution strategies within SPI and a related process for managing SPI risks. Second, we contribute to knowledge on risk management within the information systems and software engineering disciplines. We propose an approach to tailor risk management to specific contexts. This approach consists of a framework for understanding and choosing between different forms of risk management and a process to tailor risk management to specific contexts. The next three sections describe the theoretical framework, the research approach, and the research practice. After that, we present the results and discuss the research in relation to criteria for CPR-based action research. We conclude by summarizing the results and their implications for both practice and research. The detailed risk and action tables for the SPI risk approach are included as an appendix.

Framework

Software Process Improvement

SPI covers a wide range of activities, from basic project management disciplines such as project planning and tracking to sophisticated continuous improvement of development processes (Caputo 1998; Grady 1997; Humphrey 1989; Zahran 1996). A major driver behind this paradigm has been the world's largest consumer and producer of software, the U.S. Department of Defense. Faced with increased reliance on software suppliers, the Department of Defense established SEI in 1984 to guide software-developing organizations toward better practices.

One characteristic that distinguishes SPI from earlier improvement paradigms is that efforts almost always are initiated by an assessment of current practices. The purpose is to find out where improvements are needed most and can be applied with the greatest effect. In the improvement cycle, such an assessment is repeated every 12 to 18 months (Dunaway and Masters 1996; Jansen and Sanders 1998; McFeeley 1996). Another characteristic is the use of an underlying model. The most influential and popular of these models is the CMM (Paulk et al. 1993), which was developed at SEI (see Figure 1).

In CMM, higher levels indicate higher maturity of the organization's capability to develop software. Each level is characterized by a set of key process areas (e.g., project management, configuration management, quality control) that an organization should practice adequately to be on that level. The assessment will characterize the level of maturity and recommend which processes to improve. These processes usually will be found within the model. The results from the assessment are used to generate a strategy for improving some or all of the areas detected in the assessment. The strategy typically has the mentioned time frame of 12 to 18 months and it involves forming several SPI teams--one for each improvement area. With some coordination among the teams, and involvement of the rest of the organization, the improvements will be piloted and, if found to be adequate for the organization, implemented and institutionalized throughout the organization. The experiences from this entire effort are analyzed and followed by a new improvement cycle.

This overall approach to SPI is well described in the IDEAL model (McFeeley 1996) developed at SEI in response to problems experienced by organizations involved in SPI. The IDEAL model provides a cyclical process for SPI that is described in five steps (see Figure 2).

1. Initiating the SPI effort. This involves setting goals, obtaining commitment, and establishing an improvement infrastructure.

2. Diagnosing current practices through a maturity assessment. This typically is based on a maturity model, which is used to characterize the current state and develop and prioritize recommendations for improvements.

3. Establishing specific, focused improvement initiatives. An SPI team is established to deal with each of the recommended improvement areas from step 2.

4. Acting out these initiatives. The SPI teams develop and implement solutions for each improvement area.

5. Learning based on the results and experiences from the initiatives. Data on the improvements are collected and preparations are made for a new maturity assessment.

In most organizations, SPI efforts consequently are organized at two levels (Grady 1997; McFeeley 1996; Zahran 1998): The organizational level with the software engineering process group (SEPG) (Fowler and Rifkin 1990) and the more specific project level with dedicated SPI teams (see Figure 3). At the organizational level, SPI is organized as a long-term effort aimed at evolutionary improvements to the maturity of the software organization. The main responsibilities of the SEPG include conducting maturity assessments, organizing and coordinating SPI teams, and ensuring management involvement and support for SPI. At the project level, the SPI teams are charged with carrying out activities to improve a specific process within the organization, as shown in Figure 3.

[FIGURE 1 OMITTED]

Despite models and frameworks developed to assist practitioners in carrying out SPI projects, such efforts remain difficult and risk-filled, as described in the introduction. We therefore agree with others that SPI initiatives can benefit from risk management (Grady 1997; Humphrey 1989; McFeeley 1996; Statz et al. 1997). Several key sources on SPI mention risk items and risk resolution actions related to specific SPI issues (Grady 1997; Humphrey 1989; McFeeley 1996). The IDEAL model prescribes, for example, that risks should be handled at both the organizational level through a document called "SPI Strategic Action Plan" and at the project level through a document called "Tactical Action Plan." However, no specific risk items or resolution actions are mentioned. Risk issues are only addressed implicitly through the advice provided by the IDEAL model. The initiating phase, for instance, recommends setting improvement goals and establishing senior management commitments and sponsorships, without which the program would have no direction and be in grave danger of cancellation. The key sources on SPI provide no comprehensive treatment of SPI risk management and no systematic advice for practitioners attempting to mitigate risks.

[FIGURE 2 OMITTED]

[FIGURE 3 OMITTED]

The only published work dealing explicitly with risk in SPI is a simple approach based on 63 risk items organized into 13 categories (Statz et al. 1997). This approach can be used to identify risks at both the organizational level and the project level. Statz et al. recommend that SPI teams conduct a post-project review during which they evaluate how successfully risks were managed during the improvement project. They also recommend that the SEPG performs similar evaluations, which should be used to update the organization's list of SPI risk items for use in subsequent applications of the approach. The approach is straightforward to use, but it provides no explicit guidance in identifying risk resolution strategies beyond calling for the project members to identify actions for each high-ranking risk. Moreover, it does not help team members develop a shared, strategic overview of the SPI project. As such, there is currently no comprehensive approach to discover and mitigate the many risks that SPI teams face.

Software Risk Management

Risk management has been adopted and developed within a variety of areas, including warfare, space exploration, nuclear reactors, security, and financial investments. In this case, we chose to focus on risk management approaches within software development. Risk management ideas have been applied successfully to software development over the past decades in response to various forms of system failure. There is, consequently, a rich and differentiated literature on software risk management (Lyytinen et al. 1998), and this literature is inspired by and draws upon insights from other areas of risk management. In addition, the SPI practitioners at Danske Bank's IT department had experience with software risk management and knew several approaches to manage software risks.

A software risk denotes a particular aspect of a development task, process, or environment, which, if ignored, will increase the likelihood of project failure (Lyytinen et al. 1998). The degree of risk is assessed either in quantitative terms as the probability of unsatisfactory events multiplied by the loss associated with their outcome, or in qualitative terms by referring to the uncertainty surrounding the project and the magnitude of potential loss associated with project failure (Barki et al. 1993). Barki et al. suggest that software project managers see risk management as a key to success. Project managers in this survey believed that their ability to shape a project (in terms of internal integration, user participation, and formal planning) to fit its risk exposure influences the project's ability to meet budgets and produce quality results. The advantages of using risk management in software projects are that it helps the practitioners focus on many aspects of a problematic situation, it emphasizes potential causes of failure, it helps link potential threats to possible actions, and it facilitates a shared perception of the project among its participants (Lyytinen et al. 1996, 1998). Risk management approaches have been developed to identify, analyze, and tackle project portfolio risks (Earl 1987; McFarlan 1981), systems development risks (Barki et al. 1993; Boehm 1988; Charette 1989; Donaldson and Siegel 2001; Fairley 1994; Keil et al. 1998; Moynihan 1996; Ould 1999; Ropponen and Lyytinen 2000), requirements risks (Burns and Dennis 1985; Davis 1982), or implementation risks (Alter and Ginzberg 1978; Keen and Scott Morton 1978; Kwon and Zmud 1987; Lucas 1981; Lyytinen and Hirschheim 1987).

To build on this knowledge, we studied the ways in which software risk management approaches are designed. Lyytinen et al. (1998) suggest that risk items are used to detect risky incidents, resolution actions help identify possibly relevant actions, and different kinds of heuristics help link identified risks with possible resolutions. Based on this understanding, we identified four different ways in which approaches to software risk management address the three elements: risk items, resolution actions, and heuristics (see Table 1). We have assessed the relative strengths and weaknesses of the four types by comparing and contrasting their key features. We have selected exemplar approaches from the literature to illustrate each type.

First, there are...

NOTE: All illustrations and photos have been removed from this article.



More articles from MIS Quarterly
Dialogical action research at omega corporation (1).(Special Issue), September 01, 2004
Small business growth and internal transparency: the role of informati..., September 01, 2004
Design principles for competence management systems: a synthesis of an..., September 01, 2004

Looking for additional articles?
Search our database of over 3 million articles.

Looking for more in-depth information on this industry?
Search our complete database of Industry & Market reports by text, subject, publication name or publication date.

About Goliath
Whether you're looking for sales prospects, competitive information, company analysis or best practices in managing your organization, Goliath can help you meet your business needs.

Our extensive business information databases empower business professionals with both the breadth and depth of credible, authoritative information they need to support their business goals. Whether it be strategic planning, sales prospecting, company research or defining management best practices - Goliath is your leading source for accurate information.