|
Article Excerpt
ABSTRACT: Effective management of knowledge is essential for a CPA firm to remain competitive. Use of computational models of judgment processes and outcomes causes knowledge to be available for use and analysis. We present a comprehensive and integrated computational model of the difficult and knowledge-intensive judgments needed for successful audit planning. The model concludes on a client's going-concern status, applicable levels of inherent, control, and planned detection risk, and appropriate levels of statement- and account-level materiality. Most importantly, the model validly identifies the cause of significant fluctuations given causal hypotheses. The context is the sales and collection cycle of a manufacturing client. The model consistently replicates causal hypothesis judgments generated by the modeled auditor who exhibits considerable judgment expertise, i.e., his (1) judgments typically coincide with actual causes. Concerning judgment expertise, the model reveals numerous linkages among judgments, subtle interdependencies in cue importance across judgments, and new findings concerning cue diagnosticity.
Keywords: analytical procedures; audit planning; audit risk; computational model; going concern; knowledge management.
Data Availability: Contact the first author.
I. INTRODUCTION
Effective management of knowledge within a CPA firm is essential for superior performance (Havens and Knapp 2001). Traditional methods of making knowledge available include large investments in extensive training, mentoring, and review practices. Contemporary knowledge management technologies are more proactive using virtual group collaboration tools and access to previous work products, as well as directories that facilitate access to professionals within a finn. However, making the contextual details of expert reasoning processes available interactively to audit personnel for specific client engagements may result in more effective auditing.
Computational modeling is designed to capture and communicate interactively an expert's detailed knowledge, including the context-dependencies of an expert's reasoning and judgment conclusions (Bigg's et al. 1993; Meservy et al. 1986). Fundamentally, computational modeling makes available a process-oriented theory of how to perform a semi-structured judgment task that requires considerable task-specific knowledge (Bailey et al. 1988; Biggs 1991; Peters 1990, 1993; Wright and Willingham 1997). (2) Planning of an audit is such a task: an opportunity for a computational model to contribute to effective knowledge management exists.
A series of crucial risk-oriented judgments is required when an auditor plans an audit (e.g., Koonce 1993; Arens and Loebbecke 1997, 218-228, 330-331). Researchers have addressed how, and how well, auditors perform several of these judgments. One productive approach is to conduct an experiment, concentrate on a single judgment, or an aspect of a single judgment, and investigate chosen cognitive hypotheses concerning judgment outcomes (e.g., Bell and Wright 1995; Libby 1995). Alternatively, more comprehensive computational models that include more than one judgment have been researched, e.g., concentration on the going-concern judgment (Biggs et al. 1993), materiality judgments (Steinbart 1987) and risk assessments (Peters 1990). Since auditors reach several crucial risk-oriented judgments simultaneously during planning of an audit, we report a computational model of audit planning that includes them.
We report new findings concerning judgment expertise, i.e., interdependencies among several of these risk-oriented judgments and new ideas concerning the importance of information cues (see Section V). For example, identification of account balances that may be misstated, and the causes thereof, may be based on going-concern, risk, and materiality judgments. Therefore, we model simultaneously use of knowledge for the judgments and the linkages among the judgments (see Figures 1 and 2). To our knowledge, this is the first study to research simultaneously the entire sequence of crucial audit-planning judgments.
[FIGURES 1-2 OMITTED]
The remainder of the paper is organized as follows. In Section II, we elaborate on computational modeling research and explain its contribution to knowledge management and CPA firm effectiveness. The specifics of the research method are presented in Section III. Section IV presents the details of the model's reasoning and use of information. Several new findings concerning cognitive interdependencies are reported in Section V. Section VI provides results of testing of the model's conclusions. The final section summarizes our findings, particularly concerning cognitive linkages among judgments and the impact of risk factors, as well as limitations of the study and ideas for future research.
II. KNOWLEDGE MANAGEMENT AND COMPUTATIONAL MODELS
Capturing the Reasoning of Experts Using Computational Modeling
Computational modeling results in a detailed judgment process representation that is specific and internally consistent, including likely use of both qualitative and quantitative information. The representation can reflect the actual complexity of audit judgments (Gibbins and Jamal 1993) compared with other methodologies such as a series of experiments or statistical modeling of past judgments. Testing of the model will determine whether the model emulates the expert's judgment process and produces the judgments expected from the modeled expert.
While the potential of computational modeling to reveal judgment processes is substantial, so are the costs (Biggs 1991, 22-27; Biggs et al. 1993, 83; O'Leary 1993; Peters et al. 1989). The method is effective in producing a process theory for complex, demanding judgment tasks. However, given the complexity of the method, the effort required by the researcher is considerable and the process is time-consuming. Also, less control over the data gathering from experts is achieved relative to other knowledge-capturing methodologies, e.g., carefully controlled experiments.
The design of computational models, especially as knowledge management tools, is different from the design of "expert" systems. Expert systems (and decision support systems), provide optimal conclusions using whatever knowledge, analytical or statistical tools may be helpful (Vinze et al. 1991). In contrast, a computational model is a detailed, authentic representation of one person's reasoning, including any imperfect heuristic mental procedures (Meservy et al. 1986). Such a model must be "true to" the reasoning of the expert; researchers of expert systems are not so constrained. Messier (1990, 104-106) provide s additional insights on this distinction.
Computational Modeling as a Knowledge Management Tool
The objective of researching a computational model is to reveal the detailed, contextual reasoning of an expert; the objective of knowledge management is to make this knowledge available to others. Knowledge management practices include making information contextually available from databases, using knowledge repositories (Markus 2001), managing social networks to facilitate awareness of, and access to, specialists (McDonald and Ackerman 1998), and exchanging ideas and work products via enterprise software applications (e.g., Lotus Notes and intranets). Knowledge-oriented decision support systems also facilitate knowledge management and organization learning (Bhatt and Zaveri 2002). However, these practices do not permit interactive access to the detailed, contextual reasoning of different experts--expertise that, given the composition of an audit team, may not otherwise be available.
Use of computational judgment models (Biggs 1991; Meservy et al. 1986) as knowledge delivery tools provides several advantages. First, each model indicates interactively expert reasoning and judgments for each specific client situation, including the information that would be used. The modeled experts would achieve high levels of recognition and differential compensation, providing incentives for them to continue to share their knowledge. Second, an auditor could achieve learning of reasoning via "what if" changes in the client situation, assumptions made and/or the information available. In this context, computational models minimize the "psychological cost of asking," i.e., the loss of status and expected reciprocity in the future, when knowledge requests are made (McDonald and Ackerman 1998, 2). Third, the models constitute part of the organization's learning and memory (Huber 1990); auditors may discuss, compare, and critique the reasoning in each model. Bhatt and Zaveri (2002, 298) indicate that "organizational learning is not a simple aggregate sum of individual learning but is an exchange and sharing of individual assumptions and models throughout the organization." CPA firms could use computational models to centralize the desired uses of evidence and knowledge, and to ensure more consistent and appropriate reasoning, more judgment consensus and fewer errors (Huber 1990, 56-57). Fourth, opportunities for making knowledge available during staff training are apparent.
III RESEARCH METHODOLOGY
To acquire the necessary judgment process data for a computational model, Peters et al. (1989, 362-364) recommend five phases: (1) task analysis, (2) exploratory interviews, (3) structured interviews, (4) detailed problem-solving sessions, and (5) final synthesis--with model building occurring throughout the process (cf., Biggs 1991, 13-15; Gibbins and Jamal 1993, 459-462). We followed these five phases with 42 formal sessions averaging three hours each with the expert (see below).
A thorough task analysis can be helpful because "We can often learn a great deal about how problems are solved by considering how they could be solved. That is, a task analysis of problems can provide information about constraints that the nature of the problem imposes on the nature of the problem solver" (Holyoak 1990, 118). We based the task analysis on the research literature (e.g., Biggs et al. 1988; Bedard and Biggs 1991 a; Koonce 1993), Statements on Auditing Standards, and auditing textbooks (e.g., Arens and Loebbecke 1997; Messier 1997; Knechel 1998), and reviewed previous research findings. The task analysis revealed important reasoning concepts.
The second phase consisted of a series of exploratory interviews conducted with a highly experienced "expert" audit partner from a Big 4 firm who decided to start teaching auditing. He continues to be very active professionally. For example, he served as an on-site reviewer for CPA firm peer reviews conducted by the Public Oversight Board. As a result of the exploratory interviews, the expert revealed: (1) the amount of judgment process detail needed and (2) different judgment processes that could be used (cf., Gibbins and Jamal 1993; Peters et al. 1989, 364). It also became apparent that focusing only on specific judgments does not capture the interdependencies of the expert's judgments during planning of an audit.
During the third phase, we used structured interviews, an effective and efficient knowledge acquisition technique (Agarwal and Tanniru 1990). The expert reviewed, and suggested changes to, an initial representation of his reasoning. He also considered information and advice from both an auditing professor and a former audit senior. The expert provided his detailed reasoning for each judgment, what information was required, and how this information should be used. We then transformed the initial representation into a more complete model.
Next, we conducted an extensive series of focused problem-solving sessions with the expert. Use of risky audit situations with considerable contextual detail focused the expert's attention on the applicable reasoning. We refined the model to reflect more accurately the expert's contextual reasoning.
In the final phase, we reviewed all of the judgment process information. This included the type of information required for each judgment, different conclusions given different situations, and the linkages among the judgments and the evidence. We discussed and resolved any inconsistencies. In addition, we performed testing to assure that the model's conclusions were consistent with those of the expert (see Section VI below).
IV. A KNOWLEDGE-BASED MODEL OF AUDIT-PLANNING JUDGMENTS
An Overview of the Model
The model first generates a representation of the client's situation (Koonce 1993) by applying declarative and procedural knowledge (Anderson 1983). Declarative knowledge, being experience-based and situational, adds contextual meaning to audit evidence. Application of procedural knowledge generates intermediate and final conclusions. Starting with the goal of completing analytical procedures, the model backward chains to obtain contextually implied subgoal conclusions. Active information therefore includes facts, declarative knowledge, direct contextual judgments, and subgoal conclusions (Holland et al. 1986, 41). Frames represent declarative knowledge and productions represent procedural knowledge. (3) The model handles missing information by applying default values suggested by the expert. Consistent with this representation, the expert frequently expressed his reasoning in if-then language (see the final section).
Concerning applicable client situations, the expert suggested that the following challenging auditing situations are manageable given our research objectives. First, the model is designed for a manufacturing client with one major product line, little revenue from services, i.e., less than 10 percent, and annual revenue of $100-$500 million. Second, the client is publicly held and has been audited by the current audit firm for at least one year prior to the year being audited. We did not include new audits because audit procedures can vary significantly for a first-time audit. Third, the client has not completed a major acquisition or disposition during the past three years. A recent acquisition or disposition could result in significant changes in the operations of the client and, therefore, would significantly affect how the audit is performed. Finally, all detailed testing is performed after the year-end numbers are available.
Assessment of the Preliminary Going-Concern Status of the Client
Research on clients' going-concern status has evolved in two directions: professional judgment (e.g., Ho 1994; Choo and Trotman 1991; Asare 1992; Ricchiute 1992; Selfridge et a1.1992; Rosman et al. 1993; Hopwood et al. 1994; Choo 1996) and statistical models (e.g., Levitan and Knoblett 1985; Bell et al. 1990; Knapp 1991). An auditor's judgment performance depends...
|
