|
Article Excerpt
The article proposes a simple Bayesian technique for auditing property appraisals to determine whether state accuracy guidelines are met. The proposed technique addresses elicitation of appraisers' prior beliefs, computation of reappraisal sample sizes and reporting of audit results. To facilitate communication of quantitative audit findings to nonstatistician stakeholders, the concept of variance appears nowhere in prior elicitation or reporting. In contrast to classical frequentist techniques, the Bayesian procedure easily integrates expert judgment and responds flexibly to the arrival of new information. In addition, the Bayesian procedure significantly reduces the number of reappraisals required to regulate appraisal systems when they are functioning well. The technique can be applied in other settings where government officials audit their own work and must convince overseers, especially the public, that accuracy requirements are satisfied.
**********
Public-sector property appraisers, whose responsibility is to compute annual property valuations for use by tax assessors and attest to their accuracy, are providers of technical expertise and what Walls and Quigley (2001) refer to as socio-technical services. Socio-technical services are those that require specialized communication skills for eliciting statistical information from nonstatisticians and for persuasively explaining quantitative issues to various stakeholders, in this case taxpayers, users of public services financed by property tax revenue and other constituencies in the local or regional political economy. The task of communicating algorithmic detail from the computation of property owners' tax bills and attesting to the effectiveness of quality control measures clearly fits the socio-technical label.
The focus of this article is the audit of public-sector property appraisals. Such audits require reappraisal of a relatively small number of properties using costly, in-depth appraisal methods and, thus, determination of an appropriate reappraisal sample size. The goal is to convince state officials and members of the public that property appraisals satisfy exogenously given standards of accuracy such as those required by state law. The auditor, or appraisal authority, must therefore confront the difficult challenge of communicating to nonstatisticians about second-moment phenomena, namely, risk and dispersion.
Diverse voices in the scientific community have remarked on the challenges of risk communication and the benefits of simplicity in a variety of economically significant settings (Simon 1982, Slovic 2000, Gigerenzer 2002). Psychologists writing in Science (Hoffrage, Lindsey, Hertwig and Gigerenzer 2000) showed that logically equivalent descriptions of disease frequencies (e.g., "three in 1,000" as opposed to 0.3%) in medical tests to screen for disease led patients to choose significantly different courses of treatment. In the design of learning systems in artificial intelligence, Simon, Valdes-Perez and Sleeman (1997) demonstrated that algorithmic complexity is often disadvantageous, not just because of computational costs, but because simple decision rules tend to be more robust in changing environments. Analyzing how economists construct persuasive arguments, McCloskey (1985) showed that the role of language reaches well beyond logical coherence and deductive chains relating axioms to theorems.
Confronting the communication issue in the context of an audit of property appraisals means having explanations for strategies used in determination of sample size, integration of expert judgment and the weighing of statistical benefits against rhetorical costs (in terms of algorithmic complexity) at the end-user stage. Complexity imposes costs whenever it impedes attainment of the audit's ultimate socio-technical goal, which is to quantitatively characterize appraisal accuracy in language that satisfies the constraints imposed by end-users' unfamiliarity with statistical jargon, including the concept of variance. In contrast to strictly technical property appraisal issues where few, if any, costs of complexity need be considered (e.g., the inherent statistical challenges of estimating the market value of infrequently traded real assets with large and correlated location-specific components), tools designed for socio-technical tasks such as public-sector audits must deal explicitly with algorithmic complexity and its effects on end-users. Complexity not only increases skill requirements (possibly requiring direct expenditures on consultants or additional in-house personnel), it can also jeopardize the political legitimacy of quantitative decision-making procedures because of difficulty in justifying intransparent black-box computations to nonexperts.
In the United States, United Kingdom and other European nations, legal definitions and customs concerning sufficiency of evidence are fundamentally ambiguous (Steele 1992). In practice, most auditors rely on rules of thumb, such as "choose n = 30," apparently with little justification. Some rely solely on expert judgment with virtually no statistical support.
In a number of U.S. states, state law specifies accuracy requirements for property appraisals. Given such exogenous requirements, compliance can be viewed as a binary outcome determined by comparing the allowable valuation error with the observed difference between two appraisals of the same property--one from an in-depth reappraisal regarded as a close approximation to true market value, the other derived from the standard, more economical appraisal procedure. In such cases, the main technical component of the audit problem is selecting an appropriate statistical model for the probability of noncompliance.
The Bayesian probability model proposed here has advantages in terms of both informational and cost efficiency. In contrast to audit methods based on classical statistics, it easily integrates the judgments of appraisal experts concerning local market conditions and information from previous audits. And because labor-intensive reappraisals are costly, the Bayesian procedure provides practical benefits by requiring smaller sample sizes in most cases. (1) Perhaps most important, the proposed procedure results in natural-language risk reporting derived from tail probabilities of the posterior distribution without invoking the concept of variance or other statistical jargon. (2)
The article is organized as follows. The next section reviews relevant work in the fields of property appraisal research, Bayesian audit methodology, Bayesian sample size determination, elicitation of expert judgment and risk communication. The third section describes classical approaches to sample size determination illustrating their limitations and, thus, the need for an alternative approach. The fourth section describes the article's main result, an algorithm for computing the minimum number of reappraisals required to achieve user-specified posterior confidence in the event of compliance. The fifth section presents examples and numerical results illustrating how the procedure works in practice. The final section concludes with a discussion of the broader issue of sufficiency of evidence in quality assurance tasks conducted on behalf of taxpayers and their representatives in local government.
Background
There is compelling evidence that commercial and public-sector property appraisals are systematically biased (Geltner 1989, Graff and Young 1999, Shiller and Weiss 1999, Dietrich, Harris and Muller 2000). Some argue that gaps between statistical moments of appraisal-value and market-value distributions are rooted in psychological biases, such as anchoring effects (Clayton, Geltner and Hamilton 2001), although there is disagreement about their magnitude and economic significance (Diaz 1997). As a rule of thumb, appraisal dispersion as indicated by standard deviation appears to be approximately 10% (Hansz and Diaz 2003), although feedback, which enables learning, and experience (Spence and Thorson 1998) can moderate this dispersion somewhat. Appraisal bias is important not only for reasons relating to disputes over property tax collection, but also in eminent domain cases (Adams, Jackson and Cook 2001) and as a potential predictor of mortgage default (LaCour-Little and Malpezzi 2003). There are also important normative issues relating to appraisal returns and risk hedging, where overly smooth appraisal-based real estate time series can unfortunately mask the true covariance structure between real estate and other asset categories (Geltner 1989, Gau and Wang 1990, Hendershott and Kane 1995, Lai and Wang 1998, Gunnelin, Hendershott, Hoesli and Soderberg 2004).
The real estate literature on appraisal technique rests largely on several classic approaches (Isakson 1986, Lusht 1987, Kang and Reichert 1991, Isakson 1998, Pace 1998). According to Roulac, Adair, Crosby and Lim (2004), however, most contemporary appraisal research is difficult for real estate professionals to put into practice. One reason for the apparent gap between theory and practice seems to be insufficient appreciation of the simultaneous importance and difficulty of communicating about risk (O'Hagan 1998, Hoffrage, Lindsey, Hertwig and Gigerenzer 2000, Gigerenzer 2002). A similar gap between theory and practice (pointed out by Pham-Gia (1997) and O'Hagan (1998)) applies both to the Bayesian audit literature (Baker 1977, Menzefricke 1984, Rohrbach 1986, Tamura and Frost 1986, Laws and O'Hagan 2002) and sample size selection literature (Hora 1978, Aigner 1979, Cox and Snell 1979, Laws and O'Hagan 2000).
Sample size selection models with attractive features that resemble...
|
