|
Article Excerpt
1. Introduction
Statistical process control (SPC) charts are often used to monitor key performance characteristics of a production process, such as the process mean, and to detect any irregularities represented by gradual or sudden shifts in those quantities. While the correct detection of shifts is of great importance, timely detection of those shifts is equally critical. The application context determines the type of performance characteristic used to track the status of the process. For example, a specialist in computer network security may want to detect network intrusions as soon as they occur by closely tracking shifts in the mean number of audit events in successive 1-second time intervals (Kim and Wilson, 2006; Kim et al., 2007). On the other hand, a manufacturing engineer seeking to use a coordinate measuring machine in an implementation of SPC may want to track shifts in the standard deviation of measurement error that might indicate the operator is having problems while using the machine.
In this article, we take the performance characteristic to be the process mean; and we seek an SPC procedure for rapidly detecting shifts in the mean of an autocorrelated process, without any assumptions about the specific functional form of the probability law governing the monitored process. We let [ARL.sub.0] denote the in-control average run length--that is, the expected number of observations taken from the monitored process when it is in control (and thus yields the desired value of the process mean) before a false out-of-control alarm is raised. Similarly, let [ARL.sub.1] denote the average run length corresponding to a specific out-of-control condition--that is, the expected number of observations taken from the monitored process before a true out-of-control alarm is raised when the process mean deviates from its in-control value by a specific amount. Among several SPC charts that yield a user-selected value of [ARL.sub.0], we prefer the chart that yields the smallest values of [ARL.sub.1] corresponding to a range of relevant out-of-control conditions.
Remark 1. Note carefully that throughout this article, the term average run length refers to the expected value of the number of individual observations, not samples, taken from the monitored process before an alarm is raised. Each time the monitored process is tested for an out-of-control condition, the sample size required to compute the relevant test statistic depends on both the SPC chart being used and the process being monitored, as explained below; and this sample size often exceeds unity. Our usage of the term average run length is necessary so that our experimental comparison of different distribution-free SPC charts can be carried out on a consistent basis.
Montgomery (2001) explains how to calculate control limits for the classical Shewhart and tabular CUSUM charts when those charts are used to monitor shifts in the mean of a process that consists of independent and identically distributed (i.i.d.) observations sampled from a known normal distribution. However, it is rarely the case that the exact distribution of the monitored process is known to the user of an SPC chart, and there is always the risk of simply assuming a wrong distribution. This can cause an excessive number of false alarms or an insufficient number of true alarms owing to miscalibrated control limits, ultimately resulting in excessive operating costs for the chart. Naturally, one can resort to distribution-free charts instead; but obtaining appropriate control limits becomes more difficult when those control limits must work for every possible distribution of the monitored process. For this reason, we study distribution-free charts whose control limits can be obtained by an automated technique that does not require either an excessively large training data set or cumbersome trial-and-error experimentation with such a training data set.
Beyond the problem of the monitored process having an unknown distribution (which is sometimes markedly non-normal), in many SPC applications it is simply incorrect to assume that successive observations of the monitored process are independent--especially in applications involving relatively short time intervals between those observations or repeated measurements taken by the same operator on the same unit or workpiece. When classical SPC charts for i.i.d. observations are applied to autocorrelated processes, those charts may perform poorly in terms of the values of [ARL.sub.0] and [ARL.sub.1] (Rowlands and Wetherill, 1991). Maragah and Woodall (1992) show that retrospective Shewhart charts generate an increased number of false alarms when they are applied to processes with positive lag-one autocorrelation. For correlated data, Runger and Willemain (1995) use non-overlapping batch means as their basic data items and apply classical Shewhart charts designed for i.i.d. normal data, exploiting the well-known property that under broadly applicable conditions, the batch means are asymptotically i.i.d. normal as the batch size increases. For brevity, the chart of Runger and Willemain (1995) is called R&W in the rest of this paper.
Johnson and Bagshaw (1974) and Kim et al. (2007) develop CUSUM charts that use individual (raw, unbatched) observations as the basic data items; and in the rest of this article, these charts are called J&B and DFTC, respectively. Lu and Reynolds (1999, 2001) investigate the performance of the exponentially weighted moving average (EWMA) and CUSUM charts for a specific class of autocorrelated processes--namely, stationary and invertible first-order autoregressive-first-order moving average processes, which will simply be called ARMA(1, 1) processes in the rest of this article. For this relatively specialized class of monitored processes, Lu and Reynolds conclude that the CUSUM and EWMA charts perform similarly when monitoring shifts in the process mean. However, the performance of such a model-based chart can be severely degraded when the hypothesized stochastic model on which the chart is based deviates significantly from the true probability law of the monitored process; and in general, definitive validation of a specific stochastic model for the monitored process can be difficult. Moreover, calibrating the control limits for a model-based chart can be extremely time-consuming unless the user is provided with an automated procedure for performing this calibration.
When developing distribution-free SPC charts, we must use one or more parameters of the monitored process, or suitable estimates of these parameters, to determine the control limits that yield the desired value of [ARL.sub.0]. Such parameters include the marginal mean, the marginal variance, and the variance parameter of the monitored process. As explained in Section 2.1, the variance parameter is the sum of covariances at all lags for the monitored process. The R&W chart uses the marginal variance of the batch means to calculate its control limits; and this quantity can be estimated by the usual sample variance of the batch means--provided the batch size is sufficiently large so that the batch means are approximately uncorrelated, and the batch count is sufficiently large to yield a stable estimator of the batch-means variance. To calculate control limits for the J&B and DFTC charts, we must know the exact values of the marginal variance, and the variance parameter of the monitored process.
In Johnson and Bagshaw (1974), Bagshaw and Johnson (1975), and Kim et al. (2007), the experimental studies of the J&B and DFTC charts were performed assuming exact knowledge of the relevant parameters of the monitored process. While such an assumption is convenient for performing simulation experiments, in most practical applications the user of an SPC chart does not know the exact values of these parameters. Instead, the process parameters must be estimated from a training data set (also called the Phase I data set) that is collected when the target process is known to be in control; then during the course of regular operation, the corresponding control limits can be used to monitor the working data set (also called the Phase II data set) for shifts that may occur in the future. When the monitored process is autocorrelated, accurately estimating the variance parameter can be substantially more difficult than accurately estimating the current mean of the process; and inaccurate variance estimators can severely degrade the performance of any SPC chart in which such estimators are used. Jensen et al (2006) provide a comprehensive literature review on the use of parameter estimation in SPC charts and recommend that the control limits should be updated as more data become available.
Fortunately, the simulation literature provides a number of variance-estimation techniques based on the following methods for analysis of steady-state simulation outputs: autoregressive representation (Fishman, 1971); non-overlapping batch means (Fishman and Yarberry, 1997); overlapping batch means (Alexopoulos et al., 2007b); spectral analysis (Lada and Wilson, 2006); and standardized time series (STS) (Schruben, 1983). Although accurate and efficient estimation of the variance parameter is an important research problem by itself, in this article we are more interested in developing automated variance-estimation procedures that can be effectively incorporated into distribution-free SPC charts. Building on the work of Kim et al. (2007), we formulate DFTC-VE, a distribution-free tabular CUSUM chart in which the marginal variance, the variance parameter, and the chart's control limits are estimated from a training data set automatically--that is, without the need for any intervention or trial-and-error experimentation by the user. We compare DFTC-VE's performance with the performance of competing distribution-free charts that also incorporate variance estimation. In addition, we study how the use of our automated variance estimators affects the performance of distribution-free SPC charts that are designed to use the exact values of the marginal variance and the variance parameter of the monitored process.
The rest of this article is organized as follows. Section 2 provides background information, including some motivating examples, notation, and key assumptions. Section 3 details the following alternative variance-estimation techniques that have been adapted from the simulation literature for automated use in DFTC-VE: (a) an overlapping area variance estimator based on the method of STS; and (b) a less computationally intensive variance estimator based on a simplified combination of the methods of autoregressive representation and non-overlapping batch means. Section 4 presents the proposed DFTC-VE: chart for rapidly detecting shifts in the mean of an autocorrelated process. Section 5 summarizes our experimental performance evaluation of DFTC-VE versus the following: (i) distribution-free charts that use the exact values of the marginal variance and the variance parameter; and (ii) distribution-free charts that incorporate either of the variance-estimation procedures (a) or (b) above. We use the following types of test processes at various points in Sections 2 and 5: stationary first-order autoregressive (AR(1)) processes; stationary first-order exponential autoregressive (EAR(l)) processes; stationary and invertible ARMA( 1, 1) processes; and an M/M/1 queue-waiting-time process. Section 6 contains conclusions and recommendations for future study. The online Appendix contains the proof of a key result underlying DFTC-VE's variance estimator (b) above, together with tables of standard errors for all the estimated ARLs reported in Section 5.
2. Background
2.1. Notation and assumptions
Throughout this article, we distinguish two sets of data: (i) a training (or Phase I) data set {[X.sub.i]: i = 1, 2, ...,n) consisting of individual observations taken from the target process when it is known to be in control; and (ii) a working (or Phase II) data set {[Y.sub.i]: i = 1, 2, ...} consisting of individual observations taken from the target process when it must be monitored for deviations from the in-control condition. We assume that the Phase I process {[X.sub.i]: i = 1, 2, ..., n} is covariance stationary with [mu] = E[X.sub.i] and [[sigma].sup.2] = E[([X.sub.i] - [mu]).sup.2] respectively denoting the marginal mean and variance of the process. The usual sample mean and variance of the training data set,
[^.[mu]] = [bar.X](n) = [n.sup.-1] [n.summation over (i = 1)][X.sub.i] (1)
and
[[^.[sigma]].sup.2] = [S.sub.n.sup.2] = [(n - 1).sup.-1] [n.summation over (i =...
|
