A Review of Artificial Neural Network Applications in Control. Chart Pattern Recognition

A Review of Artificial Neural Network Applications in Control Chart Pattern Recognition M. Perry and J. Pignatiello Department of Industrial Engineering FAMU - FSU College of Engineering 2525 Pottsdamer St Tallahassee, FL 32310-6046 Abstract On-line automated process analysis is an important area of research since it allows the interfacing of process control with computer integrated manufacturing (CIM) techniques. The inflexibility and high computational costs of traditional SPC pattern recognition methodologies have led researchers to investigate artificial neural network applications to control chart pattern recognition. This paper addresses the current state of control chart pattern recognition using artificial neural networks and presents areas for future research. Keywords: Computer-Integrated Manufacturing, Statistical process control, Pattern recognition, Artificial neural networks, Back propagation 1. Introduction Statistical process control (SPC) involves the collection, analysis and interpretation of data for use in quality monitoring activities. Control charts are powerful techniques that can provide information about process means and variances and detect the presence of special causes. Western Electric (1956) described 15 pattern classes that can represent the occurrence of special causes (i.e., trends, mixtures, cycles, systematic variation, etc). To simplify the problem of pattern recognition, Western Electric employed a heuristic approach for detecting some systematic behavior in control chart patterns. A problem with this approach to pattern recognition is that as the number of runs rules increases, so does the false-alarm rate. Additionally, the implementation of the heuristic approach was accomplished manually. This resulted in high computational costs, tediousness and inflexibility due to the difficulty of incorporating new heuristics (Hwarng, 1992). Control chart pattern recognition has received considerable attention in the literature including the application of syntactic approaches, fuzzy-expert systems and artificial neural network models. The objective of this paper is to review the current state of control chart pattern recognition using artificial neural networks (ANNs). 2. Artificial Neural Networks Artificial neura l networks evolved from the use of mathematical formulations to model biological nervous system operations. They are highly parallel computational systems comprised of interconnected artificial neurons or processing units. Neural networks use logical parallelism combined with serial operations as information in one layer is transferred to neurons in another layer. Rummelhart (1986), Russel & Norvig (1995) and Ripley (1996) give an in-depth discussion of neural network theory. ANNs consist of three types of units: input, output and hidden. Input units receive inputs from sources external to the system under study while the output units send signals out of the system. Hidden units are those whose inputs and outputs are within the system and are necessary for the network to learn interdependencies in the model. The

units in a neural network are connected by links and each link has a numeric weight associated with it. Weights are the primary means of long-term storage in ANNs and learning occurs through changes to the weights. Each unit s computation is split into linear and nonlinear components. The linear component, called the input function, computes the weighted sum of the units input values. The non-linear component, called the activation function, transforms the weighted sum into the unit s final activation value. Figure 1 shows a single processing node where F(l) is the linear component and F' ( l) is the nonlinear component. Figure 1. Computations in a processing node. There is a variety of network architectures useful in ANN applications. Two common structures are feed-forward and recurrent architectures. In a feed-forward network, links are unidirectional and there are no feedback cycles present. A feed-forward network involves a simple computation of a function of the input values that is dependent upon the weight settings. The only internal state is that of the weights themselves. Recurrent networks have shortterm memory capabilities as a result of feedback cycles or recurrent connections between layers. In this architecture type, a layer may receive its own output or the output of subsequent layers. A network with one or more hidden layers is referred to as a multi-layered network. With one layer of hidden units, it is possible to represent any continuous function of the inputs. With two layers discontinuous functions can also be represented (Russell and Norvig 1995). Neural networks require some method of computing the weight adjustment. There exists a variety of different methods or learning rules for accomplishing this task. Among these are back propagation (BPN), simulated annealing (SA), learning vector quantization (LVQ), probabilistic neural network (PNN), and cascade correlation learning. Back propagation is the most common in the literature due to its ability to successfully classify patterns based on their general characteristics. Back-propagation is a gradient descent algorithm that compares actual outputs with desired outputs. If an error exists, its reduction is accomplished by back propagating the error through the network and adjusting the weights. A general expression of the network error is Err = T O, where T ando are target and actual network outputs, respectively. Some commonly used network error-types are mean square error, mean absolute error and hyperbolic squared error. 3. Application of ANNs to SPC There is a growing interest to interface process control with computer integrated manufacturing (CIM) techniques. ANNs can be applied to SPC pattern recognition to automate the task of interpreting the control chart. Jacobs and Luke (1993) state that the desired characteristics of a real-time SPC system in a h ighly automated and integrated manufacturing environment are accurate representation of the process without oversimplification and adaptability to new changes. Hwarng (1992), Hwarng & Hubele (1993a,b), Pham & Oztemel (1994), Cheng (1997) and Perry, Spoerre & Velasco (2001) propose ANNs as a potential solution to SPC pattern recognition.

The application of ANNs to SPC can also be beneficial when prior knowledge about the probability distribution of the process data is unknown. If enough data is available for efficient training, ANNs can extract regularities in data sets without any priori assumptions. 3.1 Structural Change Identification The earliest applications of ANNs to SPC were published by Pugh (1989, 1991). Pugh trained ANNs to detect 3 conditions: (1) in-control mean, (2) upward shift and (3) downward shift. Pugh simulated xbar charts and found that with control limits at 2 σ, ANNs were equal in performance to a control chart with regard to the ARL and Type I _ X errors. However, the neural model proved superior when type II errors were considered. Smith (1994) found that a single ANN could model xbar and R charts simultaneously. Smith trained an ANN to identify mean and variance shifts. Smith showed that when large shifts were considered, the ANN performed equal to that of a standard control chart with 3 σ limits. However, when small shifts were considered, the ANN model performed better. _ X Much of the early research focused on detecting mean and variance shifts using similar approaches to Pugh (1989,1991) and Smith (1994), including Guo and Dooley (1992) and Cheng (1995). Ho and Chang (1999) developed an integrated neural network approach for monitoring process mean and variance shifts. 3.2 Identification of the Underlying Distribution Following the pioneering works of Pugh (1989,1991), many authors investigated the application of ANNs to the detection of other non-random behavioral patterns. These patterns include sudden shifts, trends, cycles, stratification, systematic variation and mixtures. Velasco and Rowe (1993) demonstrated the potential of ANN application in the analysis of quality control charts. They developed an ANN to recognize patterns that indicate out-of-control situations as specified by the Western Electric Handbook rules. This heuristic approach showed the potential application of ANNs to real-time, on-line automated analysis of control charts. A problem with this approach is that the generation of training patterns can be tedious since there is no formal means of generating these types of patterns (Perry et al, 2001). Additionally, as the number of runs rules applied to a chart increases, so does the false alarm rate. Also, runs rules do not explicitly indicate the nature of the underlying distribution of the process. Hwarng and Hubele (1993a, 1993b) developed a back propagation pattern recognizer for the identification of up/down trends, cycles, systematic up/down, stratification, mixtures and sudden shifts. Instead of using a heuristic approach, the training data was generated by the use of a pattern generator, expressed by x ( t) = µ + r( t) + d( t), where x (t) is the observation at time t, µ is the process mean when the process is in control, r (t) is a random noise component and d (t) is a special disturbance due to some assignable cause. Hwarng & Hubele (1993a) show that the trained ANN maintains a reasonable balance between Type I and Type II errors. The authors stated that the methodology presented in their research is suitable for any potential patterns that frequent traditional Shewhart control charts. The authors emphasized the major advantages of the ANN approach to control chart pattern recognition over traditional approaches are its flexibility and high-speed computation. Pham and Oztemel (1994) presented a class of control chart pattern recognizers to detect normal patterns, trends, sudden shifts and cycles. These networks used learning vector quantization (LVQ) as a method of weight adjustment. Their results showed poor performance for a standard LVQ network. However, an LVQ model with a conscience was able to classify 95.98% of the training data and recognize 92.71% of the test data. Cheng (1997) developed two types of pattern recognizers based on different neural network architectures. A back propagation ANN and a modular ANN were trained to identify trends, sudden shifts, systematic variation, mixtures and normal patterns. Highlighted in this research are the compactness of input/output representation, the ability to detect unnatural patterns when a low signal-to-noise ratio exists, the capability to detect unnatural patterns starting

anywhere in the sequence of the data and a directional invariance property which permits patterns of different orientations to be equally detected. Perry, Spoerre and Velasco (2001) developed two back propagation ANNs for the detection of trends, mixtures, cycles and systematic variation. Their methodology used heuristics to train one ANN and data generated from pattern generators to train the second ANN. The idea was to equip the recognition system with early detection capabilities by training an ANN to recognize any violation of the four formal tests for unnaturalness as specified by Western Electric (1956). Palm (1990) found these tests to have a good balance between sensitivity and false-alarm rate. The second network was trained to recognize patterns such as trends, cycles, systematic variation and unstable mixtures. This component gave explicit pattern information by providing information on the underlying distribution of the data. 3.3 Autocorrelated Processes Cook and Chiu (1998) developed an ANN to detect shifts in the means of correlated process parameters. Their results showed that the ANN model performed substantially better than the available SPC time-series control charts for correlated data. Cook, Zobel and Nottingham (2001) discuss the development of ANN models to detect shifts in the variance of correlated process parameters. Training and test sets were generated for each combination of variance shift size and correlation coefficient. Their results showed that across all correlation coefficients, the shifted data were correctly classified on the first observation at least 95% of the time. 4. Conclusions and Future Research If out-of-control behavior in control charts is quickly detected, corrective action can be taken prior to the onset of problems in product quality. ANN outputs can serve as a link to process parameters in a closed-loop, automated process control system. In this way, adjustments to a process can be made on-line and quality problems averted. In implementing a closed-loop, automated process control system, a necessary first step is detecting the presence of an assignable cause. Such an activity is traditionally accomplished by skilled personnel who are trained in control chart analysis. Because of the high computational cost, tediousness and inflexibility of conventional computerized pattern recognition systems, more efficient computerized alternative were needed. This led researchers to investigate the application of ANN technology to SPC pattern recognition. The ANN models reported in this paper show promising results. However, it is evident that ANNs may not be able to offer complete solutions. Therefore, integration with other techniques such as fuzzy-logic and genetic algorithms could be explored for promis ing approaches to the problem of control chart pattern recognition. With the implementation of a fuzzy-neural system, classification of patterns with simultaneous special disturbances present may be possible. Finally, an important activity in SPC chart analysis is determining the magnitude of parameters associated with a particular special disturbance, such as the magnitude of a process mean shift or the period of a cycle. The task of determining parameter magnitudes through interpolation or extrapolation to learned points may be possible due to the function approximation capabilities of neural networks. References Cheng, C. S. (1995). A multi-layered neural network model for detecting changes in the process mean. Computers and Industrial Engineering., 28(1), 51-61. Cheng, C. S. (1997). A neural network approach for the analysis of control chart patterns. International Journal of Production Research, 35(3), 667-697. Cook, D. F., and Chiu, C.C. (1998). Using radial basis function neural networks to recognize shifts in correlated manufacturing process parameters. IIE Transactions, 30, 227-234.

Cook, D. F., Zobel, C. W., and Nottingham, Q. J. (2001). Utilization of neural networks for the recognition of variance shifts in correlated manufacturing process parameters. International Journal of Production Research, 39(17), 3881-3887. Guo, Y. & Dooley, K. J. (1992). Identification of change structure in statistical process control. International Journal of Production Research, 30(7), 1655-1669. Jacob, D. A. and Luke, S.R. (1993). Training artificial neural networks for statistical process control in The Tenth Biennial University Government Industry Microelectronics Symposium IEEE, Piscataway, NJ, USA, pp. 235-239. Ho, E.S. and Chang, S.I., (1999). An integrated neural network approach for simultaneous monitoring of process mean and variance shifts a comparative study. International Journal of Production Research, 37(8), 1881-1901 Hwarng, H. B. (1992). Pattern recognition on Shewhart control charts using a neural network approach. PhD. Thesis, Arizona State University. Hwarng, H. B. & Hubele, N. F. (1993a). Back propagation pattern recognizers for x-bar control charts: Methodology and performance. Computers and Industrial Engineering, 24(2), 219-235. Hwarng, H. B., & Hubele, N. F., (1993b). X-bar control chart pattern identification through efficient offline neural network training. IIE Transactions, 25(3), 27-40. Palm, A. C. (1990). Tables of run length percentiles for determining the sensitivity of Shewhart control charts for averages with supplementary run rules. Journal of Quality Technology, 22(4), 289-298. Perry, M.B., Spoerre, J.K., and Velasco, T. (2001). Control chart pattern recognition using artificial neural networks. International Journal of Production Research 39(15, 3399-3418). Pham, D. T. & Oztemel, E. (1994). Control chart pattern recognition using learning vector quantization networks. International Journal of Production Research, 23(3), 721-729. Pugh, G. A. (1989) Synthetic neural networks for process control. Computers and Industrial Engineering, 17(1-4), 24-26. Pugh, G. A. (1991) A comparison of neural networks to SPC charts. Computers and Industrial Engineering, 21(1-4), 253-255. Ripley, B. D. (1996) Pattern Recognition and Neural Networks. Cambridge, MA: Cambridge University Press. Rummelhart, D. E. (1986) Parallel Distributed Processing. Cambridge, MA: Cambridge University Press. Russell, S., Norvig, P. (1995). Artificial Intelligence: A Modern Approach. (Upper Saddle River, NJ: Prentice- Hall). Smith, A. E. (1994) X-bar and R control chart interpretation using neural computing. International Journal of Production Research, 32(2), 309-320. Velasco, T., Rowe, R. (1993) Back propagation artificial neural networks for the analysis of quality control charts. Computers and Industrial Engineering, 25(1-4), 397-400. Western Electric (1956) Statistical Quality Control Handbook. (Princeton, NJ: AT&T).