Analysis and Prediction of Multi-Stationary Tie Series

摘要

In the analysis and prediction of real-world systems, two of the key problems are nonstationarity (often in the form of switching between regimes). and overfitting (particularly serious for noisy processes). This article addresses these problems with gated experts, consisting of a (nonlienar) gating network, and serveral (also nonlinear) competing experts. Each expert learns to predict the conditional mean, and each expert adapts its width to match the nosie level in its regime. The gating network learns to predict the probability for each expert, given the input. This article focuses on the case where the gating entworks bases its decision on infroamtion from the inputs. This can be contrasted to hidden Markov model where the decision in based on the preious state(s) (i.e., on the output of the gating networks at the previous time step) as well as to averging schemes over several predictors. In contrast, gated experts soft-partition the inptu space, and only learn to mode their region. Tis article discussed the underlying statistical assumption, derives the weight update rules and discussed how gated experts discuver regimes and avoid overfitting. Applications to several time series (computer-generated, laboratory, and real world) are reproted in Weigend and Mangeas~21.