Nonlinear state-space models

In many cases, measurements originate from a dynamical system and form time series. In such cases, it is often useful to model the dynamics in addition to the instantaneous observations. We have extended the nonlinear factor analysis model by adding a nonlinear model for the dynamics of the sources in (Valpola and Karhunen, 2002). This results in a state-space model where the sources can be interpreted as the internal state of the underlying generative process. The variational Bayesian method developed in (Valpola and Karhunen, 2002) is called nonlinear dynamic factor analysis (NDFA).

An important advantage of the NDFA method in (Valpola and Karhunen, 2002) is its ability to learn a high-dimensional latent source space. We have also reasonably solved computational and over-fitting problems which have been major obstacles in developing this kind of unsupervised methods thus far. Potential applications of the NDFA method include prediction and process monitoring, control and identification. A drawback of the original NDFA method is that it is computationally quite demanding. On the other hand, it often provides very good results.

Detection of process state changes

One potential application for the nonlinear state-space model and the NDFA method introduced in (Valpola and Karhunen, 2002) is process monitoring. In (Ilin et al., 2004), we have shown that the NDFA method can learn a model which is capable of detecting an abrupt change in the underlying dynamics of a fairly complex nonlinear process. The NDFA method performs in this application clearly better than the compared standard approaches (Ilin et al., 2004).

Stochastic nonlinear model-predictive control

For controlling a dynamical system, control inputs are added to the nonlinear state-space model (NDFA) introduced in (Valpola and Karhunen, 2002). In (Raiko and Tornio, 2005), we study three different control schemes in this setting. Direct control is based on using the internal forward model directly. It is fast to use, but requires the learning of a policy mapping, which is hard to do well. Optimistic inference control is a novel method based on Bayesian inference answering the question: "Assuming success in the end, what will happen in near future?" It is based on a single probabilistic inference but unfortunately neither of the two tested inference algorithms works well with it. The third control scheme is stochastic nonlinear model-predictive control, which is based on optimizing control signals based on maximising a utility function.

Simulation resultss with a cart-pole swing-up task in (Raiko and Tornio, 2005) confirm that selecting actions based on a state-space model instead of the observation directly has many benefits: First, it is more resistant to noise because it implicitly involves filtering. Second, the observations (without history) do not always carry enough information about the system state. Third, when nonlinear dynamics are modelled by a function approximator such as an multilayer perceptron network, a state-space model can find such a representation of the state that it is more suitable for the approximation and thus more predictable (Raiko and Tornio, 2005).

The animation shows a successfull swingup performed by nonlinear model-predictive control using a prediction horizon of 40 time steps. The top subfigure shows the cart-pole system at current time in black and predictions in grey. The middle figure shows the development of hidden states and their predictions such that the current time is marked by the vertical dashed line. The bottom figure shows the place and speed of the cart, angle and angle speed of the pole, and the force applied to the cart.

References

A. Ilin, H. Valpola, and E. Oja, "Nonlinear dynamical factor analysis for state change detection". IEEE Trans, on Neural Networks, vol. 15, no. 3, 2004, pp. 559-575. Pdf (766k).

T. Raiko and M. Tornio, "Learning nonlinear state-space models for control". In Proc. Int. Joint Conf. on Neural Networks (IJCNN'05), Montreal, Canada, July 2005, pp. 815-820. Pdf (192k).

H. Valpola and J. Karhunen, "An unsupervised ensemble learning method for nonlinear dynamic state-space models". Neural Computation, vol. 14, no. 11, 2002, pp. 2647-2692.Pdf (937k).