A simple model for learning in volatile environments

by Payam Piray, Nathaniel D. Daw

Sound principles of statistical inference dictate that uncertainty shapes learning. In this work, we revisit the question of learning in volatile environments, in which both the first and second-order statistics of observations dynamically evolve over time. We propose a new model, the volatile Kalman filter (VKF), which is based on a tractable state-space model of uncertainty and extends the Kalman filter algorithm to volatile environments. The proposed model is algorithmically simple and encompasses the Kalman filter as a special case. Specifically, in addition to the error-correcting rule of Kalman filter for learning observations, the VKF learns volatility according to a second error-correcting rule. These dual updates echo and contextualize classical psychological models of learning, in particular hybrid accounts of Pearce-Hall and Rescorla-Wagner. At the computational level, compared with existing models, the VKF gives up some flexibility in the generative model to enable a more faithful approximation to exact inference. When fit to empirical data, the VKF is better behaved than alternatives and better captures human choice data in two independent datasets of probabilistic learning tasks. The proposed model provides a coherent account of learning in stable or volatile environments and has implications for decision neuroscience research.

Make more money selling and advertising your products and services for free on Ominy market. Click here to start selling now

Paper source
Plos Journal

READ MORE  Evaluation of a city-wide school-located influenza vaccination program in Oakland, California, with respect to vaccination coverage, school absences, and laboratory-confirmed influenza: A matched cohort study

Ominy science editory team

A team of dedicated users that search, fetch and publish research stories for Ominy science.

Enable notifications of new posts OK No thanks