Bayesian integration in sensorimotor learning

Bayesian integration in sensorimotor learning

Introduction Learning new motor skills Variability in sensors and task Tennis: Velocity of ball Not all are equally probable over time Increased uncertainty: fog -> rely on prior knowledge Bayes strategy Prior belief & uncertainty

Apparatus 6 male & 4 female subjects Reach a visual target with the finger Projection mirror prevented direct view Reaching movements on a table with an Optotrak 3020 tracking system measuring the position of their right index finger Showing projection of finger -> displace feedback of finger location Feedback midway: white sphere Green: target - Blue: starting point

Procedure Feedback given four different ways : No added uncertainty Each subject: 2000 trials: : Medium uncertainty : Large uncertainty : Feedback withheld Giving the final deviation at the end of only clear feedback trials Only last 1000 trials for the analysis Shift was randomly drawn from gaussian distribution (True prior) Mean 1 cm to right Standard deviation 0.5

Possible Computation Models Full compensation model Bayesian probabilistic model Mapping model

Full Compensation Model Full compensation by subjects for the visual estimate of the lateral shift The pointing variability but not the average location affected by increasing the lateral shift feedback uncertainty No requirement of estimation for the visual uncertainty or the prior distribution of shifts.

Bayesian Model Optimal use of information about the prior distribution and the uncertainty of the visual feedback to estimate the lateral shift

Sense for Bayesian statistics Sensed a lateral shift of 2 cm due to any of the many true lateral shifts True lateral shift: 1.8 cm (Error of +0.2 cm) vs 2.2 cm (Error of -0.2 cm) Errors equally probable for gaussian visual feedback noise 1.8 cm lateral shift more probable than a 2.2 cm shift given the prior distribution having mean of 1 cm. Depends on two factors: Prior distribution Degree of uncertainty in the visual feedback.

Mapping model Learn mapping from the visual feedback to an estimate of the lateral shift Error minimized over repeated trials and without any explicit representation of the prior distribution or visual uncertainty Requires knowledge of the error at the end of the movement to learn mapping The shifted finger position revealed at the end of the movement on the clear feedback trials ( ). The same mapping applied to the blurred conditions (, ). The average shift of the response towards the mean of the prior same for all amounts of blur

Comparison between 3 models Basis: The effect of the visual feedback on the final deviation from the target Model 1: The average cursor lateral deviation from the target should be zero for all conditions. Model 2: The estimated lateral shift should move towards the mean of the prior by an amount that depends on the sensory uncertainty. Model 3: Predicts that subjects should compensate for the seen position independently of the degree of uncertainty. Comparing theoretical MSE for the three models shows that it is minimal for model 2. Even though model 1 is on average on target, the response variability is higher than in model 2, leading to a larger MSE

Bayesian Model Analysis Prior Likelihood Posterior

Prior

Sensed observation

Combined estimate Given that we know ; we can estimate the uncertainty in the feedback by linear regression

MSE Model 1: Model 2:

Visual feedback noise variation At minimal noise, the estimate distribution follows the sensory distribution. As noise increases, the estimate distribution starts to approach the prior distribution.

Slope variations for individual subjects The slopes for the linear fits are shown for the full population of subjects. Planned comparisons of the slopes between adjacent uncertainty levels were all significant

Bias vs slope No deviation from the target if the true lateral shift is at the mean of the prior Predicts that the sum of the slope and offset should be zero x(estimated) = w1*1 + w2*x(sensed) Bias = E[x(estimated)-x(true)] = w1[1-x(true)] Offset = w1, Slope = -w1

Inferring Prior The derivative of this posterior with respect to x(true) must vanish at x(estimated). Using derivative of log(p(x(true))) and integrating with certain operations, the true prior can be obtained

Bimodal distributions New prior Is learning complex distributions possible? Two gaussians

Bimodal distributions Dashed line: Single gaussian Solid line: Two gaussian fit Shows a nonlinear relationship Showing one subject

Bimodal distributions Trying to fit data with linear regression Showing the non-linear relationship Shows average over subjects

Conclusion Proposed: Nervous system fits with bayesian Model: Sensorimotor consistent with neurophysiological studies analyzing that estimating reward has uncertainty Only tested on visuomotor displacement Expected to work with all sensorimotor control & learning Tennis match : Prior knowledge about other player