The Effect of Explicit Prior Judgment Elicitation on Belief Change

Speaker: Dr Mark Himmelstein

Summary

In standard judge advisor system studies a judge reports a prior belief, receives advice, and then revises their initial estimate, providing a tractable measure of quantitative belief revision. However, what happens if prior beliefs are not elicited? Past research has identified clear differences in posterior belief distributions depending on whether priors were elicited or not, implying the mere elicitation of a prior has a treatment-like effect.

However, without prior judgements, we are restricted to studying posterior judgements, rather than belief change. We propose a new method that treats judges’ priors as planned missing data and employs imputation techniques to generate estimates of those priors without ever having to ask them to directly report their priors, thereby circumventing this treatment-like effect.

We first use simulation studies to demonstrate the feasibility of our method, then apply it in two advice taking experiments. In a calorie estimation task, we show judges are both more willing to consider advice and weigh it more heavily when they are not anchored by having reported an explicit prior. However, in a probability forecasting task, neglecting to elicit a prior induced latent confirmation bias, causing detrimental effects on judgement accuracy.

To subscribe to our seminar announcements, please go to the Jiscmail mailing list. Some of the earlier talks are also available on our vimeo channel. A list of upcoming seminars is also available.