# Asking goal-oriented questions and learning from answers (@ CogSci 2019)

### Anselm Rothe, Brenden M. Lake, and Todd M. Gureckis

This paper presents a model for how people ask goal-oriented questions
and update their beliefs from answers. They work in the context of a
battleship game, where the set of questions that a person can ask are
pre-defined, as are the answers (e.g. the color on a given board cell,
or whether the ship of a given color is horizontal or vertical).
The hypothesis space $\mathcal{H}$ of boards is bounded ($\approx 1.6 million$),
so it's feasible to have an oracle that answers questions deterministically.

The novelty in this paper is considering that questions are asked with a
goal in mind, not to figure out the state of the board.
A goal defines a projection of the board space. For example,
the goal of "finding out which ships are touching the left border"
corresponds to the power set of ships: the possible answers for
the target question. Each such answer is associated with
a set of boards for which that answer is true.

Under this modelling, you can be Bayesian and maximize expected information
gain to reduce uncertainty for the particular goal in mind. The user
computes the probability of a certain answer by using a uniform prior
over all boards for which the answer would be consistent.
This model seems to explain people's behavior quite well. The model's
scores correlates strongly ($r = 0.84$) with people's performance.
If the goal is inverted for the model, its performance correlates negatively
($r = -0.14$) with the person's performance (with flipped goals),
showing that the goal is in fact a very significant component of this
question-asking procedure. It also predicts well the follow-up questions
a person asks.

This is an interesting scenario that can be used to model how people
pick questions when they have a goal in mind. It turns out that the
intuitive explanations work in practice. An extra challenge for building
systems that interact with people is, of course, how to formulate
questions and interpret free-form answers, and how to work with massive or unbounded
hypothesis spaces (e.g. natural language answers),
which are out of the scope of the proposed model,
but remain as practical challenges.