I'm exploring a few projects in detecting uncertainty, formulating and asking questions, and interpreting answers, and automated tutoring seems to be a fertile environment for these ideas. Some of them have been explored in Automated Tutoring software in the past, and AutoTutor is one of them to have actually reached classrooms, in the early 2000's.
It's targeted at guiding students through solving open-ended problems in natural language. One of the domains they tested was intuitive physics problems. For example, this dialogue from the paper illustrates it well:
AutoTutor: When a car without headrests on the seats isstruck from behind, the passengers often suffer neck in-juries. Why do passengers get neck injuries in this situa-tion? (MAIN QUESTION)
Student: The passenger experiences a backward accelera-tion, while the car moves forward.
AutoTutor: And can you add to that? (PUMP)
Student: This is because the passenger experiences a forceopposite in direction to the force on the car.AutoTutor: Um, yes, correct! (POSITIVE FEEDBACK)
When the neck provides a large force, what within theneck would get damaged? (HINT)
Student: The spine.AutoTutor: Umm (NEUTRAL NEGATIVE FEEDBACK),I think you can get this. Damage within the neck will occurin the ligaments and the . . .? [spoken with rising intona-tion and accompanied with a prompting hand gesture](PROMPT):
Student: Muscles.
AutoTutor: Brilliant! (POSITIVE FEEDBACK) Ligamentsand the muscles. (PROMPT RESPONSE) Let’s keep going.(DIALOGUE ADVANCER) Additionally, when there isno headrest present in the car, what accelerates the head ofthe passenger? (HINT)
Student: The impact from behind.
AutoTutor: I am sure you can get this. The head of the pas-senger will be accelerated by the passenger’s . . . ? [spokenwith rising intonation and accompanied with a promptinghand gesture] (PROMPT)
Student: Car.
AutoTutor: Wrong! (NEGATIVE FEEDBACK) By thepassenger’s neck. (PROMPT RESPONSE):[moving ahead 22 turns in the dialogue]
AutoTutor: Do you perhaps have any general conceptualphysics questions? (QUESTION PROMPT)
Student: No.
AutoTutor: How would you answer my first question, hav-ing gone over what we’ve just talked about? (FINAL AN-SWER PROMPT)
Student: The passengers get neck injuries because ... [81additional words].
AutoTutor: Great! (POSITIVE FEEDBACK) To recap,when a car is struck from behind, the force of impact willcause the car to experience a large forward acceleration.The car seat is attached to the car and will also have thesame large acceleration. When the car is hit from behind,the seat will exert the same acceleration to the torso of thebody. In order for the head to go along with the torso, itwill need to be accelerated at the same rate as the torso.Since the head has no acceleration being applied to it by aheadrest, the neck gets stretched, damaging its musclesand ligaments in an attempt to produce the required largeacceleration. (SUMMARY)
It requires a fair bit of annotation from human experts, and it's quite deterministic. It uses LSA to compare the students answers with a set of goals that the answer is expected to meet, and a state machine to guide the student into completing unfinished goals (that starts by asking more general questions, then asks questions that give hints, and ultimately will give out the answer if needed). This guides the algorithm in what questions to ask next - again, from a human-written set.
AutoTutor seems to have been applied to other problem domains, but this format doesn't seem very flexible. Nevertheless, for this particular shape of problems, it illustrates how far you can get with very simple heuristics grounded on observations of humans solving the problem (they watched expert tutors and tried to mimick their process).
Unfortunately, it seems these automated tutors in general have been kind of forgotten. I tried searching for ways to play with AutoTutor and others from its generation, but it seems they're mostly living on papers as of now.