How a game of draughts helped unravel the brain’s reward system

One of the winners of the 2017 Brain Prize reveals the 1955 computer program that helped transform our understanding of the human brain

In 1955, American computer pioneer Arthur Samuel unveiled a draughts-playing program that human opponents described as "tricky but beatable". The achievement sounds quaint today given the subsequent decisive triumphs of machines over humans at chess, Jeopardy, Go and poker. But according to Prof Peter Dayan, a computational neuroscientist at University College London and one of the recipients of the 2017 Brain Prize, Samuel had hit on "one of the first good ideas in AI" and a concept that has transformed our understanding of the human brain.

Samuel's program used a souped-up form of Pavlovian reinforcement to learn how to play draughts. Pavlov's dogs learned the simple association between hearing a bell and the arrival of food, but in a game like draughts there are many steps on the path to victory or defeat. This raises the question of how we (or a computer) learn which moves contribute to victory and should be repeated in the future.