In less recent times (circa 1960’s), this problem was posed and considered in the case where the payoff mechanisms had a very simple structure: each slot machine is a coin flip with a different probability $ p$ of winning, and the player’s goal is to find the best machine as quickly as possible. Herbert Robbins, one of the first to study bandit learning algorithms.
