Some argue that reward or punishment must follow the behavior of interest by no more than 0.5 second if learning is to occur. Others argue that animals can learn with delays between behavior and consequence of even a couple of minutes or more. Where do these numbers come from and which is more accurate?
The earliest studies that we can find that address delay of reinforcement and learning are studies by Wolfe in 1934, Perin in 1943 and Grice in 1948. Wolfe and Grice used performance in T-mazes to measure learning. Perin used a Skinner box and all used rats as subjects. In Wolfe and Grice’s studies, the animals had to run to the correct arm of the maze (black or white) to get a food reinforcement. In both studies the animals were delayed in getting the reinforcer after they made the correct choice. Wolfe found that with delays of 150 seconds some rats were still able to learn the task.
Perin’s study used rats in Skinner boxes and had the animals move a lever either left or right to get a food reward. These rats were delayed for different time periods before reinforcement. He found that rats could learn the task with delays up to 30 seconds, beyond that, there was no learning.
Grice thought that the earlier studies allowed for the possibility that stimuli were present which could act like conditioned reinforcers and thus extended the effectiveness of the of primary reinforcement despite the long delay. For example, in the Wolfe study the rats were prevented from eating for various time periods while sitting in the white goal box. So the white color, which was correlated with the food, could act as a conditioned reinforcer. Grice tried to control for the occurrence of conditioned reinforcers by having the rats delayed from getting their reinforcement while sitting in a neutral color box. He found that delays longer than 0.5 second produced significant interruption and more than 5 seconds produced no learning at all.
In addition to the presence or absence of conditioned reinforcers, research has shown that the magnitude of reinforcement (small vs. large) the magnitude of deprivation (long or short) and the salience of the stimuli associated with the reinforcement can all interact with delay of reinforcement to effect learning.
So it appears that when all other variables that can influence performance are controlled for, learning is optimal with delays shorter than 0.5 second and won’t occur with delays longer than 5 seconds. But, if stimuli are present that can act as conditioned reinforcers for the animal, if large reinforcers are used, if animals are more severely deprived of the reinforcer, or if very salient stimuli are present during learning such as a bright light, then learning can occur with longer delays.
To our knowledge, the interaction of all these variables hasn’t been investigated in dogs, so we don’t know how long a delay in reinforcement can still produce good learning under various conditions.
So what does it mean for the average pet owner trying to train her dog? It appears the best advice to clients is to catch the animal in the act and deliver the consequence immediately (or deliver a conditioned reinforcer such as a ‘click’ from a clicker immediately, followed soon after by the primary reinforcer). Delays in delivering the consequence will impede learning but we can’t say how much because it may depend on the levels of these other variables.
You can find a good discussion of all this in Flaherty, C.F. (1985) Animal Learning and Cognition, New York, NY: Alfred A. Knopf, pp. 165-171, see also p. 221 for the effects of delay of punishment on learning. References to the Grice, Wolfe and Perin studies can be found there. There is a much briefer discussion of delay of reinforcement in Domjan, M. & Burkhard, B. (1986) The Principles of Learning and Behavior, 2nd Ed., Pacific Grove, CA: Brooks/Cole Publishing., pp.120-122.
You can find practical information like this every week in BEN in our new articles, webinars and Facebook discussions. Go to our Home Page to learn more. Join us!
Leave A Response