Continued Responding During an Extinction Procedure

J Exp Anal Behav. 2006 Jan; 85(1): 23–39.

Resistance to Extinction following Variable-Interval Reinforcement: Reinforcer Rate and Amount

Received 2004 Oct 30; Accepted 2005 Jun 3.

Abstract

Rats obtained food-pellet reinforcers by nose poking a lighted key. Experiment 1 examined resistance to extinction following single-schedule training with different variable-interval schedules, ranging from a mean interval of 16 min to 0.25 min. That is, for each schedule, the rats received 20 consecutive daily baseline sessions and then a session of extinction (i.e., no reinforcers). Resistance to extinction (decline in response rate relative to baseline) was negatively related to the rate of reinforcers obtained during baseline, a relation analogous to the partial-reinforcement-extinction effect. A positive relation between these variables emerged, however, when the unit of extinction was taken as the mean interreinforcer interval that had been in effect during training (i.e., as an omitted reinforcer during extinction). In a second experiment, rats received blocks of training sessions, all with the same variable-interval schedule but with a reinforcer of four pellets for some blocks and one pellet for others. Resistance to extinction was greater following training with the larger (four pellets) than with the smaller (one pellet) reinforcer. Taken together, these results support the principle that greater reinforcement during training (e.g., higher rate or larger amount) engenders greater resistance to extinction even when the different conditions of reinforcement are varied between blocks of sessions.

Keywords: resistance to extinction, reinforcer rate, reinforcer amount, behavioral momentum, omitted reinforcers, key poke, rats

If one's aim is to engender responding that persists during extinction, would it be better to reinforce the response with a variable-interval (VI) schedule that provides reinforcers at a high rate (i.e., a rich VI) or one that provides reinforcers at a low rate (i.e., a lean VI)? The answer to this seemingly straightforward question turns out not to be a simple one. Under some conditions, training with the richer VI schedule engenders the more persistent responding during extinction; under other conditions, training with the leaner VI schedule does so.

Consider first what happens if the training is carried out with a multiple VIx VIy schedule, that is, each of two VI schedules is correlated with a particular discriminative stimulus, and the two discriminative stimuli, along with their corresponding VI schedules, alternate every minute or so throughout each 1- or 2-hr training session. After sufficient training to achieve stable responding in the presence of the discriminative stimuli (e.g., 20 daily sessions), one or more extinction sessions are introduced. That is, the discriminative stimuli alternate as during training but the reinforcers are no longer given for responding. Response rate declines during extinction in the presence of both discriminative stimuli, but not equally. Instead, the nearly invariable result is that response rate declines less relative to its training baseline level in the presence of the discriminative stimulus that had been correlated with the richer VI schedule (i.e., the one in the presence of which reinforcers occurred more frequently) than in the presence of the discriminative stimulus that had been correlated with the leaner VI schedule. In other words, with this kind of multiple-schedule arrangement, training with the richer VI schedule engenders the more persistent responding during extinction (for reviews see Nevin, 1992; Nevin & Grace, 2000).

In contrast, when the different VI schedules are presented in separate training blocks—for example, where each block consists of 20 consecutive training sessions with a single VI schedule followed by an extinction session—the relation between resistance to extinction and frequency of reinforcement during training is negative. That is, resistance to extinction is greater following training with a leaner VI schedule than following training with a richer VI schedule (Cohen, 1998; Cohen, Riley, & Weigle, 1993). This effect of VI intermittency appears to be an extension of the familiar partial-reinforcement-extinction (PRE) effect: A response is more resistant to extinction if only some occurrences of the response have been reinforced than if all occurrences have been reinforced (for reviews, see Lerman & Iwata, 1996; Mackintosh, 1974; Nevin, 1988; Svartdal, 2000; Zarcone, Branch, Hughes, & Pennypacker, 1997).

The results from multiple-schedule training and single-schedule training thus lead to opposite conclusions regarding the kind of reinforcement history that should be given to enhance resistance to extinction. The multiple-schedule results indicate that the best way to enhance resistance to extinction is to reinforce the response at a high rate during training; the single-schedule results (i.e., the PRE-like results) suggest that it would be best to reinforce at a low rate. Moreover, the multiple-schedule results, along with those from other studies, support a broader principle relating reinforcement history and persistence: The greater the reinforcement during training (e.g., either through reinforcing at a higher rate or with a larger amount), the more resistant the response will be to all sorts of attenuating influences, not simply to extinction (Killeen & Hall, 2001; Nevin, 1999; Nevin & Grace, 2000; Smith, 1974). The results from single-schedule training, of course, appear to contradict this principle because the response that receives the less frequent reinforcement during training is the more resistant to extinction.

Despite the apparent contradiction, it might nevertheless be possible to reconcile the single-schedule results with the principle that greater reinforcement leads to greater resistance to attenuation. According to an interpretation elaborated by Nevin and his colleagues (Grace, McLean, & Nevin, 2003; Nevin, 1988; Nevin & Grace, 1999, 2000, 2005; Nevin, McLean, & Grace, 2001), extinction is a complex procedure for reducing response rate. It not only eliminates the strengthening effect of reinforcers but also changes the discriminative stimulus context supporting the response (cf. Mackintosh, 1974; see Drew, Yang, Ohyama, & Balsam, 2004, for a related two-process interpretation of extinction in an autoshaping procedure). During training, responses are reinforced in the context of a particular distribution of time intervals since the last reinforcer (or, equivalently, in the context of a particular rate of reinforcers). During extinction, then, as time elapses without a reinforcer, the stimulus context becomes increasingly dissimilar to that during training, and response rate declines due to this increasing stimulus disparity. Moreover, this stimulus change will occur more rapidly during extinction if the training VI has been rich than if it has been lean. In everyday terms, we might say that with a lean VI schedule, the subject often has experienced reinforcement following long intervals since the last reinforcer. Consequently, it will take a long time during extinction before the change in conditions can be detected. With a rich VI during training, in contrast, responses have been reinforced following relatively short time intervals since the last reinforcer, so the change to extinction can be detected quickly. To the extent that responding during extinction is determined by this kind of stimulus disparity, resistance to extinction will be negatively related to the rate of reinforcement during training. And this negative relation could outweigh, and thus mask, any positive relation between reinforcement and resistance to extinction that also might be present in the data.

Therefore, the negative relation between resistance to extinction and rate of reinforcers during training found with the single-schedule procedure does not necessarily contradict the principle that greater reinforcement engenders greater resistance to attenuation. The results simply might indicate that discriminative stimulus control based on rate of reinforcement plays a particularly strong role with the single-schedule procedure. Why that role should be so much stronger with the single-schedule procedure than with the multiple-schedule procedure is a question that will be deferred to the General Discussion section. For now, the focus will be on whether credible evidence can be found with the single-schedule procedure for a positive relation between resistance to extinction and the level of reinforcement during training. In Experiment 1, we searched for such evidence in a set of data obtained from extinction sessions following single-schedule training with different VI schedules (and VI schedules with an added response requirement intended to alter the response unit) ranging from VI 16 min to VI 0.25 min. The baseline data were reported in an earlier paper (Shull, Grimes, & Bennett, 2004). Because any positive relation between resistance to extinction and rate of reinforcers was, presumably, masked by the stimulus disparity effect based on rate of reinforcers, the evidence was sought in various transformations of the data.

In Experiment 2, we attempted to see if a positive relation between resistance to extinction and level of reinforcement would appear directly in the untransformed extinction data following single-schedule training if the schedules differed in the amount of the reinforcer (i.e., number of food pellets) rather than in the rate of reinforcers. Because the VI schedules were the same for these comparisons, there should be no differential effect of stimulus change based on rate of reinforcement to mask a possible positive relation between resistance to extinction and level of reinforcement. And any differential effect of stimulus change based on reinforcer amount seemed likely to be small. Data summarized by Case (2000) suggest that evidence of a positive relation might, indeed, be found with different reinforcer amounts. The point of carrying out the comparison here was to ensure that a positive relation between resistance to extinction and amount of the reinforcer (Experiment 2) could be found under the same set of conditions (e.g., apparatus, rats as subjects, and methods of data analysis) that generated a negative relation between resistance to extinction and rate of reinforcers (Experiment 1). Such results would demonstrate that a negative relation is not an inevitable outcome of the single-schedule procedure. They would thus tend to support the view that the negative relation is a special case, dependent on using different VI schedules (i.e., different rates of the reinforcer) as the method of varying level of reinforcement during training.

Experiment 1

Method

The baseline data were reported in a previous paper (Shull et al., 2004). The relevant portions of the Method section from that paper are repeated here.

Subjects

The subjects were 4 male Long Evans hooded rats, about a year old at the start of the experiment. Obtained as juveniles (about 150 g), they were gradually (over several months) brought to a weight of 335 g (± 15 g) and maintained at that level by free access to food blocks in their home cages for 1 to 1.5 hrs after each session. (Ator, 1991, provides a rationale for this method of food deprivation for rats. As discussed by Ator, percentage of free-feeding body weight usually is not a useful measure of a rat's deprivation level because rats given free access to food will grow continuously.)

A rat's home cage was a plastic box with a metal grate cover through which access to a water bottle was continuously available. The home cages were kept in a room that maintained a temperature of about 22 °C. The overhead lights in the room were on from about 6:00 a.m. until 6:00 p.m. each day; the experimental sessions were conducted during the lights-on periods. The rats previously had served as subjects in an undergraduate laboratory course, in projects that demonstrated basic behavioral phenomena such as shaping, discrimination, and behavioral chains.

Apparatus

Each rat was assigned to one of four similar experimental chambers for the duration of the experiment. These chambers, located in a separate room from where the home cages were kept, were 300 mm wide by 320 mm deep by 300 mm high. They were constructed of sheet metal (top and three sides), clear plastic (rear door), and 7-mm diameter stainless steel rods spaced 10 mm apart (floor). Food pellets (45 mg Noyes, Formula A, obtained from Research Diets, Inc.) were delivered into a small metal food tray located behind a square opening (44 mm by 44 mm) in the middle of the front panel, 43 mm above the floor. The operation of the pellet dispenser made a click, and the pellet landing in the tray made a "plink" sound a fraction of a second later.

The operandum was a translucent plastic key (a Lehigh Valley Electronics pigeon key), mounted behind a 19 mm diameter round hole through the left wall. The center of the key was 51 mm toward the rear of the chamber from the front wall and 62 mm above the floor. For a response to be recorded, the key had to be pushed a distance of about 1.5 mm (measured at the center) with a force of at least 0.18 N (three of the four chambers) or 0.3 N (the fourth chamber; Rat C3). A recordable response produced a brief click from a small snap-action switch connected to the key. Each rat was observed to poke its key mostly with its nose but occasionally with its paw. At the start of each session, the key was transilluminated with a white light; the light remained on until the end of the session.

A metal drinking spout extended into the chamber through a small hole in the left wall near the back left corner (approximately 240 mm from the front wall and 25 mm above the floor). The spout was attached to a water bottle suspended outside the chamber and provided continuous access to water.

The four chambers were placed on a cart, two to a shelf. No attempt was made to shield any of the chambers from sounds made in the others; it was apparent that each rat quickly learned to go to its food tray only when its own feeder operated. The four chambers operated concurrently, although out of phase.

The experimental sessions were conducted with the room darkened except for low-level light that entered from the corridor through a translucent window. The keylight provided the only other source of illumination in the chamber (i.e., there was no houselight or feeder light).

Four special-purpose computers (Walter & Palya, 1984), one for each chamber, controlled the experimental events and recorded the data. These computers were connected to a desktop computer for uploading programs and downloading data.

Procedure

Key poking was established by reinforcing successively closer approximations (shaping) with food pellets. Following shaping, the rats were given a few days' training on progressively longer VI schedules until the value reached was a VI 1-min schedule, which was the first condition of the experiment.

Throughout the experiment, sessions were conducted daily. Each session lasted for 2 hrs or until 101 food pellets (at one food pellet per reinforcer) had been delivered, whichever occurred first. A condition consisted of a block of 20 consecutive sessions during which a particular schedule of reinforcement was in effect. Based on visual inspection, total response rate typically appeared to stabilize before the 15th session (with two exceptions, Conditions 7 and 14, as discussed in Shull et al., 2004).

For one set of conditions, reinforcement of key poking was arranged by VI schedules of reinforcement. That is, throughout each session the first key poke that occurred after a variable interval of time elapsed produced a food pellet. The first interval in a session was timed from the start of the session; all subsequent intervals were timed from the previous pellet delivery. The intervals in a session were drawn randomly (with replacement) from a list of 16 that was constructed to provide a roughly constant probability of reinforcement in time since the last reinforcer (Fleshler & Hoffman, 1962; Hantula, 1991) except that the shortest interval in the list was never longer than 4 s regardless of the mean VI. The mean VI duration differed among conditions.

For another set of conditions, the schedule operated exactly like a VI schedule except that after the end of the interval some unpredictable number of responses, from 2 through 8 (mean = 5), was required to obtain the assigned reinforcer. (This arrangement required the one response specified by the VI component plus an average of four additional responses.) Each whole number from 2 through 8 had an equal likelihood of being selected. Because there was no stimulus change to indicate the end of the interval, this schedule is referred to as a tandem VI variable-ratio (VR) 4 schedule. The Appendix includes a list of the training schedules (VI and tandem VI VR 4 schedules) along with the order in which they occurred.

After the end of each condition, the rats were given a single 2-hr session of extinction. The extinction sessions operated exactly like the baseline sessions except that no food-pellet reinforcers were delivered.

Results

One way to get a sense of how response rate changed over the course of an extinction session is to present the data in the form of a cumulative record (i.e., a plot showing the cumulative number of responses as a function of elapsed time). Typically for extinction, such a plot is increasing and negatively accelerated, indicating that response rate declines as time elapses during extinction (Skinner, 1938). Figure 1 (top row) shows examples of such cumulative response plots for the 4 rats. The records were selected to show the effects of training with a moderately rich VI schedule (VI 1 min) and a moderately lean one (VI 8 min) and also of including or not including the VR 4 response requirement during training. From these plots it is easy to see which training schedule of a pair produced the larger number of responses by the end of the 2-hr extinction session: More responses occurred following training with the richer VI than with the leaner VI schedule, and for a given VI schedule, more responses occurred following training with than without the VR 4 requirement. It is hard to determine from these plots (top row), however, how rapidly response rate declined following the different training schedules. To do so requires comparing the rates of change in the slopes of the different cumulative response plots, and this comparison is made difficult by the large differences in total responses.

An external file that holds a picture, illustration, etc. Object name is jeab-85-01-02-f01.jpg

Cumulative responses plotted as a function of elapsed time during selected extinction sessions for each of the 4 rats.

The top row shows the absolute cumulative response counts; the bottom row shows the cumulative response counts as a proportion of the total count for the session. The line style indicates the particular training schedule that preceded the extinction session.

One way to facilitate comparison of slopes is to express the cumulative response values as a proportion of the total number of responses that occurred by the end of the session, so that each plot is normalized relative to its total response output (Mellgren & Elsmore, 1991; Schaal, 1996). The plots in the bottom row of Figure 1 show such normalized cumulative response plots for the data shown in the top row of plots. From these normalized plots, it is evident that response rate declined more quickly following training with the richer VI (VI 1 min) than with the leaner VI (VI 8 min) schedule. Moreover, the presence or absence of the VR 4 requirement during training appears not to have had a systematic effect on how quickly response rate declined. The major determiner appears to have been the rate of reinforcement (i.e., the VI duration) during training.

The normalized cumulative response plots suggest one way to quantify resistance to extinction: Determine the time to emit half of the total number of responses during the extinction session. If response rate were constant throughout the session, it would take half the session (i.e., 60 min) to generate half the responses. If, instead, response rate declined throughout the extinction session, half the responses would occur before half the session elapsed. More generally, the faster response rate declined, the less time it would take for half the responses to be emitted. Visually, for the plots in the bottom row of Figure 1, this measure of resistance to extinction is the value along the x axis directly below the point where the plot crosses an imaginary line that extends horizontally from 0.5 on the y axis. Consider, for example, the plots for Rat C1 (lower left panel). That rat took less than 15 min to emit half its total responses when the training schedule was VI 1 min but about 45 min when the training schedule was VI 8 min. By this measure, then, resistance to extinction was greater following training with the leaner VI (VI 8 min) than with the richer VI (VI 1 min) schedule. Again, the presence or absence of the VR 4 requirement during training had little effect on this measure of resistance to extinction.

The time taken to emit half of the total responses was calculated for all the extinction sessions and plotted in Figure 2, bottom row, as a function of the rate of reinforcers obtained during training (mean over the last five sessions). Consistent with the trends indicated by the illustrative cumulative records, the full set of data shows a negative relation between resistance to extinction and rate of reinforcers during training and no apparent effect due to the presence (circles) or absence (triangles) of the VR 4 requirement during training.

An external file that holds a picture, illustration, etc. Object name is jeab-85-01-02-f02.jpg

Three measures of resistance to extinction plotted as a function of the obtained rate of reinforcers during training (mean over the last five sessions) for each of the 4 rats.

The top row shows the logarithm (base 10) of the response rate during the extinction session as a proportion of the baseline response rate. The middle row shows the number of minutes until response rate during the extinction session dropped below 10% of the baseline response rate. The bottom row shows the number of minutes taken for half of the extinction-session responses to occur. Circles show data from the VI schedules; triangles show data from the tandem VI VR 4 schedules.

The plots in the middle and top row of graphs in Figure 2 show a similar negative relation with two other measures of resistance to extinction. The measure in the middle row is the time taken during the extinction session for response rate to drop below 10% (i.e., one base-10 log unit) of its baseline level (mean over the last five sessions of training; listed in the Appendix). (For reasons discussed in Shull et al., 2004, the postreinforcer pause was excluded from the baseline response rate calculations.) To determine these time values, we divided each extinction session into consecutive 15-min segments, calculated the response rate in each segment, and then expressed each of these response rates as a proportion of the baseline response rate. Next, for each extinction session, we plotted the logarithms (base 10) of these proportions as a function of elapsed time and connected the points with straight lines. These within-session extinction plots tended to decline, and most dropped below a log value of −1 (i.e., below 10% of baseline). The time taken for a plot to do so was determined by linear interpolation to the nearest 0.5 min. (For a discussion of similar measures see Killeen & Hall, 2001, and Nevin, 1988.)

Estimates could not be obtained this way for the conditions involving training with the longest VI schedules (VI 8 min and VI 16 min) because, for those conditions, response rate never dropped below 10% of its baseline level. Accordingly, no points are included in Figure 2, middle row, for extinction following training with the VI 16-min and the tandem VI 16-min VR 4 schedules. For extinction following training with the VI 8-min and the tandem VI 8-min VR 4 schedules, the maximum time value of 120 min was assigned and plotted in Figure 2, middle row.

The measure of resistance to extinction in the top row of graphs is the logarithm (base 10) of the response rate over the whole 2-hr extinction session expressed as a proportion of the baseline response rate (i.e., log[B_x /B_o ], where B_x is the response rate during extinction and B_o is the baseline response rate; cf. Nevin & Grace, 2000). With this measure, a value of zero indicates no change from baseline (i.e., complete resistance to extinction), and increasingly negative values indicate progressively lower resistances to extinction (i.e., smaller proportions of baseline response rate).

Again, with all three measures, and regardless of the presence or absence of the VR 4 requirement, higher reinforcer rates during training tended to generate responding that was less resistant to extinction. As should be apparent from trends in Figure 2, the three measures of resistance to extinction were positively correlated. All 12 Pearson r values (correlating each of the three measures with the other two for each of the 4 rats) were .80 or higher, and all but two were .90 or higher. These high correlations were expected and show merely that trends shown in Figure 2 were not due to some unsuspected quirk of one particular summary measure.

Discussion

The results shown in Figure 2 further substantiate Cohen's (1998) conclusion that with single-schedule training, resistance to extinction (measured as rate of decline in response rate from an initial or baseline level) is negatively related to the rate of reinforcers from VI schedules during training. Cohen demonstrated such a negative relation by comparing the results of extinction following training with VI 2 min and VI 0.5 min schedules. The data in Figure 2 extend Cohen's findings by showing that the negative relation holds over the range of training VIs from VI 16 min to VI 0.25 min.

The data also show that approximately the same negative relation holds whether or not the training schedule imposed an additional VR 4 requirement, even though imposing the VR 4 requirement usually increased baseline response rate (Shull et al., 2004; and Appendix). There is reason to think that adding the VR 4 requirement increased response rate primarily by causing the size of an operant unit (specifically, the number of key pokes per bout of key poking) to increase (Shull et al., 2004). If so, the consistency of the extinction data in Figure 2 between the VI and the tandem VI VR 4 training schedules suggests that resistance to extinction after VI reinforcement might be independent of the size of the operant unit, at least over some range of sizes—an outcome reminiscent of the response-unit account of resistance to extinction following different schedules of intermittent reinforcement (Mowrer & Jones, 1945; Skinner, 1938).

The negative relation between resistance to extinction and rate of reinforcers during training appears contrary to the principle that greater reinforcement leads to greater resistance to extinction. But recall Nevin's argument (e.g., Nevin, 1988; Nevin & Grace, 1999, 2000, 2005), summarized in the Introduction, that the change from training to extinction is more readily discriminated following a rich VI schedule than following a lean one and that this difference in discriminability could mask a positive relation between resistance to extinction and rate of reinforcers during training. If so, it might be possible to detect a positive relation if the magnitude of stimulus change from training to extinction could somehow be equated across the different training schedules.

One approach toward this end is based on the idea that the magnitude of the stimulus change from training to extinction is proportional to the number of mean baseline interreinforcer intervals that has elapsed since the start of extinction (cf. Gallistel & Gibbon, 2000; Kakade & Dayan, 2002; Nevin & Grace, 2005). Speaking more colloquially, the idea is that the rat detects extinction based on the number of "omitted reinforcers"—that is, on the number of reinforcers that would have occurred had the baseline schedule continued during extinction. For a given duration of extinction, this number, and thus the magnitude of stimulus change, will be larger following training with a rich VI than following training with a lean VI schedule. (During the first 15 min of extinction, for example, the number of omitted reinforcers will be about 30 following VI 0.5-min training but only about two following VI 8-min training.)

Figure 3 shows three "adjusted" measures of resistance to extinction plotted as functions of the rate of reinforcers during training. These measures are analogous to the ones plotted in Figure 2, but in Figure 3 they are expressed relative to the number of omitted reinforcers. To determine the number of omitted reinforcers, we first computed the mean interreinforcer interval obtained over the last five training sessions of the relevant schedule (i.e., the reciprocal of the rate of reinforcers listed in the Appendix, expressed in minutes) and then divided that interval into the time elapsed during the extinction session. Unlike the plots in Figure 2, those in Figure 3 all show a positive relation between the measure of resistance to extinction and the rate of reinforcers during training. (The two outliers in the top row, one for Rat C2 and the other for Rat C4—the closed circle at the upper left in each panel—are from the VI 16-min condition. As discussed in Shull et al., 2004, there is reason to suspect that performance might not have reached stability under that condition.) In other words, it took more omitted reinforcers to bring about a given decline in response rate if the training VI had been rich than if it had been lean, consistent with data reported by Nevin and Grace (2005) obtained with the multiple-schedule procedure and consistent with the principle that greater reinforcement during training results in greater resistance to extinction.

An external file that holds a picture, illustration, etc. Object name is jeab-85-01-02-f03.jpg

Three "adjusted" measures of resistance to extinction plotted as a function of the obtained rate of reinforcers during training (mean over the last five sessions) for each of the 4 rats.

The measures are the same as shown in Figure 2 except calculated relative to the number of omitted reinforcers during extinction (or, equivalently, calculated relative to the number of mean baseline interreinforcer intervals that occurred during extinction). For the top row, the adjustment is the number of omitted reinforcers during the 2-hr extinction session, which is equivalent to 2 times the obtained reinforcers per hour (r) during training.

A related set of ideas is given formal expression in Nevin and Grace's (2000) Equation 17 (Equation 1 here):

equation image

where the term on the left of the equality sign is the measure of resistance to extinction that was plotted in the top row of Figure 2 (i.e., the logarithm of the response rate during extinction [B_x ] as a proportion of the response rate during training [B_o ]). In the expression on the right of the equality sign, the numerator represents the attenuating effects of extinction due to suspending the response-reinforcer contingency (c) and to the magnitude of the stimulus change from training to extinction (dr) due to withholding reinforcers, with r indicating the rate of reinforcers during training and d representing a proportionality constant. x is a scaling constant representing the number of extinction sessions or their duration; because we gave a single fixed-duration extinction session, we set x equal to 1.0 for all subsequent analyses. The denominator (r^a ) represents the strengthening effect from a given rate of reinforcers (r) during training—the effect that makes behavior resistant to attenuation by extinction and other variables; a represents the sensitivity of this effect to the rate of reinforcers. (Because dr represents an effect of extinction, the r in that term could be interpreted as representing the number of omitted reinforcers per hour of extinction rather than the reinforcers per hour during training [Nevin & Grace, 2005; Nevin et al., 2001]. So interpreted, Nevin and Grace's Equation 17 would join with other accounts, cited above, in treating the discriminability of extinction as proportional to the number of omitted reinforcers [i.e., to the number of mean training-duration interreinforcer intervals].)

Equation 1 expresses the point explicitly that arranging a higher rate of reinforcers during training would have opposing effects on resistance to extinction. Extinction would become more discriminable (dr), thereby reducing resistance extinction, but the response would be further strengthened (r^a ) in the sense of becoming more resistant to extinction. If the effect of discriminability dominated, the trend in the data would be negative, as in Figure 2, over much of its range. A successful fit of Equation 1 to the extinction data, therefore, could be taken as support for an additional positive relation between resistance to extinction and rate of reinforcers.

We fitted Equation 1 to the data shown in the top row of Figure 2 (i.e., a single fit to the data from both schedule types). The fits were carried out with Microsoft® Excel Solver, set to adjust the free parameters so as to minimize the sum of the squared deviations between the data points and the fitted function (i.e., the sum of the squared residuals). For these fits, the value of x was set to 1.0 so that only c, d, and a were free to vary; both c and d were constrained to be equal to or greater than zero. Table 1 shows the values of c, d, and a, as well as the percentage of variance accounted for (VAC) by these fits. The value of c is zero for 3 of the 4 rats and nearly so for the 4th (Rat C3).

Table 1

The values of the best-fitting free parameters of Equation 1, along with the proportion of variance accounted for (VAC) for Equations 1 and 2. The set on the left shows the values based on all the data; the set on the right shows the values based on all the data except those from the VI 16-min schedule.

Rat	All data					Without VI 16-min data
	Equation 1				Equation 2	Equation 1				Equation 2
	c	d	a	VAC	VAC	c	d	a	VAC	VAC
C1	0.00	0.31	0.76	0.63	0.81	0.00	0.33	0.78	0.59	0.88
C2	0.00	0.16	0.58	0.82	0.38	0.00	0.18	0.61	0.81	0.83
C3	0.01	0.14	0.60	0.91	0.78	0.01	0.14	0.60	0.91	0.78
C4	0.00	0.30	0.74	0.70	0.52	0.00	0.35	0.77	0.67	0.95

The top row of Figure 4 reproduces the top row of Figure 2 and includes the best-fitting version of Equation 1 for each rat. Equation 1 appears to characterize the data trends fairly well. To the extent that it does so, the existence of a positive relation between resistance to extinction and rate of reinforcers during training gains support.

An external file that holds a picture, illustration, etc. Object name is jeab-85-01-02-f04.jpg

Top row: A measure of resistance to extinction (the logarithm of the response rate during extinction as a proportion of baseline response rate) plotted as a function of the obtained reinforcer rate (mean of last five training sessions) for each rat.

The solid curve in each panel represents the best-fitting version of Equation 1 [i.e., Nevin & Grace's (2000) Equation 17]. Table 1 lists the values of the parameters from these fits and the proportion of variance accounted for. Bottom row: An adjusted measure of resistance to extinction derived from the fits of Equation 1. The solid curve in each panel represents Equation 2, with the value of a derived from the fits of Equation 1, as listed in Table 1.

Indeed, it should be possible to extract and display the positive relation by adjusting the measure of resistance to extinction so as to eliminate the differential effect of the discriminability of extinction. Within the framework of Equation 1, the rationale for this adjustment can be seen by dividing both sides of the equation by the term representing the attenuating effects of extinction (and eliminating x by setting it equal to 1.0) so that:

equation image

The expression on the left of the equality sign is the adjusted log resistance measure. The adjustment was carried out for each rat's data set using the values of c and d obtained from fitting Equation 1 (Table 1). (Because c was virtually zero for this data set, the adjusted resistance measure was, effectively, log(B_x /B_o )/dr, which is similar to the adjusted measure in the top row of Figure 3.) The plots in the lower row of Figure 4 show that for each rat the adjusted log resistance measure increased as a function of the rate of reinforcers during training. The solid curves represent Equation 2. They were derived from the parameters resulting from the fits of Equation 1 and do not, therefore, represent separately fitted functions. Yet they appear to characterize the trends of the data points fairly well—except for an outlier in the upper left of three of the plots, most notably for Rats C2 and C4, less so for Rat C1. These outliers came from the VI 16-min schedule for which baseline response rate might not have been entirely stable, as noted above. To assess the possible influence of the VI 16-min data, the fits of Equation 1 were carried out again but without the VI 16-min data. As indicated in Table 1, the parameters and VAC from the fits of Equation 1 were similar whether or not the VI 16-min data were included (as we hoped they would be). But the fits of Equation 2 to the adjusted resistance data, as measured by the VAC, were better without the VI 16-min data (mean VAC of 0.86 vs. 0.62).

Extinction data are understandably variable, so it is not surprising that the VAC values for Equations 1 and 2 (even with the outliers removed) are not particularly high (see Table 1). Yet the residual plots in Figure 5 show little evidence that the deviations from Equation 1 were systematic (with the possible exception of Rat C1). Equation 1 thus appears to offer a fair, although imperfect, description of the trend in the data set for each rat. Equation 1 has received extensive empirical support as an account of extinction data obtained from multiple schedules (Grace et al., 2003; Nevin & Grace, 1999, 2000, 2005; Nevin et al., 2001). The present data suggest that it applies well to data from the single-schedule procedure, as predicted by Nevin and Grace (1999).

An external file that holds a picture, illustration, etc. Object name is jeab-85-01-02-f05.jpg

Residual plots for Equation 1.

The difference between the obtained and predicted values of log(B_x /B_o ) are plotted for each rat as a function of the obtained reinforcer rate during baseline. The horizontal line in each panel indicates no difference between obtained and predicted values.

To recapitulate, extinction data obtained with the single-schedule procedure appeared to contain evidence of a positive relation between resistance to extinction and rate of reinforcers during training (Figures 3 and 4) provided that the resistance-to-extinction measures were adjusted to cancel the presumed differential effect of reinforcer rate on the discriminability of extinction. The two kinds of adjustments that we made were based on assumptions about how the rate of reinforcers during training might affect the discriminability of extinction. Although those assumptions have substantial theoretical and empirical support, they might, nevertheless, be wrong. Thus our interpretation of the trends in Figures 3 and 4 rests on assumptions of plausible but uncertain validity.

Experiment 2

There is an implication of the discriminability hypothesis that does not depend on assuming a particular function relating discriminability of extinction to the rate of reinforcers during training. Suppose that the comparison training conditions provided different amounts of the reinforcer (e.g., different numbers of pellets) but at the same rates (i.e., with the same VI schedule). This method of varying reinforcement should induce different strengths (due to the different reinforcer amounts), and it might do so without engendering the large differences in the discriminability of extinction that presumably occur when the rate of reinforcers is varied among conditions. That is, the change from training to extinction might be more discriminable when the reinforcer during training had been large than when it had been small. But this difference in discriminability seems likely to be substantially smaller than the difference engendered by different rates of reinforcers during training. If so, a positive relation might be evident, even with the single-schedule procedure, between resistance to extinction and the amount of the reinforcer during training. Adjusting the resistance measures might be unnecessary for observing this positive relation because the differential discriminability of extinction might be too small to require an adjustment. Experiment 2 carried out this kind of comparison by varying the amount of the reinforcer between training conditions.