Uncertainty as Wealth, 2003
Ainslie, G., (2003) Uncertainty as wealth. Behavioral Processes, 64, 369-385.
http://www.picoeconomics.org/HTarticles/Uncer/Uncertainty2.html

 

`

 

Uncertainty as Wealth

George Ainslie
Veterans Affairs Medical Center, Coatesville PA, USA
University of Cape Town, South Africa
George.Ainslie@va.gov

 

 Published in Behavioural Processes, 64, 369-385, 2003

This material is the result of work supported with resources and the use of facilities at the Department of Veterans Affairs Medical Center, Coatesville, PA, USA. The opinions expressed are not those of the Department of Veterans Affairs of the US Government.

 

Abstract

Emotional satisfaction cannot be increased above “normal”—the same normal as the caveman’s—for any length of time, but the wealth and consumption style of modern civilization may systematically reduce some people’s satisfaction below normal.  Hyperbolic discounting of delayed, expected rewards suggests causes for this reduction in humans, and for how we often respond to it, while conventional exponential discounting does not.  Hyperbolic discounting has been well demonstrated by four experimental routes; and there is moderate evidence that it motivates impulse control by an intertemporal bargaining technique, proposed as the mechanism of willpower.

A theoretical model is described in which emotion is a reward-dependent behavior rather than a stimulus-bound respondent.  Positive emotion is then limited by premature satiation of the appetite for it, a relentless process motivated by the impatience that is described by hyperbolic discount curves.  This satiation can be restrained only by using adequately rare and unpredictable occasions as cues for the emotion.  Willpower not only is helpless against the urge for premature satiation, but it exacerbates the satiation problem by making anticipation more thorough.  The result is an asymmetrical contest between systematic attempts to vouchsafe satisfying events and impetuous attempts to put them at risk.  Despite their adversarial relationship, both may to some extent be in the person’s long range interest.

Keywords:  Boredom; emotion; happiness; hyperbolic discounting; impulsive behavior; respondents; risk-taking; self-control; will

 

 

God, if only I was hungry…  I’d walk into the restaurant, my stomach would start growling right as I sat down, and I’d proceed to stuff my face for a good hour or so.  I’d kill to have room for something like that…  Curse this full belly!   – Cal Link, The Onion, October 10, 2001

Webster defines “impetuous” as “acting suddenly with little thought; rash; impulsive.”   The trait has been declining in favor among modern societies, which are increasingly apt to label it as a character disorder or attention deficit disorder (Wright, 1999).  It appears to be the opposite of rationality, which as refined by modern utility theory (“rational choice theory”) demands consistency of choice over time (Sugden, 1991; Korobkin & Ulen, 2000; Coleman, 1990).  Consistent people are certainly more apt to avoid impulses and self-defeating behaviors, and thus compete best in the marketplace.  If we equate a person’s welfare with her economic wealth, we will have to conclude that the consistency achieved by individuals through self-discipline, and by organizations through the application of systematic guidelines to members’ choices, is an outcome that ought to be maximized.  Conversely, impetuosity must be a maladaptive trait, and ought to be cured or trained out of those who suffer from it.

The Need to Reappraise What Constitutes Welfare

Mounting evidence shows rational choice theory to be incomplete at best.  It makes many kinds of observed human behavior anomalous, among them gambling behavior, the need for self-control, and altruism (Jolls et.al. group these anomalies under the headings of “bounded rationality, bounded willpower, and bounded self-interest”—1998, p. 1476).  I have addressed these problems generally in a larger work (Ainslie, 2001); here I will develop a long but specific line of hypotheses that suggest a role for impetuosity beyond the one in which it is usually cast, that of maladaptive symptom.  The necessary steps can be summarized in short sentences:

 Research has confirmed the wisdom of ages that prosperity does not increase happiness.  All over the world, self-reports of happiness have not increased as material wealth has (Diener, 2000; Easterlin, 1995; Myers, 2000).  Indeed, changes of an individual’s physical circumstances as radical as winning a major lottery or having a leg amputated do not change her self-reported happiness for more than a few months (Brickman et.al., 1978). Wealth in material objects or even physical capacity is only tangentially connected with subjective state of satisfaction.  And many people are aware of this.  Citizens of developed countries report “post-materialist value orientations (Abramson & Inglehart, 1995), and even among adolescents the mentally healthy have relatively nonmaterialistic “life priorities (Cohen & Cohen, 1996).”

At first glance the finding of relative constancy in subjective well-being despite differences in objective condition offends common sense.  In the scientific theory that most closely parallels common sense, cognitive psychology, happiness is governed by judgments about facts.  Thus Diener (2000) explains this constancy by pointing out that people change their expectancies as their material circumstances change, and hypothesizing that they judge well-being by comparing their circumstances with these expectancies.  Similarly, to explain clinical depression, an inverse of well-being that seems to be increasing by the decade, Schwartz (2000) hypothesizes that modern culture makes expectations change faster than circumstances improve, leading to disappointment.  But the seemingly counterintuitive failure of subjective satisfaction to grow with objective prosperity merely reminds us that modern, “higher” satisfactions depend on a homeostatic system just as much as the satisfaction of hunger and thirst; as with the satisfaction of hunger and thirst, other satisfactions will have a physiological upper limit.  The processes subtending satisfaction presumably evolved to motivate choice, and would have no use for levels above what were needed to differentiate alternatives.

The rising expectations to which authors refer may reflect needs for increasing stimuli just to maintain optimal satisfaction, following the same logic as an addict’s need for increasing doses of drug.  This is easy to understand.  The hard question is, what principles govern how events occasion satisfaction?  This question remains important despite caps to satisfaction:  Short-term variations that do not change aggregated happiness can nevertheless dominate choice, because of an overvaluation of imminent events that I will discuss presently; and although great increases in aggregate happiness (above “normal”) are not at stake, great decreases may be.  That is, the higher satisfactions may still vary downward (below normal) to an extreme degree, just like those from food, drink, oxygen, and temperature.

 

The conventional theory of welfare is ripe for re-examination.  Psychologists, having noted that happiness does not vary with conventional wealth, have not gone on to suggest how it does vary.  As for economists, while they acknowledge that utility may be highly individual and dependent on non-material goods, their actual analyses have been restricted to concrete rewards and punishments.  Even Becker’s pioneering work on preferences for non-monetary goods dealt with physical transactions like drug addiction and theft (1976).  Why, then, do the fields that study welfare continue to interpret wealth as an ample store of goods?

The answer is probably that there are good tools for studying this kind of wealth.  Although this is a case of the drunk looking for his keys in the light rather than in the dark where he lost them, the alternative of trying to hypothesize a mechanism of larger satisfactions has been daunting.  It seems to lie in the area of emotional experience, which some authors regard as the source of most motivation.  (For instance, Izard says, “Emotions constitute the primary motivational system for all levels of behavior (1991, p. 84),.” echoing Sartre who despite his mistrust of its influence said, “No great work is ever accomplished without passion.”  If you subtract the relief of physical hungers from the set of strongly motivated human goals, what is left are events that “induce” emotion.  The great novels are about romantic love and love of family, the desire for glory and the desire for revenge, the gratification that comes of dominating others and the resentment, or satisfaction, that comes of being dominated.  The great religions teach openness to empathic experience and escape from the slavery of concrete rewards.  Most schools of psychotherapy aim to make clients more aware of the immediacy of experience and less bound by rigid personal rules called “conditions of worth,” “cognitive maps,” or “a punitive super-ego,” (Corsini, 1984).  This is not to neglect such lofty pursuits as intellectual culture or social idealism; but without emotional inspiration such things become empty exercises, as John Stuart Mill realized after years of pursuing both for their own sake:

The whole foundation on which my life was constructed fell down.  All my happiness was to have been found in the continual pursuit of this end.  The end had ceased to charm, and how could there ever again be any interest in the means? (1873/1974, p. 94).

Thus the greater part of wealth should perhaps be equated with the prospect of emotional experience.  “Wealth” also implies substantial duration—even money would not be called wealth if you could have it only temporarily.  Emotion-based wealth is then the prospect of having rewarding emotional experience over a period longer than the immediate future.  It excludes transient feelings, and feelings that lure your attention but are preponderantly unrewarding.  However, this is an absurdly rudimentary way to describe the motivational engine that drives most human endeavor.

We know a great deal about the properties of the processes that get called emotions.  The boundaries of this class are a matter of debate, but a core definition probably excludes those that lack motivational valence (Ortony et.al., 1988) and those that have valence but lack a self-perpetuating quality—that is, those that do not temporarily lower the threshold for further activity of the same kind (Panksepp, 2000, p. 143).  It is probably the tendency to self-perpetuation over a period of time that gives them a substantive quality, leading us to name them with nouns (joy) rather than verbs (joying; Frijda, 2000, p. 66).  Innate constraints on the plasticity of many emotions have been suggested by the demonstration of unlearned facial expressions, even in neonates and the congenitally blind (Galati et.al., 1997), which are recognizable across cultures (Ekman & Friesen, 1986).   In addition, these basic (“blue-ribbon, grade-A”) emotions—rage, fear, sadness, joy, affection--  have long been induceable by electrical brain stimulation (Delgado, 1969) and are now known to have specific brain circuits using specific neurotransmitters (Panksepp, 2000).   Many more experiences have been named emotions, but they tend to have indistinct boundaries and to be acknowledged in one culture but not another, or even in one historical period but not another within the same culture (Lutz, 1988; Stearns, 1986). 

 

 

Emotion as Reward-Dependent Behavior

The original behaviorist model of emotion had it evoked as a conditioned response to innately determined stimuli (Watson, 1924).  However, it proved to be hard to trace the emotional impact of a stimulus to a conditioning event.  Even in the laboratory fear is the only emotion that has been conditioned; actual phobias are rarely a consequence of trauma involving the object feared, and trauma rarely leads to phobia (Rachman, 1977).  The belief that an emotion is determined by a distant releasing stimulus linked to the immediate occasion by a chain of associations was an act of faith, not a conclusion required by the evidence.

Current notions of what induces emotion are less specific, but still imply that it is driven by external givens that a person encounters—if not innate releasing stimuli then belief  that she faces a condition that contains these stimuli.  Emotion is still a reflex of sorts, albeit usually a cognitively triggered reflex, a passive response to events outside of her control—hence “ passion.”  In reviewing current cognitive theory, Frijda notes that the trigger may be as nonspecific as “whether and how the subject has appraised the relevance of events to concerns, and how he or she has appraised the eliciting contingency (2000, p. 68);” but this and the other theories of induction he covers still involve an automatic response to the motivational consequences of the event, not a choice based on the motivational consequences of the emotion itself.  Even though emotions all have such consequences,  “the individual does not produce feelings of pleasure or pain at will, except by submitting to selected stimulus events (ibid p. 63).”   That is, all emotions reward or punish, but they are not chosen because of this consequence.  In every current theory they are not chosen at all, but evoked. 

The seemingly universal theoretical acceptance of the automaticity of emotions ignores a fair amount of both common experience and data.  Granted that emotions are usually occasioned by events outside of your voluntary control; the theory that they are governed by such events runs afoul of the widespread acknowledgment that they are trainable: You can “swallow” your anger or “nurse” it, learn to inhibit your phobic anxiety (Marks & Tobena, 1990), panic (Clum et.al., 1993; Kilic et.al., 1997) or grief (Ramsay, 1997), refrain from rejoicing or “give yourself over to it.”  Techniques to foster or inhibit emotions in everyday life have been described (Parrott, 1991), as has their use in preparing yourself for particular tasks (Parrott, 1993).  Most schools of acting teach an ability to summon emotion deliberately (e.g. Strasberg, 1987; Downs, 1995), because even in actors actual emotion is more convincing than feigned emotion (Gosselin et.al., 1998).  The frequent philosophical assertion that emotions have a moral quality—good or bad (e.g. Hume as presented by Baier, 1991)—implies motivated participation; some philosophers have gone so far as to call the passions voluntary (e.g. Sartre, 1939/1948).   In sum, emotions show signs of being goal-directed processes that are ultimately selected by their consequences, not their antecedents.  That is, they are at least partially in the realm of operant behaviors, not conditioned responses; they are pulled rather than pushed.  Even “negative” emotions like fear and grief seem to be urges that lure you into participating in them, rather than automatically imposed states

Of course, emotions cannot be willed like a leg movement.  You could not beat a rapid rhythm with, say, bursts of anger; but this is also true of the urinary sphincter (a smooth muscle), and certainly the various autonomically-controlled and central nervous system processes that can be brought under voluntary control with biofeedback (Heyman et.al, 1999; Kotchoubey et.al., 2001; Nakao et.al., 1997).  This does not mean that such slower-responding processes are unmotivated.  Even the absence of deliberate volition does not mean that a process is unmotivated.  Like the changes in posture that occur unconsciously to maintain comfort, even during sleep, behaviors like salivation, sexual arousal, and the emotions represent semi-autonomous processes that often respond to the prospect of reward without being willed—indeed that can be inhibited by the will only with training.  Such processes can be thought of as foraging for possible rewards just as animals forage for food, and responding to available rewards more like your livestock than like your muscles.  The Roman physician Galen already knew this, pointing out that anger was tamed like a horse, but that the “concupiscible power,” like a wild boar or goat, had to be controlled by starvation (1963, p. 47).

Instances of entertaining emotions in a goal-directed fashion are usually dismissed as examples of self-conditioning.  Actors, for instance, use rehearsal of significant emotional memories to learn the necessary control, and psychotherapists often use guided imagery to influence emotions.  According to conditioning theories you find the right conditioned stimulus and provoke your own reflex with it, like hitting your own knee with a rubber hammer to produce a jerk.  It is true that in a given instance the operant sequence of

   cue → response → reward

can always be interpreted as the respondent sequence of

  conditioned stimulus → conditioned response → unconditioned or lower-order conditioned stimulus

and vice versa.  However, if the conditioning stimulus is not repeated on successive trials, a true conditioned response will extinguish.  The memory or image will stop evoking the emotion.  If the response grows and comes more readily, like the actor’s emotion as she learns to summon it, it must have come under the control of a different selection agent, which probably means that it has been learned as an operant.  Learning to induce an emotion follows the same course as a bulimic’s learning to vomit at will—the gagging stimulus of a spoon or finger becomes less and less necessary, and eventually can be dispensed with altogether.

It is a striking fact that the operant quality of emotions has not even been considered in the literature of behavioral science.  This omission may have partly reflected society’s recognition of only the least manipulable of emotions as authentic, which serves our need for ascertaining otherwise hidden personal facts about each other.  In reading others’ character and intentions, people disregard emotions that seem to be generated voluntarily (Frank, 1988); perhaps as a result, the culture stigmatizes such emotions as artificial.  It is the processes on the least controllable end of the continuum that define the usual concept of emotion.

However, theoretical problems implicit in the concept of operant reward have probably been a greater obstacle, even though they have not been discussed directly.  These theoretical problems follow from the conventional utility-based model of motivation.  If you could produce “feelings of pleasure or pain at will,” why not overdose on the pleasure and skip the pain, without regard to the outside world?  If an emotion is aversive and avoidable, what induces people to entertain it?  If an emotion is pleasurable and readily accessible, what keeps people from indulging in it ad lib?  Of course there are sometimes reasons why emotions are useful for practical purposes (Parrott, 1993), but these reasons pale beside their pure hedonic potential.  The ability to choose emotions as behaviors might let any behaving organism reward itself autistically, ignoring the demands of its environment. 

However, modification of utility theory in light of the findings of parametric research on how prospects are discounted with delay will suggest a solution.  The modified model shows how particular risks and opportunities in the environment can invite emotions in a relatively free internal marketplace of motivation, without absolutely overriding their competitors.

A Precis of Hyperbolic Discounting

It is now well documented that behaving organisms have a robust tendency to devalue expected incentives in a hyperbolic curve.  Such a curve represents a radical departure from the exponential curve that has been the explicit assumption of behavioral psychology and classical economics, and is implied by the “rational choice theory” that has become the norm in all behavioral sciences that depend on utility theory (Sugden, 1991; Cooter & Ulen, 2000).   Four kinds of experiment have demonstrated this phenomenon:

1. Given choices between rewards of varying sizes at varying delays, both human and nonhuman subjects express preferences that fit curves of the form,

a hyperbola, better than the form,         

an exponential curve (where V is motivational value, A is amount of reward, D is delay of reward from the moment of choice, and k is a constant expressing impatience; Grace, 1996; Green, Fry & Myerson, 1994; Kirby, 1997; Mazur 2001).  It has also been observed that the incentive value of small series of rewards is the sum of hyperbolic discount curves from those rewards (Brunner & Gibbon, 1995; Mazur, 1986; Mitchell & Rosenthal, 2003).

2. Given choices between smaller-sooner (SS) rewards and larger-later (LL) ones available at a constant lag after the SS ones, subjects prefer the LL reward when the delay before both rewards is long, but switch to the SS reward as it becomes imminent, a pattern that would not be seen if the discount curves were exponential (Ainslie & Herrnstein, 1981; Ainslie & Haendel, 1983; Green et.al., 1981; Kirby & Herrnstein, 1995).  Where anticipatory dread is not a factor (with nonhumans or with minor pains in humans), subjects switch from choosing SS aversive stimuli to LL ones as the SS ones draw near (Dinsmoor, 1998; Novarick, 1982; Solnick, 1980).

3. Given choices between SS rewards and LL ones, nonhuman subjects will sometimes choose an option available in advance that prevents the SS alternative from becoming available (Ainslie, 1974; Hayes et.al., 1981).  The converse is true of punishments (Deluty et.al., 1983).  This design has not been run with human subjects, but it has been argued that illiquid savings plans and other choice-reducing devices serve this purpose (Laibson, 1997).  Such a pattern is predicted by hyperbolic discount curves, while conventional utility theory holds that a subject has no incentive to reduce her future range of choices (Becker & Murphy, 1988).

4. When a whole series of LL rewards and SS alternatives must be chosen all at once, both human (Kirby & Guastello, 2002) and nonhuman (Ainslie and Monterosso, 2003a) subjects choose the LL rewards more than when each SS vs. LL choice can be made individually.  The effect of such bundling of choices is predicted by hyperbolic but not exponential curves.

In short, exponentially discounted prospects do not change their relative values however long they are delayed or however many are summed together; but hyperbolically discounted SS rewards become disproportionately valued as they draw near, and can lose much of this differential value when the choices are bundled into series.

The most obvious implication of hyperbolic discounting is that people’s preferences are not intrinsically stable—that they can be expected to change as a function of elapsing time unless the person adopts some means of forestalling future changes of preference.  Such changes offer a ready explanation for a broad range of impulsive choices (Monterosso & Ainslie, 1999), especially addictive choices in people who are “trying” to be sober (Bickel et.al., 1999; Madden et.al., 1997; Mitchell, 1999; Vuchinich & Simpson, 1998).  The frequent experience of emotional processes as independent-minded animals living in our bodies also makes more sense with hyperbolic discounting.  Impetuosity is not a pathological development; it is the starting place of all decision-making.

Furthermore, the absence of intrinsic stability and the likelihood of frequent, similar choices that are not in the person’s long range interest have suggested the first explicit mechanism for willpower: the increased motive to avoid impulses recruited by the perception of current choices as precedents for future choices in similar situations.  This perception can be expected to foster tacit intertemporal bargaining somewhat like the bargaining that stabilizes repeated prisoner’s dilemmas (PDs; Ainslie, 2001, pp. 90-104).  Briefly, perceiving a choice as a precedent recruits the (summed, discounted) incentive value of the whole series of relevant choices, the prospect of which requires the present self to “cooperate” with future selves by choosing the LL alternative.  Conspicuous features of the situation become criteria for personal rules, classes of cues defining what behaviors the person herself will see as defections in this tacit bargaining.  This mechanism depends on discount curves with relatively high tails, which can become dominant when the curves from multiple rewards are bundled together; it would not operate if discounting were exponential instead of hyperbolic (see figure 1).

 

 

It thus looks possible that people’s self-control functions are not inborn, but evolve in an internal marketplace in the manner of Adam Smith’s unseen hand.  The most radical change that hyperbolic discounting suggests for a model of human personality is that a person may not have a unitary self; rather she may comprise a population of processes that have been shaped by the rewards they obtain, although within inborn constraints on what is rewarding and what can be learned (Ainslie, 2001, pp. 39-47).  Then it would be not only emotions that resembled independent-minded animals, but all motivated processes.  The perception of common elements might herd them into systematic patterns that would be experienced as deliberate, but individual processes would remain influenceable by imminent prospects for reward.

The properties of a recursive mechanism that can generate complex self-control are best understood, and perhaps best studied, by examining the incentives that bear on the analogous interpersonal situation.  In fact repeated two-person PDs evoke patterns of cooperation and defection similar to those that have been described for a barely adequate will (Monterosso et.al., 2002).   However, the threat of retaliation that is a cornerstone of strategy in the two person PD is not literally possible in an intertemporal game among successive selves.  A modification of the PD to model the intertemporal case more precisely was used in a pilot experiment I did with John Monterosso and Pamela Toppi Mullen:

 A roomful of male residents in a substance abuse program were told that the experimenter would call on the seated residents in order, and would ask each one to choose between having every resident get ten cents for that turn, or getting a dollar just for himself.  Then the next subject would get the same choice, and so on until an unpredictable point after at least twenty subjects had chosen.  To prevent a predictable endpoint, the turns might start to go around a second time, but most subjects chose only once.  The incentives each subject faced were roughly those of an individual deciding whether to try willpower against a recurring temptation:  Successive choice-makers choose only once.  If a “cooperation” decision (cf. ten cents for all) is necessary and sufficient to move subsequent choice-makers to cooperate, each will do better by cooperating than by defecting (cf. getting the dollar).  If many previous choice-makers have defected, one cooperation will probably not seem sufficient to reverse the trend.  Even if most have cooperated, with luck a current cooperation may turn out not to be necessary; but defection will be a gamble that may set off a stampede of defections.  This risk is the intertemporal equivalent of the threat of retaliation in the two-person PD.  

We had hoped that this design could test predictions about the logic of bargaining, such as the possibility that a defection by a ringer who was conspicuously different from most subjects (the only outsider, say) would be less apt to provoke subsequent defections—the analog of a good rationalization for giving in to a particular temptation.  However, the method proved impractical, and is described only to illustrate the contingency structure of intertemporal bargaining.

Although the possible implications of hyperbolic discounting are far-reaching, they are hard to study by means more direct than thought experiments, modeling, and simply fitting theory to patterns observed in psychiatric patients and ordinary people (Ainslie, 2001, pp. 117-140).  They permit simplifications of motivational theory, but these simplifications entail changes in our habitual distinctions between behavior attributes, even such basic ones as rewarding/aversive, reward-dependent/conditioned, and rational/irrational.  The rest of this article will develop one of these implications, revising utility theory in the light of  hyperbolic discounting to predict how impetuosity affects welfare.  I do not claim to have experimental support for the resulting model, much less proof.

Hyperbolic Discounting as the Constraint on Emotion

Hyperbolic discounting makes possible a fully operant model of emotions, in contradistinction to the (usually tacit) assumption that they are classically conditioned.  Classical conditioning has several flaws as a selective process for responses-- including mental responses-- beyond just the association of sequences of stimuli.  To enumerate problems discussed elsewhere (Ainslie, 1992, 41-48; 1999a; 2001, 18-22):  Where proper feedback is possible “conditioned” responses can be modified and even reversed by operant incentives (Heyman et.al, 1999; Kotchoubey et.al., 2001; Nakao et.al., 1997; Ainslie & Engel, 1974); conditioned responses must compete with each other (O’Brien et.al., 1986) as well as operant incentives, making comparability of selective principles a necessity (Donahoe, 1993, p. 21); and conditioned responses are not just copies of the corresponding unconditioned responses but are shaped afresh, presumably by some other selective principle (Rescorla, 1988).  Those problems aside, the fact that unconditioned stimuli all have motivational valence as well (Miller, 1969) and can support what look like conditioned responses when treated as operant incentives in computer models (Donahoe et.al. 1997) makes them desirable targets for Ockham’s razor, that is, for being subsumed under operant reward (discussed further in Ainslie, 2001, pp. 18-22).  The principle obstacle to this simplification is the theoretical problem raised above in the case of emotions:  What would make organisms entertain painful experiences, or limit their indulgence in pleasurable ones?

Negative emotions without conditioning  The model of what imposes pain is not necessary to the point of this article; but  since I will argue that a conditioning or other automaticity mechanism is unnecessary in positive emotions, I should summarize how it can also be dispensed with in negative emotions.  This is the strong form of a self-reward hypothesis that is permitted for the first time by hyperbolic discounting.  A weak form that merely depends on the ready accessibility of emotion-inducing stimuli would largely preserve my argument for positive emotions, but would leave negative emotions dependent on a stimulus-driven mechanism like conditioning.

 

The argument for seeing negative emotions as operants involves the commonalities of aversive emotions and addictive rewards (Ainslie, 2001, pp. 90-104).  Although both are usually avoided from a distance, both are seductive when they might occur in the near future.  That is, however much you know that a binge will cost more than it is worth or that a fear is unfounded, it is sometimes hard not to participate in them.  The examples reviewed above, of a learnable skill of inhibiting negative emotions, demonstrate that these emotions are based on urges that are resistible but hard to resist—not on obligatory reflexes or other unmotivated processes—just as addictive behaviors are.

Addictive behaviors can be well explained by imminent highs that are valued temporarily, because of  hyperbolic discounting, above the more delayed rewards of sobriety (Mitchell, 1999; Vuchinich & Simpson, 1998).  How the opposite rewarding and unrewarding incentives for negative emotions are mixed together to attract attention but deter approach in general is still unclear.  The similarity to addictive behaviors suggests that the urge to succumb to panic, anger, anguish, and even physical pain might be based on a rapidly recurring but very brief reward, lasting long enough to command attention but not deliberate choice, and fused in perception with longer, unrewarding consequences to form an experience both vivid and aversive (Ainslie, 1992, pp. 100-114).  Other bases for the mixture of attraction and aversion are certainly possible, but they must entail a broader concept of reward than just pleasure (an outcome desirable at any distance); reward must simply be whatever increases the frequency of the choices it follows, including those processes, like panic, that are avoided at most distances and, when selected, are instantly regretted.  Preliminary fMRI studies do show some responsiveness in brain reward centers to painful stimuli (Mirenowicz & Schultz, 1996), but very short temporal sequences still cannot be studied.

Positive emotions without releasing stimuli  Given the rich array of stimuli available in a cosmopolitan society, the converse problem-- what limits positive emotion?—would exist even if emotion had to be elicited by stimuli-- conditioned-- instead of shaped by reward.  You could deluge yourself with stimuli for positive emotions (in the weak theory) just as well as you could learn to generate the emotions as behaviors, and the question in either case is, what limits your incentive to do so?  The obvious answer in either case is that rehearsing a positive emotion ad lib attenuates its effect to the level of a daydream.  The problem then becomes, why should this be so. 

This problem has received little attention.  To explore it we need to recognize that emotion depends on a readiness—a curiosity or suspense or longing or even foreboding-- that could be called appetite.  Maximizing emotional reward seems to depend on building appetite for it.  In the laboratory this appetite can be induced artificially by brain stimulation or infusion of a drug.  For instance, Schachter & Singer famously showed that a nonspecific appetite for anger or euphoria could be stimulated by epinephrine, and that the emotion that most subjects then emitted depended on whether their situation contained elements suggesting the anger or the euphoria (1962; see also Sinclair et.al., 1994).  In ordinary life the only substantial way to build appetite for a positive emotion is by deferring its consumption—which suggests that the reward is not simply proportional to this deferral but increases disproportionately as deferral increases (figure 2).  It would make no sense to “work up an appetite,” as people often do for physical rewards, if the rewarding power did not increase disproportionately as deprivation increased.  It is well known that the consumption of physical rewards should be paced to get the most effect from them.  This is also known about the tangible stimuli for emotions—You should not learn whodunit before you see the movie, or read ahead in the book.  But when emotion does not need a stimulus, what keeps you from entertaining it whenever the idea strikes you, without occasion? 

 

The problem of insufficient appetite has long been a vague complaint of the rich, but, like the operant nature of emotion, it has rarely been examined.  Konrad Lorenz’ Nobel Prize acceptance speech about the ennui of the younger generation is an exception (1970):

The normal rhythm of eating with enjoyment after having become really hungry, the enjoyment of any consummation after having strenuously striven for it, the joy in achieving success after toiling for it in near-despair-- in short the whole glorious amplitude of the waves of human emotions, all that makes life worth living-- is dampened down to a scarcely perceptible oscillation between scarcely perceptible tiny displeasures and pleasures.  The result is an immeasurable boredom.

This is because

the mechanisms equilibrating pleasure and displeasure are thrown off balance because civilized man lacks obstacles which force him to accept a healthy amount of painful, toilsome displeasure.

In short:

To expend any joy down to the point of full exhaustion is downright bad pleasure-economy.

Considering the apparent magnitude of this problem, it is again striking that modern culture has not discussed it more.  Three reasons come to mind:

1.   Lack of prompt reward.  Solutions to the problem do not pay off quickly, and hence suffer from the same discounting as the emotional rewards themselves.  The prospect of deferring satisfaction must by its very nature promise heavily discounted rewards, resulting in value that is probably even less than that of immediate satisfaction of attenuated appetite.  This is, on a much smaller scale, the same consideration that often in history kept farmers from discovering the value of letting fields lie fallow:  An exhausted field grows few crops, but a fallow field grows none at all that year.

2.  Lack of conceptual tools.  Premature satiation of appetite should not occur if future experiences are discounted as common sense seems to demand, that is, exponentially instead of hyperbolically.  A conventionally rational person could just estimate what her optimum level of appetite was, and wait to induce the relevant emotion until that point was reached.  There would be no urge to “end the suspense,” because the expectation of satisfaction would be greater for holding off, up to the very point when not holding off would be most rewarding.  Just as there would be no need for personal rules against reading ahead in a book, there would be no need to actively avoid premature emotion.  Thus under the usual assumptions about the value of prospective reward, mirrored in conventional utility theory, the problem of insufficient appetite makes no sense. Failure of appetite is familiar enough, but without hyperbolic discounting to explain why people avoid that "healthy amount of painful, toilsome displeasure" it is unaccountable.

3. Actual counterproductiveness of analysis.  People seem to avoid a direct approach to positive emotions, as if recognizing the novelist Hawthorne’s insight that

happiness is a butterfly, which, when pursued, is always just beyond your grasp, but which, if you will sit down quietly, may alight upon you… Happiness in this world, when it comes, comes incidentally—Make it the object of pursuit, and it leads us a wild-goose chase, and is never attained (Browns, 1964, p. 257).

Manipulating our emotions brings us closer to gratifying them at will and thus trivializes them, so we learn to avert our gaze, and study only the tasks that may let them come “incidentally.”

To state the problem in concrete terms:  If our emotional apparatus is ready at hand like the keys of a piano, how do we decide when to play?  The hyperbolic discounting hypothesis is that the intrinsic rewards for emotionality are delivered on a DRL schedule—differential reinforcement of low rates.  The more often you strike a note, the less it rewards you, and disproportionately so.  The greatest reward as evaluated at a distance comes from greatly delayed strokes, from chords left unresolved for long periods of time.  However, hyperbolic overvaluation of immediate rewards motivates us to always play the next note quickly, and to keep pounding on it rather than exploring elsewhere on the keyboard, even though this pattern greatly reduces our reward over time.

This self-reward [now endogenous reward -added July 30, 2019] hypothesis is that the free availability of emotion as a behavior confronts people with an elementary and pervasive self-control problem.  In some areas we solve it readily.  Where playing our mental keys is tied to events outside of our control, like the enjoyment of food, we often learn to pace our access to these events.  Where a stimulus is necessary, it is possible to stay away from that stimulus until the right time, or use willpower to pace our use of it.  Even when no stimulus is necessary for our behavior—and this will be true of most emotionality—the very most unproductive playing patterns will extinguish naturally.  The choice of repeating ad libitum will be so unrewarding that any variable external cues become signals for hitting the note, and compete successfully with the choice of playing with no cue at all; generating feelings in response to even common occasions beats out generating feelings continuously.  This is probably all that is needed to ensure that nonhuman animals have their affective reactions mostly in response to external cues; but with greater imagination we humans can feed ourselves cues, and thus unsolve the self-control problem.  It is those occasions for emotion that are not continuously available, but are still too common to let appetite build up much, which are the enemies of long range satisfaction.

Surprise as the Only Way to Forestall Anticipation

The will maneuver—the recruitment of motivation against impulses by seeing current choices as precedents—can be of some use against premature satiation.   It can pose obstacles to consumption in the form of resolutions or even beliefs—especially the more socially constructed ones (Ainslie, 1993; 2001, pp. 175-179).   However, the will cannot simply forbid easy emotionality.  It can only make the person’s selection of occasions for emotion a little less responsive to her momentary wishes.

Rarity alone will be of limited value as a principle for pacing emotion.  This is because a rare event that is certain to happen at a known time will be anticipated.  Anticipation happens with the speed of attention; there is no pausing to test whether we will it or not.  We cannot keep ourselves from thinking ahead to a prospective emotional occasion any more than we can deliberately not think about white bears (Wegner, 1989).  And this anticipation will deflate our suspense, spoil the joke we have heard before, congeal drama into ritual, and convert tickling to mere touching.  It will cause the reward spike in the nucleus accumbens to occur early and then not when the objective reward occurs (Hollerman et.al., 1998; Schultz et.al., 1997).  We seem like the nearly omnipotent god who cannot build a wall so high that even he cannot climb over it.

Here is a self-control problem that the will not only fails to solve, but exacerbates.  For, if the intertemporal bargaining hypothesis is true, the will is based on perceiving regularities—choices that resemble each other so as to form precedents for a single large category of choice, choices that must be made always in the same direction on pain of reducing the credibility of the will and thus its power.  But insofar as our future behavior is predictable we anticipate it, and whatever emotions it will entail we feel in advance, diminished.  Will is more the enemy than the friend of emotional appetite.

There is a way to control the relentless impulse to peer ahead, and that is to occasion emotions with surprises.  We have to learn to gamble on uncertain outcomes in order to avoid Lorenz’ limbo of continual satiety.  The gambles can be great, like falling in love with a dangerous person, or petty, like seeing if you can get a task done before the hour strikes.  They can be long range, like seeing how a child will turn out, or momentary, like sparring with competitors in debate or traffic.  The part of the day that does not involve adjusting physical comfort is arguably structured by multiple gambles of various sizes.  The narrower the range of outcomes, the less surprise a given gamble can deliver.  Tic tac toe soon becomes boring, and checkers (draughts) not long afterward.  Chess continues to support emotional engagement by virtue of its complexity.  A roll of the dice commands suspense only if a great deal is staked on it; and although casino gambling can become an addiction, it lacks the dimensionality to maintain adults’ interest without money riding on it.  Granted that part of the art of gambling is to make stakes seem bigger than they are—for instance to give the impression of danger without the reality, as in roller coasters and bungee jumps-- The greater art is to arrange the complexity, the texture of gambles so as to restrain the relentless pull of anticipation.

I have argued elsewhere that the richest source of texture is to gamble on vicarious experience (Ainslie, 1995, 2001, pp. 179-186).  Although a person is free to sample many sources of this experience, thus introducing the danger of arbitrariness, the emotions suggested to her by a given perception are fixed—either the same emotions as her object is experiencing, or, in the case of negative empathy, an obvious converse emotion like gloating at the object’s chagrin.  Recent neurophysiological data suggest that just watching another person generates highly specific signals about what she is experiencing via the automatic stimulation of “mirror neurons” in your own cortex (Iacoboni et.al., 1999).  Insofar as this activity is surprising, it should serve to pace emotion.  Insofar as the person finds rationales to make the choice of object less arbitrary, she creates occasions that are both surprising and rare. Given the evocative power of what happens to even randomly selected strangers in news reports, to say nothing of fictional portraits, reducing her freedom of object choice by committing to uniquely determined others (one mate, two parents, etc.) should obviously create gambles of great impact.

Conversely, avoidance of major gambles on other people leads to characteristic pathologies.  Empathic impairment is a major factor in schizoid personality, in a subset of schizophrenia, in autism, and to a lesser extent in Asperger’s syndrome (Bowler, 1992); but the purest form of failure to gamble per se is narcissistic personality.  Unlike the other syndromes, narcissism occurs in someone with an intact ability to perceive and interpret emotion in others.  The problem is a reluctance to invest importance in their experience, that is, to gamble on any experience in which they stand a significant chance of not getting their hoped-for outcome.  The narcissist surrounds herself with undemanding friends and competitors of lesser ability than she, and generally takes on only tasks in which her success is assured.  The resulting complaint is of unaccountable boredom (Wink & Donahue, 1997).  The narcissist feels empty despite “having everything I ask for.”  The seeming paradox of narcissism represents a natural experiment that falsifies the conventional notion of wealth.

A Long Range Motive to Evade the Will

The commonsense view of welfare that became utility theory evolved over centuries when the average farmer had to spend half his energy just getting food (Braudel, 1981, pp. 129-145), life expectancy was in the thirties, and the preoccupying concern of popular culture was the control of fear (Muchembled, 1985). Even then theology and philosophy warned of the emptiness of material wealth (Segal, 1991), but the problem is bound to be more pressing in societies that are comfortable and safe, and most pressing of all if the comfort and safety come from the very systemization that makes life predictable.  Certainly the hyperbolic shape of discount curves will teach a person what satisfies appetites well before it reveals what creates them.  But most importantly, the will maneuver must be much more effective against behavioral impulses that lose satisfactions than against the impulse to anticipate, which spoils emotional appetite.  The systematic acquisition and organization of knowledge, and the consistent application of this knowledge to the task of obtaining satisfactions, is very stuff of willpower.  Appetite is often best refreshed by the impulses that willpower forbids—to gamble enough that you lose sufficiently often.

Consider the plight of a very successful sports team, the “damn Yankees” of the 1950s, the contemporary Australian cricket team, or a team even stronger than them.  A team that wins too often disappoints almost as much as one that loses too often—its games become boring in both cases-- but there is no obvious way for the team itself to deal with that.  It cannot choose inferior strategies or bench its best players or tell its players not to try so hard, without undermining the principles of choice that let them play at all.  Choosing by the principle of always trying to win generates clear choices or at least clear methods of choice; not always doing this opens up alternatives that cannot be easily weighed against each other.  How much should winning weigh against increasing the longing upon which the emotional impact of future wins will depend?  But also, how much will legitimizing this kind of choice create rationalizations for taking the easy road, for being lazy?  And even if the team retains adequate playing effectiveness despite blurring its discipline, how satisfying will the less frequent wins be to fans who suspect them of having been scripted?  Clearly the team cannot just pull its punches.   It must subject itself to a league or other organization that has the power to redistribute resources from extreme winners to extreme losers, or be stuck in the paradoxical failure of its very success.

If such a league did not exist, the owner could hire an impetuous manager if his impetuosity were not too extreme, on the pretext that it represented charisma or some other ostensibly desirable quality.  Then everyone could still do his best, including the manager, but the impetuosity would provide the necessary handicap.  Owners may not have actually done this, because leagues solve the problem better, and possibly also because the problem in sports is at least partially a commons problem—a monotonously winning team still does better than the regular losers, at the expense of the sport as a whole.  However, when reward-seeking processes compete within an individual, they also face the paradox of success, and some analog of the impetuous manager solution may be common.

Of course reward-seeking processes can enter into leagues with each other—this is the will maneuver.  However, willpower cannot restrain premature satiation.  On the contrary, the effectiveness of a person’s will depends on her grouping similar choices into categories and making them in a consistent way, with clear criteria that tell her when she has cooperated with this arrangement and when she has not.  She must reduce the novelty that invites miscalculations and misinterpretations, and for the same reasons reduce her own spontaneity, although to keep her occasions for emotion fresh she would have to arrange surprises.  Furthermore, a maximally effective will establishes criteria for success, and forestalls the temporary preferences for the short term pleasures that make success less likely.  The most effective criteria concern concrete accomplishments whose presence or absence cannot be a matter of argument, and which, even better, can be counted up.  Thus the will maneuver favors the building of conventional wealth and recruits incentive not to be impetuous; but if intertemporal bargaining is the basis of willpower, extensive use of it makes the person compulsive.  The leagues that participate in the will do not refresh appetite; they smother it (this and other side effects of willpower discussed in Ainslie, 1999b and 2001, pp. 143-160). 

The necessary response to this loss of surprise is to find gambles that will again put the person’s satisfactions at risk.  Of course, the motives to increase her gambles must operate without her recognition, for once she recognizes them she will have to confront them with the rationale of her will, which will make the gambles seem like contradictions to the long range plans she believes in: to save money, court promotion at work, build reputation, etc.  If she does not find some higher principle that requires her to gamble—a struggle against a superior force, devotion to a cause, a love that overrides mere money--  she is apt to develop “unconscious” behaviors that thrive despite their apparent maladaptiveness because they actually serve her long range interest by refreshing her appetite.

That is, a person's will may have become so confining that a pattern of regular lapses actual­ly makes her better off in the long run.  There is clinical lore that attributes bingeing to a patient's inhibitedness in the rest of her life; her general overcontrol is said to set up periodic episodes of breaking loose. The model of intertemporal bargaining predicted by hyperbol­ic discount curves provides a specific rationale for this pat­tern:  Personal rules that prohibit any large source of emotional reward will create a proportional motive for the person to bypass or break those rules.  Even her long range interest will lie in partially escap­ing from them.  Thus the compulsions that come from willpower potentially create alliances between long and short range interests.  The person's occasional binge comes to serve as a correction to the comparative sterility of excessive rules, a means of providing richer experiences while perhaps still limiting the scope of impulsive behaviors.  The longest range interest of an alcoholic who is too rigid when sober may be to tacitly foster the cycle of drunkenness and sobriety, rather than be continuously imprisoned by her rules.  People are often conscious of a need to slow or delay consumption of rewards, to savor them (Loewenstein, 1987); but needs that are unconscious because they are frustrated by conscious recognition may be a much larger factor in human choice, a factor that incidentally complicates questionnaire studies of preference.

Uncertainty as wealth, at last  If wealth is a store of future prospects for reward, emotional appetite must be regarded as a form of wealth, and emotional appetite can only survive in an atmosphere of at least partial uncertainty.  It does not seem to be a form of wealth that can be counted up in combination with the more conventional forms—to a large extent it contradicts them.  Also, unlike them, it is not monotonic.  The value of conventional wealth grows as goods grow, progressively flattening out, it is true, but not falling.  However, emotional appetite at some point becomes seriously aversive, even where it is storing up future satisfactions.  Uncertainty becomes insecurity or fear, and longing becomes misery.  Of course even wretchedly unhappy experiences might set up a long-lasting appreciation of conditions that would otherwise be humdrum; but there is probably a limit to how much even someone who recognizes the value of appetite can accept imminent discomfort to get a prospect of years of happiness.  It is hard to sign up for a month of Outward Bound or a non-required student rotation in surgery.  The extremely spare lifestyle of primitive hunter-gatherers like the !Kung of the Kalahari Desert is often said to go along with cheerfulness and apparent emotional vigor, but given the choice they regularly change it for the squalid lower reaches of western civilization (Thomas, 1989; see also Glantz & Pearce, 1989; Smith, 2002).

Thus for civilized people the most important requirement for realizing actual reward may be impossible to seek consciously in any great quantity.  It may often be that costly gambles that were sought unconsciously are the biggest factor in keeping emotional appetite fresh.  There are undoubtedly many details that determine the difference between refreshing appetites and turning them into nagging pains.  What I have suggested is just a starting point for examining the neglected half of psychological welfare, with theoretical guidance from the finding of hyperbolic discounting.

 

Conclusions

The elementary hyperbolic nature of how future prospects are evaluated is a likely explanation for the robustness of impulse disorders.  At first glance this model might seem to call for tighter controls on impulses, both by society and by rational individuals.  However, the intertemporal bargaining hypothesis to which it leads suggests that the most powerful controls we know of, society’s laws and an individual’s willpower, have side effects that may make impulses worse.  The drawbacks of using law have been dealt with elsewhere (Ainslie & Monterosso, 2003, pp. 856-859; see also Sunstein, 1995, pp. 991-996).  The main drawback of willpower is that both its failure and its success create incentives for contrary behaviors—the failure by encapsulating situations where further failure is likely (Ainslie, 1999b; 2001, 148-149), the success, if too thoroughgoing, by making the future too predictable.  A person’s behaviorally relevant evaluation of the costs and benefits of impetuosity probably comes to include estimates of its effect on refreshing appetite as well as its effect on satisfying appetite.  Because of the incompatibility of these two components, a person is probably not conscious of the overall evaluation.  She will then be unable to explain episodes of impetuosity, or  to explain why she is not more successful at avoiding them; but her “heart may have reasons that [her] head knows not of.” 

 

Acknowledgements

I thank John Monterosso for comments on the manuscript; him, Ron Pekala and Andrew Henry for help with references; and Wanda Sandoski for help preparing the manuscript.

 

Notes

 

1. Both “passive” and “passion” come from the Latin verb for “to suffer,” and took on their implications of separate kinds of undergoing from the 14th to 16th centuries (Ayto, 1990, p. 385).

2. It has been argued in the case of fears that conditioned responses which fail to extinguish are serving as their own unconditioned stimuli (Bersh, 1980; Eysenck, 1968).  However, as with so many elements in the operant/classical debate, this mechanism can be interpreted just as well as an operant one.  If the prospect of fear results in further fear, the mechanism might be either that fear serves as both a conditioned and an unconditioned response, or that fear rewards the operant response of fear (presumably transiently, as in gratifying the “urge to panic”).  Insofar as fear is not an obligatory response, this positive feedback effect must depend on the emotion’s being in some sense rewarding.  In the case of a visceral response that may be learned purposively, like bulimic vomiting, the selective factors are both an expectation of vomiting that makes vomiting more likely, and instrumental reward; the instrumental reward is either added to the former factor to produce learning, or is not added to it, leading the vomiting response to extinguish.  This summation effect suggests that the two selective factors use the same currency and thus may have the same mechanism.
      The classical conditioning interpretation has usually prevailed for lack of a way that fear could be seen as a reward; but the section on negative emotions following the précis of hyperbolic discounting supplies the missing hypothesis.  The ultimate choice of models may depend more on parsimony than on critical experiments.

3. Social demands in this setting and the lack of repeated experience made the contingencies relatively ineffective.

4. An additional complication is that some emotions are not always positive or negative; there can be an ambivalent valence, or it can change signs in different circumstances.  Even “negative” emotions are cultivated under some circumstances—fear and grief in horror films and tragedies, and anger sometimes even as a personal leitmotif.  Thus Panksepp asks, “is rage intrinsically a positive or a negative emotion?  Most investigators assume that anger is a negative emotional state, but it is easy to envision that consistent winning may make this a positive emotional state (2000, p. 146).”  This protean quality adds evidence that negative emotions contain positive elements.

5. Observed activity in response to both rewards and aversive stimuli has been interpreted as evidence that a center serves an alerting rather than a rewarding function; but since animals will work to obtain stimulation in the same centers, the hypothesis that they serve reward after all, and that aversive stimuli must have a rewarding component to attract attention, seems a better bet.

6. Even negative emotions need an appetite, the factor that is reduced by medications like anxiolytics and mood stabilizers, and the potential for which (“thinness of skin") varies with genetic endowment (e.g. Emde et.al., 1992).  However, the risk that organisms will indulge in ad lib induction of negative emotions is probably small, so that there should have been no selective pressure for these appetites to attenuate with multiple inductions.  It is the positive emotions that seem exquisitely sensitive to repetition.

 

Bibliography

 

Abramsom, P. R. & Inglehart, R., 1995.  Value Change in Global Perspective.  University of Michigan. 

Ainslie, G., 1974.  Impulse control in pigeons.  Journal of the Experimental Analysis of Behavior, 21:  485-489.

Ainslie, G., 1992.  Picoeconomics:  The Strategic Interac­tion of Successive Motivational States within the Person.  Cambridge University Press.

Ainslie, G., 1993.  A picoeconomic rationale for social constructionism.  Behavior and Philosophy,21:  63-75.

Ainslie, G., 1995  A utility-maximizing mechanism for vi­carious reward:  Comments on Julian Simons "Interpersonal alloca­tion continuous with intertemporal allocation" Rationality and Society 7: 393-403.

Ainslie, G., 1999a.  The intuitive explanation of passion­ate mistakes, and why it’s not adequate.  In J. Elster, Ed.,  Addiction: Entries and Exits.  Sage, 209-238.

Ainslie, G., 1999b.  The dangers of willpower: A picoecono­mic understanding of addiction and dissociation.  In J. Elster and O-J. Skog, Eds. Getting Hooked: Rationality and Addiction.  Cambridge U., 65-92.

Ainslie, G., 2001.  Breakdown of Will.  Cam­bridge University.

Ainslie, G. & Engel, B. T., 1974. Alteration of clas­sically conditioned heart rate by operant reinforcement in mon­keys. Journal of Comparative and Physiological Psychology 87: 373-383.

Ainslie, G. & Haendel, V., 1983.  The motives of the will.in E. Gottheil, K. Druley, T. Skodola, H. Waxman (Editors), Etiology Aspects of Alcohol and Drug Abuse, Springfield,Ill.: Charles C. Thomas,  pp. 119-140.

Ainslie, G. & Herrnstein, R., 1981.   Preference reversal and delayed reinforcement. Animal Learning and Behavior, 9:  476-482.

Ainslie, G. & Monterosso, J., 2003a.  Building blocks of self-control: Increased tolerance for delay with bundled rewards.  Journal of the Experimental Analysis of Behavior79, 83-94.

Ainslie, G. & Monterosso, J., 2003b.  Will as intertemporal bargaining: Implications for rationality.  University of Pennsylvania Law Review 151, 825-862.

Ayto, J., 1990.  A Dictionary of Word Origins.  Arcade.

Baier, A., 1991.  A Progress of Sentiments: Reflections on Hume’s Treatise.  Harvard University.

Becker, G., 1976.  The Economic Approach to Human Behavior.  Chicago University.

Becker, G. & Murphy, K., 1988.  A theory of rational addiction.  Journal of Political Economy, 96:  675-700.

Bersh, P., 1980.  Eysenck’s theory of incubation: A critical analysis.  Behaviour Research and Therapy,  18:  11-17.

Bickel, W. K., Odum, A. L., & Madden, G.  J., 1999.  Impulsivity and cigarette smoking: Delay discounting in current, never, and ex-smokers.  Psychopharmacology,  146:  447-454.

Bowler, D. M., 1992.  “Theory of mind” in Asperger’s syndrome.   Journal of Child Psycyology and Psychiatry, 5:  877-893.

Braudel, F., 1981.  The Structures of Everyday Life: The Limits of the Possible  (S. Reynolds, tr.)  Harper and Row.

Brickman, P., Coates, D., & Janoff-Bulman, R., 1978.  Lottery winners and accident victims:  Is happiness relative?  Journal of Personality and Social Psychology, 36:  917-927.

Browns, R. E., 1964.  The New Dictionary of Thoughts.  Standard.

Brunner, D. & Gibbon, J., 1995.  Value of food aggregates: Parallel versus serial discounting.  Animal Behavior, 50:  1627-1634.

Clum, G. A., Clum, G. A., & Surls, R., 1993.  A meta-analysis of treatments for panic disorder.  Journal of Consulting and Clinical Psychology, 61:  317-326.

Cohen, P., & Cohen, J., 1996.  Life Values and Adolescent Mental Health.  Erlbaum.

Coleman, J. S.,  1990.  Foundations of Social Theory,  Belknap.

Cooter, R. & Ulen, T., 2000.  Law and Economics.  Addison-Wesley.

Corsini, R.J., 1984. Current Psychotherapies.  Third Edition.  Peacock.

Delgado, J. M. R., 1969.  Physical Control of the Mind:  Toward a Psychocivilized Society.  Harper & Row.

Deluty, M.Z., Whitehouse, W.G., Mellitz, M., & Hineline, P.N., 1983.  Self-control and commitment involving aversive events.   Behavior Analysis Letters, 3:  213-219.
 
Diener, E., 2000.  Subjective well-being:  The Science of happiness and a proposal for a national index.  American Psychologist, 55:  34-43.

Dinsmoor, J. A., 1998.  Punishment.  In W. T. O’Donohue (Editor), Learning and Behavior Therapy.  Allyn & Bacon.

Donahoe, J. W., Burgos, J. E., & Palmer, D. C., 1993.  A selectionist approach to reinforcement.  Journal of the Experi­mental Analysis of Behavior 60, 17-40.

Donahoe, J. W., Palmer, D. C., & Burgos, J. E., 1997.  The S-R issue: Its status in behavior analysis and in Donahoe and Palmer's Learning and Complex Behavior.  Journal of the Experi­mental Analysis of Behavior 67, 193-211..

Downs, D., 1995.  The Actor's Eye:  Seeing and Being Seen.  Applause Theatre Books.

Easterlin, R., 1995.  Will raising the incomes of all increase the happiness of all?  Journal of Economic Behavior and Organization, 27:  35-47.

Ekman, P. &  Friesen, W. V., 1986.  A new pan-cultural facial expression of emotion.  Motivation and Emotion, 10:  159-168.

Emde, R. N., Plomin, R., Robinson, J., Corley, R., & DeFries, J.,  1992.  Temperament, emotion, and cognition at fourteen months: the MacArthur longitudinal twin study.  Child Development, 63:  1437-1455.

Erdelyi, M. H., 1990.  Repression, reconstruction and defense:  History and integration of the psychoanalytic and experi­mental frameworks in Singer, Jerome L., Ed., Repression and Dissociation:  Implications for Personality Theory, Psychopatholo­gy, and Health.  University of Chicago.

Eysenck, H. J.,  1968.  A theory of the incubation of anxiety/fear response.  Behaviour Research and Therapy,  6:  309-321.

Frank, R.H., 1988.  Passions Within Reason, Norton.

Freud, S., 1915/1956.  Instincts and their vicissictudes in J. Strachey and A. Freud (Editors), The Standard Edition of the Complete Psychological Works of Sigmund Freud (Vol. 14).  Hogarth.

Frijda, N. H., 2000.  The psychologists’ point of view.  In M. Lewis & J. M. Haviland-Jones (Editors), Handbook of Emotions 2d Edition.  Guilford.

Galati, D., Scherer, K. R., & Ricci-Bitti, O. E., 1997.  Voluntary facial expression of emotion: Comparing congenitally blind with normally sighted encoders.  Journal of Personality and Social Psychology, 73:  1363-1379.

Galen, 1963.  (Harkins, P.W., trans.) Galen on the Passions and Errors of the Soul.  Ohio State University.

Glantz, K. & Pearce, J., 1989.  Exiles from Eden:  Psychotherapy­ from an Evolutionary Perspective.  Norton.

Gosselin, P., Kirouac, G., & Dore, F.Y., 1998.  Components and recognition of facial expression in the communication of emotion by actors.  Journal of Personality and Social Psychology, 68:  83-96.

Grace, R., 1996.  Choice between fixed and variable delays to reinforcement in the adjusting-delay procedure and concurrent chains.  Journal of Experimental Psychology: Animal Processes, 22:  362-383.

Green, L., Fisher, E.B., Jr., Perlow, S. and Sherman, L., 1981.  Preference  reversal and self-control:   Choice as a function of reward amount and delay.  Behaviour Analysis Letters1, 43-51.

Green, L., Fry, A., & Myerson, J., 1994.  Dis­counting of delayed rewards: A life-span comparison.  Psychological Science, 5:  33-36.

Heyman, S., Wexner, S. D., Vickers, D., Nogueras, J. J., Weiss, E. G., & Pikarsky, A. J., 1999.  Prospective, randomized trial comparing four biofeedback techniques for patients with constipation.  Diseases of the Colon and Rectum, 42:  1388-1393.

Hollerman, J.R., Tremblay, L., & Schultz, W., 1998.  Influence of reward expectation on behavior-related neu­ronal activity in primate striatum.  Journal of Neurophysiology, 80:  947-963.

Iacoboni, M., Woods, R. P., Brass, M., Bekkering, H., Mazziotta, J. C. & Rizzolatti, G., 1999.  Cortical mechanisms of imitation.  Science,286:  2526-2528.

Izard, C., 1991.  Four systems for emotional activation:  Cognitive and noncognitive processes.  Psychological Review,100:  68-90.

Jolls, C., Sunstein, C. R. & Thaler, R., 1998.  A behavioral approach to law and economics.  Stanford Law Review 50, 1471-1550.

Kasser, T., 2002.  The High Price of Materialism.  MIT.

Kilic, C., Noshirvani, H., Basoglu, M., & Marks, I., 1997.  Agoraphobia and panic disorder: 3.5 years after alprazolam and/or exposure treatment.  Psychotherapy and Psychosomatics, 66:  175-178.

Kirby, K. N., 1997.  Bidding on the future: Evidence against normative discounting of delayed rewards.  Journal of Experimental Psychology: General, 126:  54-70.

Kirby, K. N., & Guastello, B., 2001. Making choices in anticipation of similar future choices can increase self-control.  Journal of Experimental Psychology: Applied, 7:  154-164.

Kirby, K. N. & Herrnstein, R. J., 1995.  Preferen­ce reversals due to myopic discounting of delayed reward.  Psych­ological Science, 6:  83-89.

Korobkin, R. & Ulen, T. S, 2000  Law and behavioral science:  Removing the rationality assumption from law and economics. California Law Review 88, 1051-1144.

Kotchoubey, B., Strehl, U.k Uhlmann, C., Holzapfel, S., Konig, M., Froscher, W. Blankenhorn, V. & Birbauer, N., 2001.  Modification of slow cortical potentials in patients with refractory epilepsy: A controlled outcome study.  Epilepsia, 42:  406-416.

Laibson, D., 1997.  Golden eggs and hyperbolic discounting.  Quarterly Journal of Economics, 62:  443-479.

Loewenstein, G., 1987. Anticipation and the valuation of de­layed consumption. The Economic Journal 97, 666-685. 

Lorenz, K., 1970.  The enmity between generations and its probable ethological causes.  Psychoanalytic Review,57:  333-377.

Lutz, C., 1988.  Unnatural Emotions:  Everyday Sentiments on a Micronesian Atoll and their Challenge to Western Theory.  University of Chicago.

Madden, G.J., Petry, N. M., Badger, G. J., & Bickel, W. K., 1997.  Impulsive and self-control choices in opioid-dependent patients and non-drug-using control patients:  Drug and monetary rewards.  Experimental and Clinical Psychopharmacology,5:  256-262.

Marks, I. & Tobena, A., 1990.  Learning and unlearning fear: A clinical and evolutionary perspective.  Neuroscience and Biobehavioral Reviews, 14:  365-384.

Mazur, J.E., 1986.  Choice between single and multiple delaye­d reinforcers.  Journal of the Experimental Analysis of Behavior, 46:  67-77.

Mazur, J. E., 2001.  Hyperbolic value addition and general models of animal choice.  Psychological Review, 108:  96-112.

Miller, N., 1969. Learning of visceral and glandular re­sponses. Science 163, 434-445.

Mirenowicz, J. & Schultz, W., 1996.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli.  Nature,379:  449-451.

Mitchell, S., 1999.  Measures of impulsivity in cigarette smokers and nonsmokers.  Psychopharmacology,146:  455-464.

Mitchell, S. H. & Rosenthal, A. J., 2003.  Effects of multiple delayed rewards on delay discounting in an adjusting amount procedure.  Behavioural Processes, 64, 273-286.

Monterosso, J. & Ainslie, G., 1999.  Beyond Discounting:  Possible experimental models of impulse control.  Psychopharmacology,146:  339-347.

Monterosso, J. R., Ainslie, G., Toppi Mullen, P., & Gault, B.,  2002,  The fragility of cooperation: A false feedback study of a sequential iterated prisoner's dilemma. Journal of Economic Psychology 23:4, 437-448.

Muchembled, R., 1985.  Popular Culture and Elite Culture in France, 1400-1750. (L. Cochrane, tr.)  Louisiana State University.

Murray, E.J. & Foote, F., 1979.  The origins of fear of snakes.  Behavior Research and Therapy,17:  489-493.

Myers, D.G., 2000.  The funds, friends, and faith of happy people.  American Psychologist,55:  56-67. 

Nakao, M., Normura, S., Shimosawa, T., Yoshiuchi, K., Kumano, H., Kuboki, T., Suematsu, H., & Fujita, T., 1997.  Clinical effects of blood pressure biofeedback treatment on hypertension by auto-shaping.  Psychosomatic Medicine, 59:  331-338.

Novarick, D.J., 1982.  Negative reinforcement and choice in humans. Learning and Motivation, 13:  361-377.

O'Brien, C. P., Ehrman, R. N., & Ternes, J.W., 1986.  Classical conditioning in human dependence.  In Steven R.Goldberg and Ian P. Stolerman, (Editors), Behavioral Analyses of Drug Depend­ence.  Academic, pp. 329-356.

Ortony, A., Clore, G., & Collins, A., 1988.  The Cognitive Structure of Emotions.  Cambridge University.

Panksepp, J.,  2000.  Emotions as natural kinds within the mammalian brain.  In M. Lewis & J. M. Haviland-Jones, eds.  Handbook of Emotions 2d Edition.  Guilford.

Parrott, W.G., 1991.  Mood induction and instructions to sustain moods.  A test of the subject compliance hypothesis of mood congruent memory.  Cognition and Emotion, 3:  41-52.

Parrott, W. G., 1993.  Beyond hedonism: Motives for inhibiting good moods and for maintaining bad moods.  In D. M. Wegner & F. W. Pennebaker  (Editors), Handbook of Mental Control 278-305.  Prentice Hall.

Ramsay, R. W., 1997.  Behavioural approaches to bereavement.  In S. Rachman & H. J. Eysenck (Editors), The Best of Behavior Research and Therapy.  Pergamon.

Rachman, S. J., 1977.  The conditioning theory of fear acquisition: A critical examination.  Behaviour Research and Therapy, 15:  375-388.

Rescorla, R. A., 1988. Pavlovian conditioning:  Its not what you think it is. American Psychologist 43, 151-160.

Sartre, J.-P., 1939/1948.  The Emotions: Sketch of a Theory  (B. Frechtman, trans.)  Philosophical Library.

Schultz, W., Dayan, P., & Montague, P.R., 1997.  A neural substrate of prediction and reward.  Science,275:  1593-1599.

Schachter, S. & Singer, J., 1962.  Cognitive, social, and physiological determinants of emotional state.  Psychological Review, 69:  379-399.

Schwartz, B., 2000.  Self-determination:  The tyranny of freedom.  American Psychologist, 55:  79-88.

Segal, J. M., 1991.  Alternative conceptions of the economic realm.  In R. M. Coughlin (Editor), Morality, Rationality, and Efficiency: New Perspectives on Socio-Economics.  M. E. Sharpe.

Sinclair, R. C., Hoffman, C., & Mark, M. M., Martin, L. L., & Pickering, T. L.,   1994.  Construct accessibility and the misattribution of arousal: Schachter and Singer revisited.  Psychological Science, 5:  15-19.

Smith, E.O., 2002.  When Culture and Biology Collide:  Why We Are Stressed, Depressed, and Self-Obsessed.  Rutgers University.

Solnick, J., Kannenberg, C., Eckerman, D. and Waller, M., 1980.  An experimental analysis of impulsivity and impulse control in humans. Learning and Motivation, 2:  61-77.  Review, 217-225.

Solomon, R. C., 2000.  The philosophy of emotions.  In M. L. Lewis & J. M. Haviland-Jones (Editors), Handbook of Emotions.  Guilford.

Stearns, P.N., 1986.  Historical analysis in the study of emotion.  Motivation and Emotion, 10:  185-193.

Strasberg, L., 1988.  A Dream of Passion:  The Development of the Method.  Dutton.

Sugden, R., 1991.  Rational choice: a survey of contributions from economics and philosophy.  Economic Journal, 101:  751-785.

Sunstein, Cass R., 1995.  Problems with rules.  California Law Review 83, 953-1030.

Thomas, E.M., 1989.  The Harmless People.  Vintage.

Toates, Frederick, 1994.  Comparing motivational systems—An incentive motivation perspective.  In Charles R. Legg & David Booth (Editors), Appetite: Neural and Behavioural Basis.  Oxford.

Vuchinich, R.E. & Simpson, C.A., 1998.  Hyperbolic temporal discounting in social drinkers and problem drinkers.  Experimental and Clinical Psychopharmacology, 6:  292-305.

Watson, J.B., 1924.  Behaviorism.  The Peoples Institute.

Wegner, D. M., 1989.  White Bears and Other Unwanted Thoughts: Suppression, Obsession, and the Psychology of Mental Control. Penguin.

Williams, C., Wright, B, & Partridge, I., 1999.  Attention deficit hyperactivity disorder: a review.  British Journal of General Practice, 49:  563-571.

Wink, P. & Donahue, K., 1997.  The relation between two types of narcissism and boredom.  Journal of Research in Personality, 31:  136-140.

 

1. Both “passive” and “passion” come from the Latin verb for “to suffer,” and took on their implications of separate kinds of undergoing from the 14th to 16th centuries (Ayto, 1990, p. 385)
2. It has been argued in the case of fears that conditioned responses which fail to extinguish are serving as their own unconditioned stimuli (Bersh, 1980; Eysenck, 1968).  However, as with so many elements in the operant/classical debate, this mechanism can be interpreted just as well as an operant one.  If the prospect of fear results in further fear, the mechanism might be either that fear serves as both a conditioned and an unconditioned response, or that fear rewards the operant response of fear (presumably transiently, as in gratifying the “urge to panic”).  Insofar as fear is not an obligatory response, this positive feedback effect must depend on the emotion’s being in some sense rewarding.  In the case of a visceral response that may be learned purposively, like bulimic vomiting, the selective factors are both an expectation of vomiting that makes vomiting more likely, and instrumental reward; the instrumental reward is either added to the former factor to produce learning, or is not added to it, leading the vomiting response to extinguish.  This summation effect suggests that the two selective factors use the same currency and thus may have the same mechanism.
      The classical conditioning interpretation has usually prevailed for lack of a way that fear could be seen as a reward; but the section on negative emotions following the précis of hyperbolic discounting supplies the missing hypothesis.  The ultimate choice of models may depend more on parsimony than on critical experiments.
3. Social demands in this setting and the lack of repeated experience made the contingencies relatively ineffective.
4. An additional complication is that some emotions are not always positive or negative; there can be an ambivalent valence, or it can change signs in different circumstances.  Even “negative” emotions are cultivated under some circumstances—fear and grief in horror films and tragedies, and anger sometimes even as a personal leitmotif.  Thus Panksepp asks, “is rage intrinsically a positive or a negative emotion?  Most investigators assume that anger is a negative emotional state, but it is easy to envision that consistent winning may make this a positive emotional state (2000, p. 146).”  This protean quality adds evidence that negative emotions contain positive elements.
5. Observed activity in response to both rewards and aversive stimuli has been interpreted as evidence that a center serves an alerting rather than a rewarding function; but since animals will work to obtain stimulation in the same centers, the hypothesis that they serve reward after all, and that aversive stimuli must have a rewarding component to attract attention, seems a better bet.
6. Even negative emotions need an appetite, the factor that is reduced by medications like anxiolytics and mood stabilizers, and the potential for which (“thinness of skin") varies with genetic endowment (e.g. Emde et.al., 1992).  However, the risk that organisms will indulge in ad lib induction of negative emotions is probably small, so that there should have been no selective pressure for these appetites to attenuate with multiple inductions.  It is the positive emotions that seem exquisitely sensitive to repetition.