For a PDF version of this article that can be printed or downloaded, click here:   

 

Picoeconomics in Neural
and Evolutionary Contexts

George Ainslie
Veterans Affairs Medical Center, Coatesville PA, USA
University of Cape Town, South Africa
George.Ainslie@va.gov

 

Published in Social Neuroscience and Public Health: Foundations for the Science of Chronic Disease Prevention (Peter A. Hall, editor) pp. 3-18.
Springer, 2013

 

This material is the result of work supported with resources and the use of facilities at the Department of Veterans Affairs Medical Center, Coatesville, PA, USA. The opinions expressed are not those of the Department of Veterans Affairs of the US Government.

 

 

Abstract

Hyperbolic delay discount curves reflect a basic psychophysical principle and are not maladaptive in nonhumans.  However, in people who plan they create conflicts between present motives and expected future motives.  Unlike conflicts between simultaneous motives, these cannot be resolved by simply weighing the alternatives against one another, but instead confront a person with sequential strategic choices.  Such choices are the subject of picoeconomics (micro-micro-economics).  

In recent centuries willpower has become the most approved means of stabilizing intertemporal conflicts, in addition to social commitment.  In willpower a variant of repeated prisoner’s dilemma can be inferred from behavioral experiments and common experience—as clarified by thought experiments—but current neuroimaging techniques cannot visualize the self-interpretations that are hypothesized.  fMRI does suggest that a unified reward network is modulated by prefrontal cortical activity, which is recruited even by the process of choice itself.

 

 Text

Picoeconomics studies the implications for motivational science of nonexponential delay discounting.  Many of these can only be mentioned in this chapter, but all except the most recent publications under the author’s control can be downloaded from www.picoeconomics.org.

Reward is the selective principle of choice, a process that has presumably been selected in turn by evolution to be a proxy for fitness.  However, addictions and other maladaptive behaviors are often strongly rewarded, raising questions about both the unity of the person and the efficiency of natural selection.  This divergence of rewardingness from fitness can be accounted for by the apparently inborn form in which we discount prospective rewards as a function of their delay.  Nevertheless, the survival of this form in evolution makes sense.   Here I will review a rationale developed elsewhere for how this form generates both motivational conflict and somewhat imperfect means of resolving it (Ainslie, 1992, 2001, 2005), and discuss relevant research, particularly the neuroimaging studies that have begun to go beyond the simple anatomy of motivation.  I will also suggest evolutionary and historical frameworks for the conflict of impulse and control.  

Motivational conflict is temporal, not spatial

The analysis of consistency over time and its failure is surprisingly new—newer than that of the functional division of the brain.  Olds had already published work on a reward center (e.g. Olds & Milner, 1954), and Penfield his map of sensory and motor centers (e.g. Penfield & Jasper, 1954) when economist Robert Strotz broke with classical economics to point out that a person’s preferred rate for consuming a good is likely to change over time, even if she has learned nothing new about her options (1956).  Even then the article was largely ignored for two decades (Grüne-Yanoff, unpublished manuscript).  It was a finding in behavioral psychology, Herrnstein’s matching law (1961), that eventually provided a tool for analyzing preference change as a function of delay.  The matching law states that choice on concurrent variable interval schedules of reward is proportional to the immediacy of the rewards as well as to their amount and rate of delivery (Chung & Herrnstein, 1967).  When this proportionality was applied to single discrete choices it predicted that the discounting of delayed rewards would obey a hyperbolic function (Ainslie, 1975), rather than the exponential function that had been explicit in economics (Samuelson, 1937) and implicit in the other behavioral sciences’ assumption that preferences tend to stay consistent in the absence of new information.  Many observations in nonhuman animals (Ainslie & Herrnstein, 1981; Mazur, 1997) and people (Green, Fry, & Myerson, 1994; Kirby & Marakovic, 1995) confirmed this prediction, as well as the related predictions that preference between a smaller, sooner (SS) and larger, later (LL) reward would often shift from LL to SS as delay becomes shorter, and that, with appropriate tools, even pigeons and rats can learn to choose a commitment that prevents them from subsequently acting on the new preference (Ainslie, 1974; Deluty et.al., 1983).  Activity in human cortical reward centers has recently been found to track delay in parallel with the hyperbolic discount curves that describe the subjects’ actual choice behaviors, but the data are still too noisy for a specific hyperbolic function for brain activity itself to be differentiated from an exponential function (Kable & Glimcher, 2007).

Economists exploring dynamic inconsistency of choice soon picked up the model of hyperbolic discounting, but proposed a modified, hyperboloid shape that grafted a steep rise in value as an SS reward becomes closer on top of a standard exponential curve for all other delays (Laibson, 1997).  This modification was prompted by the difficulty of using hyperbolic curves in economic models rather than by data (Angeletos, et.al., 2001, p. 50);  but hyperboloid curves have gained intuitive support from the phenomenon in which some rewards are augmented by emotional arousal or appetite, a property sometimes called viscerality (Loewenstein, 1996), which suggests a mechanism for the steep rise in a reward’s value when it is close.  However, many examples of temporary preferences for SS rewards do not involve arousal, such as simple procrastination (Ainslie, 2010), short-sighted job seeking (Paserman, 2008), and failure to save for retirement.  Also, a hyperbolic shape has been observed where the closer alternative is months or decades away (Cropper, et.al., 1992; Green et. al., 2005).  Nevertheless, the hyperboloid variant is widely accepted, especially in economics.  Because most of the difference between exponential and hyperbolic curves is observed in the period just before the SS reward is due, hyperboloid curves plotted as the sum of a very steep and a shallower exponential curve can fit experimental data as closely as pure hyperbolic curves (e.g. McClure et.al., 2007), although the hyperboloid curve requires two parameters while the hyperbolic curve needs only one.  Hyperbolic and hyperboloid shapes each can account for a person’s inconsistent preference over time, as well as for an incentive for her to commit herself in advance to wait for an LL alternative.  However, a pure hyperbolic shape is arguably necessary to motivate people’s progression from the discount function we are born with to rational adult patience.  

 

Internal self-control requires intertemporal bargaining

Hyperbolic discounting of prospective reward divides a person into competing interests, not based on competing reward centers but on changing command of a unitary reward network over time.  Interests based on delayed rewards will be weaker than interests based on imminent rewards, but they have the advantage of foresight.  If such an interest can motivate precommitment of choice or keep a future self from coming too close to an SS reward, it will get the LL reward on which it is based.  If it fails, the SS reward will become stronger and have the last word.  Extrapsychic precommitments include medicines that change appetite, contracts, illiquid investments (Laibson, 1997), and especially social environments.  Intrapsychic commitment in advance is also possible, but to a limited extent:  A dominant interest can restrict attention or inhibit specified responses for a period of time, but can remain vigilant only so long against the weighing of alternatives.  Like price controls in an otherwise free market, restricted attention builds up contrary motives. Response inhibition can be seen in experiments where subjects have to resist an urge, for instance saying the color names instead of the print colors in a Stroop task or waiting for a signal in a go/no-go task.  The many studies  of this kind of task have found it to be associated with activity in the dorsolateral and ventrolateral prefrontal cortices and anterior cingulate gyrus (e.g. Chambers et.al., 2009), but it is almost certainly not the process that stabilizes intentions over long periods of time (Monterosso et.al., 2010).  People can also learn what trains of thought lead to the appetite for an impulse—for instance, the Catholic church’s “venial sins” (Holton, 2009)—and derail them before they become too attractive; but again this method requires forestalling the impulse in advance.

A need for commitment in advance implies a fragility of internal self-control, sometimes called weakness of will.  Willpower does more than commit against temptations.  With willpower a person tests herself against temptations while “both alternatives are steadily held in view” (James, 1890, p. 534), and feels an emotional loss—guilt—if she fails.  A mechanism that does not involve separate motivational faculties has been elusive:  What self can be said to control what other self?  However, a rationale can be derived from the high the tails of hyperbolic discount curves that depict the value of LL alternatives at relatively long delays.  The tails of hyperbolic curves are much higher than those of exponential curves—and of hyperboloid curves, which, by definition, are the same as exponential curves when the rewards are not imminent.  The difference is especially pronounced where a person interprets the value of a current choice to include the value of a bundle of similar choices that she expects to make in the future.  To illustrate the difference in the values of bundled rewards, figure 1 shows series of four rewards, discounted exponentially versus hyperbolically, at rates adjusted to make the value of a reward of amount 10 worth 1 at ten units of delay. 

 

Even added together, the exponentially discounted value of the bundle soon falls to a tiny fraction as delays get longer, just as that of single reward does (e.g. the last reward in the series if the curve were not augmented by the others).  By contrast, the hyperbolically discounted value remains relatively high, falling more and more slowly as delays get longer.

 

 

I have argued that this bundling effect is what lets people learn to follow the rational norm for exponential discounting, as long as the consequent present deprivation is not too great (Ainslie, 1991):  In a choice between an SS and LL reward, if she notices that her current choice is a good predictor of how she will make similar choices in the future, her expectation of that whole bundle of future rewards will come to depend the meaning she finds in her current choice.  That is, to the extent that she interprets her current choice as a test case for a bundle of later rewards, the discounted values of the whole bundle will depend on, and therefore contribute to, her choice (figure 2; discussed further in Ainslie, 2012).  This hypothesis has two parts: that choosing a bundle of rewards all at once will increase the value of the LL options; and that a person’s interpreting her current choice as a test case will have the effect of creating such a bundle, much as a player’s current move in a repeated prisoner’s dilemma is based on her expectation of how that move will affect the whole string of her partner’s future moves. 

 

 

There is evidence that the current discounted values of future rewards are additive (Kirby, 2006; Mazur, 1986), and there are experiments showing that choosing a series of rewards all at once increases preference for the LL alternatives over what it is when subjects choose between the same pairs one at a time, both in people (Kirby & Guastello, 2002: Hofmeyr et.al., 2010) and in rats (Ainslie & Monterosso, 2003).  The rat experiment is especially valuable in showing that increased patience for bundled rewards is not an artifact of culture or experimenter suggestion, but presumably based on the raw rewarding effect depicted by the discount curves.  

However, the second part of the hypothesis is harder to test.  The person’s weighing of alternatives is proposed to be recursive, so if she chooses against the current alternative she reduces her expectation of subsequently choosing LL rewards in similar situations, which may make choice of the current LL alternative relatively more attractive; but this will be true if and only if she expects choosing the SS alternative to reduce her likelihood of getting later LL rewards, and expects choosing the LL alternative to increase this likelihood.  In exploring the problem she may think of a rationale whereby the current choice is exceptional, and therefore not a test of future prospects; or she may have such a bad record of giving in to temptation that one LL choice will not create much hope for future choices.  The logic of this intertemporal bargaining is much like that of the repeated prisoners’ dilemma that defines self-enforcing contracts between individuals (Telser, 1980)—the deterrent to defection being not revenge but the loss of expected cooperation in future transactions.  Experiments on this internal dialogue are hard to design because they represent exceptional cases by their very nature.  Nevertheless, there was a finding that suggests recursive self-prediction in each of the two human experiments just cited:  Telling a group of subjects who chose between an SS and LL reward every week that their future free choices were apt to be the same as their current choice led to more LL choices than in a control group, though not as much as in a group that had to make all their choices at once in the first week.  The phenomenon is better demonstrated by means of a loan from the philosophy of mind, the thought experiment (discussed in Ainslie, 2007).  Simplest is Monterosso’s problem: 
Consider a smoker who is trying to quit, but who craves a cigarette. Suppose that an angel whispers to her that, regardless of whether or not she smokes the desired cigarette, she is destined to smoke a pack a day from tomorrow on. Given this certainty, she would have no incentive to turn down the cigarette— the effort would seem pointless.  What if the angel whispers instead that she is destined never to smoke again after today, regardless of her current choice?  Here, too, there seems to be little incentive to turn down the cigarette—it would be harmless.  Fixing future smoking choices in either direction (or anywhere in between) evidently makes smoking the dominant current choice. Only if future smoking is in doubt does a current abstention seem worth the effort. But the importance of her current choice cannot come from any physical consequences for future choices; hence the conclusion that it matters as a precedent. (Monterosso & Ainslie, 1999) 

It does not matter that the negative effects of some habits, such as smoking, do not come repeatedly and soon after the positive ones, hangover fashion, but only in the far future (as Rick & Loewenstein have objected, 2008).  The prospect of future health still forms a stake that is at risk in every choice that the person sees as evidence of her pattern of future choices.

In other contexts feedback from self-testing is a familiar experience.  Visceral processes such as anger, panic, nausea, sleep (in insomniacs), and urination (in men with prostatic hypertrophy) are promoted by signs that they are already happening, a phenomenon first described by Darwin, James, and Lange but mistakenly held to be the origin of these processes (Rolls, 2005, pp. 26-28).  The importance of self-testing in willpower may not be evident when the stakes are low, as in resolving to clean up your office; but it becomes clear when large amounts of incentive hinge on the test, as when a recovering alcoholic decides whether to try drinking just once.  The latter case follows the same logic as the decision of a party to a self-enforcing contract to cheat her partner; such a defection by the current self leads to the notorious abstinence violation effect (Marlatt & Gordon, 1980; for dieters, see Polivy & Herman, 1985).  Furthermore, where an appetite-based consumption is restrained by willpower, upticks in the person’s appetite may cause reductions in her certainty of control that induce further appetite.  Such a vicious circle may produce the sudden cravings that are often implicated in relapses, which have been imperfectly explained by the conventional theory, classical conditioning (discussed in Ainslie, 2010). 

Where neuroimaging might detect intertemporal bargaining

Response inhibition tasks such as the Stroop are easy to study with fMRI, but the imaging of valuation-based self-control is harder, and is less than a decade old.  The first fMRI study of SS vs. LL choice appeared to show that delayed rewards were evaluated only in frontal cortical sites, not the limbic ones that responded to immediate rewards, a finding that might be interpreted as showing separate reward centers (McClure et.al., 2004, 2007).  However, other SS/LL studies have shown that all reward-sensitive sites in humans discount delay of reward equally (Kable & Glimcher, 2007).  The activity in sites associated with human self-control, particularly the dorsolateral prefrontal cortex (PFC) and posterior insula, seem to modulate rather than compete with comprehensive reward valuation centers such as the ventromedial PFC and ventral striatum (Hare et.al., 2009; Monterosso & Luo, 2010).  The imaging of the modulation process is far from providing clear mechanisms, but some suggestive studies have been done.

First of all, the process of weighing alternatives per se has been found to alter their value, in a way that favors LL rewards:  When subjects anticipate individual SS and LL rewards for which they both previously and subsequently express equal preference, activity in brain reward centers is less when they expect the LL reward than when they expect the SS reward (Luo et.al., 2009).  Similarly, disruption of left lateral PFC function with transcranial magnetic stimulation decreases choice of  LL rewards that had previously been equally preferred to SS alternatives-- without changing subjects’ reported valuations of the rewards when considered singly (Figner et.al., 2010).  These findings imply that the process of intertemporal choice itself augments the relative value of LL rewards. 

There have begun to be reports of relationships among centers that are specifically associated with LL choice.  Combined valuation of food and prospective health in the ventromedial PFC is modulated by activity in the dorsolateral PFC when subjects are exercising self-control (Hare et.al., 2009).   The oft-noted increase in patience from adolescence to mid-adulthood is accompanied by a greater connectivity of the ventromedial PFC with the dorsolateral PFC and parietal and insular cortices during LL choices (Christakou et.al, 2011).  When subjects try not to be tempted by cigarettes or food, increases in lateral PFC activity and decreases in reward center activity are correlated with reported decreases in craving, an effect fully modulated by one of the reward areas (ventral striatum—Kober et.al., 2010).  A further study found that when subjects have to repeatedly reject stimuli previously conditioned to SS rewards in order to get an LL reward, activity in a region of the anterior PFC varies inversely with activity in reward centers, to a greater degree the more successfully a subject resists the lure (Diekhof & Gruber, 2010); however, interpretation of this last finding is complicated by its resemblance to go-nogo tasks.  Finally, subjects who show more spontaneous alternations of preference between an equally preferred pair of SS and LL rewards have more activity in another region that is often observed to be active in self-control (left insula/inferior frontal gyrus) when making LL choices, suggesting that inconsistency may elicit more executive function (Luo et.al., 2011).

Reports that executive functions in frontal centers modify activity in valuation centers have led to the proposal that there is a third possibility beyond single-valuation and dual-valuation hypotheses, “self-control” (Figner et.al., 2010).  However, even though lateral PFC activity does not track activity in the ventromedial PFC and other reward centers, it must still depend on the common currency of reward.  Some competitive process must still weigh, for instance, whether it is worth the risk to try a single cigarette after a month without smoking, or whether gratifying an angry impulse is worth the harm it would do to your self-image.  Executive functions must still compete in the marketplace.  Abstract and long term value must arise somewhere, and be weighed against the value from more tangible sources.  This somewhere might even be the same ventromedial PFC that has been seen to weigh tangible rewards, perhaps in a continuous pattern that we cannot presently detect with our episodically based experimental designs.  The ventromedial PFC is part of a set of wide, overlapping networks that subtend autobiographical memory, vicarious experience, future projection, and undirected thought (Spreng et.al., 2009)—in short, imagination.  It has been reported to modify other rewards (Peters & Büchel, 2010), but might well be capable of generating reward in its own right, constrained only by the tendency of self-generated reward to habituate (Ainslie, in press).  Whatever the source, more patient choice has been found to be correlated with activity in the ventromedial PFC when subjects imagine future events (Mitchell et.al., 2011).  Similarly, presenting subjects with words naming their own expected future events during an intertemporal choice task causes more patient choice, accompanied by activity in the ventromedial PFC and anterior cingulate gyrus (an “episodic imagery network”) and increased coupling between this gyrus and the hippocampus (Peters & Büchel, 2010).  These findings are tantalizing, but the motivational contingencies that induce and constrain the modulating activity of imagination cannot themselves be seen.  As long as fMRI can take only snapshots, not movies, direct observation of internal dialog such as recursive self-prediction will not be practical, even if good markers for semantic content (e.g. “this choice is a test case”) can be found.  Meanwhile, the interaction of a person’s alternative prospects might be partially modeled by the fMRI of interacting pairs of subjects—so-called “second person neuroscience” (Schilbach et.al., in press)--by analogy to modeling intertemporal prisoner’s dilemmas with interpersonal ones (Monterosso et.al., 2002).

Evolutionary and more recent history of self-control

Both the steepness and the curvature of our inborn discount curves look maladaptive.  They have been implicated in such problems as the named addictions (e.g. Bickel & Marsch, 2001) and some less obvious ones such as short term preferences for overeating, procrastination, passive entertainment, and social disengagement.   The question immediately arises of how they could have survived natural selection, but an answer is not hard to find.  By the time humans evolved, the basic math of perception was long established.  Differences in elementary psychophysical quantities—brightness, weight, loudness-- are experienced proportionately to an index amount, that is, hyperbolically, a phenomenon known as the Weber-Fechner law (Gibbon, 1977).  For instance, we perceive a change in brightness proportionately to the starting level of the brightness.  If delay or some dimension incorporating delay were experienced the same way it would not have caused a problem for nonhuman species, in which long term interests are protected not by planning but by instinctual incentives to hoard, mate, migrate, and so forth, gratification of which pays off immediately (see Ainslie, 1992, pp. 85-88).  Reward does not imply adaptiveness; it is only an evolved proxy for adaptiveness, and may be slow to itself adapt to changed contingencies of natural selection.  Hyperbolically discounted reward motivates adaptive long term choices perfectly well when these pay off immediately, in the gratification of instinctive urges.  Where rewardingness diverged from adaptiveness was in the radical increase of intelligence that let humans steal pleasure from evolved instincts, and for the first time subjected our welfare to our hyperbolic discounting of future prospects. 

Addictions are just conspicuous examples of a widespread phenomenon, capture by short term rewards, which evolution and even cultural selection have not had time to counteract.  Growth of biological immunity to specific addictions is certainly possible.  For instance, the prolonged aldehyde dehydrogenase metabolic phase that makes alcohol aversive to many east Asians (Agarwal & Goedde, 1989) could, over millennia, spread and become general.  Cultural responses can be faster, and arguably have adapted European behavior toward alcohol over the centuries, in contrast to the devastation wrought by its sudden introduction to native American cultures.  But the cheap, concentrated substances that cause high addiction rates—distilled grain alcohol, synthetic opiates, purified cocaine, amphetamines-- date back no further than the seventeenth century (Austin, 1978), and new, fast-paying activities continue to be introduced without our having any idea of their addictive potential (but see King et.al., 2011).  Modern culture has been slow even to learn about the addiction-prone aspect of human nature, much less to evaluate new hedonic inventions for how they might be exploiting it.
At the genetic level an evolutionary response to impulsiveness might be seen in compensatory processes such as the larger prefrontal cortices, which seem to be crucial for the process of self-control, in Homo sapiens than in Homo heidelbergensis (DuBreuil, 2009).  Similarly, delay discounting rate has been reported to be inversely proportional to lateral frontal cortex volume (Bjork et.al., 2009).  However, even increased self-control may fail to increase fitness in the Darwinian sense because it fosters long term reward-maximizing solutions that do not prioritize the increase of gene copies-- for instance with the choice to use birth control in societies where most offspring survive, and, to a lesser degree, adoption of measures to prolong life into old age.  When impulsive behaviors evade control, evolutionary fitness may sometimes increase.  To that extent society will have to deal with the consequences of hyperbolic discounting culturally, without net assistance from natural selection.
Historically, the popularity of willpower as a means of impulse control has been associated with the growth of individualism in western society.  As late as the sixteenth century most decision-making was a social process, in which individual interests were overshadowed by those of the family and clan (e.g. Stone, 1977).  Reliance on social influence for self-restraint is still widespread, and is correlated with personality, gender, and other factors (Gilligan, 1977; Smith et.al., 1997), but in a cosmopolitan society this extrapsychic device has three notable weaknesses: It leaves the person open to exploitation by others, it does not affect concealable impulses, and it is useless when the person’s group as a whole tolerates an impulse.  In the sixteenth and seventeenth centuries increasing attention to the individual conscience—the subject of most early diaries (Carroll, 1981; Shea, 1968)—went along with the theology of predestination, in which a person’s whole expectation of salvation was staked on her every choice (Weber, 1904/1958).  Minus the divine mediation, this process is simply an extreme example of the recursive self-prediction that recruits willpower—making each choice a test case for your expectation of a bundle of later rewards.  This nonlinear process makes a person’s choices unpredictable in principle from a knowledge of the incentives she starts with, a serious flaw from the viewpoint of economic analysis (discussed in Ainslie, 2012), but a solution to the old philosophical conundrum of free will, which demanded that a choice be either uncaused or caused linearly by prior conditions (discussed in Ainslie, 2011).  Willpower itself, however, is no more an ideal correction to hyperbolic discounting than social control is.  It makes lapses hard to recover from, creates an incentive to limit awareness of one’s own choice-making, and is apt to make a person compulsive (discussed in Ainslie, 2001, pp. 143-160).  The development of solutions to the problematic interaction of technical skill with hyperbolic delay discounting may be said to have only started.

Conclusions

With an intact nervous system the simultaneous conflict of motives is not likely to elicit self-control, only a comparison of values.  The incentive for self-control is the prospect that this comparison will come out differently in subsequent choices. Short term committing devices such as response inhibition have clear fMRI correlates, but correlates of the intertemporal bargaining implied by willpower have barely begun to be explored.  Social influence is the other major impulse-controlling factor, but this, too, struggles to keep up with an environment that has moved far beyond the one in which our motivational faculties evolved.  Since the way to maximal long term reward seems to lie in balancing imperfect strategies, the best societal response would seem to be the study and teaching of their counterintuitive motivational bases.

 

Highlights

• Human motivational conflict is best analyzed in the relationship between present and expected future selves, rather than between separate motivational centers.

• People have inherited a delay discount curve that is probably a pure hyperbola, making us prone to addictions and impulsive behaviors.

• The motivational force of willpower comes from seeing a current choice as a test case that predicts future choices in similar cases (recursive self-prediction).

• Current neuroimaging techniques can reveal the interaction of motivational centers in self-control, but not their semantic content, such as the hypothesized recursive self-prediction.

• Hyperbolic discount curves have survived in evolution because they have a deeply rooted psychophysical form, and are harmless in species whose future planning is instinctive.

• There is no dimension of impulse control that is best maximized, since the major available strategies, social pressure and willpower, both have serious limitations.

 

Acknowledgments

I am grateful to John Monterosso and Shan Luo for comments and suggestions.

References

Agarwal, Dharam P. and Goedde, H. Werner  (1989)  Human aldehyde dehydrogenases: Their role in alcoholism. Alcohol 6, 517-523.

Ainslie, George (1974) Impulse control in pigeons. Journal of the Experimental Analysis of Behavior 21, 485-489.

Ainslie, George (1975) Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin 82, 463-496.

Ainslie, George  (1991)  Derivation of "rational" economic behavior from hyperbolic discount curves.  American Economic Review 81, 334-340.

Ainslie, George  (1992)  Picoeconomics:  The Strategic Interac­tion of Successive Motivational States within the Person.  Cam­bridge: Cambridge U.

 Ainslie, George  (2001)  Breakdown of Will.  New York, Cam­bridge U.

Ainslie, George (2005) Précis of Breakdown of Will. Behavioral and Brain Sciences 28(5), 635-673.

Ainslie, George  (2006)  Motivation Must Be Momentary.  In J. Elster, O. Gjelsvik, A. Hylland and Moene, K, eds., Understanding Choice, Explaining Behaviour: Essays in Honour of Ole-Jorgen Skog.  Oslo: Unipub Forlag, pp. 11-28.

Ainslie, George (2007)  Can thought experiments prove anything about the will?   In D. Spurrett, D. Ross, H. Kincaid and L. Stephens, Eds., Distributed Cognition and the Will: Individual Volition and Social Context, pp. 169-196. MIT.

Ainslie, George  (2010)  Procrastination, the basic impulse.  In Andreou, Chrisoula and White, Mark, eds., The Thief of Time: Philosophical Essays on Procrastination. New York, Oxford, pp. 11-27.

Ainslie, George  (2011)  Free will as recursive self-prediction:  Does a deterministic mechanism reduce responsibility?  In Jeffrey Poland and George Graham (eds.)  Addiction and Responsibility. MIT Press, pp. 55 - 87.

Ainslie, George  (2012)  Pure hyperbolic discount curves predict “eyes open” self-control.  Theory and Decision 73, 3-34.  10.1007/s11238-011-9272-5

Ainslie, George  (in press)  Grasping the impalpable: The role of endogenous reward in process addictions.  Inquiry.

Ainslie, George and Haendel, V. (1983) The motives of the will.  In E. Gottheil, K. Druley, T. Skodola, H. Waxman (eds.),Etiology Aspects of Alcohol and Drug Abuse, Springfield,Ill.: Charles C. Thomas, pp. 119-140.

Ainslie, George and Monterosso, John  (2003)  Building blocks of self-control: Increased tolerance for delay with bundled rewards.  Journal of the Experimental Analysis of Behavior 79, 83-94.

Austin, Gregory A. (1978) Perspectives on the History of Psychoactive Substance Abuse.  Rockville, MD: National Institute on Drug Abuse.

Angeletos, George-Marios, Laibson, David, Repetto, Andrea, Tobacman, Jeremy, and Weinberg, Stephen  (2001)  The hyperbolic consumption model: Calibration, simulation, and empirical evaluation.  Journal of Economic Perspectives 15, 47-68.

Atance, C. M. and O’Neill, D. K.  (2001)  Planning in 3-year-olds: A reflection of the future self?  In The Self in Time: Developmental Perspectives C. Moore and K. Lemmon, eds.,  Erlbaum.

Bechara, Antoine  (2004)  The role of emotion in decision-making: Evidence from neurological patients with orbitofrontal damage.  Brain and Cognition 55, 30-40.

Berridge, Kent C.  (2003)  Pleasures of the brain.  Brain and Cognition 52, 106-128.

Berridge, Kent  (2009)  Wanting and liking: Observations from the neuroscience and psychology laboratory.  Inquiry 52, 378-398.

Bickel, Warren K. and Marsch, Lisa A.  (2001)  Toward a behavioral economic understanding of drug dependence: delay discounting processes.  Addiction 96, 73-86.

Bjork, J. M., Momenan, R., and Hommer, D. W.  (2009)  Delay discounting correlates with proportional lateral frontal cortex volumes.  Biological Psychiatry 65, 710-713.

Calder, A. J. (2003). Disgust discussed. Annals of Neurology 53: 427-428.

Carroll, John  (1981)  The role of guilt in the formation of modern society: England, 1350 – 1800.  British Journal of Sociology 32, 459-503.

Carter, R. McKell, Meyer, Justin R., and Huettel, Scott A.  (2010)  Functional neuroimaging of intertemporal choice models: A review.  Journal of Neuroscience, Psychology, and Economics 27-48.

Chambers, Chistropher D., Garavan, Hugh, and Bellgrove, Mark A.  (2009)  Insights into the neural basis of response inhibition from cognitive and clinical neuroscience.  Neuroscience & Biobehavioral Reviews 33, 631-646.

Christakou, Anastasia, Brammer, Mick and Rubia, Katya  (2011)  Maturation of limbic corticostriatal activation and connectivity associated with developmental changes in temporal discounting.  Neuroimage 54, 1344-1354.

Chung, S. and Herrnstein, R. J. (1967) Choice and delay of rein­forcement. Journal of the Experimental Analysis ofBehavior 10, 67-74.

Cropper, Maureen L., Aydede, Sema K., and Portney, Paul R.  (1992) Rates of time preference for saving lives.  American Economic Review 82, 469-472.

Diekhof, E. K. and Gruber, O.  (2010)  When desire collides with reason: Functional interactions between anteroventral prefrontal cortex and nucleus accumbens underlie the human ability to resist impulsive desires.   The Journal of Neuroscience 30, 1488-1493.

DuBreuil, Benoit,  (2009) Paleolithic public goods games: The evolution of brain and cooperation in Mid-Pleistocene hominins.  http://african.cyberlogic.net/bdubreuil/pdf/PPGG.pdf.

Ekman, P. (1999). Basic emotions. In T. Dalgleish & M. J. Powers (Eds.), Handbook of emotion and cognition. New York: John Wiley and Sons LTD.

Figner, Bernd, Knoch, Daria, Johnson, Eric J., Krosch, Amy R., Lisanby, Sarah H., Fehr, Ernst, and Weber, Elke U.  (2010)  Lateral prefrontal cortex and self-control in intertemporal choice.  Nature Neuroscience 13, 538-539.

Frederick, Shane, Loewenstein, George, and O’Donoghue, Theodore  (2002)  Time discounting and time preference: A critical review.  Journal of Economic Literature 40, 351-401.

Freud, S. (1920/1956)  Beyond the pleasure principle. in J. Strachey and A. Freud (Eds.), The Standard Edition of the Complete Psychological Works of Sigmund Freud. vol. 18.  Hogarth.

Gibbon,  John.  (1977) Scalar expectancy theory and Weber’s law in animal timing.  Psychological Review 84, 279-325.

Gilligan, Carol  (1977)  "In a different voice: Women’s concep­tions of self and morality."  Harvard Educational Review, 47, 481-517.

Glimcher, Paul. W.  (2009)  Choice: towards a standard back-pocket model.  In Paul W. Glimcher, Colin Camerer, Russell Alan Poldrack, and Ernst Fehr, eds., Neuroeconomics: Decision making and the brain. Elsevier, pp. 503 – 521.

Green, Leonard, Fry, Astrid, and Myerson, Joel  (1994)  Dis­counting of delayed rewards: A life-span comparison. Psychological Science 5, 33-36.

Green, L., Myerson, J., & Macaux, E. W. (2005). Temporal discounting when the choice is between two delayed rewards. Journal of Experimental Psychology: Learning, Memory, & Cognition 31, 1121-1133.

Grüne-Yanoff, Till  (unpublished manuscript) The shape of temporal discounting: Interdisciplinary exchanges between economics and psychology.  Royal Institute of Technology, 2012.

Hare, Todd A., Camerer, Colin F., and Rangel, Antonio  (2009)  Self-control in decision-making involves modulation of the vmPFC valuation system.  Science 324, 646-648.

Herrnstein, Richard  (1961) Relative and absolute strengths of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior 4,267-272.

Hofmeyr, André, Ainslie, George, Charlton, Richard and Ross, Don  (2010)  The relationship between addiction and reward bundling: An experiment comparing smokers and non-smokers.  Addiction 106, 402-409.

Holton, Richard (2009)  Determinism, self-efficacy, and the phenomenology of free will.  Inquiry, 52, 412-428.

James, W. (1890) Principles of Psychology. Holt.

Kable, Joseph W. and Glimcher, Paul W.  (2007)  The neural correlates of subjective value during intertemporal choice.  Nature Neuroscience 10, 1625-1633.

King, D. L., Delfabbro, P. H., and Griffiths, M. D.  (2011)  The role of structural characteristics in problematic video game play: An empirical study.  International Journal of Mental Health and Addiction 9, 320-333.

Kirby, Kris N.  (1997)  Bidding on the future: Evidence against normative discounting of delayed rewards.  Journal of Experimental Psychology: General 126, 54-70.

Kirby, Kris N.  (2006)  The present values of delayed rewards are approximately additive.  Behavioural Processes 72, 273-282.

Kirby, Kris N., and Guastello, Barbarose  (2001)  Making choices in anticipation of similar future choices can increase self-control.  Journal of Experimental Psychology: Applied  7, 154-164.

Kirby, Kris N. and Marakovic, Nino, N.  (1995)  Modeling myopic decisions: Evidence for hyperbolic delay-discounting within subjects and amounts.  Organizational Behavior and Human Decision Processes 64, 22-30.

Kober, Hedy, Mende-Siedlecki, Peter, Kross, Ethan F., Weber, Jochen, Mischel, Walter, Hart, Carl L. and Ochsner, Kevin N.  (2010)  Prefrontal-striatal pathway underlies cognitive regulation of craving.  Proceedings of the National Academy of Scineces 107, 14811-14816.

Kuhl, Julius  (1994)  Motivation and volition.  In G. d'Yde­walle, Bertelson, and Eelen, Eds., International Perspectives on Psychological Science vol.2.  Hillsdale, NJ Erlbaum, 311-340.

Laibson, David  (1997)  Golden eggs and hyperbolic discounting.  Quarterly Journal of Economics, 62, 443-479.

Loewenstein, George  (1996)  Out of control:  Visceral influences on behavior.  Organizational Behavior and Human Decision Processes 35, 272-292.

Luo, S., Giragosian, L., Ainslie, G., Monterosso, J.  (2009)  Behavioral and neural evidence of incentive bias for immediate rewards relative to preference-matched delayed rewards.  Journal of Neuroscience, 29(47):14820-14827. PMCID: PMC2821568

Luo, Shan, Ainslie, George, Pollini, Drusus, Giragosian, Lisa, and Monterosso, John R.  (2011)  Moderators of the association between brain activation and farsighted choice.  Neuroimage. doi: 10.1016/j.neuroimage.2011.08.004

Marlatt, G. Allen and Gordon, Judith R. (1980) Determinants of relapse: Implications for the maintenance of behavior change, in Park O. Davidson and Sheena M. Davidson (eds.),Behavioral Medicine: Changing Health Lifestyles. Pergamon, pp. 410-452.

Mazur, J.E. (1986) Choice between single and multiple delaye­d reinforcers. Journal of the Experimental Analysis of Behavior 46, 67-77.

Mazur, James E.  (1997)  Choice, delay, probability, and condi­tioned reinforcement.  Animal Learning and Behavior 25, 131-147.

Mazur, James E., & Biondi, Dawn R. (2009). Delay-amount tradeoffs in choices by pigeons and rates: Hyperbolic versus exponential discounting. Journal of the Experimental Analysis of Behavior, 91(2), 197-211.

McClure, Samuel M., Laibson, David I., Loewenstein, George, and Cohen, Jonathan D.   (2004)  The grasshopper and the ant: Separate neural systems value immediate and delayed monetary rewards.  Science 306, 503-507.

McClure, Samuel  M., Ericson, Keith M., Laibson, David I., Loewenstein, George, and Cohen, Jonathan D.  (2007)  Time discounting for primary rewards.  The Journal of Neuroscience 27, 5796-5804.

Mitchell, Jason P., Schirmer, Jessica, Ames, Daniel L., and Gilbert, Daniel T.  (2011)  Medial prefrontal cortex predicts intertemporal choice.  Jounal of Cognitive Neuroscience 23, 857-866.

Monterosso, John and Ainslie, George  (1999)  Beyond Discounting: Possible experimental models of impulse control. Psychopharmacology 146, 339-347.

Monterosso, John Robert, Ainslie, George, Toppi- Mullen, Pamela, and Gault, Barbara  (2002)  The fragility of cooperation: A false feedback study of a sequential iterated prisoner's dilemma. Journal of Economic Psychology 23:4, 437-448.

Monterosso, John R. and Luo, Shan  (2010)  An argument against dual valuation system competition:  Cognitive capacities supporting future orientation mediate rather than compete with visceral motivations.  Journal of Neuroscience, Psychology, and Economics 3, 1-14.

Monterosso, John, Mann, Traci, Ward, Andrew, Ainslie, George, Bramen, Jennifer, Brody, Arthur, and London, Edythe D. (2010)  Neural recruitment during self-control of smoking: A pilot fMRI study.  In Don Ross, Harold Kincaid, David Spurrett, and Peter Collins, eds., What is Addiction, pp. 269-289.

Mulcahy, N. J. and Call, J.  (2006)  Apes save tools for future use.  Science 312, 1038-1040.

Olds, J. and Milner, P.  (1954)  Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain.  Journal of Comparative and Physiological Psychology 47, 419-427.

Paserman, M. Daniele  (2008)  Job search and hyperbolic discounting: Structural estimation and policy evaluation.  The Economic Journal 118, 1418-1452.

Penfield, Wilder and Jasper, Herbert  (1954)  Epilepsy and the Functional Anatomy of the Human Brain.  Oxford, England: Little, Brown & Co.

Platt, Michael L. and Padoa-Schioppa, Camillo  (2009)  Neuronal representations of value. In Paul W. Glimcher, Colin Camerer, Russell Alan Poldrack, and Ernst Fehr, eds., Neuroeconomics: Decision making and the brain. Elsevier, pp. 441 – 462.

Polivy, J. and Herman, C.P. (1985) Dieting and binging: a causal analysis. American Psychologist 40, 193-201.

Rick, Scott and Loewenstein, George  (2008)  Intangibility in intertemporal choice.   Philosophical Transactions of the Royal Society B 363, 3813-3824.

Rolls, Edmund T.  (2005)  Emotion Explained. Oxford U.

Ross, Don  (2010)  Economic models of pathological gambling.  In Don Ross, Harold Kincaid, David Spurrett, and Peter Collins, eds., What is Addiction, pp. 131-158.

Ryan, Richard M., Kuhl, Julius, and Deci, Edward L.  (1997)  Nature and autonomy: An organizational view of social and neuro­biological aspects of self-regulation in behavior and develop­ment.  Development and Psychopathology 9, 701-728.

Samuelson, P.A. (1937) A note on measurement of utility. Review of Economic Studies 4, 155-161.

Schilbach, Leonhard, Timmermans, Bert, Reddy, Vasudevi, Costall, Alan, Bente, Gary, Schlicht, Tobias, and Vogeley, Kai  (in press)  Toward a second-person neuroscience.  Behavioral and Brain Sciences.

Shea, Daniel B., Jr.  (1968)  Spiritual Autobiography in Early America.  Princeton.

Smith, Peter B., Dugan, Shaun, and Trompenaars, Fons  (1997)  Locus of control and affectivity by gender and occupational status: A 14 nation study.  Sex Roles 36, 51-77.

Sperry, Roger W.  (1984)  Consciousness, personal identity and the divided brain.  Neuropsychologia 22, 661-673.

Spreng, R. Nathan, Mar, Raymond A., and Kim, Alice S. N.  (2009)  A common neural basis of autobiographical memory, prospection, navigation, theory of mind, and the default mode: A quantitative meta-analysis.  Journal of Cognitive Neuroscience 21 (3), 489-510.

Stone, Lawrence  (1977)  The Family, Sex, and Marriage: England, 1500-1800.  New York: Harper.

Strotz, R.H. (1956) Myopia and inconsistency in dynamic utility maximization. Review of Economic Studies 23,166-180.

Telser, L.G. (1980) A theory of self-enforcing agreements. Jou­rnal of Business 53, 27-45.

Weber, M. (1904/1958) The Protestant Ethic and the Spirit of Capitalism. New York:  Charles Scribners Sons.