What I am proposing here is an approach that recognizes the great human capacity to coin endogenous reward.  Our huge cortical apparatus does not just evaluate rewarding stimuli, but directly activates whatever brain centers create reward-- indeed, that the large, overlapping cortical areas that are active in autobiographical memory, fantasy, and prospection (Buckner et.al., 2008; Spreng et. al., 2009) must generate reward in order to compete for our attention.  When this reward is conspicuous we often name it an emotion (see Rick & Loewenstein, 2008), but we can also be aware of motivation that does not have much emotional quality, which we describe in terms such as importance or interest.  Our experience of this reward is often, indeed usually, attached to an external event; but this does not mean that the event either has intrinsic rewarding properties or predicts the occurrence of another event that does.  What I propose is an alternative rationale for this attachment.  It falls far short of the parametric models that have been developed on the basis of predicting extrinsic rewards, but I hope it opens the way to explore rewards that are constrained differently.

 

II. Endogenous reward is limited by appetite(s) for it

Behavioral scientists began to think about reward as a discrete mechanism around the time of William James, but the first to look at how we might generate our own reward was the psychologist Raymond Dodge.  Dodge had just begun to study the phenomenon of “autogenic reinforcement” (1917, p. 103) when the behaviorists banished introspection as a source of observation, making him not only first but just about last such psychologist, until educational psychologists recently took up a similar task (Renninger & Hidi, 2011; Silvia, 2006).  He thought it was “quite as significant for psychology as the action of adrenin to which Cannon has introduced us” (ibid.).  The property that offered him a handle on this process was its limiting factor, “fatigue,” which he thought might be characteristic of all nervous tissue.  “The routine is regularly less alluring than the unusual.  Mankind in general prefers new scenes, new plays, new walks, new jokes, new styles, new investigations.” (ibid., p. 104).  He ascribed this observation to a “relative refractory phase” in the relevant mental process.  “Repetitions of all sorts seem to be avoided whenever practicable.  The repetition of questions, courses, lectures, phrases, and even words is possible enough, but except for special reinforcing circumstances, it is postponed until the effect of the initial case is somewhat worn off” (ibid., p. 103).  Certainly such a refractory phase occurs in mental activity, as shown by the perseveration syndrome that occurs in its absence—some brain-damaged patients repeat the same phrase hundreds of times in a row, as if they had freshly come upon it each time (Sandson & Albert, 1984).  But this disinhibition of perseveration shows also that the refractory phase does not come literally from neuronal fatigue. 

Dodge missed the important distinction between the rewarding power that is renewed by rest—as in recovery from a refractory phase—and the rewarding power of a learned sequence that is never entirely renewed, which is actually characteristic of most of his examples: “The popular song, the clever phrase, the good joke, soon finds us refractory to the point of desperation” (ibid., p. 104).  Rest never restores to such rewards the impact that they had when new.  The loss of rewarding power because of learning is not fatigue but habituation. Dodge did observe a telling counterexample to the decline of rewarding power due to repetition, that “one of the marks of good art is the constancy of its appeals.”  In contrast to the transitory examples he had just given, “the great classics in music and literature may be heard over and over with increasing satisfaction” (ibid, p. 104).   Here he raised a question that still must be answered:  When we have learned how to harvest endogenous reward from, say, a good story, why do we not rehearse this story repeatedly, getting better and better at experiencing its rewards each time?  Conversely, why are we able to do just that with some stories, pieces of music, and paintings?

 

Although the key to the effectiveness of endogenous reward is the appetite for it, this appetite is so impalpable that no common name seems to specify it well.  Sometimes there are names for particular appetites within this category: curiosity, suspense, impatience, a wish to resolve ambiguity.  In general this appetite could be called a sense of importance; but this may differ radically from a person’s objective judgment of importance.  It might be called interest or even interestingness-- to distinguish it from the meaning of her objective or “best” interest. [This might better be called hedonic importance. -Added July 30, 2019]  Interest has begun to be studied again by educational psychologists, but not usually with reference to the concept of reward, which is taken to imply external manipulation (e.g. Ryan & Deci, 2000, p. 16; Silvia, 2001, p. 278)—an unnecessary theoretical limitation for which behaviorism has only itself to blame.  The concept of interest in educational, vocational, and aesthetic psychology is clearly what we are discussing.  A student’s interest in a new content area is first “triggered” as a “situational interest” (Renninger, 1992)—presumably by relevance to an existing interest—and then develops in stages into an “individual interest” in its own right.  Renninger and Hidi even say that interest functions as a reward, and note its correspondence with activity in brain reward circuits (2011).  Still, when the authors in these fields speak of intrinsic motivation they just mean whatever governs free choice among interests in various topics; they do not ask where this motivation comes from.

Both Dodge’s and the educators’ examples are complex, involving sensory stimulation, social considerations, and larger purposes, which might admit of many explanations.  To get closer to the mechanisms involved, we need to simplify.  Many illustrations of the minimal requirements of endogenous reward are provided by the psychology of boredom, which has recently been augmented by social psychology experiments on strength of will.  Here a subject must stay vigilant on a simple, repetitive task such as detecting letter patterns, adding numbers, or reporting stimuli while disregarding a prominent quality (as in the Stroop task, reading color names printed in a different color), thereby depleting her willingness to do similar tasks and eventually her ability to do them correctly (Hagger et.al., 2010).  Subjects get an external reward—money or course credit—for the whole task, but need to find ways from moment to moment to keep their attention from straying (Sansone & Harackiewicz, 1996).  In analogous real life tasks even an overwhelming reward contingency, such as detecting or failing to detect enemy airplanes on a radar screen, has often failed to maintain vigilance (Mackworth, 1948).  Presumably what ongoing vigilence requires is finding, or generating, endogenous reward with minimal input.

Vigilance tasks still involve contingencies that are externally imposed.   It will be better to take as our exemplar a solo activity with no exogenous consequences—for instance, the game of solitaire.  I create my appetite for solitaire out of a small, unnamed purse of motivation that is at my arbitrary disposal.  I make the game important to the extent that I bet this motivation on an unknown outcome.  As I play, the interestingness or importance of the game increases; in effect my bet increases.  The words are generic, but suggest something about the nature of the process:  I could say I have taken an interest in the outcome, or that I have made the outcome important, at least to the extent that it commands my attention.  The significance of these words is in their ambiguity about my degree of control.  Once I have taken an interest in the game I find it interesting.  Once I have given it importance, that importance lingers to be perceived when I re-evaluate it.  In effect, I can pull up a rewarding activity by its own bootstraps—but I am constrained to do this gradually.  If an eccentric billionaire offers me a million dollars to make the next turn of the cards extremely important to me, and will pay me for physiological evidence of this importance regardless of what card comes up, I will be unable generate the necessary suspense.  Yet in stages people cultivate the importance of intrinsically worthless outcomes all the time.

Once created, the interest or importance has an inertial quality, as is evident in a game that is more complex than solitaire, or that can be a component of some larger game.  If I do Sudoku puzzles my appetite may grow with repetition, so when I begin a new game I find it more important than I found an earlier game.  I can add or take away importance deliberately, but my appetite is also intrinsically limited by some particular properties.  If I play several games in a row my interest will wane; it will become less important to me. This is Dodge’s “refractory period.”  Conversely, if I conclude that the game is a waste of time and forbid myself to play any more that day, the importance I formerly built is refreshed, so the opportunity to play becomes a recurring temptation.  And as I become a better player, puzzles at a difficulty level that was once exciting become boring, so I must find tougher ones; this is habituation.  The process of investing importance in activities lacks an inclusive name.  It seems to be what Freud meant by a person’s directing libido to “sit on” an activity, which Ernest Jones translated as cathecting it (Freud, 1916-1917, pp. 335-357); but this has complex theoretical connotations.  By analogy to our solitaire example I will refer to it here as betting on the activities.

 

IV. The best occasions are singular and surprising

Given interest, events within a game pay off in proportion to their singularity and their surprisingness.  Singularity is the combination of their rarity and the extent to which they stand out from other kinds of event that are less rare.  In baseball a hit by a pitcher at bat is more singular than a hit by a fielder, and a number of scoreless innings in a row is more singular than a haphazard sequence.  Announcers try to increase the appearance of singularity by finding ways that the current play is statistically unusual—the first hit by a left-handed batter against this pitcher this season—but the net effect is to deflate the singularity of similar statistical finds by increasing their frequency.  The game itself may be more or less singular: a real game more than one generated by a video simulation program, a game with your home team more than with another, a game in post-season playoffs more than a regular one, today’s game or one shown within the day more than a game from the past, even if you do not know the outcome of the past game.  In general, the emotional impact of the outcome of a gamble is inversely proportional to its probability (Mellers et.al., 1999). 

Surprisingness is important because anticipating outcomes undermines your appetite for them (suspense).  If you know the outcome of the past game its occasions will pay off less, and less still if you have previously seen the whole game.  The same principles can be seen operating in other spectator activities.  Encountering a singular Stephen Sondheim rhyme such as “the hands of the clock turn but don’t play a nocturne” occasions more reward than hearing moon rhymed with June, but most when you first encounter it, and of course proportionately as you have taken interest in musical theater.  News items occasion reward according to surprisingness but also according to singularity, as measured by how recently and how close to home the events occurred.  If you watch the news only once a day it will have more impact than if you frequent all-news channels; the hedonic deterioration from waiting for news to consuming “infotainment” at will is essentially the same as from belief to fantasy.

When psychologist D. E. Berlyne reviewed a behavioral implication of interestingness—attention-- from a behavioral viewpoint, he found that it was “selected for” (?rewarded) by four stimulus properties: novelty, uncertainty, conflict, and complexity (1960, pp. 18-44).  All four can be characterized as blocking anticipation.  Ability to participate in an activity also makes it more interesting, but by inviting you into specific bets on outcomes and thus increasing their singularity; it is more fun to play solitaire than to watch it being played.  Recent experiments have pointed out that the pleasantness of stimuli does not necessarily go along with their interestingness (Turner & Silvia, 2006), but this finding merely confirms that unpleasant experience can reward attention

 

V. Texture is the aptness of an environment to provide good occasions

Endogenous reward becomes arbitrary without occasions, and then interestingness cannot build.  In prisons solitary confinement is a major punishment, more aversive than the din of a cellblock (Suedfeld et.al., 1982).  Experimental subjects deprived of sensory input develop psychotic symptoms in a matter of hours (Bexton et.al., 1954).  Even a child whose fantasy life has only begun to habituate needs a ball thrown or a stick to manipulate or mud to sculpt.  Conversely, even one-year-olds can be observed to use minimal cues such as circles moving on a screen to occasion emotions (Johnson et. al., 2007).  With experience we gain skill at investing importance—betting-- but our results depend on the availability of good occasions in the environment, a property that might be called its texture.  Rich texture permits complex occasioning, for instance in a challenging puzzle or plot or human relationship.  Durable texture is not worn away by familiarity, for instance in art.  As endeavors and environments compete for our investment, this durability over time may be the determining factor for which endeavors survive.    In Dodge’s examples, the popular song, the clever phrase and the good joke are rewarding in the short run, but with repetition anticipation erodes their usefulness as occasions.  Tic-tac-toe (naughts and crosses) engages a child’s interest, but is replaced by draughts/checkers and then chess as the contingencies become predictable.  As she becomes more able to predict complex patterns of play, these become the elements of wider puzzles.  The most satisfying puzzles are those whose solution reveals new puzzles.  Silvia describes the same pattern of unfolding “cognitive conflict” in the “selective persistence of interest” (2001, pp., 280-281).

The texture of occasions for endogenous reward provides a framework on which interestingness can grow, much as a trellis attracts vines by increasing the surface area that can be exposed to sunlight. But in addition to the obvious differences, trellises merely have to increase exposure to sunlight with no appetite-like limitation, whereas good texture has to improve a person’s combination of appetite and reward.  Development of such texture for endogenous reward has become a growth industry.  For instance, video games have grown from being the distractions of a few minutes to foci of engagement that last for hours or days.  This growth has historical parallels in the growth of short literary forms such as ballads and stories to novels, but automated games add responsiveness to players’ moves, and research on the behavioral effects of various textures permits systematic increases in their motivational impact (Jackson, 2012; King et.al., 2011).  Manipulation of texture alone has brought to video games an endogenously rewarding potential similar to that of gambling for money (Griffiths, 2005).  Here we are reminded that, although many textures invite a growth of interest that increases long term reward, some may yield intense reward up front that is followed by emotional impoverishment, just as in substance addictions.  The case of gambling should have shown society that the absence of a substance does not preclude the development of addictive preferences.  Computers have created many new patterns of occasioning reward, including not only video games but also simulated reality, media on demand,  internet cruising, and social networking.  Entrapping effects have been described by Sherry Turkle and others since the 1980s (Turkle, 2011), but the most direct evidence for an impulse control problem is the popularity of precommitting programs such as Self-Control and Freedom, which lock out a person’s own access during specified times.  I will not catalog the activities where reward that is obviously endogenous may be addictive, but I will now point out that its role is not apparent at first in the field where it is spread widest.

 

VI. Background for indirection: Prediction of exogenous reward also creates singularity

So far we have not considered the activities that engage most people most of the time, the ones with instrumental value—those that obtain rewarding consequences without necessarily being rewarding in their own right.  We have been talking about hedonic value—the value of an activity in its own right—and within this category only endogenous rewards, those that are not governed by the expectation of stimuli that are hardwired to be rewarding (food, sex, pain…).  We have seen in commonplace examples that endogenous reward does not have the properties that have been demonstrated in exogenously based rewards.  However, this does not mean that the two forms of reward usually occur in isolation from each other.  Even well-off people do not spend most of their time in activities with no instrumental purpose. 

Identification of endogenous reward in the presence of external incentive is difficult.  There are many activities in which the determination of reward by exogenous events is arguable or simply unknowable.  To what extent is money an endogenous reward?  Psychologists Stephen Lea and Paul Webley showed several ways that money has a “drug” effect over and above its “tool” value as a means of buying goods, especially a “money illusion” in which people adjust money’s subjective value in accordance with its nominal value or even physical properties (2006).  The irrational impatience people sometimes report for money—annualized discount rates of thousands of percent (Ainslie & Haendel, 1983; Kirby, 1997; Rosati et.al., 2007)—may sometimes reflect simply an impatience to spend it, for instance in the case of customers for an English loan company that advertizes a 1734% annual interest rate (www.QuickQuid.co.uk, June, 2012); but recent experiments trading off small amounts of money against delays of just seconds have found even steeper discount curves—hyperbolic in one case (Wittmann et.al., 2010), ambiguous in another (Gregorios-Pippas et.al., 2009)—despite the fact that the money was not to be delivered until the end of the experiments.  The valuations were consistent with fMRI brain recordings of subjects’ striatal reward centers, but by definition reflected drug-like rather than tool-like utility, which Wittmann et.al. analogized to points in a video game (2010).  With longer delays experimental dissociation of money’s hedonic value from its instrumental value will obviously be harder.

The example of money raises the general question of how much we should count as endogenous those token rewards that are ostensibly exogenous.  Outcomes in activities such as solitaire are obvious because they lack any credible connection with exogenous rewards.  However, what about the large variety of activities that are performed in the name of exogenous rewards, but that are often so distant that their discounted value would be negligible?  Using conventional financial discount rates, planners of welfare wonder how investments over decades are ever motivated (Karp, 2005).  The relatively high tails of hyperbolic discount curves improve this picture only partially (ibid.), especially if we consider the discount rate for actual experiences, which may be close to those of nonhuman animals.  Experiments with middle class subjects that use money as a reward are offering surplus value, not something that affects the subjects’ current comfort.  The discount rate for obviously exogenous rewards would be how much bad traffic, smoking deprivation, boredom, or noxious noise people would accept at a delay to avoid it in the immediate future.  At least in the case of noise, undergraduates choose immediate relief for a fraction of the duration they could get only tens of seconds later (Navarick, 1982; Solnick et.al., 1980), numbers similar to the cost in effort that pigeons will accept to postpone current effort (Mazur, 1996).  In one study of immediate versus delayed food, humans’ patience was on the same order of magnitude as chimpanzees’ (Rosati et.al., 2007).   Compared with the value of current comfort, the present value of a good that is twenty years away is vanishingly small.  When we seek a distant, expected reward whose discounted value is virtually nil, some process other than simply weighing it must be making it desirable.

Furthermore, distant prospects are often so unlike what we have actually experienced that they must be subject to extensive interpretation, or even wholly constructed: retirement, a dream vacation, getting a dread disease, getting a heavenly or infernal reward.  Discounting aside, explanation in terms of secondary or conditioned reward--by analogy to behavioral experiments--is not adequate for novel scenarios constructed from imagination.  In some cases an endogenous component can be shown to exceed the value that the expected rewards would have if imminent, as seen in the process of savoring desirable scenarios (Loewenstein, 1987) or dreading painful ones (Berns et.al., 2006).  It may arise even when the person does not believe the scenarios will actually take place, as in savoring the vacation that could never fit into her schedule or dreading the phobic object that she knows is not dangerous.  These believed-and-not-believed scenarios differ from fantasy to the extent that they are singular—licensed, as it were, by whatever has made them stand out from the common ruck of the person’s imaginings.  Such licensure may even lead her to seek signs of progress toward a goal that detain her from getting the goal itself, as when she prefers good grades in a subject to actually mastering it, or acquiring valuables for sale over actually selling them (see Sansone, 2009). The connection between current choices and the ostensibly exogenous rewards in the above examples is too tenuous to be accounted for by conditioned or secondary reward.  The obvious alternative is reward that is not induced, released, or limited by external events—reward that is endogenous.  Distant expectations are apt to be shaped by the endogenous reward they permit.  That is, when we can construct singular scenarios that map the future, these can serve as present occasions for reward, somewhat like the process that Rick and Loewenstein call “immediatizing” (2008).  The observation that hyperbolic value relationships are preserved within all ranges of delay must depend on the way we discern singularity, but this has not been specifically modeled (see Ainslie, 2006).  My point is that, although these choices are usually made in the name of maximizing some exogenous reward, this cannot be what they actually depend on.

[Even the boundaries of endogenous reward are debatable.  At the extreme it might be said to include all reward, at least in humans—in that all potential reward is subject to interpretation.  People can learn to resist being emotionally drawn into even intense physical stimulation, as in dental audioanalgesia (Melzack et.al., 1963), or when a hypnotist suggests interpreting a surgeon’s knife as warm sun (Hilgard & Hilgard, 1994).  But it is also likely that many seemingly endogenous rewards will turn out to have some exogenous component.  This is especially true when we use the behavior of other people as an occasion.  We give importance to a companion’s smile, but there may be an innate tendency to do this—a hardwired readiness to experience reward when we see a smile.  We know that our brains create vicarious experiences automatically (Iacoboni & Dapretto, 2006).  But only the readiness is inborn, not the motivational valence.  We may choose to make these models important or not, and choose whether the importance is sympathetic (as in shared experience) or invidious (as in envy or schadenfreude).  That is, we are not blank slates upon which any scenario is as apt to be written as any other; but our innate preparedness is only a groove in the slate, from which we may find it rewarding to divert the chalk. -Added Aug. 21, 2019]

 

VII. Endogenously rewarded activities may parasitize instrumental ones, leading to indirection

Even if we assume that activities with an expressed instrumental purpose are rewarded by the expectation of achieving that purpose—exogenously-- we should expect a component of endogenous reward as well.  Recall that endogenously rewarded interest tends to grow around occasions that are singular and surprising.  Occasions chosen purely for their hedonic value tend to deteriorate because their arbitrariness tempts the person to redefine them or replace them when they are paying off poorly.  This consideration suggests that occasions that are tied to actual productiveness in the real world will be more robust than those that are contingent on their near-term emotional effects.  Granted that a dedicated fantasist such as Coover’s J. Henry Waugh can commit himself to strict imaginary rules, and video games such as Second Life have discovered how to invite extensive commitment; in most people instrumental demands are apt to induce even stricter commitment.  Therefore steps toward exogenous rewards will often be the best occasions for endogenous reward.  The pursuit of exogenous rewards then offers two separate incentives:  One is the discounted value of the expected exogenous reward; the other is the endogenous reward occasioned by steps toward the exogenous reward. 

Here is the crux of why reward may function differently in people than in both nonhuman animals and in expected utility theory.  People seek external events somewhat as conventionally believed, but not entirely so.  As we seek events we learn that the pursuit itself creates occasions for additional reward:  As long as we follow a strict convention for belief, we can pursue distant goals and experience substantive current reward.  We can occasion reward with prospects that we have merely deduced as expectable.  As just described, we often experience more reward than the expected event would induce if present.  And we may find that the occasions we have authenticated as milestones can compete against the goals that provided the rationale for them to begin with.  However, these departures from instrumental efficiency must not be so great as to undermine our belief that they are realistic:  To cut them loose from the path to exogenous rewards would be to undermine their singularity, just like cheating at solitaire.

Furthermore, belief itself can depend on other factors than predictive realism.  I have argued elsewhere that the experience of belief—as opposed to make-believe—is the awareness of constraints on what we can choose (Ainslie, 2009, pp. 151-155).  Recursive self-prediction reaches equilibrium in personal rules, which, to the extent that we are committed by them, may be felt as facts.  The following example shows how they reward by singularity, quite apart from the ostensible rewards involved:

The belief that you have found a bargain can be instantly rewarding…  However, maintenance of this belief requires behavior that is consistent with it.  If you have stocked up on a food at a good price or bought a concert series at a discount, you may face an incentive to eat the food when you are tired of it or attend a concert you do not expect to enjoy in order to avoid recognizing a loss.  And yet you may be fully conscious of the unpleasant prospect.  The belief in the bargain is really a personal rule for playing a game, the wins in which occasion [endogenous] reward that is related only tangentially to the reward of tasting the food or listening to the concert.  The relationship is that the prospect of this consumption authenticates the bargain-hunting as an instrumental activity rather than a mere game, even though, once so authenticated, the bargain-hunting is a self-sufficient source of reward and has requirements that sometimes contradict those of optimally consuming the ostensible reward (ibid., p. 154).

Differences in the principles governing exogenous and endogenous reward create two kinds of motive—to maximize instrumental effectiveness and to refresh appetite.  Even though both may serve long range goals, they tend to be incompatible.  Instrumental effectiveness is best served by willpower, which lets us get to well-defined goals as efficiently as possible (see Ainslie, 2001, pp. 161-174; 2005).  Appetite is best refreshed by being confronted with delay and surprise.  The singularity of milestones in realistic instrumental plans make them excellent occasions for endogenous reward—but just to the extent that we have appetite for that reward.  If our plans have worked so well that they no longer excite us, conventional wisdom tells us to take this as the best possible accomplishment; however, from a hedonic standpoint it is suboptimal.  If realistic obstacles arise, creating suspense, the accomplishment will be less but the hedonic outcome may be better.  The hedonic problem, of course, is that we have been relying on the instrumental importance of our plan to make its occasions for reward singular, that is, more than just the results of a game or fantasy.   If we catch ourselves introducing obstacles to make the task more exciting, we belie that importance, a disincentive more significant than the obstacles’ immediate cost.  To maximize the plan’s effectiveness in pacing our reward, we have to believe both in its instrumental value and in the necessity of its obstacles.  Thus the experiences that have had the most sustained importance in a person’s life—the ones that she might say have “given it meaning”—are those that have entailed the most massive obstacles, experiences that she might never deliberately seek to have again (well illustrated in George Loewenstein’s rebuttal to a study basing well-being on the absence of unpleasantness—2009, especially pp. 92-100).  A major source of post-traumatic stress disorder in war veterans, besides traumatic memories, is difficulty in taking civilian pursuits seriously again (see Broyles, 1984).

Probably the largest component of endogenous reward is not what we indulge in deliberately for consciously hedonic reasons.  The largest component is what we cultivate in conjunction with instrumental tasks that depend on exogenous reward.  And we must expect these tasks to actually produce exogenous reward, unless they have some equally good claim to singularity-- a hallowed cultural imperative, for instance, or a longstanding habit. Once they can no longer be seen as instrumental (or otherwise dictated by a singular rationale), they lose most of their value as occasions for endogenous reward.  This consideration creates an incentive not to notice the endogenous contribution to the value of the activity, an incentive that pushes it into the Freudian dynamic unconscious; that is, it invites repression and denial of the endogenous motive.    

To restate this hypothesis:  Refreshing your emotional appetite without having to contradict what you have resolved often requires you to believe in some seemingly rational, or arguably necessary, activity that is incompatible with the direct routes to a reward.  That is, you need to find indirect routes to success: dummy activities that are not actually optimal for their ostensible purpose, but stay desirable insofar as they maintain appetite by creating good gambles.  In general you will need to believe in some larger quest that requires you to put your satisfaction at risk.  Instrumental and hedonic effectiveness can both be bases for choosing the same activity, but the hedonic component motivates you to under-perceive its own value and over-perceive that of the instrumental component.  

The incentive for indirection created by endogenous reward makes it hard to discriminate the two rationales in practice.  The best empirical indicator is probably the stubborn irrationality of the choices people make in ostensibly instrumental tasks—stubborn in the sense that people persist in displaying it even after extensive argument, reflection, and experience, that is, at “reflective equilibrium” (Slovic & Tversky, 1974).   Examples of obvious endogenous reward patterns embedded in instrumental activities are often trivial.  For instance, some people habitually risk being late.  Ostensibly they are optimizing use of their time: maximizing the value of [efficiency from tighter scheduling minus risk of objective loss].  However, they may also be getting an endogenous payoff from a solitary gambling game, the interestingness of which is proportional to the risk, and inversely proportional to the idle time spent after arriving early.  The determinant of actual behavior will be the sum of the exogenous and endogenous prospects. 

I know of no controlled research on subjects’ paying more for outcomes than the outcomes are objectively worth—some analog of hunting for pennies off on gasoline, or of defending a belief in bargain purchases as in the above passage—but a recent experiment does show the value of having a rationale per se for an endogenously rewarded activity:  Students were promised one of two kinds of chocolate for turning in some paperwork, either at a nearby site or at a place that was a 15 minute walk away.  If (and only if) one kind of chocolate could be had just at the distant site, a majority of subjects chose to walk to it rather than stay idle—whichever that kind was, and even though the subjects had been found to value both kinds equally (Hsee et.al., 2010).  In the language of the present discussion, the experimenters’ suggestion that walking to a dummy goal had an instrumental purpose let subjects give it importance, a way for them to occasion endogenous reward for the task.  Significantly, the subjects who walked reported a happier experience than those who chose idleness, indicating that the mere pretext conferred value.  Elsewhere I have presented an argument that gambling for money is a paradigmatic example of an ostensibly instrumental activity that is actually dominated by the endogenous contingencies of reward, with the chance of winning money serving as a MacGuffin to give the activity a rationale (Ainslie, 2013).

 

VIII. Summary: Endogenous reward is governed differently from exogenous reward

In this paper I have suggested that humans can generate a great deal of reward—the selective factor for choice—independently of any information predicting causes that lie outside of our control.  That is, we are physically able to coin reward through imagination, and this reward stands on its own, without needing endowment by an external releaser.  Except for its endogenous origin, this is the same reward conceived by classical behaviorism.  An ability to cultivate it jibes with common experience, but in conventional theory this ability should be impossible because of the circularity of reward’s selecting itself.  I have hypothesized that this kind of vicious circle is usually prevented by our hyperbolic impatience to gratify our appetite, which undermines ad lib self-reward and creates an incentive to set up obstacles to gratification.  I have outlined how constraints must operate on any kind of reward that depends only on the appetite for it, and how both conflict and symbiosis with instrumental reward is apt to result, sometimes creating process addictions. 

The important factors in endogenous reward have been partially recognized in behavioral psychology and classical economics, but beyond reward itself the concepts increasingly diverge from those proposed here (table 1).

 

Table 1: key concepts in endogenous reward and their closest equivalents in two other approaches

Picoeconomics	Behavioral Psychology	Classical Economics
Reward	Process reward	Utility
Endogenous reward	Process reward	Unaccounted tastes
Interestingness	Drive, appetite	Demand
Occasion (as a noun)	Secondary reward	Predictive information
Singularity	[no concept]	Scarcity (but negative)
Texture	[no concept]	[no concept]
Indirection	[no concept]	X-inefficiency (March, 1978)

• Exogenous and endogenous rewards share a dependence on appetite; however, the appetite for endogenous reward does not depend on physical deprivation, but rather on how the person mentally bets on outcomes. 
• Satisfaction of an endogenous appetite—harvesting it-- does not depend on a physical turnkey, but on the occasions that the person has bet on.  In the absence of an actual turnkey, satisfaction is conventionally thought to come from the secondary rewarding properties of these contingencies; but endogenous reward can function on its own, rather than being a soft currency that must be backed by the hard currency of external turnkeys.
• Beyond the degree of appetite, the value of occasions is determined by their singularity and surprisingness.  Singularity denotes infrequency and distinctness from other possible occasions, as in criteria for winning a game or accomplishing a task.  Like the criteria of a personal rule, its function is to regulate a temptation, in this case the urge to harvest appetite prematurely.  Surprisingness is an ordinary speech term; it prevents harvesting appetite in anticipation.
• Texture refers to the distribution of occasions for endogenous reward in a situation or environment.  Rich texture permits complex occasioning, for instance in a challenging puzzle or human relationship.  Durable texture is not effaced by familiarity, a characteristic of art, or of those puzzles and relationships that present new challenges when earlier ones are mastered.
• Indirection is the strategy of pursuing an instrumental task by a route that has better texture than a more instrumentally efficient route.  Not acknowledging this strategy preserves the singularity of the indirect route.