`

 

Nutcracker Secundus

Abbreviated intro, text & links for HD

Roll mouse over text in dark red to reveal explanatory boxes.

-------


Introductory section will go here, ending with summary such as below:

.


My argument can be outlined as follows. Detailed discussion of each point follows below.

 

-------

 

Hyperbolic Discounting

How to explain choices: top-down versus bottom-up.  Some authors think that sophisticated human choice-making comes from subtle faculties that defy explanation.  Some others think it can be explained by rational choice theory (RCT).  But in both of these approaches we govern our animal instincts from the top down. So What?

Many people have been averse to bottom-up explanations.Explaining human choices from lawful combinations of simple processes has been called reductionism.  Fear of reductionism has limited the science of choice.

We evolved to seek reward.   The reward process itself evolved because of its adaptiveness — selecting individual behaviors is more adaptive than selecting whole organisms; but within an organism reward is the ultimate determinant of choice, regardless of adaptiveness.  Just as individual organisms can be seen as vehicles for transmitting their genes, behaviors can be seen as mechanisms for obtaining reward. So What? [Pico 179-180: “A parallel [that] has been proposed… fittest over time”]{People seek reward}  Note: Picoeconomics is not HTML linkable.  For the moment, I have linked to a similar passage in the BBS-McKay commentary ("Non-instrumental belief based on singularity").

Reward selects the processes it follows.  Reward that follows a mental process is what selects that process to get repeated. Rewards are events that induce the mental mechanism, reward. So What?

 

Done to here

 

Reward is a unitary phenomenon.  Although different parts of the brain seem to specialize in evaluating rewards at different time ranges of delays, the evaluations are consistent among parts, and consistent with hyperbolic discount curves.  [Social Neuro: Motivational conflict is temporal… “synchronic conflict (Ross, 2010)”]

The internal marketplace of reward.  Alternative mental processes (some leading to behaviors) compete on the basis of expected, discounted* reward.  They are tried out for how they feel, one after the other, a process called vicarious trial and error (VTE).  [OxfordMacGuffin: “The neural mechanics… imagining something else”]

Hyperbolic delay discounting.  In contrast to RCT, four decades of controlled experiments have now shown that the selecting power of a reward* is inversely proportional to its expected delay.  A graph with delay on the x axis and selecting power on the y axis forms a hyperbola.  [EconMod: “Psychologist Herrnstein…neglected the fourth prediction.”]

Alternatives to hyperbolic discounting.  A fundamentally hyperbolic discount curve is not the only reason preference might change as possible rewards get closer.  Some simple possibilities are easy to dismiss. [Munich “Hyperbolic discounting seems… social demand or other theoretical notions] We will later deal with a more complicated one, beta-delta discounting. {Sudden appetites are positive feedback phenomena}

A foundational experiment:

Change of preference over time in pigeons.  When choosing between an SS food reward and an LL reward four seconds later, birds shift from choosing SS to LL as the delay before the SS goes from 0 to 12 seconds.  [Ainslie & Herrnstein: Method… figure ] A&H 81 not in HTML. I used a link to the PDF version online.


A foundational experiment:

Change of preference over time in people.  A majority of both college students and substance abuse patients reported preference for an SS of $50 over an LL of $100 changed their preference as the delay before the SS was increased.  [Ainslie & Haendel, Self-reports of temporary preference section, with figures 9-4 and 9-5-- currently not in Word]  Not online in any format.

How did hyperbolic discount curves survive natural selection? Hyperbolic curves create impulsiveness, the tendency tend to prefer smaller, sooner (SS) rewards to larger, later (LL) rewards temporarily, when the SS rewards are imminent.  And foresight alone can’t prevent these temporary preferences.[“Hyperbolic discounting raises… self come into question soon after]

 

Done to here

 

 

 

References

Notes of references to be added

Robbins (1932/1985)

 

 

I will be arguing that many choices get rewarded when they are not at all pleasant—the urge to panic rewards attention to it, at least, and maybe the falling for it besides. {see ??}
Authors often assert that behaviors are selected for adaptiveness.  This calls forth “just so” stories, the explanatory style that used to be called functionalism.  The process of selection by reward clearly had to have been adaptive in past eons, but what gets rewarded today may have drifted far from the contingencies that existed when the reward process evolved.
If we can understand all choices as built on simple processes, we can analyze our “animal” urges and our higher mental functions  in the same terms.
Thus we might give value to an event that has no reward value of its own because it “allows” us to deliver endogenous reward to ourselves.
A repeated series of very brief but very intense rewards may command attention, while preventing us from doing or attending to anything else – and thus be aversive.
The notion of free will arises from our sense that our decisions are not predicatable. The delicate interplay among bundled clusters of future rewards and the question of to what extent we can anticipate our future choices produces just such a sense of unpredicatbility, even though the choice mechanism itself is deterministic.
As in a prisoner's dilemma game, the overall outcome is best if we cooperate with our "future selves" (which can make future choices either impulsively or non-impulsively). My present self avoids a tempting impulse based on a prediction that this will set a precedent for future choices, and the future self cooperates by not breaking that precedent. That cooperation feeds back recursively to strengthen my confidence that my present choices do indeed predict future choices.
Discount curves from bundled LL rewards summate to overcome impulsive choice.
A choice is impulsive when a smaller reward is chosen over a larger reward because it comes sooner. If the delay is greater, the larger reward will be chosen.
A discount curve shows the momentary value of a reward as a function of time to reward. Hyperbolic curves are steeper than exponential curves, but the tail is higher.
Organisms distribute choices in direct proportion to those choices' reward value and in inverse proportion to the reward delay. This is known as the matching law, and has been validated by hundreds of experiments.
Hyperbolic decay means that the momentary value of a reward declines in inverse proportion to the amount of the delay. It is contrasted with exponential decay, in which the momentary value declines as a constant percentage of the undelayed value.