Dopamine Appetite and Cognitive Impairment in Attention Deficit/Hyperactivity Disorder

19 Pages·2014·3.54 MB·English

NEURALPLASTICITY VOLUME 1,NO. 1-2,2004
Dopamine Appetite and Cognitive Impairment in
Attention Deficit/Hyperactivity Disorder
JonathanWilliamsl,2andEric Taylor3
DepartmentofChildandAdolescentPsychiatry, InstituteofPsychiatry
DeCrespignyPark, DenmarkHill, LondonSE58AF, UK
SUMMARY KEYWORDS
The underlying defects in ADHD (Attention dual pathway, methylphenidate, computational
Deficityperactivity Disorder) are notyetclear. cognitivedeficit, stateregulation,ADHD
The current paper tests three existing theories:
State Regulation, Cognitive Deficit, and
Temporal Difference (TD) learning.We present INTRODUCTION
computational simulations of the Matching
Familiar Figures Task and compare these with The Diagnostic and Statistical Manual of
the experimental results reported by Sonuga- Mental Disorders (DSM-IV-TR) (American
Barke (2002). The TD model contains four Psychiatric Association, 2000) offers three
parameters: the learning rate, discounting for subtypes of attention deficit hyperactivity disorder
future rewards, brittleness (randomness) of (ADHD): Predominantly inattentive; Predominantly
behavior, and action bias. The results show that hyper-active-impulsive; and Combined, implying
the basic TD model accounts well for control thattherearetwo maincausesofADHD,whichcan
performance in trials of5sec, 10sec, and 15sec co-occur. A similar view has been codified as the
duration; but not for the deficits in ADHD "dual pathway model" (Sonuga-Barke, 2002b). The
performance at 5sec and 15sec. Extending the dual causation view has support from several
TD model to incorporate either a state sources. One is the finding that ADHD children
regulation deficit, or working memory deficit withasusceptibilityconferring7-repeatalleleofthe
and delay in starting trials, can provide a good dopamine D4 receptor gene are remarkably free of
account of both control and ADHD results, at the slow, highly variable response times that
all trial-lengths. We discuss the significance of characterize ADHD children lacking that allele
the results for theories of ADHD and make (Swanson et al., 2000a). Furthermore, two quite
suggestions forfutureexperimentation. different tasks, the Choice-Delay Task (CDT) and
the Stop Signal Task (SST) together form a very
sensitive and specific predictor of the clinical
Reprintrequeststo:JonathanWilliams,DepartmentofChild diagnosis of ADHD. Yet the two measures show
and Adolescent Psychiatry, Institute of Psychiatry, De very little correlation between themselves (Solanto
CrespignyPark,DenmarkHill,LondonSE58AF,UK;e-mail:
et al., 2001), supporting the independence of the
[email protected]
Financialsupportreceivedfrom: tLondonPostgraduateMedical two causative pathways. The SST is seen as
Deanery; 2Gatsby Charitable Foundation; 3Medical Research revealing a component ofexecutive control, which
Council. may be related to general cognitive level; and the
(C)2004Freund&Pettman,U.K. 115
116 J.WILLIAMSANDE.TAYLOR
CDT appearsto measuredelayaversion,whichmay attention and frontal function (Doyle et al., 2000;
becausedbyanabnormalityofrewardprocessing. Grodzinsky and Barkley, 1999; Lockwood et al.,
Although several etiological factors have been 2001). Areduced activation ofvarious frontal areas
associated with ADHD, the mechanisms by which has been described in ADHD children during
they act are still largely mysterious. However, a Stroop, stop, and motor timing tasks (Bush et al.,
new level of theoretical clarity was achieved 1999; Rubia et al., 1999). Abnormalities ofevent-
recently with the head-to-head testing of four related potentials in acontinuous performancetask;
explanatory theories ofADHD, usingthe Matching of visuomotor perception; and of verbal memory
Familiar Figures Task (MFFT), in both a severely and learninghavealsobeendescribed(Oie&Rund,
affected clinical sample and a mild population 1999; Raggio, 1999; Sunoharaetal., 1999).
sample matching criteria for combined type Subgroups can be particularly affected
ADHD (Sonuga-Barke, 2002a). In this task, seven (Swanson et al., 2000b). Perhaps half of all
pictures arepresentedsimultaneouslyonacomputer children with ADHD have significant deficits in
screen, and the subject has a specified number of motor or perceptual skills, without clear mental
seconds to discover which picture matches the top retardation or major neurological disability
one. Each of the four theories (State Regulation (Gillberg, 2003). Dyslexia and other language
Deficit, (General) Cognitive Deficit, Premature problems are seen in about 50% ofchildren with
TaskDisengagement, and EcologicalNiche) ledto ADHD (Gilger & Kaplan, 2001) and also often
distinct predictions regarding ADHD performance verbal-performancediscrepancyonIQtesting.
onthe MFFT. It is not clear whether there is a visual search
In short, the control groups’ performance deficit in ADHD, but it seems likely (Mason et al.,
steadilyimprovedasthetrialswere lengthenedfrom 2003; Malone & Swanson, 1993). More relevantto
5 to 10 then to 15 seconds. In comparison, the MFFT, however, children with ADHD are delayed
ADHD performance was inferior to that of the in the initiation of a serial visual search and have
controls at 5 see; then equivalent at 10 see, and deficits in spatial working memory (Karatekin &
again inferior at 15 see (shown below). This effect Asarnow, 1998;Kemptonetal., 1999).
was seen in both of the very different samples, Difficulties after rewards can be expected for
suggesting that we are seeing an important effect. two reasons. First, reward may slow down ADHD
The data will be used in the current paper as the childrenmorethancontrol children(Scheresetal.,
basis for assessing the validity of three distinct 2001). Also, set-shifting problems have been
computational simulations of decision-making in reported in preschool children with ADHD
ADHD, which can be briefly described as learning- (Sonuga-Barke et al., 2003), and these may slow
based,appetite-based,andcognition-based. down the transition from processing reward to
startingthenexttrial.
CognitivedeficitsinADHD
DopamineappetiteandstateregulationinADHD
It is well known that children with ADHD
often suffer fromacademic impairment(Barkleyet It has been suggested thatthe underlying cause
al., 1991; Faraone et al., 1993). Some impairments ofADHD is adefectinthe regulationofactivation:
persist from preschool to college age (DuPaul et "To counteract a performance decrement, subjects
al.,2001; Heiligensteinetal., 1999). have to regulate their state: they have to inhibit
Neuropsychological deficits have also been activationwhenstimuliarerapidlypresented,andto
describedinADHD,particularlyintestsofselective exciteactivationwhenstimuliareslowlypresented"
DOPAMINEAPPETITEANDCOGNITIVEIMPAIRMENTINADHD 117
(van der Meere et al., 1999). The proposed parameters) controlling learning and behavior. The
regulation can be described as indirect because a mappingofthese parametersto brainstructures(as
predictor ofneed (the stimulus rate) is being used in Fig. 1) is considered mainly where the
to regulate activation rather than the level of experimental data is strongest, namely with the
activation being used to regulate itself. In the dopaminergic neuromodulatory system originating
control of respiration, where regulation has been intheventraltegmental area(VTA).
well studied, such indirect regulation does exist, The link from the TD learning method to
controlled by emotion, body position, movements, dopamine function was originally made by
and predicted activity level (Shea, 1996); but Montague et al. (1996; see also Friston et al.,
direct control by hypercapnia is simpler and more 1994) to account for data on the activity of
important. We have therefore studied direct dopamine cells in the VTA and substantianigraof
regulation as a potentially more parsimonious monkeys during the learning of an operant
explanationforADHD. conditioning task (Schultz, 1998). The idea is that
Stimulants like methylphenidate (MPH) block the monkeys are constantly learning to predict
dopamine transporters (Volkow et al., 1998) and future reinforcement within a trial, and that the
are used to treat ADHD. in which they improve phasic activity of dopaminergic cells signals
academicperformanceandnormalizevariousneuro- mismatches in these predictions. The model
psychological measures (Elia et al., 1993; Haenlein accounts well for a wide variety of data on the
& Caul, 1987; Sunoharaetal., 1999;Tannocketal., dopamine system in learning (Schultzet al., 1997),
1989). Stimulants cause an immediate, unlearned, forwhichevidence is also accumulatinginhumans
and fully reversible reduction in hyperactivity in (Friedetal.,2001).
ADHD and in control children (Flungund et al., Figure l(a) shows the basic TD model. In it, a
1979), leading to the possibility that the children representation of the current state is available in
have an ’appetite’ for dopamine, which can be cortex. The basal ganglia learn to associate these
satisfiedbystimulants. states, and potential actions, with reinforcements.
Dopaminergic systems are activated by reward The ’action’ (suchas inspectingapictureorlooking
but also by various other factors, notably novelty around the room) that appears likelyto produce the
(Ljungberg et al., 1992; Cloninger, 1987; Horvitz, largest reinforcement is in general chosen by the
2000; Schultz, 1998). Indeed, increased novelty- basal ganglia. The ’predicted reinforcement’ is used
seeking is found in ADHD (Downey et al., 1996; in conjunction with information about immediate
Young et al., 2000), and associations have been reinforcement to create the ’prediction error’ signal
found between a specific allele of the dopamine that, in turn, is used to alterthe prediction that will
D4 receptor and novelty-seeking (Malhotra & be made when, in future, the same situation is
Goldman, 2000). Hyperactivity inADHD has been encounteredagain.
repeatedly found to be ameliorated by novelty The model gradually learns to choose
(Iabonietal., 1995;Felicetti&Julliard,2000; Sleator behaviors that leadto rewards. Details are given in
& Ullmann, 1981) and by high reinforcement rates the Appendix. The basic TD model used in this
(Douglas& Parry, 1994; Carlson& Tam.z,.,,2000). paper has already been validated against the
Choice-Delay Task (CDT) (ofSonuga-Barke et al.,
TheTemporalDifference(TD)modelofADHD 1992; Williams & Dayan, unpublishedobservations).
However, a weakness ofthe model is that it does
This model is primarily concerned with the not provide an obvious account ofthe immediate
behavioral effects of variations in constraints (or effects of stimulants. The MFFT data are
118 J.WILLIAMSANDE.TAYLOR
particularly appropriate fortesting the model, as it matches the index, versus inspecting anon-match.
is quite different from and considerably more We consider the latter first. When choices are
complicatedthanthe CDT. superficially similar (i.e. in difficult trials),
The current study presents a computational recognizing differences is more likely than
simulation of the task used by Sonuga-Barke recognizing the identical match. So the program
(2002a), in order to determine whether state keepstrackotthe currentestimatedprobability for
regulation deficits and/or cognitive impairment each ofthe six target pictures. These start at 1/6.
and/or the TD model are able to account for When adifference is found, the probability ofthat
ADHD performance in the MFFT. Before starting picture matching the index drops toward zero, at a
this work, we expected, based on results described rate determined by the ’learning rate’, and the
above, thatthe TD model neededto be extendedto probabilitiesarere-distributedtototal 1.
incorporateaStateRegulationdeficit. However with very simple pictures (e.g. stick
figures), subjects may be able to tell ’at a glance’
that they have found the right one. Our simulation
EXPERIMENTAL allows such recognition in easy trials only.
Without such an ’at a glance’ method, the model
ModelingtheMFFT cannot achieve learning as fastasthe experimental
controls (though such fast learning could also be
The challenge here is to simplify the task accounted for by controls having a special quick-
sufficiemly to allow it to be programmed, without scan strategy for short trials.) Note that the
losing any aspect that is critical to ADHD. The distribution ofdifficulty levels ofthe trials affects
task is implemented using a computer program. performance(seeAppendix).
Time is simulated as steps, with each time-step The overall approach just described can be
notionally one second. At each time-step, the implemented in many ways. Our program
model has three options: to inspect the next ofsix incorporated temporal difference learning, with its
target pictures to see whether it matches the index parameters described below. It also incorporated
picture; to similarly inspect a randomly chosen parameters suggested by the state regulation and
picture; ortodosomething else(notspecified). cognitive deficit theories (also described below),
The purpose ofthe inspections is forthemodel in such a way that any or all of them could be
to steadily improve its estimation ofthe likelihood disabled, and the effects ofthe remaining one or
that each target picture is a match for the index two factors could be investigated in isolation. The
picture. After all the time-steps in the trial have program is written in Matlab (available by email
elapsed, the model is interrogated regarding its fromthe firstauthor).
choice of best match. It then has two options: to
give its bestguess orjusttogivearandomanswer. Applying the Basic Temporal Difference (TD)
Complicated cognitive tasks, such as task ModeloflearningtotheMFFT
acquisition and visual search, are not simulated in
this work. Visual search in particular has been the The TD model’s behavior is controlled by the
subject of a large amount of theoretical and reward size and by four key parameters. In this
simulation work, but none has directly addressed context, parameters are numerical measures of
the MFFT. long-term aspects ofan organism’s or the model’s
The model treats quite differently the behavior. These are generally unchanging during
situations of inspecting a target picture that an experimental episode and may be genetically
DOPAMINEAPPETITEANDCOGNITIVEIMPAIRMENTINADHD 119
controlled. Although the parameters interact to it is received. In reality, it will be a nonlinear
determine overall behavior ofthe model, we study function ofthe objective reward size. We included
primarily the simple case of their effects in this factor to allow the model to account for any
isolation. child who tended to experience rewards as less
1. Perceivedrewardsize. This isconceived as rewarding than other children, as hypothesized as
the amount ofpleasure elicited by a reward when one possible cause ofADHD (Blum et al., 2000;
sensoryinputs sensoryinputs
Cognitive
deficits
.
Discount Discount
Actionbias Actionbias
Brittleness Brittleness
Actionselection Actionselection
andpredictedI andpredicted
reinforcement
prediction prediction
error error
direct (dopamine) direct (dopamine)
reinforcement reinforcement
Dopam=ne
appetite
ModelA Model B
TheBasicTemporalDifferenceModel TheExtendedTemporalDifferenceModel.
Differencesfrom ModelA arethe inclusionof
dopamineappetite and cognitivedeficits.
Fig. 1: Modelsusedinthisstudy. (OFC:orbitofrontalcortex;VTA:VentralTegmentalArea)
120 J.WILLIAMSANDE.TAYLOR
seediscussion in Solantoetal., 2001). The explicit 3.Brittleness. The predictability ofthe model’s
reward provided by the experimenter at the end of behavior is dependent on three main factors: the
successful trials was modeled. We did not model, reliability ofenvironmental predictors ofreinforce-
however, implicit rewards in the task, such as the ment, the extent to which such lessons have been
novelty of seeing new pictures, the pleasure of learned, and finally brittleness, the extent to which
finding each difference, and the pleasure in behavior is based on those learned lessons.
becomingcertainofamatch. Brittleness being too high allows single learning
2. Rewarddiscounting. The idea here is that a experiences to cause sudden changes from one
reward expected inthe future isworthlessthanthe consistent behavior to another, and a comparative
same reward delivered now. Repeatedly in life, inability to persist with one behavior in an
individuals face choices between small, immediate unpredictable environment. On the other hand,
rewards and delayed large rewards (e.g. in brittleness being too low would indicate a
dieting). We quantify this by multiplying the comparative inability to persist with one behavior,
reward by a number D (between 0 and 1), at each even when the subject has collected adequate
time-step. At 9 pm, an immediate reward mightbe informationaboutreinforcementavailability.
worth R, but at 8 pm, it is worth RD, and at 7 pm, In principle, a subject might be more
it is worth even less, RD2. In our model, ADHD predictable in some areas ofdecision-making than
children have a smaller D, so future reinforce- in others. The first is the brittleness of the
ments are discounted more, and delayed rewards decision-making at every time-step. Here the
will influence their behavior less than in control subject must choose whether to (a) methodically
children. Indeed, increased discounting has been inspect the next picture in sequence; (b) randomly
demonstrated in ADHD (Barkley et al., 2001; choose another picture to inspect; or (c) do
Sagvolden et al., 1998); Sagvolden, Aase et al something else. The second is in communication
(2000) proposed that a "shorter and steeper delay of the choice at the end of each trial, when the
gradient" in ADHD children may lead to the subject must choose whether to (a) tell the
development ofoveractivity, increased behavioral interviewerthe most likelymatch; or(b) guess.
variability, motor impulsiveness, and impaired 4.Learning Rate. For consistency with the
sustainedattention. computational literature, this termistakento mean
Such a gradient has also been correlated with themaximalrateatwhichthemodel isabletoalter
impulsive behavior in general psychiatric out- its predictions about the environment, rather than,
patients (Crean et al., 2000). Males discount future as would be more natural from a behaviorist
rewards more than females, and addicts more than perspective, the rate atwhichbehavior changes. In
non-addicts (Wilson & Daly, 2003; Kollins, 2003). the model, prediction error causes changes in
It has been suggested (Williams & Dayan, 2004) predictions and future actions. Ifthe errorsignal is
that the discounting rate may to some extent be multiplied or divided by a factor, then this has the
learned from the degree of unpredictability in an sameeffectaschangingtheresponsetthatsignal,
individual’s environment; such unpredictability, i.e. the learning rate. Although we model the
however, may be highly context-specific (Aloise & learning rate as primarily dependent on various
Miller, 1991). Over a much shorter time course, heritable factors associated with dopamine, the
discountingcanbe increased(i.e. D canbereduced) rate may also reflect intrinsic synaptic factors.
by exposure to certain pleasurable stimuli (Wilson Therefore, the terms ’dopamine multiplier’ and
&Daly,2003). ’learningrate’ are interchangeable.
DOPAMINEAPPETITEANDCOGNITIVEIMPAIRMENTINADHD 121
There are several lessons learned by subjects Stateregulationdeficit
in this task, including (a) the rules of the task;
(b) strategies to use; (c) within an individual trial, We have included dopamine appetite as a
the characteristics of the index picture; and simple example ofstate regulation. This takes the
(d) which pictures are different from the index form ofa ’dopamine level’ that is increased when
picture. Both (a) and (b) were not needed to reward is received at the end of trials, and then
explain the data from Sonuga-Barke because the slowlydecaysaway.
order oftrial lengths was randomly determined for The dopamine appetite must be linked to some
each subject. For simplicity, we have excluded (c) output to complete a regulatory loop. In principle,
andmodeledonly(d). appetite could act by increasing or decreasing any
A high learning rate is a disadvantage in tasks other parameter (or parameters) in the model. The
for which previous learning canexcessivelyimpair ’gearing’ ofthis influence is a numeric value that
exploration. Such is the case with the Delayed we can adjust in the simulation. We have simulated
Response Time Task (DRTT; Williams & Dayan, appetite effects on action bias and brittleness,
2004) and in exploring an environment to find because these parameters can have immediate and
maximal reward. T his would be relevant to the reversible effects on behavior (see Fig. 1), as
MFFT in trials in which a subject erroneously suggestedbythe immediate resultofstimulants. We
identified a match, and so then stopped exploring have not simulated appetite effects on learning rate
and did not have a chance to correct his mistake. or discount, because the model does notallowthese
This sort ofmistake has not been included in the tohaveimmediatereversibleeffects.
currentsimulation.
5.Action Bias. Action bias is ameasure ofthe Cognitiveimpairment
child’s preference for action over inaction. Action
bias is greater than zero if the child, for any Performance of the MFFT involves many
reason, has an innate bias to act, such as finding subtasks. We have modeled the effect of several
that action itself reinforcing. Co nversely, action changes that might be expected to accompany
bias is negative ifthe child prefers to be inactive. general cognitive impairment. Each is potentially
A non-zero action bias can force the child to make very complicated, so we have notmodeledthem in
suboptimal decisions. Such a preference could in detail. Instead, we have represented each by a
theorybe innate orlearnedorboth. singlenumber.
There is little opportunity within the MFFT 1. orking memory. ADHD is often accom-
task for physical action, and so we have chosen to panied by an impairment in working memory
defineactionbias asthetendencyto preferactivity (Sonuga-Barke et al., 2003). In real subjects,
awayfromthetask. working memory is important for holding features
ofthe index picture in mind, for comparison with
Extensions to the Temporal Difference Model the others. Working memory is also important for
usedinModelB holding the ’result’ until it is requested by the
experimenter. We have modeled only this latter
The TD model was extended to incorporate aspect. We model working memory as holding
state regulation problems and cognitive impairment estimates ofthe likelihood ofeachpicture matching
to see whether these allow a better fit to the the index picture. These estimates are subject to
experimental results. These extensions are shown slight ’blurring’ during each second in which the
inFig. (b)andare discussedbelow. modelisoff-task(seeAppendixfordetails).
122 J.WILLIAMSANDE.TAYLOR
We expect that most children are motivated the difference at one trial-length, extra factors
bothbytheirdesireto succeed andbytheirdesireto would be required for the other two trial-
please the investigator. After the children become lengths.
certain that they have found the right answer, the
question of succeeding disappears, and they may RESULTS
become interested insomething else. Wemodelthis
by stopping all activity on the task, once an exact ModelA
match has been found. In long trials where the
model finds amatchearly,the subsequent fadingof 1. Perceived reward size. A reduction in
itsworkingmemorymaylimitperformance. perceived reward size impairs performance across
2.Delay at the start oftrials. This delay is the awiderangeoftrial-lengths(resultsnotshownbut
time used between finishing one trial and starting very similar to Figure 2(a)). This occurs because,
the search through the object pictures. The delay in the model, a substantial reward increases the
consists of the Post-Reinforcer Pause (PRP) plus likelihood of(i) at each time-step, staying on-task
the time used in any initial scan of the index and exploring the pictures carefully; and (ii) atthe
picture and will include the time needed for set- end of each trial, telling the investigator the
shifting, which has been shown to be an area of correctanswer.
deficiency in ADHD (Sonuga-Barke et al., 2003). 2. Reward discounting. Increased discounting
During this time, the model is losing opportunities has minimal effect in short trials; impairs
tocollectinformationaboutthe picturestomatch. performance somewhat in longer trials; and has
little effect in very long trials (result not shown).
Othercognitivefactors The deterioration is caused by a reduction in
incentive, during the early part of long trials, for
a) The ability to search from picture to picture the individual to work at the task. In very long
methodically, avoiding time-wasting repetition. trials, this wasted time does not matter. All results
This ability helps performance in short trials shownusenodiscounting(i.e. D 1).
but not in long trials; in any event, it cannot 3. Brittleness. Groups with low brittleness of
account for anabsolutedecline inperformance communication have little tendency to tell the
in longertrials(resultsnotshown). interviewer what they know (this would be quite
b) The ability tofind any difference between the different from having an oppositional inclination
indexpicture and thepicture being inspected. to tell him a falsehood). Their recorded
This ability helps performance at all trial performance is very poor and is little affected by
lengths, but like (a) cannot account for an what they may have learned (results not shown).
absolute decline in performance in longer Groups with high brittleness ofcommunication,on
trials,so is notshown. the other hand, reliably tell the interviewer what
c) Speed ofthinking. Many studies have shown they know. This allows the interviewer to obtain a
apparently slowedthinking/responses inADHD good measure of their ability. All results shown
(e.g. Mason et al., 2003; Solanto et al., 2001; use ahighlevel ofbrittleness(6).
Berman et al., 1999). We have not modeled 4. Learning Rate. A higher learning rate is
this in detail because it cannot provide a generally beneficial in this simulation, as we have
parsimonious accountoftheMFFTdifferences not modeled erroneous identification of matches.
between ADHD and controls. Although The exception is in cases for which it leads to
stretching oftime can be used to account for premature task disengagement (results not shown;
DOPAMINEAPPETITEANDCOGNITIVEIMPAIRMENTINADHD 123
b d
Model A Model Bwith state Model B with
regulation differences cognitive differences
AB=0 one-tallAB=0.0 memblur=0
AB=0.8 one-tailAB=1.6 memblur=0.017
2 4 6 8 10 12 14 16 25 40
Triallength(sec)
two-tailAB=0.0
two-tailAB=I.7
2 4 6 8 10 12 14 16 25 40 2 4 6 8 10 12 14 16 25 40
Triallength(sec) Tdallength(sec)
Fig.2: Simulations ofthe MatchingFamiliarFiguresTask(MFFT), comparedwithreal data. Each graph showsthemeans and
standard deviations from Sonuga-Barke (2002) Study 1, redrawn unchanged in everygraph as thin lines (solid=control;
dashed=ADHD). Superimposedonthis, eachgraph showstheperformanceoftwo simulated groupsofchildren, asthick
lines(solid--modelofcontrols;dashed--modelofADHD).Allthesimulationsusethesamevaluesforallparameters,except
fortheoneortwoexplicitlymanipulatedforeachfigureandshowninthelegendboxes,andModelAdoesnotincludeany
workingmemorydeficit, delayatthestartoftrials,orstateregulationmechanism. Eachsimulateddatapointresultsfrom
5000trials.Notethattherightendsofthex-axesarecompressed.(a)EffectonthemodeloffixeddifferenceinActionBias.
(b)Effectonthemodel,offixeddifferenceinresponsetorewarddeficit,increasingActionBias.(c)Effectonthemodel,of
fixeddifference in response to nonoptimal reward rate, increasingAction Bias. (d) Effecton themodel, ofreducingthe
retentivenessofworkingmemory. (e)Effectonthemodel,oftwofixeddifferences: start-trialdelay andworkingmemory
retentiveness.
Abbreviations: AB ActionBias(seetext). One-tailAB:Theextenttowhichnon-optimalrewardrateincreasestheActionBias.
Thisis"one-tailed"becausereduction,butnotincreaseintherewardrateincreasesActionBias.Two-tailAB:Theextentto
whichnon-optimalrewardrateincreasestheActionBias. Thisis"two-tailed"becauseincreaseintherewardrateincreases
Action Bias,just as reduction does. Memblur: the extent ofblurring ofworking memory each second (seeAppendix).
Startdelay: Thenumberofsecondsnotusedatthestartofeachtrial.
124 J.WILLIAMSANDE.TAYLOR
see Discussion). All results shown use a high that children could have their performance
learningrate(1). impaired both by becoming over-excited at high
5. Action bias. Figure 2(a) is shown as an ratesanddisinterestedatlowrates.
example of the behavior of Model A, in the The possibility ofa link from reward excess or
Matching Familiar Figures Task. The figure shows deprivation to reduced brittleness (rather than to
the effect of increasing the action bias on the actionbias) was also brieflyexplored. Inthe model,
performance of the model. This impairs such alinkwas much less impairingthanthe linkto
performance on the task, as time away from the increasedActionBias(resultnotshown).
taskreducestimespentinspectingthepictures.
ModelB,with differenceinCognitiveImpairment
Summaryofresultsfrom ModelA betweenthetwogroups
Leaming creates a general trend to better 1. Working memory. Figure 2(d) shows that
performanceinlongertrials, seeninthesimulations. impaired retentiveness of working memory has a
This effect succeeds in very roughly approximating small effect in short trials, but a major effect in
the improvement seen in real ADHD and control long trials. This is because after the model has
groups (see Fig. 2(a)). The parameters of the TD reached its criteria for success in atrial, ittends to
model can provide various explanations for the choose non-task activities, during which its
tendency for ADHD performance to be somewhat working memory fades. Other simulations (not
worse than the control group. These explanations shown) showed that having a lower threshold for
include reductions in perceived reward size, certainty, i.e. disengaging from the task before the
brittleness, or learning rate, and increases inreward answer was well known, preferentially impaired
discountingoractionbias. performanceonlongtrials, as expected.
However, as shown for example in Fig. 2(a), 2. Delay at start oftrials. Figure 2(e) shows
simple changes to the TD parameters are not able the effect ofhypothesizing two related differences
to account for the absolute deterioration inADHD between ADHD and control: working memory
performanceseenempirically inlongtrials. retentiveness and start-trial delay. The upper solid
line indicates the performance ofthe model when
Model B, with a Difference in State Regulation not losing any such search time. The lower solid
betweenthetwogroups line indicates the performance ofthe model when 3
seconds are lostatthe startofeverytrial. The figure
We introduced a link from deficient reward shows that the effect ofsuch disruption reduces as
(i.e. below the set-point) to an increased Action triallengthisincreased,asonewouldexpect.
Bias. In practical terms, this reflects the tendency
ofa child to become more active when he has not
received much stimulation recently. The effect of DISCUSSION
this factor is to selectively impair performance on
longertrials, as expected(Fig. 2(b)). Main results
Figure 2(c) shows a two-tailed version ofthe
same mechanism. In this case, Action Bias is We have attempted to explain the difference
increased whenever the reward rate (modeled as between ADHD and control performances on a
dopamine level) is outside an acceptable range. high-level cognitive task, the MFFT, using the
This is intended to model previous suggestions Temporal Difference (TD) model ofADHD. The