Beyond arousal and valence: The importance of the biological versus social relevance of ...

25 Pages·2012·0.79 MB·English

CognAffectBehavNeurosci(2012)12:115–139
DOI10.3758/s13415-011-0062-x
Beyond arousal and valence: The importance
of the biological versus social relevance
of emotional stimuli
Michiko Sakaki&Kazuhisa Niki&Mara Mather
Publishedonline:1October2011
#PsychonomicSociety,Inc.2011
Abstract The present study addressed the hypothesis that theinterconnectionbetweentheamygdalaandvisualcortex
emotional stimuli relevant to survival or reproduction supportsenhancedattentionallocationtobiologicalstimuli.
(biologically emotional stimuli) automatically affect cogni- In contrast, socially emotional images evoked greater
tive processing (e.g., attention, memory), while those activityinthemedialprefrontalcortex(MPFC)andyielded
relevant to social life (socially emotional stimuli) require stronger functional connectivity between the amygdala and
elaborative processing to modulate attention and memory. MPFC than did biological images. Thus, it appears that
Results of our behavioral studies showed that (1) biolog- emotional processing of social stimuli involves elaborative
ically emotional images hold attention more strongly than processing requiring frontal lobe activity.
do socially emotional images, (2) memory for biologically
emotional images was enhanced even with limited cogni- Keywords Social emotion.Biological emotion.
tive resources, but (3) memory for socially emotional Attention.Memory encoding.Amygdala.Medial
images was enhanced only when people had sufficient prefrontal cortex.fMRI
cognitiveresourcesatencoding. Neither images’subjective
arousal nor their valence modulated these patterns. A
Emotion has a major impact on cognitive processing (see
subsequent functional magnetic resonance imaging study
Dolan, 2002, for a review). To understand these effects,
revealed that biologically emotional images induced stron-
researchers have focused on two orthogonal dimensions of
ger activity in the visual cortex and greater functional
emotion (Anderson,Christoff,Stappen etal.,2003;Russell
connectivity between the amygdala and visual cortex than
& Carroll, 1999): arousal (how exciting or calming) and
did socially emotional images. These results suggest that
valence (how positive and negative). Studies based on this
two-dimensional approach demonstrate the importance of
both arousal and valence.
This research was supported by Grant-in-Aid for JSPS Fellows Intermsofvalence,positiveandnegativeemotionsdiffer
(19–8978) and by a grant from the National Institute on Aging
in how they affect various kinds of cognitive processing:
(R01AG025340).
memory encoding (Kensinger, 2009; Mather & Carstensen,
Electronicsupplementarymaterial Theonlineversionofthisarticle 2005; Mickley & Kensinger, 2008; Ochsner, 2000; Talmi,
(doi:10.3758/s13415-011-0062-x)containssupplementarymaterial,
Schimmack, Paterson, & Moscovitch, 2007), the scope of
whichisavailabletoauthorizedusers.
: attention (Fenske & Eastwood, 2003; Fredrickson &
M.Sakaki(*) M.Mather
Branigan,2005;Rowe,Hirsh,&Anderson,2007),cognitive
UniversityofSouthernCalifornia,
flexibility (Isen & Daubman, 1984; Isen, Johnson,Mertz,&
3715McClintockAve.,
LosAngeles,CA90089,USA Robinson, 1985), creative problem solving (Isen, Daubman,
e-mail:[email protected] & Nowicki,1987; Subramaniam, Kounios,Parrish,& Jung-
Beeman,2009),cognitivecontrol(Dreisbach,2006),knowl-
K.Niki
edge retrieval (Bäum & Kuhbandner, 2007), and perceptual
NationalInstituteofAdvancedIndustrialScienceandTechnology,
Tsukuba,Ibaraki,Japan processing (Kuhbandner et al., 2009).
116 CognAffectBehavNeurosci(2012)12:115–139
Arousalalsohaseffectsonmemoryandotheraspectsof could have a different intention, goal, or emotional
cognitive processing. People have enhanced memory for feeling, depending on the contexts and other individuals.
emotionally arousing materials (Bradley, Greenwald, Petry, Because of the complex nature of social situations, the
&Lang,1992;Dolcos&Denkova,2008;Dolcos,LaBar,& meanings,outcomes,andcausesofemotionareembedded
Cabeza, 2004; Hamann, Ely, Grafton, & Kilts, 1999; in each stimulus context.
Kensinger & Corkin, 2004) and their intrinsic features These considerations suggest the following hypothesis:
(D’Argembeau & Van der Linden, 2004; Doerksen & Emotional materials related to survival/reproduction (bio-
Shimamura, 2001; Kensinger & Corkin, 2003a; Mather, logically emotional materials) imply clear meanings and
2007; Mather & Nesmith, 2008; Nashiro & Mather, 2011), direct physical outcomes, and therefore, their emotional
but arousal does not either enhance or impair memory nature can be detected even with just automatic processing
information peripheral to the emotional aspect of an event (seeFig.1a).Incontrast,emotionalmaterialslessrelatedto
(Kensinger, 2009; Kensinger, Garoff-Eaton, & Schacter, survivalorreproductionandmorestronglyrelatedtosocial
2007;Mather,Gorlick,&Nesmith,2009;Payne,Stickgold, life (socially emotional materials) have ambiguous mean-
Swanberg, & Kensinger, 2008). Highly arousing stimuli ings and outcomes shaped by their context. Therefore,they
also recruit attention (Anderson, 2005; Schimmack, 2005), needtobeinterpretedbyeachindividualineachcontextin
which interrupts cognitive processing of competing less- order to elicit an emotion. Thus, socially emotional
salient stimuli (Arnell, Killman, & Fijavz, 2007; Dolcos & materials should require effortful cognitive processing to
McCarthy, 2006; Ihssen, Heim, & Keil, 2007; Ihssen & elicit emotions and any subsequent effects of emotion on
Keil, 2009; Keil & Ihssen, 2004; Mather et al., 2006; K. J. cognition (see Fig. 1b). The present study aims to address
Mitchell, Mather, Johnson, Raye, & Greene, 2006; Most, this hypothesis.
Chun, Widders, & Zald, 2005). In general, emotional
arousalmodulatescognitiveprocessing,enhancingprocess-
ing of salient stimuli while reducing processing of Effects of survival or reproduction relevance
nonsalient stimuli (Mather & Sutherland, 2011; Suther-
land & Mather, 2011). Consistent with the hypothesis, past studies have
Althoughthistwo-dimensionalapproachcanaccountfor reported preferential processing of survival-relevant
many effects, arousal and valence may not be sufficient to stimuli. People tend to detect threats to survival (such
explain all of the effects of emotion on cognition. One as a gun or a snake) more automatically than other
possibly important factor that past studies have not neutral stimuli (e.g., Blanchette, 2006;Brosch&Sharma,
addressed well is the motivational relevance (see Larson 2005; Carlson, Fee, & Reinke, 2009; Fox, Griggs, &
&Steuer,2009,forrelatedarguments).Emotionalreactions Mouchlianitis, 2007; Öhman, Flykt, & Esteves, 2001).
are often induced by stimuli related to primary motives, Similar tendencies were reported in preschool children
such as survival (e.g., foods; Lang et al., 1998; Morris & (LoBue & DeLoache, 2008) and infants from 5 to
Dolan, 2001) and reproduction (e.g., sexual images; 18 months old (DeLoache & LoBue, 2009; LoBue &
Hamann, Herman, Nolan, & Wallen, 2004). However, DeLoache, 2008, 2010; Rakison & Derringer, 2008),
people can also feel emotions when they encounter social suggesting the possibility that humans have an innate
stimuli that are not directly related to survival or reproduc- predispositiontoprocesssurvival-relatedstimulipreferentially.
tion (e.g., Eisenberger, Lieberman, & Williams, 2003; Althoughpastresearchhasfocusedmostlyontheeffects
Singer et al., 2004). These two kinds of emotional stimuli ofthreateningstimuli,recentresearchhasdemonstratedthat
might influence cognitive processing in different ways. the effects of survival relevance are not limited to negative
For example, many studies on the effects of emotion stimuli. Positive stimuli related to reproduction, such as
on cognition have used picture stimuli obtained from sexual materials (Lykins, Meana, & Kambe, 2006) and
the International Affective Picture System (IAPS: Lang, babies’ faces (Brosch, Sander, Pourtois, & Scherer, 2008;
Bradley, & Cuthbert, 1997). The IAPS involves arousing Brosch, Sander, & Scherer, 2007), captured people’s
pictures depicting sexual behaviors, severe injuries, dead attention automatically. Likewise, infants’ faces more
bodies, or aimed guns. These stimuli are highly related to quickly induced strong activity in reward-related regions
survivalorreproduction,representingsituationsthatimply inthebrainthandidadults’faces(Kringelbachetal.,2008;
direct physical outcomes (either positive or negative), Nitschke et al., 2004). There is also evidence that basic
suchasdeath,injuries,andsexualexperiences.Incontrast, motives, such as hunger or thirst, produce preferential
other pictures in the IAPS are less relevant to survival or processing of positive reinforcers that satisfy those needs
reproduction and are more relevant to social life (e.g., (Drobes et al., 2001; Mogg, Bradley, Hyare, & Lee, 1998;
smiling children, crying people). These stimuli represent Morris & Dolan, 2001). Furthermore, studies have demon-
social situations with other individuals, each of whom strated preferential attention and memory encoding for
CognAffectBehavNeurosci(2012)12:115–139 117
Fig. 1 Proposed mechanisms (a) (b)
bywhichbiologicallyand
sociallyemotionalstimulimod-
ulatecognitiveprocessing.(a) Biologically emotional stimuli Socially emotional stimuli
Biologicallyemotionalstimuli (survival- or reproduction- (stimuli relevant to
implyclear/directphysicalout- related stimuli) social life)
comes.Therefore,theiremo-
tionalnaturecanbedetected
evenwithjustautomaticpro-
cessing,andtheycanmodulate
cognitiveprocessingwithout Automatic processing Elaborative Processing
elaborativeprocessing.(b)In
contrast,sociallyemotional
stimulihaveambiguousmean-
ingsandoutcomes.Thus,each
socialstimulushastobeinter-
pretedbyeachindividualin
Emotional reaction Emotional reaction
eachcontextinordertoelicitan
emotionandtomodulatecogni-
tiveprocessing
Cognitive processing Cognitive processing
(e.g., attention, memory) (e.g., attention, memory)
taboo words, as compared with other emotional words 2004).Althougheachstudydealtwithadifferentaspectof
(Anderson, 2005; Jay, Caldwell-Harris, & King, 2008; social emotion, such as social pain, moral judgments,
Kensinger & Corkin, 2003b; MacKay et al., 2004; empathy, or compassion, they each suggested differences
Mathewson, Arnell, & Mansfield, 2008). Although taboo in underlying processing between biologically and socially
words are defined by social norms, many of them refer to emotionalstimuli.
sexual acts or body products (Foote & Woodward, 1973; For example, observing others’ physical pain induced
Jay, 2009). Thus, it appears that people can process earlier activity in emotion-related regions in the brain
survival-/reproduction-related stimuli preferentially even than did observing others’ social pain (Immordino-Yang
after they are converted to an abstract/verbal format. et al., 2009). Studies also revealed that biologically
Insummary,paststudiessuggestthatmaterialsrelatedto emotional images (e.g., sexual images or mutilation)
survival or reproduction receive preferential processing. modulate activity in the visual cortex (Bradley et al.,
Most of these studies, however, did not compare survival 2003), skin conductance response, and startle reflex
relevance with social life relevance. Thus, it is unclear (Bradley, Codispoti, Cuthbert, & Lang, 2001) more
whether survival relevance impacts cognitive processing strongly than do socially emotional images (e.g., happy
more automatically than does social relevance. families).Inaddition,processingofbiologicallyemotional
films(i.e.,apizzacommercial,woundedbodies)produced
greater activity in brain regions involved in visceral
Social cognitive neuroscience studies on biological responses (Britton et al., 2006), while processing of
versus social emotions socially emotional stimuli (e.g., comedy show, poignant
bereavementscene,picturesinvolvingpeopleorfaces)has
Recent studies in social cognitive neuroscience have been associated with cortical regions that implement
started to examine how biological and social emotions higher order cognitive processing (Britton et al., 2006;
are processed in the brain (Eisenberger & Lieberman, Norris, Chen, Zhu, Small, & Cacioppo, 2004). These
2004; Eisenberger et al., 2003; Immordino-Yang, McColl, results seem consistent with our hypothesis, suggesting
Damasio,&Damasio,2009;Jackson,Meltzoff,&Decety, that people process the emotional implications of biolog-
2005; Moll, de Oliveira-Souza, Bramati, & Grafman, ically emotional stimuli automatically but engage in more
2002; Moll, de Oliveira-Souza, Eslinger et al., 2002; elaborative processing when presented with socially
Moll, Eslinger, & Oliveira-Souza, 2001; Singer et al., emotional stimuli.
118 CognAffectBehavNeurosci(2012)12:115–139
Since these studies targeted physical or neural the stimulus type (i.e., biological or social stimuli) and
responses induced by biological and social emotional arousal or valence.
stimuli, however, it is not clear whether relevance to
survival or reproduction versus relevance to social life
affect cognitive processing (e.g., attention or memory Study 1
encoding) differently. In addition, there has been no
clear agreement about how to define biologically and InStudy1,weexaminedtheeffectsofbiologically/socially
socially emotional stimuli, leading different studies to emotional stimuli on attention. Previous studies revealed
use different ways to categorize biologically and that emotional stimuli have more impact on attention
socially emotional stimuli. Furthermore, most of the disengagement than on attention capture (Fox, Russo,
past studies did not match arousal and valence between Bowles, & Dutton, 2001; Fox, Russo, & Dutton, 2002),
social versus biological emotional materials. Thus, it is especially when they compete with other stimuli for
unclear whether the results were due to the social/ attention (Anderson, 2005; Buodo, Sarlo, & Palomba,
biological nature of stimuli or arousal/valence. 2002; Pratto & John, 1991; Schimmack, 2005). On the
basis of these findings, we investigated the effects of
biologically and socially emotional stimuli on attention
Overview of the present study disengagement, by using a dot-probe task combined with
other competing stimuli. To enhance participants’ engage-
We tested our hypothesis that, as compared with socially ment,aproblem-solvingtaskwasemployedasacompeting
emotional stimuli, biologically emotional materials impact task (see Schimmack, 2005, for a similar procedure).
cognitive processing more automatically, using several On each trial (Fig. 2a), participants were shown a
different paradigms. In Study 1, we compared the effects riddle, which was followed by either a biological, social,
of biological versus social emotional materials on attention neutralpictureorasterisksatthecenterofadisplay.After
and memory encoding. In Study 2, we further examined 150 ms of the picture or the asterisks, participants saw a
memory encoding of biological and social emotional dot-probe at one of eight possible locations (Fig. 2b), all
materials, manipulating cognitive resources available at of which were different from the picture’s location. The
encodingtoseewhethercognitiveresourcesaffectmemory participants’ task was to indicate the location of the dot-
encoding of biologically and socially emotional materials probe as quickly and as accurately as possible. If
differently. Finally, in Study 3, we investigated the brain participants have more difficulty disengaging their atten-
regions associated with processing biological and social tion from biologically emotional pictures than from
emotional stimuli by using functional magnetic resonance socially emotional ones, their reaction times to detect the
imaging (fMRI). dot-probe should be slower after biological pictures than
Given the absence of clear definitions about biolog- aftersocialones.Study1alsoincludedasurprisememory
ical and social emotional materials in the literature, test of pictures to examine the effects of biologically/
across all studies, we adopted a bottom-up approach to socially emotional stimuli on memory, as well as
define biologically and socially emotional stimuli to attention.
reduce any arbitrary biases introduced by the research-
ers. That is, we selected stimuli on the basis of people’s Method
ratings on relevance to survival/reproduction and rele-
vance to social life. Although there might be emotional Participants Twenty-twoJapaneseundergraduateandgrad-
stimuli rated high in relation to survival/reproduction uate students at the University of Tokyo took part in the
and to social adaptation, the primary purpose of the experiment (14 males; M = 21.09, SD = 2.36).
age
present study is to identify the effects of biological and
social relevance separately. Therefore, as biologically Materials: Pictures On the basis of pilot ratings (see the
emotional stimuli, we used stimuli rated high in relation Supplementary Materials), we selected 10 biological (5
to survival/reproduction and low in relation to social positive, 5 negative), 10 social (5 positive, 5 negative),
life. Socially emotional stimuli were also defined as and 10 neutral pictures. Biological and social pictures
those rated high in relation to social life but low in were matched in arousal and valence (see S-Table 1).
relation to survival/reproduction. We also carefully Examples for each category were sexual images or
matched arousal and valence across the biologically appetizing food for biological positive pictures; a snake,
and the socially emotional stimuli to avoid confounding a skull, or a man who commits suicide for biological
CognAffectBehavNeurosci(2012)12:115–139 119
Fig. 2 (a) Schematic represen- (a)
tationofproceduresinStudy1.
Oneachtrial,participants
viewedariddleandthensawa
picture(inthebiological,social,
What is so fragile that when
andneutralconditions)oraster-
you say its name you break it?
isks(inthecontrolcondition).
After150msofthepictureor
theasterisks(150ms),thered
dotappearedatoneofeight
possiblelocations.Participants
wereaskedtoindicatetheloca-
tionofthedotasquicklyandas
accuratelyaspossible.Immedi-
atelyaftertheyansweredthe
correctlocationofthedot,the
dotwasreplacedbythesolution
totheriddle.(b)Eightpossible
locationsforthedotprobeand Silence
thecorrectkeyresponseforeach
ofthem
(b)
LEFT MIDDLE RIGHT
KEY KEY KEY
negative ones; smiling people, celebrating athletes, or Results
money for social positive ones; KKK or neo-Nazi for
social negative ones. The recognition test included an Effectsofpicturesonreactiontimetodetectthedot-probe In
additional 5 biological positive, 5 biological negative, 5 this and the following studies in the present article,
social positive, 5 social negative, and 10 neutral pictures outlier response times were identified using Tukey’s
as foils. (1977) criterion of three times the interquartile range (the
hinge-spread) higher than the third quartile in each
Procedure We employed a modified version of the dot- condition. The remaining reaction times were submitted
probe task to examine the effects of biological and social to a one-way analysis of variance (ANOVA).1 This
emotional stimuli on attention (Fig. 2; see the Supplemen- ANOVA revealed a significant main effect of stimulus
tary Methods for details). After participants finished the type, F(3, 63) = 7.74, R2 = .01, p < .01. Post hoc
attention task, they worked on a mathematical calculation comparisons (Tukey’s HSD) revealed that reaction times
task for 5 min, which was followed by a surprise
recognition test for the pictures. In the memory test,
0Becauseparticipantspressedtheleftkeymorequicklythantheother
participants were shown pictures used in the attention task
two keys(ps< .01), response key (left, middle, right) was used as a
and new pictures and were asked to indicate whether they
covariate. The response key did not modulate the effects of the
saw each picture or not. stimulustype(p>.80).
120 CognAffectBehavNeurosci(2012)12:115–139
were significantly longer after biological pictures than neutralones(M=.19),respectivets(42)=3.13,4.15,SEs=
aftersocialpictures,neutralpictures,orasterisks(Fig.3a), .05, ps < .05. There was no significant difference for
respectivets(63)=2.78, 3.30,4.44, SEs=10,ps< .05.In biological and social pictures (p > .40). Further analyses
contrast, the reaction time did not differ across social, revealed no significant effects of valence and arousal (see
neutral, and control conditions (ps > .30). In addition, the the Supplementary Results).2
difference between biological and social pictures was not
modulated by valence and arousal (Fig. 3b; see the
Supplementary Results for details). These results suggest Discussion
that people have more difficulty disengaging their atten-
tion from biologically emotional pictures than from Study 1 revealed that biologically emotional stimuli hold
socially emotional pictures, regardless of valence and attention more than do socially emotional stimuli, regard-
subjective arousal. lessofvalenceandarousal.Likethebiologicallyemotional
pictures,sociallyemotionalpictureshadhigherarousalthan
Memory of pictures Next, we examined participants’ did neutral pictures—yet the social pictures did not slow
memoryforbiologicallyversussociallyemotional pictures. reaction times more than neutral pictures did. Thus, the
A one-way repeated measures ANOVA on the hit rates of relevancetosurvivalorreproductionhadalargerimpacton
pictures revealed a significant effect of picture type, F(2, attention than did arousal or valence. In contrast, partic-
42) = 8.77, R2 = .19, p < .01. Post hoc Tukey’s HSD tests ipantsrememberedbothbiologicallyandsociallyemotional
revealed that participants remembered biological (M = .33) stimulisignificantlybetterthanneutralstimuli.Infact,they
and social (M = .39) pictures significantly better than remembered socially emotional stimuli as well as biologi-
cally emotional stimuli. These results suggest that memory
for socially emotional stimuli and memory for biologically
(a) emotional stimuli are facilitated through different mecha-
nisms. Study 2 addressed this possibility.
500
One possible concern about Study 1 is that each person
S) 450 might have different evaluations about what is related to
M
ME ( 400 BIOLOG social adaptation and what is related to survival/reproduc-
TI SOCIAL tion,dependingonhis/herexperiences.Inaddition,Study1
ON 350 NEUTRAL mightnothaveenoughstatisticalpowertodetecttheeffects
CTI 300 CONTROL of arousal and valence, because of the relatively small
A
E number of trials in each condition. The statistical power
R 250
issue was exacerbated by the fact that dichotomization of
200 continuous variables (i.e., arousal and valence in this case)
results in the loss of the statistical power (Irwin &
(b) McClelland, 2003). However, in a supplemental study (S-
Study 1; see the Supplementary Studies for details), we
500
S) replicated the results from Study 1 while determining each
M 450 picture’s biological/social relevance, arousal, and valence
ME ( 400 byeachparticipant’sevaluation.Inthissupplementalstudy,
N TI BIOLOG we treated arousal and valence as continuous variables to
O 350 SOCIAL
TI increase the statistical power; as in Study 1, we found a
C
A 300 significanteffectofbiologicalrelevance,butnotofvalence
E
R and arousal (S-Fig. 1). Thus, this supplemental study
250
200
NEG POS HIGH LOW
VALENCE AROUSAL 0Thefalsealarmrate(M=.06)wasnotsignificantlydifferentacross
stimulustype(p>.40).Inaddition,asimilaranalysisonthecorrected
Fig. 3 Effects of stimulus type (biologically vs. socially emotional recognitionmeasure(i.e.,hitrateminusfalsealarmrate)confirmeda
stimuli) onattention in Study 1.Error barsrepresent standarderrors. significanteffectofstimulustype,F(2,42)=8.83,R2=.17,p<.01;
(a) Reaction times to detect the dot-probe were slower after participants had better memory for biological (M=.27)andsocial
biologically emotional pictures than in other conditions, while the (M = .34) stimuli than for neutral stimuli (M = .10), ts(42) = 2.95,
reaction times did not differ across social, neutral, and control 4.20,ps<.05.Thedifferencebetweenbiologicalandsocialstimuli
conditions. (b) The valence category and subjective arousal did not wasnotsignificant(p>.25).Thus,hitrateandcorrectedrecognition
modulatetheresultsinreactiontimes measuresyieldedthesamepatternofresults.
CognAffectBehavNeurosci(2012)12:115–139 121
providesadditionalsupportfortheimportanceofbiological differences between remember/know and old/new judg-
relevance in attention. ments in the recognition test, and the other did not press
keys during the encoding session.
Study 2 Materials On the basis of the pilot picture-rating study, we
chose32biological(16positive,16negative),32social(16
Past research suggests two different mechanisms by which positive, 16negative), and32 neutralpictures. Biologically
emotionenhancesmemory(Anderson,Christoff,Panitz,De and socially emotional pictures were selected to have
Rosa, & Gabrieli, 2003; Kensinger & Corkin, 2004; Talmi matched valence and arousal levels (see S-Table 1). In the
et al., 2007): (1) Emotional materials tend to attract and recognition task, 78 nonstudied foils (13 biological posi-
holdattentionmoreautomaticallythandoneutralmaterials, tive, 13 biological negative, 13 social positive, 13 social
and (2) emotional materials recruit more effortful semantic negative, and 26 neutral) were used.
elaboration than do neutral materials. In Study 1, we found
that biologically emotional stimuli held attention more Procedure Participants saw each of 96 pictures for
strongly than socially emotional ones when the pictures 2,500 ms in a randomized order with a 4-sec intertrial
were not relevant to the primary task. This automatic interval. The encoding session consisted of four blocks. In
capture of attention by biologically emotional stimuli, but both the full- and divided-attention conditions, participants
not socially emotional stimuli, suggests that memory for wereaskedtomakeajudgmentaboutwhethertheylikedor
biologically emotional stimuli would depend on the disliked each picture as quickly and as accurately as
automatic attention mechanism more than would socially possible, using their right hand to press keys.
emotional stimuli. In contrast, if each socially emotional In addition to this picture judgment task, participants in
stimulus has to be interpreted by each individual in each thedivided-attentionconditionlistenedtoasoundsequence
context to elicit an emotion (as we posited above), the and worked on a sound-pitch judgment task (Gilbert &
semantic elaborative process should be more crucial in Silvera, 1996) throughout the session. The sound sequence
memory for socially emotional stimuli than for biological consisted of a low-pitched, a medium-pitched, and a high-
stimuli. Study 2 addressed these predictions. pitched tone in randomized order with variable intervals
Half of the participants in Study 2 viewed emotional or (approximately 1–3 s). Participants’ task was to keep track
neutral pictures while working on a secondary task of the sound sequence and to press a button with their left
(divided-attentioncondition),whereastheotherhalfviewed handimmediatelyaftertheydetectedaspecificsequenceof
pictures without any additional task (full-attention condi- three tones: low-, medium-, and high-pitched tones in that
tion). This encoding session was followed by a surprise order, regardless of interval duration.
recognitiontestofpicturememory.Toexaminehowvividly The encoding session was followed by a mathematical
participants remembered each picture, we included remem- calculationtaskfor3min.Finally,participantsweregivena
ber/know judgments (Gardiner, Ramponi, & Richardson- surprise recognition test for the pictures. They indicated
Klavehn, 1998), as well as old/new judgments. If memory whether they remembered seeing the picture (i.e., old) or
encoding of socially emotional materials depends on not (i.e., new). For any item that received an old decision,
effortful elaborative processing, participants should show participants were asked to indicate whether they vividly
worse memory for social stimuli in the divided-attention remembered seeing the picture in the first session (i.e.,
condition than in the full-attention condition. In contrast, if remember) or sensed that the picture was familiar but did
memory for biologically emotional materials is enhanced not remember any details about its prior presentation (i.e.,
through automatic attention mechanisms, participants’ know).
memory for biologically emotional pictures should be less
influenced by the attention manipulations than their
memory for socially emotional stimuli. Results
Method Effects of attention and stimulus type on memory A 2
(attention: full vs. divided) × 3 (stimulus type: biological,
Participants Forty-eight Japanese undergraduate and grad- social, or neutral) ANOVA on the hit rate3 revealed
uate students at the University of Tokyo participated (21
males; M = 22.19 years, SD = 2.03). They were
age 0Becauseparticipantsreportedbettermemoriesforpicturesinthelast
randomly assigned to either the full-attention (n = 25) or
two blocks than in the first two blocks (ps < .01), we added block
the divided-attention (n = 23) condition. Data from 2
order as a covariate. There were no significant interactions between
participants were not used; one did not understand the blockorderandstimulustype.
122 CognAffectBehavNeurosci(2012)12:115–139
significant effects of attention, F(1, 44) = 9.66, p < .01, p < .05, and neutral , t(44) = 2.21, SE = .04, p < .05,
R2=.18,andstimulustype,F(2,88)=5.53,p<.01,R2= stimuli, and there was no significant difference between
.33, and an interaction between attention and stimulus social and neutral stimuli (p > .95).4 These results suggest
type, F(2, 88) = 3.36, p < .05, R2 = .17. The attention that cognitive resources contribute more to encoding
manipulation had a significant effect for social,F(1, 88) = socially emotional materials than to encoding biologically
16.08, p < .01, and neutral, F(1, 88) = 4.31, p < .05, emotional materials.
stimuli, but not for biological stimuli (p > .10; Fig. 4a).
Participants in the full-attention condition remembered Effects ofattention andstimulustype onremember rates A
biological and social stimuli significantly better than similar analysis on the proportion of remember responses
neutral stimuli, ts(44) = 3.16, 3.94, SEs = .02, ps < .01, also revealed significant effects of attention, F(1, 44) =
with no significant difference between biological and 5.75, p < .05, R2 = .10, and of stimulus type, F(2, 88) =
social stimuli (p > .60; Tukey’s HSD). In contrast, 8.28, p < .01, R2 = .22, and an interaction between them,
participantsinthedivided-attentionconditionremembered F(2, 88) = 5.27, p < .01, R2 = .10. The attention
biologicalstimulibetterthansocial,t(44)=2.39,SE=.04, manipulation had a significant effect on memory for social
and neutral stimuli, Fs(1, 88) = 11.35, 4.33, ps < .05, but
(a) not for biological stimuli (p > .30; Fig. 5a). Participants in
the full-attention condition were more likely to vividly
1
remember social stimuli than biological or neutral stimuli,
0.8 ts(44)=3.35,4.53,SEs=.02,.03,ps<.01,whiletherewas
no significant difference between biological and neutral
0.6 BIOLOG
HIT SOCIAL stimuli (p > .20; Tukey’s HSD). When participants’
0.4 NEUTRAL attention was focused elsewhere in the divided-attention
condition,however, they did not show enhanced remember
0.2
rates for social stimuli, as compared with neutral or
0 biological stimuli (ps > .20). Instead, they produced a
FULL DIVIDED greater proportion of remember responses to biological
(b) stimuli than to neutral stimuli, t(44) = 3.16, SE = .03, p <
1 .05. Thus, it appears that vivid memory for socially
0.8 emotional stimuli depends more on cognitive resources
thandoesvividmemoryforbiologicallyemotionalstimuli.5
0.6 FULL
T
HI DIVIDED
0.4
Like/dislike judgments in the encoding session To examine
0.2 whether the attention manipulation influenced like/
dislike judgments during the encoding session, the
0
POS NEG POS NEG reaction times of like/dislike judgment swere submitted
to a 3 (stimulus type) × 2 (attention) ANOVA. This
BIOLOG SOCIAL
(c)
1
0A similar analysis on the false alarm rates did not find a significant
0.8
interactionbetweenstimulustypeandattention(p>.45).Theaccuracy
0.6 measureofrecognition(i.e.,hit–false-alarmrates)alsoproducedsimilar
HIT patternstothehitrates;participants’memorywassignificantlyimpaired
0.4 in the divided-attention condition, as compared with the full-attention
0.2 condition,forsociallyemotionalstimuli,F(1,88)=17.37,p<.01,but
notforbiologicallyemotionalstimuli(p>.05).
0 0The remember rates to new items did not show a significant
HIGH LOW HIGH LOW interaction between stimulus type and attention (p > .70). The
BIOLOG SOCIAL corrected remember rates (i.e., remember rates to old items minus
remember rates to new items) also produced similar patterns; vivid
Fig.4 ResultsofthehitratesfromthepicturememorytestinStudy memory for socially emotional pictureswas impaired inthe divided-
2.Errorbarsrepresentstandarderrors.(a)Whenpeoplehaveenough attentioncondition,ascomparedwiththefull-attentioncondition,F(1,
cognitive resources at encoding, they remembered both biologically 88) = 10.84, p < .01, while there were no significant differences
andsociallyemotionalstimulibetterthanneutralstimuli.Incontrast, between divided- and full-attention conditions in memory of biolog-
when participants’ attention was focused elsewhere, it impaired their icallyemotionalpictures(p>.05).A2(attention)×3(stimulustype)
memoryforsocialstimuli,butnotforbiologicalstimuli.(b)Valence ANOVA on the proportion of know responses did not reveal a
and(c)subjectivearousaldidnotmodulatethepatterns significantinteractionbetweenattentionandstimulustype(p>.30).
CognAffectBehavNeurosci(2012)12:115–139 123
(a)
make like/dislike judgments than do biologically emotional
1
stimuli.
0.8 We also calculated the proportion of pictures for which
R the participants’ judgments were consistent with the
E 0.6 BIOLOG
EMB SOCIAL valence categories (i.e., liked positive pictures and disliked
EM 0.4 NEUTRAL negative pictures), to examine the accuracy of participants’
R
judgments. This consistency measure was submitted to a 2
0.2
(attention) × 2 (stimulus type: social vs. biological)
0 ANOVA. Because it is hard to define correct judgments
FULL DIVIDED for neutral pictures, we did not involve neutral pictures in
(b) this analysis. The ANOVA revealed a main effect of
1
stimulustype,F(1,44)=4.20,p<.05,R2=.06,indicating
0.8 that participants were more accurate when they made
R
MBE 0.6 FULL judgments about biological pictures than about social ones
E (M =.76vs.M =.72).Noneoftheothereffectswere
M DIVIDED bio social
E 0.4
R significant (ps > .20).
0.2
0
POS NEG POS NEG Discussion
BIOLOG SOCIAL
When participants had enough attention (i.e., full-attention
(c) condition), socially emotional stimuli were remembered as
1
wellasbiologicallyemotionalstimuli.Inaddition,whenwe
0.8 looked at remember rates, socially emotional stimuli
R produced even higher remember rates than did biological
BE0.6
M stimuli. However, dividing attention impaired memory for
E
M0.4 socially emotional stimuli more than for biologically
E
R
emotional stimuli. As a result, in the divided-attention
0.2
condition, biologically emotional stimuli were more likely
0
to be recognized than socially emotional stimuli. Further
HIGH LOW HIGH LOW
analyses confirmed similar patterns irrespective of valence
BIOLOG SOCIAL
and arousal (Figs. 4b, c; 5b, c; see the Supplementary
Fig.5 Resultsoftherememberratesfromthepicturememorytestin Results for details). These results suggest that memory for
Study2.Errorbarsrepresentstandarderrors.(a)Attentionalresources socially emotional stimuli is enhanced through effortful
influencedencodingdetailedmemoriesforsociallyemotionalstimuli
elaboration, while memory for biologically emotional
more than for biologically emotional stimuli; although social stimuli
stimuli is enhanced through automatic attention allocation,
producedhigherrememberratesthandidbiologicalstimuliinthefull-
attention condition, dividing attention impaired memory for socially regardless of subjective arousal and valence.
emotional stimuli, but not for biologically emotional stimuli. Similar We also found that during the encoding session,
resultswereobtainedregardlessof(b)valenceand(c)subjectivearousal
participants took longer to make like/dislike judgments for
socially emotional stimuli than for biological stimuli. This
ANOVA revealed a significant main effect of stimulus result is also consistent with our hypothesis that the
type, F(2, 88) = 26.24, p < .01, R2 = .02, with no other affective nature of biologically emotional stimuli can be
significant effects (ps > .30). Participants took longer to detected more automatically than the affective nature of
make judgments about neutral pictures (M = 1,178 ms) socially emotional stimuli.
than about social (M = 1,092 ms) and biological (M =
1,048ms;Tukey’sHSD)pictures,ts(88)=4.12,7.12,SEs=
18,ps<.01.Thisresultisnotsurprising,becausewedidnot Study 3
allow participants to judge pictures as neutral in their like/
dislike judgments. More interestingly, participants took Study 3 examined the neural mechanisms underlying
longer to make a decision about social pictures than about processing of biologically and socially emotional stimuli.
biologicalpictures(M =1,092msvs.M =1,048ms), Given previous evidence showing that the amygdala
social bio
t(88) = 2.40, SE = 18, p < .05, suggesting that socially responds to both social and nonsocial emotional stimuli
emotional stimuli require deeper cognitive processing to (e.g.,Adolphs,2003;Brittonetal.,2006;Moll,deOliveira-
124 CognAffectBehavNeurosci(2012)12:115–139
Souza, Eslinger et al., 2002; Norris et al., 2004), we emotional words recruit different brain regions from
expected that both biologically and socially emotional biologically emotional pictures but induce similar activa-
stimuli would induce similar activity in the amygdala. tion patterns to socially emotional stimuli, while socially
According to the results from Studies 1 and 2 and our emotional stimuli activate similar brain regions, regardless
hypothesisasoutlinedinFig.1,however,wealsoexpected of the stimulus format. By including biologically and
different brain regions activated and different functional socially emotional words, we addressed these predictions.
connectivitywiththeamygdaladependingonstimulustype
(i.e., biologically and socially emotional stimuli). Method
Past studies revealed that the amygdala is reciprocally
interconnected with the visual cortex (Amaral, Behniea, & Participants Sixteen Japanese University of Tsukuba un-
Kelly, 2003; Emery & Amaral, 2000) and that these two dergraduate and graduate students took part in the exper-
brain regions influence each other, which results in iment(12males;M =21.2years,SD=1.78).Theygave
age
enhancedperceptualprocessingofemotionalstimuli(Bradley informed consent in accordance with the MRI ethics
etal.,2003; Schupp, Junghöfer, Weike, & Hamm, 2003). If committee ofAIST.Prospectiveparticipants wereexcluded
biologically emotional images hold visual attention more iftheyhadanymedical,neurological,orpsychiatricillness.
strongly than do socially emotional images, as revealed in One participant judged more than 70% of social positive
Study 1, biologically emotional pictures should produce pictures and more than 60% of social negative pictures to
greateractivityinthevisualcortexandstrongerconnectivity be neutral during the task. Data from this participant were
between the amygdala and visual cortex than do socially not included.
emotional pictures.
In contrast, previous research has suggested that the Materials: Emotional pictures Twenty-eight biologically
medialprefrontalcortex(MPFC)implementsmetacognitive emotional pictures (14 positive, 14 negative), 28 socially
orelaborativeoperationsaboutemotionalaspectsofstimuli emotional pictures (14 positive, 14 negative), and 28
(Amodio & Frith, 2006; Cunningham, Johnson, Gatenby, neutral pictures were selected on the basis of the pilot
Gore, & Banaji, 2003; Ochsner, Knierim et al., 2004). If picture-rating study. Social and biological pictures were
sociallyemotionalstimulitriggermoreelaborativeprocess- matched not only in arousal and valence (see S-Table 1),
ing than do biologically emotional stimuli, MPFC should but also in luminance (measured by Adobe Photoshop).
show stronger activation in response to socially emotional Neutral pictures were also matched in luminance to the
stimulithantobiologicallyemotionalstimuli.Inaddition,if emotional pictures. We also included 14 scrambled images
MPFCplaysacriticalroleinprocessingtheaffectivenature asnonsensestimuli.Theywerecreatedonthebasisofthree
of social stimuli, the amygdala should have stronger picturesfrombiologicalpositive,biologicalnegative,social
functional connectivity with the MPFC for socially emo- positive, and social negative pictures and the two neutral
tional stimuli than for biological stimuli. pictures used in the experiment.
Study 3 also employed both pictorial stimuli and verbal
stimuli to examine whether biologically and socially Materials: Emotional words On the basis of a pilot rating
emotional stimuli recruit similar brain regions regardless study of words (see Supplementary Materials), 28 biolog-
of the stimuli format (i.e., word or picture). Two different ical (14 positive, 14 negative), 28 social (14 positive, 14
predictions can be made concerning the brain regions negative), and 28 neutral words were chosen (see S-
underlying processing of words. First, if the conceptual Table 3). There were no differences in arousal or valence
meaning of the stimuli is critical, biologically emotional scores between the biological and social stimuli. In
stimuli would activate similar brain regions, regardless of addition, 14 nonsense words were used in the experiment.
the format of the materials. Similarly, socially emotional Biological, social, and neutral words were matched on
words and pictures would activate overlapping regions of familiarity obtained from published norms (Amano &
thebrain.Analternativepossibilityisthatoncebiologically Kondo, 2000).
emotional stimuli become abstract (i.e., verbal stimuli, in
this case), physical outcomes would be less evident than in Behavioral procedure In each trial, participants saw either
pictorial stimuli, and therefore, they might need similar a word or a picture for 1,800 ms. Words and pictures from
elaborative processing as socially emotional stimuli to thedifferent emotion categorieswererandomlyintermixed,
evoke emotional reactions. In contrast, socially emotional with interstimulus intervals ranging from 6 to 8 s. After
stimuli would not imply direct physical outcomes, regard- each stimulus disappeared, participants were asked to
less of the stimulus formats, and both socially emotional indicate whether they liked, disliked, or were neutral about
words and pictures would needsimilar elaborativeprocess- each stimulus by pressing buttons (see the Supplementary
ing. Thus, an alternative prediction is that biologically Results for behavioral results).