Reward and punishment learning in daily life

18 Pages·2017·7.88 MB·English

RESEARCHARTICLE
Reward and punishment learning in daily life:
A replication study
VeraE.Heininga1*,EeskevanRoekel1,2,MariekeWichers1,AlbertineJ.Oldehinkel1
1 DepartmentofPsychiatry,InterdisciplinaryCenterPsychopathologyandEmotionregulation,Universityof
Groningen,UniversityMedicalCenterGroningen,Groningen,Groningen,TheNetherlands,2 Departmentof
DevelopmentalPsychology,Tilburguniversity,Tilburg,Noord-Brabant,TheNetherlands
*[email protected]
a1111111111
a1111111111
a1111111111 Abstract
a1111111111
a1111111111 Day-to-dayexperiencesareaccompaniedbyfeelingsofPositiveAffect(PA)andNegative
Affect(NA).Implicitly,withoutconsciousprocessing,individualslearnabouttherewardand
punishmentvalueofeachcontextandactivity.Theseassociativelearningprocesses,in
turn,affecttheprobabilitythatindividualswillre-engageinsuchactivitiesorseekoutthat
OPENACCESS context.Sofar,implicitlearningprocessesarealmostexclusivelyinvestigatedincontrolled
laboratorysettingsandnotindailylife.Hereweaimedtoreplicatethefirststudythatinvesti-
Citation:HeiningaVE,vanRoekelE,WichersM,
OldehinkelAJ(2017)Rewardandpunishment gatedimplicitlearningprocessesinreallife,bymeansoftheExperienceSamplingMethod
learningindailylife:Areplicationstudy.PLoSONE (ESM).Thatis,usinganexperience-samplingstudywith90timepoints(threemeasure-
12(10):e0180753.https://doi.org/10.1371/journal.
mentsover30days),weprospectivelymeasuredtimespentinsocialcompanyandamount
pone.0180753
ofphysicalactivityaswellasPAandNAinthedailylivesof18-24-year-oldyoungadults
Editor:PabloBrañas-Garza,MiddlesexUniversity,
(n=69withanhedonia,n=69withoutanhedonia).Multilevelanalysesshowedapunish-
UNITEDKINGDOM
mentlearningeffectwithregardtotimespentincompanyoffriends,butnotareward
Received:January17,2017
learningeffect.Neitherrewardnorpunishmentlearningeffectswerefoundwithregardto
Accepted:August18,2017 physicalactivity.Ourstudyshowspromisingresultsforfutureresearchonimplicitlearning
Published:October4,2017 processesindailylife,withtheprovisoofcarefulconsiderationofthetimescaleused.Short-
termretrospectiveESMdesignwithbeepsapproximatelysixhoursapartmaysufferfrom
Copyright:©2017Heiningaetal.Thisisanopen
accessarticledistributedunderthetermsofthe mismatchnoisethathampersaccuratedetectionofassociativelearningeffectsovertime.
CreativeCommonsAttributionLicense,which
permitsunrestricteduse,distribution,and
reproductioninanymedium,providedtheoriginal
authorandsourcearecredited.
DataAvailabilityStatement:Allrelevantdataare
withinthepaperanditsSupportingInformation Introduction
files.
Emotionsarecentraltoeverydaylife,astheybringflavortoday-to-dayexperiences.These
Funding:Researchreportedinthispublicationwas flavorsguideindividualstodeterminetheirsubsequentactions.Imagine,forexample,toexpe-
fundedbyaVicigrantfromtheNetherlands
riencemomentsofjoyandexcitementwhilebeinginthecompanyoffriends.Withoutcon-
OrganizationforScientificResearch(NWOVici
sciousprocessing,theco-occurrenceofthiscontext(i.e.,friends)anditspositiveinternal
016.001/002;http://www.nwo.nl/)awardedtoAJO,
‘seasoning’(i.e.,feelingjoyfulandexcited)teachesyouthatspendingmoretimewithfriendsis
andtheBrainFoundationfellowshipgrant(F2012
(1)-03;https://wetenschap.hersenstichting.nl/ potentiallyrewarding.Asaresult,theprobabilitythatonewillseekoutthecompanyoffriends
subsidievormen/fellowships)awardedtoMW. againinthefutureincreases.
Theideathataffectiveexperiencesmotivateactionsthroughimplicitassociativelearning
Competinginterests:Theauthorshavedeclared
thatnocompetinginterestsexist. originatesfromoperantconditioningtheory.AccordingtoPavlovianorclassicalconditioning,
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 1/18
Rewardandpunishmentlearningindailylife
individualsimplicitly(i.e.,withoutconsciousprocessing)learnabouttheassociationsbetween
contextualcuesandinternalcuessuchastheconcurrentlevelsofaffect[1,2].Therewarding
orpunishingvalueofeachassociation,inturn,affectstheprobabilityofre-engagementinthat
activity[3,4].Althoughtheemotionalseasoningofdailylifeprovidesanimportantbiasfor
subsequentmotivatedaction,theseimplicitlearningprocessesareusuallystudiedinacon-
trolledlaboratorysettingandnotindailylife.
Inlaboratorysettings,implicitlearningprocessesareassessedbyexperimentaltasks,such
astheprobabilisticrewardtask[5];theprobabilisticselectiontask[6];andtheprobabilistic
reversallearningtask[7].Insteadofassociationsbetweencontextualcuesandpositiveornega-
tiveemotions,associationsbetweenstimuliandmonetarygainsorlossesarecreatedand
manipulatedtoinduceimplicitpreferencesforcertainstimuli.Forexample,whenstimulusA
issubliminallyrewardedwithmoremoneythanstimulusB(i.e.,withoutconsciousprocess-
ing),individualsarefoundtodevelopapreferenceforAoverB[8–10].Althoughthelabora-
torysettingfortheseexperimentsallowsforprecisecontrol,thestimuliandthemonetary
gainsandlossesareapaleshadowofrewardingorpunishingexperiencesindailylife.Most
importantly,itremainsunknownhowtheresponsebiasesfoundinlaboratorytaskstranslate
tomotivatedactionsinreal-life.
TheExperienceSamplingMethod(ESM)providesanexcellenttooltoinvestigatesuch
learningprocessesbecause,contrarytothesnapshotofanindividual’sresponsebiasinlabora-
torysettings,ESMallowsinvestigationofthephenomenaoveralongerperiodoftime.Thatis,
ESMenablesresearcherstoinvestigatesmallbutrecurrentbehavioralandemotionalchanges
acrosshours,daysorweeks,includingbehavioralconsequencesofrewardingandpunishing
experiences.Insteadofinvestigatingmonetarygainsandlossestoabstractstimuliinsimula-
tiontasks,trackingreal-lifepositiveandnegativeaffectiveexperiencestorealcontextualsti-
mulianddailyactivateswouldenableresearcherstoinvestigateresponsebiasesinthereal
world,withfindingsofhighecologicalvalidity.Takentogether,theapplicationofESMto
investigatereinforcementlearninginreallifeseemsverypromising.
Recently,Wichersandcolleagues[11]werethefirsttoshowthatimplicitreinforcement
learningprocessescanbestudiedinreal-lifebytheESM.Theyuseddataof621womenwho
werepagedat10semi-randomtimepointsaday,andassessedtheiremotionalstate,theextent
towhichtheywerephysicallyactive,andtowhatextenttheyappreciatedthecompanythey
wereinatthetimeofthebeep.PositiveAffect(PA)andNegativeAffect(NA)experiencedin
pleasantcompanyaffectedthedegreetowhichtheindividualswouldseekoutpleasantcom-
panythereafterinthisstudy.Hence,thesefindingsyieldedsupportforimplicitrewardand
punishmentlearningprocesseswithregardtoengagementinsocialcompany.Theassociations
werefoundbothacrossconsecutivebeepsandacrossconsecutivedays.Withregardtophysical
activity,Wichersandcolleagues[11]foundsimilarlearningeffects,butacrossathree-day
cycleinsteadofthenextbeeporday.Asapossibleexplanation,theauthorsproposedthatthe
bodyoftenneedsoneortworecoverydaysaftersports,duringwhichrenewedengagementin
physicalactivityisunlikely.
ThreecharacteristicsofthemethodsanddesignusedbyWichersandcolleagues[11]limit
theinterpretationoftheirfindings.Thefirstconcernsthesuboptimalmeasureofsocialcon-
text.Insteadoftheamountofsocialinteractionortimespentinacertainsocialcontext,social
companywasmeasuredastheextenttowhichparticipantslikedthecompanytheywerein.
ThelevelofappraisalofcompanywasfoundtopartiallyoverlapwithlevelsofPA(r=.30),
whichmayhaveinflatedthefoundrewardlearningeffectswithregardtosocialcompany.The
secondlimitationconcernsthetimescale.Thatis,theintervalsduringthedaywererelatively
short(onaverage90minutesapart),andtheactivitiesonbeep-levellikelyoftenreflectcontin-
uedratherthannewlystartedactivities.Finally,astheauthorsnoted,itwouldbehighly
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 2/18
Rewardandpunishmentlearningindailylife
informativetostudyrewardandpunishmentlearningprocessesinindividualssufferingfrom
mentalhealthproblemsinsteadofasampleofmerelyhealthyindividualsliketheirs.
TheaimofthecurrentstudywastoreplicatethisfirstExperienceSamplingMethod(ESM)
studyintoimplicitlearningprocessesinreallifewhileovercomingthelimitationsdiscussed
above.Insteadofameasureofsocialbehaviorthatisinterwovenwithappraisal,weusedmore
neutralmeasuresofsocialbehavior(i.e.,timespentincompany;amountofsocialinteraction),
andweincludedspecificsocialcontexts(i.e.,timespentincompanyoffriends,partner,or
family)toexplorelearningacrossdifferenttypesofsocialcontexts.Furthermore,wereduced
theprobabilityofspill-overeffectsduetocontinuationofactivitiesinsteadofre-engagement
byusingbeepsthatwereapproximatelysixhoursapart.Finally,weextendedthestudyof
Wichersandcolleagues[11]bycomparingtherewardandpunishmentlearningeffectsinindi-
vidualsnotdiagnosedforamentaldisorderorincurrenttreatmentforamentaldisorderto
thoseinindividualssufferingfromanhedonia.Anhedoniaisoneofthecoresymptomsof
depression,anddefinedbytheDiagnosticStatisticalManualasamarkedlossofpleasureor
interest[12],characterizedbyanimpairedabilitytopursue,experience,andlearnabout
reward[13–16].Anhedonicindividualsoftenshowreducedrewardlearningandpunishment
learninginprobabilisticrewardtasks[17–20],possiblybecausetheyarelessabletoexploit
affectiveinformationthatguidesbehavior.Nonetheless,towhatextentthesealteredresponses
inanhedonicindividualstranslatetomotivatedactionsinreal-lifehasneverbeeninvestigated.
Basedontheabove-describedconsiderations,wehypothesizedthatlevelsofPAandNA
duringsocialactivitywouldrespectivelyincreaseanddecreasetheprobabilityofsubsequent
engagementinthatactivity,bothwithindaysandacrossdays.Forphysicalactivity,wehypoth-
esizedthatreinforcementlearningwouldfollowa3-daycycleinsteadofa1-dayorwithin-day
cycle.Furthermore,wehypothesizedthatindividualswithanhedoniawouldshowimpaired
associativelearningratesascomparedtoindividualswithout.
Materialandmethods
Sample
ThesampleispartofthelargerNoFunNoGory(NFNG)study:arandomizedcontrolledtrial
toexploretheeffectsofpersonalizedlifestyleadviceandtandemskydivesonpleasurelevelsin
anhedonicyoungadults(see[21]).TheNFNGstudyisregisteredintheDutchClinicalTrial
Register(NTR5498)andwasapprovedbytheDutchMedicalEthicsCommitteefromtheUni-
versityMedicalCenterGroningen(no.2014/508).Participantswereselectedthroughan
onlinescreeningsurveyamong2937youngadults(Mage=21.4years,SD=1.9,78%women)
fromtheNorthernpartoftheNetherlands.Thefinalsampleconsistedofatotalof138ESM
participantswhomettheinclusioncriteriaandwererandomlyselectedfortheESM-partof
thestudy:69withanhedonia,and69matchedcontrols.Foramoregraphicrepresentationof
theinclusionandexclusionofparticipants,pleaseseetheflowchartinFig1;formoredetails
onthedemographicsoftheanhedonicindividualsandmatchedcontrols,pleaseseeTable1.
Inclusioncriteriafortheanhedoniagroupwere1)persistentanhedoniaand2)willingness
toperformaskydive.PersistentanhedoniawasdefinedbasedonthreeitemsfromtheDo-
mainsofPleasureScale(DOPS;[23])assessing(1)thelevelofpleasureinthepasttwoweeks,
(2)whetherthislevelrepresentedachangecomparedtowhatisconsiderednormalforthis
individual,and(3)thedurationofthelossofpleasure,ifany.Inordertobeeligibleforinclu-
sionoftheanhedoniagroup,aparticipantwasrequiredtoreportalevelofpleasurebelowthe
25thpercentile,whichwasexperiencedaslowerthannormal,andhadtosufferfromthisloss
ofpleasureforatleasttwomonths.Forthecontrolgroup,theinclusioncriteriawere1)noloss
ofpleasure(i.e.,pleasureratingabovethe50thpercentile,ratedas‘justasmuch’,‘more’or
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 3/18
Rewardandpunishmentlearningindailylife
Fig1.Flowchartoftheparticipantenrollment.Dottedlineindicatesmatchingprocedure.
https://doi.org/10.1371/journal.pone.0180753.g001
‘muchmore’comparedtotheirnormallevels)and2)willingnesstoperformaskydive.Partici-
pantswithanhedoniawhowerewillingtoperformaskydive(answeryesormaybe)didnot
differsignificantlyfromthosewhowerenotwillingtodoso(N=25)withregardtothesever-
ityofanhedonia(t=1.30,p=.13),thelevelofconsummatorypleasure(t=-0.84,p=.40),and
depressivesymptoms(t=0.30,p=.77).Exclusioncriteriawere:inabilitytokeepanelectronic
diarythreetimesaday;currentprofessionaltreatmentforpsychiatricproblems;currentuse
ofpsychotropicmedication;epilepsy;pregnancy;conditionsthatmakeitimpossibletobe
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 4/18
Rewardandpunishmentlearningindailylife
Table1. Demographicsofthematchedanhedonicandnon-anhedonicsample.
Noanhedonia(controls) Anhedonia
Men(%) 14 (20) 14 (20)
Meanageinyears(SD) 21.51 (1.90) 21.45 (1.96)
Currenteducation(%)
Tertiaryeducation:University 41 (59) 40 (58)
Tertiaryeducation:Highervocational 26 (38) 25 (36)
Secondaryeducation 2 (3) 3 (4)
Lowersecondaryeducation 0 1 (2)
Compliancerate(%) 83.52 (92.8) 84.33 (93.7)
Agereflectstheageinyearsonthedaybeforetheintroductorysession.N=69inbothgroups.Duetodifferencesineducationalsystems,currenteducation
isgroupedbasedonFig1inVeldmanetal.[22].Highestlevelofcompletededucationofthethreeparticipantswithoutcurrenteducationwere:lowerand
highersecondaryeducation,andhighervocationaleducation.Threeparticipantsreportednocurrenteducation,andwerecategorizedonthebasisoftheir
highestlevelofeducationattained(namelylowersecondaryeducation;highersecondary;highervocational).Complianceratewastheaveragenumberof
assessmentsfilledoutof90,averagedoverallparticipantswithinthatgroup.
https://doi.org/10.1371/journal.pone.0180753.t001
attachedtothetandemmaster(e.g.,looseprostheses);heightofmorethantwom;weightof
morethan95kg;inabilitytoraiseone’slegs90˚(neededforsafelandingaftertandemsky-
dive);significantvisualorhearingimpairments;experiencewithskydiving,bungeejumping,
basejumping,orskyjumping;andcardiovascularproblems.Allcriteriawereincludedasques-
tionsinthescreeningsurvey.Afteronemonthofdiarystudy,allparticipantsreceivedacom-
pensationof75eurosfortheirparticipationundertheconditionthattheycompletedthe
monthlyquestionnairesandfilledinatleast72ofthe90momentaryassessments(i.e.,80%).
Inaddition,individualsintheanhedoniagroupneededtoprovidebloodsamplesbeforeand
afterthemonthofdiarystudytoreceivethecompensation.
Procedure
Individualswhowereeligiblefortheinterventionstudyandgavepermissiontobecontacted
forfurtherresearchreceivedadetailedinformationletterandconsentformbyemail.The
informationletterincludedinformationaboutthestudyprocedures,possiblerisks,insurance
regulationsandthepossibilitytostopthestudyatanypoint.Afterindividualsreturneda
signedconsentform,participantswereinvitedforanintroductorymeeting,andaskedtofillin
anonlinequestionnaireathomeontheprecedingday.
Duringtheintroductorysession,participants’age(checkedinID),exclusionandinclusion
criteriawerecheckedagain,studyprocedureswereexplained,andparticipantswereinstructed
onhowtofillinthemomentaryassessmentsontheirsmartphone.Ifstilleligibleandjudged
abletoadheretotheprotocol,participantsreceivedaninteractiveweblinkinatextmessage
(ande-mails,ifdesired)atthreesettimepointsperdayfor30consecutivedays.Theweblinks
weresentsecurelyviaawebapplicationbyRoQua(www.roqua.nl),asystemspeciallydesigned
forroutineoutcomemonitoringinmentalhealthcarethatmeetsthehighestprivacystan-
dards.Exceptforthenights,theintervalbetweentheassessmentswasalwayssixhours,butthe
exacttimeswereadaptedtothelifestyleoftheindividualparticipant(e.g.,9am–3pm–9
pm).Whenadiarywasnotfilledinatthescheduledtime,participantsgotareminderafter
onehourandanotherafteranhourandahalf.Thelinkbecameunavailabletwohoursafter
thefirstnotification.Compliancewasmonitoreddaily,andparticipantswerecontactedwhen
theymissedmorethanthreeassessmentsonthreeconsecutivedays,orwereclosetomissing
morethan20%ofthetotalnumberofassessments.
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 5/18
Rewardandpunishmentlearningindailylife
Measures
Affect. AffectwasassessedwithitemssimilartothoseusedintheUncoveringthePositive
PotentialofEmotionalReactivity(UPPER)study[24],whichwereadaptedfrompreviouseco-
logicalmomentaryassessmentstudies[25,26],andconsistsofbothhigharousalandlow
arousalitems.Participantswereaskedtoratetheextenttowhichtheyexperiencedacertain
emotionbymovingaslideralongthecontinuumofaVisualAnalogueScale(VAS),thatwas
anchoredwithnotatall(left)andverymuch(right).Thelocationofthesliderwasconverted
intoascorebetween0and100.Intheafternoonandeveningassessments,affectwasmeasured
inretrospect(i.e.,“Sincethelastassessment,Ihavefelt[insertadjectiveemotion]”).Toprevent
distortionbysleep,morningaffectwasmeasuredinthemoment(i.e.,“Ifeel[insertadjective
emotion]”).Positiveaffect(PA)wascalculatedbyaveragingtheVAS-scoresofthefollowing
10items:feelinginterested,joyful,determined,calm,lively,enthusiastic,relaxed,cheerful,sat-
isfied,andenergetic.Negativeaffect(NA)wascalculatedbyaveragingthescoresonthefollow-
ingeightitems:feelingupset,gloomy,sluggish,anxious,bored,irritated,nervous,andlistless.
Cronbach’salphawas.94forPAand.86forNA(calculatedoverallassessments).
Socialcompany. Socialcompanywasoperationalizedinfivedifferentways,namely:1)
theamountofsocialinteraction;2)theamountoftimespentincompany;3)theamountof
timespentwithromanticpartner;4)theamountoftimespentwithfamily;and5)theamount
oftimespentwithfriends.Thefirstoperationalizationwaswithanadapteditemfromthe
UncoveringthePositivePotentialofEmotionalReactivity(UPPER)study[24],thatread:“[-
Sincethelastassessment,]howmuchdidItalktoothers?”,followedbyVASanchoredwith
notatall(left)andverymuch(right).Theotherfourtypesofcompanywereassessedbya
probequestioninwhichparticipantswereaskedtoindicatewhomtheyspenttimewithsince
thelastassessment(answercategories:Alone/Withpartner/Withfamily/Withfriends/With
fellowstudents/Withcolleagues/Withacquaintances /Withstrangers),foreachoptionticked
followedbythequestion:“HowmuchtimedidIspendwith[typeofcompany]?”,whichcould
beansweredonaVASsalerangingfromverylittle(left)toalot(right).TheVAS-scoreswere
convertedintoanumberbetween0and100.Fortheprobequestion,anuntickedboxwasreg-
isteredas0(i.e.,notimespentinthatcompany)andthetimeregisteredfor“Alone”was
invertedtoreflectthetimethattheparticipantwasnotalone,andthusincompanyofothers.
Physicalactivity. Physicalactivitywasadaptedfrompreviousecologicalmomentary
assessmentstudies[24,27,28]andassessedbythequestion:“[Sincethelastassessment,]Ihave
beenphysicallyactive:”,followedbyaVASanchoredwithnotatall(left)andverymuch
(right).Thelocationofthesliderwasconvertedintoascorebetween0and100.
Dataanalyticplan
FollowingWichersandcolleagues[11],weusedmultilevelmodellinginwhichESMobserva-
tions(level1)wereclusteredwithinparticipants(level2).Afterweperson-centeredthelevel1
predictorsinSPSSversion24.0.0.0[29],weconductedmultilevelregressionanalysesinMPlus
version7.4[30].Inordertoinvestigatetheeffectofimplicitrewardandpunishmentlearning
processesonsubsequentactivities,socialandphysicalactivityatTwasoperationalizedasa
functionofreinforcementlearningatT-1.Becausemorningaffectthatwasreportedinthe
momentcouldneverco-occurredwithactivitiesreportedsincethelastassessment,morning
assessmentsofaffectwereexcludedfromtheanalyses.Rewardandpunishmentlearningwere
operationalizedastheinteractioneffectof,respectively,thelevelofaffect(i.e.,PAorNA)at
thepreviousassessment(i.e.,T-1)withthelevelactivityatT-1(i.e.,socialorphysicalactivity)
onthelevelofthatparticularactivityatT.Toexploregroupdifferencesintheseeffects,we
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 6/18
Rewardandpunishmentlearningindailylife
addedindividuals’anhedonicstatusasalevel2predictor(controlgroupversusanhedonia
group).Mathematically,therandomslopemodelwasasfollows:
Level1:
Activity ¼b þb ðAffect Þþb ðActivity Þþb ðAffect (cid:2)Activity Þþr
ij 0j 1j T(cid:0) 1 2j T(cid:0) 1 3j T(cid:0) 1 T(cid:0) 1 ij
Level2:
b ¼g þg ðanhedonicstatusÞþu
0j 00 01 0j
b ¼g þg ðanhedonicstatusÞþu
1j 10 11 1j
b ¼g þg ðanhedonicstatusÞþu
2j 20 21 2j
b ¼g þg ðanhedonicstatusÞþu
3j 30 31 3j
Inadditiontobeeplevel,andstillmirroringtheanalysesofWichersetal.[11],thereward-
andpunishment-drivenbehavioralactivitieswerealsoinvestigatedondaylevel.Themathe-
maticalmodelsoftheanalysesondaylevelwerethesameasforbeeplevelexceptthat,before
calculatingthelaggedeffects,thelevelofaffectandactivitywereaveragedoverdays.Forthe
laggedday-levelvariableofco-occurrence(i.e.,interactionterm),beep-levellaggedvariables
wereusedtoensureco-occurrencewithinthesametimeframe(e.g.,morningPAwasmulti-
pliedbybehaviorinthemorning).
Intotal,wetested26reinforcementlearningeffects.Onbeeplevel,wetested6interaction
effectsbetweenlevelofaffectatT-1andlevelofsocialorphysicalactivityatT-1onsubsequent
behavioratTforPAand6interactioneffectsforNA.Wethenrepeatedthese12beep-level
analysesonday-level,making24learningeffectsintotal.Additionally,wetestedtwo3-day-
levelinteractioneffectswithregardtophysicalactivity,makingatotalof26effectstested.
Tomaintainafamilywiseerrorrateof.05overallanalyses,aBonferroni-correctionofa0 ¼
1
1(cid:0) ð1(cid:0) aÞVeffLiwasappliedwithVeffLi beingthe‘effectivenumber’ofindependenttestscor-
rectedforthecorrelationamongstthedifferentpredictors(inthiscase:theinteractionterms).
UsingtheapproachproposedbyLi&Ji[31],theeffectivenumberofindependentvariablesof
our26correlatedvariableswas19.Theexperiment-widesignificancethresholdthatisrequired
tokeepTypeIErrorRateat5%with19independentvariablesis:0.0027.
Results
Descriptivestatistics
Thestudyhadhighcompliancerates.Onaverage,bothanhedoniaandcontrolparticipants
respondedtomorethan81ofthetotal90beeps(i.e.,>90%).Giventhematchingprocedure,
groupsdidnotdifferingendercomposition,meanage,andeducationallevel(seeTable1).
AsshowninTable2,controlparticipantshadspentonaveragearound15%oftheirtime
withtheirpartner,18%withtheirfamily,and22%withtheirfriendssincethelastassessment.
Intotal,theyhadbeenincompanyofothers65%ofthetime,andreportedtohavebeenmod-
eratelysociallyinteractiveonaverage.Withregardtophysicalactivity,participantsfromthe
controlgroupreportedtobelowtomoderatelyactiveonaverage.Acrossthe30daysofstudy,
controlgroupparticipantsreportedanaveragePAlevelof64.44(SD=9.36),andanaverage
NAlevelof14.95(SD=7.23).
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 7/18
Rewardandpunishmentlearningindailylife
Table2. Descriptivestatisticsforkeystudyvariables.
Participants
Noanhedonia(controls) Anhedonia Difference
Mean (SD) Mean (SD) t p
(136)
PA 64.44 (9.36) 54.56 (9.67) -6.10 ***
NA 14.95 (7.23) 23.84 (8.98) 6.40 ***
SocialInteraction 52.19 (11.98) 50.05 (10.92) -1.10
Timespentincompany 65.47 (11.75) 67.25 (13.68) 0.82
Timespentwithpartner 14.97 (17.12) 17.31 (19.87) 0.74
Timespentwithfamily 17.72 (12.93) 20.68 (13.45) 1.32
TimespentwithFriends 21.87 (12.95) 19.74 (12.11) -1.00
PhysicalActivity 32.46 (10.12) 29.45 (9.41) -1.81 ~
PA=PositiveAffect;NA=NegativeAffect.Meansreflectindividuals’day-levelmeans,aggregatedover30dayswithinpersons.ForPAandNAthe
morningassessmentwereaskedmomentaryinsteadofretrospective,andwerethereforeexcluded.N=69inbothgroups.Differencesbetweengroups
weretestedwithindependentt-tests.
~p=.05-.10
*p<.05
**p<.01
***p<.001
https://doi.org/10.1371/journal.pone.0180753.t002
AnhedoniawasassociatedwithaloweraveragelevelofPA,andahigheraveragelevelof
NA,butnottodifferencesinaveragesocialorphysicalactivity(seeTable2).
Rewardandpunishmentlearning
Withregardtoimplicitrewardandpunishmentlearning,andasshowninFig2andTables3
and4,thecontrolgroupdidnotshowmodulatedbehaviorasafunctionofassociativelearning
betweenlevelofaffectandlevelofactivityonbeep-and3-day-level.Thatis,noneofthe
rewardandpunishmentlearningeffectsontheselevelssurvivedtheBonferroni-correctedsta-
tisticallevelofapproximatelyp<.003.
Ondaylevel,however,asignificantpunishmentlearningeffectwasfoundwithregardto
theamountoftimespentwithfriends(p<.001;seeTable4).Thiseffectreflectsthatif,com-
paredtotheirownaverage,non-anhedonicindividualshadspentmoretimeincompanyof
friendsandsimultaneouslyhadexperiencedhigherthanaveragelevelsofNA(i.e.,withinthe
sametimeframe),theytendedtospentlesstimewiththeirfriendsthenextday.
Rewardandpunishmentlearninginanhedonicindividuals
Comparedtotheirnon-anhedoniccounterparts,individualsintheanhedoniagroupdidnot
differinrewardandpunishmentlearning(seeTables3and4),regardlessofthelevelof
investigation.
Discussion
InthisstudyweaimedtoreplicatethepioneerfindingofWichersandcolleagues[11],while
overcomingtheirlimitations.Basedontheirfindings,andbasicprinciplesofreinforcement
theory[3,4],weexpectedhighlevelsofPAandNAthatoccurredduringsocialorphysical
activitieswouldrespectivelyboostorlimitre-engagementintheseactivitiesthroughassocia-
tivelearningprocesses.Furthermore,basedonfindingsfromlaboratoryresearch[17–20],we
expectedthattheseprocesseswouldbeimpairedinindividualswithanhedonia.However,we
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 8/18
Rewardandpunishmentlearningindailylife
Fig2.Beta-coefficientsforrewardandpunishmentlearningatbeeplevel,daylevel,and3daylevel.Barsrepresentthebeta-coefficientsofaffect
atT-1byactivityatT-1onactivityatT,andreflectsthemagnitudeoftheserewardpunishmentlearningeffectsonbeeplevel(Figs2a&2b),dayand3day
level(Figs2c&2d).
https://doi.org/10.1371/journal.pone.0180753.g002
foundneitherrobustevidenceofrewardlearningorpunishmentlearningprocessesinnon-
anhedonicindividuals,norsignsofdeviationsoftheseabilitiesinindividualswithanhedonia.
Oftheintotal52rewardandpunishmentlearningeffectstested,onlyonesurvivedBonferroni
correction:participantswhohadspentmoretimeincompanyoffriendsandexperiencedhigh
levelsofNAduringthesametimeintervalthatday,tendedtospentlesstimewithfriendsthe
nextday.
Perhaps,ourmeasureofassociativelearningismoreecologicallyvalidthanlaboratoryfind-
ingsbutalsohardertocaptureduetothelimiteddegreesoffreedomindividualshaveto
changetheirbehaviorindailylife.Inreallife,humanshaveoftentightschedulesofappoint-
mentsthatleaveonlylittleroomforadhocchangesbasedonwhatisrewardingorpunishing
andwhatisnot.Forexample,forthosewhohadapartner,itislikelythattheywantedtosee
him/hermoreoften,butcommitmentsonbothsidesmadethatimpossible.However,despite
possiblelimiteddegreesoffreedom,wewerestillabletofindasmallbutsignificantpunish-
mentlearningeffectwithregardtofriends.Thatwefoundapunishmentlearningeffect,but
notarewardlearningeffectmaybeduetotheinexplicablyintertwinednaturebetweenfriends
andPA[32].Thatis,experiencinghighPAinthisspecificcontextisprobablytheveryreason
whyfriendsarelabelledassuch.ExperiencingNAwhilebeingincompanyoffriendsislikely
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 9/18
Rewardandpunishmentlearningindailylife
Table3. Estimatesofrewardlearninginindividualswithoutanhedoniaanddifferencesofthoseestimatesinindividualswithanhedonia.
Anhedoniastatus
Noanhedonia(controls) Anhedonia
Est. 95%CI p Est. 95%CI p
BEEP-LEVEL
AmountofsocialInteraction 51.35 [48.29, 54.40] *** -1.99 [-6.11, 2.12]
PA onInteraction 0.23 [0.15, 0.32] *** -0.08 [-0.18, 0.03]
T-1 T
SocialInteraction onInteraction 0.22 [0.18, 0.27] *** 0.01 [-0.06, 0.07]
T-1 T
(PA ×Interaction )onInteraction 0.00 [0.00, 0.01] 0.00 [-0.01, 0.00]
T-1 T-1 T
TimespentinCompany 63.69 [60.40, 66.99] *** 2.21 [-2.74, 7.17]
PA onCompany 0.19 [0.09, 0.29] *** 0.01 [-0.12, 0.14]
T-1 T
Company onCompany 0.30 [0.26, 0.35] *** -0.01 [-0.07, 0.06]
T-1 T
(PA ×Company )onCompany 0.00 [-0.01, 0.00] 0.00 [0.00, 0.01]
T-1 T-1 T
TimespentwithPartner 18.09 [13.18, 23.00] *** 3.26 [-4.34, 10.86]
PA onPartner 0.09 [0.01, 0.16] * -0.04 [-0.14, 0.07]
T-1 T
Partner onPartner 0.48 [0.40, 0.57] *** 0.01 [-0.09, 0.11]
T-1 T
(PA ×Partner )onPartner 0.01 [0.00, 0.01] * -0.01 [-0.01, 0.00] *
T-1 T-1 T
TimespentwithFamily 18.82 [15.53, 22.11] *** 2.70 [-2.09, 7.49]
PA onFamily -0.03 [-0.10, 0.05] 0.00 [-0.11, 0.10]
T-1 T
Family onfamily 0.47 [0.41, 0.53] *** -0.01 [-0.08, 0.07]
T-1 T
(PA ×Family )onFamily 0.00 [-0.01, 0.01] 0.00 [0.00, 0.01]
T-1 T-1 T
TimespentwithFriends 22.64 [19.21, 26.07] *** -2.14 [-6.70, 2.41]
PA onFriends 0.25 [0.15, 0.36] *** -0.10 [-0.24, 0.04]
T-1 T
Friends onFriends 0.33 [0.29, 0.38] *** 0.01 [-0.06, 0.07]
T-1 T
(PA ×Friends )onFriends 0.01 [0.00, 0.01] * 0.00 [0.00, 0.01]
T-1 T-1 T
PhysicalActivity 31.19 [28.77, 33.61] *** -3.78 [-7.13, -0.43] *
PA onPhysicalActivity 0.16 [0.09, 0.24] *** -0.07 [-0.17, 0.03]
T-1 T
PhysicalActivity onPhysicalActivity 0.07 [0.02, 0.12] ** 0.02 [-0.05, 0.09]
T-1 T
(PA ×PhysicalAct )onPhysicalAct 0.00 [0.00, 0.00] 0.00 [0.00, 0.00]
T-1 T-1 T
DAY-LEVEL
AmountofsocialInteraction 51.83 [48.90, 54.75] *** -2.03 [-5.96, 1.91]
PA onInteraction 0.25 [0.15, 0.35] *** -0.05 [-0.17, 0.08]
T-1 T
SocialInteraction onInteraction 0.11 [0.05, 0.16] *** -0.04 [-0.12, 0.04]
T-1 T
(PA ×Interaction )onInteraction 0.01 [0.00, 0.01] * 0.00 [-0.01, 0.00] ~
T-1 T-1 T
TimespentinCompany 65.38 [62.51, 68.25] *** 1.67 [-2.69, 6.03]
PA onCompany 0.28 [0.14, 0.41] *** -0.08 [-0.25, 0.08]
T-1 T
Company onCompany 0.22 [0.17, 0.27] *** -0.01 [-0.07, 0.06]
T-1 T
(PA ×Company )onCompany 0.00 [-0.01, 0.01] 0.00 [-0.01, 0.01]
T-1 T-1 T
TimespentwithPartner 14.93 [10.89, 18.97] *** 2.50 [-3.70, 8.70]
PA onPartner 0.18 [0.08, 0.28] *** -0.08 [-0.21, 0.06]
T-1 T
Partner onPartner 0.24 [0.18, 0.31] *** 0.01 [-0.09, 0.11]
T-1 T
(PA ×Partner )onPartner 0.00 [0.00, 0.01] 0.00 [-0.01, 0.01]
T-1 T-1 T
Timespentwithfamily 17.80 [14.80, 20.80] *** 2.73 [-1.63, 7.09]
PA onFamily 0.07 [-0.04, 0.18] 0.01 [-0.14, 0.16]
T-1 T
Family onfamily 0.37 [0.32, 0.43] *** -0.01 [-0.08, 0.06]
T-1 T
(PA ×Family )onFamily 0.00 [0.00, 0.01] 0.00 [-0.01, 0.01]
T-1 T-1 T
TimespentwithFriends 21.37 [18.37, 24.38] *** -1.75 [-5.95, 2.45]
PA onFriends 0.17 [0.05, 0.29] ** -0.04 [-0.19, 0.11]
T-1 T
Friends onFriends 0.22 [0.16, 0.27] *** -0.04 [-0.12, 0.04]
T-1 T
(Continued)
PLOSONE|https://doi.org/10.1371/journal.pone.0180753 October4,2017 10/18