Affective Information Processing

347 Pages·2009·10.828 MB·English

by  Jianhua Tao, Tieniu Tan (auth.), Dr Jianhua Tao, Professor Tieniu Tan BSc, MSc, PhD (eds.)

Jianhua Tao
Tieniu Tan
(Eds)
T
a
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
o

·
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
T
a
n
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

(
E
d | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
s
)
| | | | | | | | | | | | | | | | | | | | | |
1
Affective
A
Information
f
f
e
c
t
i
v
Processing
e

I
n
f
o
r
m
a
t
i
o
n

P
r
o
c
e
s
s
i
n
1 3
g
Affective Information Processing
Jianhua Tao Tieniu Tan
•
Editors
Affective Information
Processing
ABC
Dr.JianhuaTao,PhD
ProfessorTieniuTan,BSc,MSc,PhD
InstituteofAutomation,ChineseAcademyofSciences
95ZhongguancunEastRoad
Haidian,Beijing,100080,P.R.China
ISBN:978-1-84800-305-7 e-ISBN:978-1-84800-306-4
DOI:10.1007/978-1-84800-306-4
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
LibraryofCongressControlNumber:2008937467
(cid:176)c Springer-VerlagLondonLimited2009
Apartfromanyfairdealingforthepurposesofresearchorprivatestudy,orcriticismorreview,aspermit-
tedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced,stored
ortransmitted,inanyformorbyanymeans,withthepriorpermissioninwritingofthepublishers,orin
thecaseofreprographicreproductioninaccordancewiththetermsoflicencesissuedbytheCopyright
LicensingAgency.Enquiriesconcerningreproductionoutsidethosetermsshouldbesenttothepublishers.
Theuseofregisterednames,trademarks,etc.inthispublicationdoesnotimply,evenintheabsenceofa
specificstatement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandthereforefree
forgeneraluse.
Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyoftheinformation
containedinthisbookandcannotacceptanylegalresponsibilityorliabilityforanyerrorsoromissions
thatmaybemade.
Printedonacid-freepaper
SpringerScience+BusinessMedia
springer.com
Preface
Affective information processing involves studies that attempt to give computers
the humanlike capabilities of observation, interpretation, and generation of affect
features. It is an important topic in human–computer interaction (HCI), because it
helpsimprovethequalityandeffectivenessofhumantocomputercommunications.
Words such as “affect,” “affective,” and “affection” are usually linked to humans
andhaverarelybeenusedforlifelessmachines.Affectiveinformationprocessingis
quitenewinHCIwhichconsidersaffectfeaturesandinvestigateswaysofcapturing
andprocessingthesebycomputers.Thecomputationofaffectivemultimodalinfor-
mationisalsoknownas“affectivecomputing”andhasbecomeaveryhotresearch
topic.
In2005,weorganized thefirstinternationalconference onaffectivecomputing
andintelligentinteraction(ACII2005)inBeijing.Therewere134papersaccepted
for the conference in total, and about 140 people from 32 countries attended the
conference. The conference proceedings were published as a volume in Springer
LNCS.In2007,thesecondACIIwasheldinLisbonwithanattendanceofaround
150.ACIIisnowaneventoftheHUMAINEAssociation.Fromthattimeon,ACII
hasbecomeacentraleventforresearchersstudyinganddevelopingtheroleofemo-
tionandotheraffectivephenomenaastheserelatetohuman–computerandhuman–
robotinteraction,graphics,artificialintelligence,robotics,vision,speech,synthetic
characters,games,educationalsoftware,andsoon.
TheinternationalenthusiasmatACIIencouraged ustoeditabookonaffective
informationprocessingbysolicitingoriginaltheoreticalandpracticalworkoffering
newandbroadviewsofthelatestresearchinthisnewandexcitingarea.Mostofthe
authorswereselectedandinvitedfromamongtheattendeesofACII2005,andsome
chapterswerewrittenbyotherleadingresearchersoutsideACII2005.Thisensures
thatthebookhasabalancedcoverageoftheimportanttopics.
Thebooksummarizessomekeytechnologiesthatareorganizedintofourparts:
“Cognitive Emotion Model,” “Affect in Speech,” “Affect in Face,” and “Affect
in Multimodal Information,” with 17 chapters. Unlike other books which often
focusonpsychologicalproblemsorcognitivemodelsforemotions,thisbookgives
more content on how to process affective information for the next generation of
v
vi Preface
HCItechnologies.Thusthemultimodalitiesofaffectiveinformationintheformof
language,speech,andfaceareemphasizedinthisbook.
Wehavenointentionofaddressingallproblemsinaffectiveinformationprocess-
ingwithinthisbook,buttrytogiveawidedescriptionandsomedetailedknowledge
inthisarea.Wehopethatthebookcanbetakenasausefulreferencebookandwill
helppromotemoreanddeeperstudiesinthisexcitingarea.
Asafinalword,wewouldliketothankalltheauthorsfortheirqualitycontribu-
tions.OurthanksalsogotoWayneWheelerandCatherineBrettfromSpringerfor
theirinvaluablehelpandguidancethroughouttheproject.Thisbookissupportedin
partbytheNationalNaturalScienceFoundationofChinaunderGrant(60575032)
Beijing,China JianhuaTao
May10,2008 TieniuTan
Contents
1 Introduction................................................ 1
JianhuaTaoandTieniuTan
PartI CognitiveEmotionModels
2 AffectandEmotionsinIntelligentAgents:WhyandHow?......... 11
AndrewOrtony
3 CognitiveEmotionModelinginNatural
LanguageCommunication.................................... 23
ValeriaCarofiglio,FiorelladeRosis,andNicoleNovielli
4 ALinguisticInterpretationoftheOCCEmotionModelforAffect
SensingfromText ........................................... 45
MostafaAlMasumShaikh,HelmutPrendinger,andMitsuruIshizuka
5 AffectiveAgentsforEducationAgainstBullying ................. 75
RuthAylett,AnaPaiva,JoaoDias,LynneHall,andSarahWoods
PartII AffectinSpeech
6 EmotionPerceptionandRecognitionfromSpeech................ 93
Chung-HsienWu,Jui-FengYeh,andZe-JingChuang
7 ExpressiveSpeechSynthesis:Past,Present,andPossibleFutures ... 111
MarcSchro¨der
8 Emotional Speech Generation by Using Statistic Prosody
ConversionMethods......................................... 127
JianhuaTaoandAijunLi
vii
viii Contents
PartIII AffectinFace
9 WhytheSameExpressionMayNotMeantheSameWhenShown
onDifferentFacesorSeenbyDifferentPeople ................... 145
UrsulaHessandPascalThibault
10 Automatic Facial Action Unit Recognition by Modeling
TheirSemanticAndDynamicRelationships ..................... 159
YanTong,WenhuiLiao,andQiangJi
11 FaceAnimationBasedonLargeAudiovisualDatabase ............ 181
JianhuaTao,PanrongYin,andLeXin
PartIV AffectinMultimodalInteraction
12 AffectinMultimodalInformation.............................. 203
AnnaEsposito
13 NonverbalFeedbackinInteractions............................ 227
KristiinaJokinen
14 EmotionRecognitionBasedonMultimodalInformation........... 241
ZhihongZeng,MajaPantic,andThomasS.Huang
15 AMultimodalCorpusApproachfortheStudy
ofSpontaneousEmotions..................................... 267
Jean-ClaudeMartinandLaurenceDevillers
16 PhysiologicalSensingforAffectiveComputing ................... 293
ChristianPeter,EricEbert,andHelmutBeikirch
17 Evolutionary Expression of Emotions in Virtual Humans
UsingLightsandPixels ...................................... 311
CelsoM.deMeloandAnaPaiva
Index .............................................................335
Contributors
AijunLi InstituteofLinguistic,ChineseAcademy ofSocialScience, Beijing,
[email protected]
AnaPaiva IST–TechnicalUniversityofLisbonandINESC-ID,AvenidaProf.
CavacoSilva–Taguspark,2780-990PortoSalvo,Portugal,[email protected]
AnaPaiva INESC-ID,TagusPark,Av.Prof.Dr.CavacoSilva,2780-990Porto
Salvo,Portugal,[email protected]
AndrewOrtony NorthwesternUniversity,2120NorthCampusDrive,Annenberg
Hall,EvanstonIL60208-2610,[email protected]
Anna Esposito Dipartimento di Psicologia, Seconda Universita` di Napoli Via
Vivaldi43,81100Caserta,ItalyIIASS,ViaPellegrino19,84019,VietrisulMare,
SA,Italy,[email protected],[email protected]
CelsoM.deMelo USC,UniversityofSouthernCalifornia,[email protected]
ChristianPeter FraunhoferInstituteforComputerGraphicsRostock,Joachim
JungiusStr.11,18059Rostock,Germany
Chung-HsienWu DepartmentofComputerScienceandInformationEngineering,
NationalChengKungUniversity,Tainan,[email protected]
EricEbert arivisMultipleImageToolsGmbH,Schwaansche Straße1,18055
Rostock,Germany
FiorelladeRosis DepartmentofInformatics,ViaOrabona4,70124Bari,Italy
Helmut Beikirch University of Rostock, Faculty of Computer Science and
ElectricalEngineering,Albert-Einstein-Str.2,18059Rostock,Germany
Helmut Prendinger Digital Contents and Media Sciences Research Division,
NationalInstituteofInformatics,2-1-2Hitotsubashi,Chiyoda-ku,101-8430Tokyo,
Japan,[email protected]
ix
x Contributors
Jean-ClaudeMartin LIMSI-CNRS,France
LaurenceDevillers LIMSI-CNRS,France
JianhuaTao NationalLaboratoryofPatternRecognition,InstituteofAutomation,
ChineseAcademyofSciences,Beijing,[email protected]
Joao Dias NESC-ID, TagusPark, Av. Prof. Dr. Cavaco Silva, 2780-990 Porto
Salvo,Portugal,[email protected]
Jui-FengYeh NationalChia-YiUniversity,ChiaYi
Kristiina Jokinen University of Helsinki and Tampere, Finland, Kristi-
ina.Jokinen@helsinki.fi
Le Xin National Laboratory of Pattern Recognition, Institute of Automation,
ChineseAcademyofSciences,Beijing
Lynne.Hall School of Computing and Technology, University of Sunderland,
Sunderland,SR60DD,UK,[email protected]
Mostafa Al Masum Shaikh Department of Information and Communication
Engineering, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-8656 Tokyo,
Japan,[email protected]
MitsuruIshizuka DepartmentofInformationandCommunicationEngineering,
University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-8656 Tokyo, Japan,
[email protected]
Maja Pantic Imperial College London University of Twente, Netherlands,
[email protected]
MarcSchro¨der DFKIGmbH,Saarbru¨cken,Germany,[email protected]
NicoleNovielli DepartmentofInformatics,ViaOrabona4,70124Bari,Italy
PanrongYin NationalLaboratoryofPatternRecognition,InstituteofAutomation,
ChineseAcademyofSciences,Beijing
PascalThibault DepartmentofPsychology,McGillUniversity,Montreal,Canada,
[email protected]
Qiang Ji Department of Electrical, Computer and Systems Engineering,
RensselaerPolytechnicInstitute,Troy,NY,[email protected]
Ruth Aylett Mathematics and Computer Science, Heriot-Watt University,
Edinburgh,EH144AS,UK,[email protected]
Sarah Woods Psychology Department, University of Hertfordshire, Hatfield,
Herts,AL109AB,[email protected]
TieniuTan NationalLaboratoryofPatternRecognition,InstituteofAutomation,
ChineseAcademyofSciences,Beijing