Table Of ContentStudies in Computational Intelligence 1060
Arash Shaban-Nejad
Martin Michalowski
Simone Bianco Editors
Multimodal
AI in
Healthcare
A Paradigm Shift in Health Intelligence
Studies in Computational Intelligence
Volume 1060
SeriesEditor
JanuszKacprzyk,PolishAcademyofSciences,Warsaw,Poland
The series “Studies in Computational Intelligence” (SCI) publishes
new developments and advances in the various areas of computational
intelligence—quickly and with a high quality. The intent is to cover the theory,
applications, and design methods of computational intelligence, as embedded in
the fields of engineering, computer science, physics and life sciences, as well as
themethodologiesbehindthem.Theseriescontainsmonographs,lecturenotesand
editedvolumesincomputationalintelligencespanningtheareasofneuralnetworks,
connectionist systems, genetic algorithms, evolutionary computation, artificial
intelligence, cellular automata, self-organizing systems, soft computing, fuzzy
systems,andhybridintelligentsystems.Ofparticularvaluetoboththecontributors
and the readership are the short publication timeframe and the world-wide
distribution,whichenablebothwideandrapiddisseminationofresearchoutput.
This series also publishes Open Access books. A recent example is the book
Swan,Nivel,Kant,Hedges,Atkinson,Steunebrink:TheRoadtoGeneralIntelligence
https://link.springer.com/book/10.1007/978-3-031-08020-3
IndexedbySCOPUS,DBLP,WTIFrankfurteG,zbMATH,SCImago.
AllbookspublishedintheseriesaresubmittedforconsiderationinWebofScience.
· ·
Arash Shaban-Nejad Martin Michalowski
Simone Bianco
Editors
Multimodal AI in Healthcare
A Paradigm Shift in Health Intelligence
Editors
ArashShaban-Nejad MartinMichalowski
Oak-RidgeNationalLaboratory(ORNL) SchoolofNursing
CenterforBiomedicalInformatics UniversityofMinnesota
TheUniversityofTennesseeHealth Minneapolis,MN,USA
ScienceCenter(UTHSC)
Memphis,TN,USA
SimoneBianco
ResearchCenter
IBMAlmaden
SanJose,CA,USA
ISSN 1860-949X ISSN 1860-9503 (electronic)
StudiesinComputationalIntelligence
ISBN 978-3-031-14770-8 ISBN 978-3-031-14771-5 (eBook)
https://doi.org/10.1007/978-3-031-14771-5
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature
SwitzerlandAG2023
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse
ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Multimodal Artificial Intelligence is a relatively new concept in high performance
computational sciences that aims at integrating multiple data streams in different
formats(e.g.,text,image,video,audio,andnumericaldata)toimprovetheaccuracy
ofinformationextractionandinference,reducebias,andgenerateanoverallbetter
representationofthephysical,medical,orsocietalprocessesdescribedbythedata.
IncorporatingmultimodalAItoprocessmultidimensionalandmultimodaldatasets
inmissioncriticaldomainssuchashealthandmedicinecanadvancehealthanalytics,
improve case finding/prediction, diagnosis, risk stratification, referrals, and follow
upanddecision-makingbyhealthprofessionalsandpolicymakers.
ThisbookaimstohighlightthelatestachievementsintheuseofAIandmultimodal
artificial intelligence in biomedicine and healthcare. The edited volume contains
selectedpaperspresentedatthe2022HealthIntelligenceworkshopandtheassociated
DataHackathon/Challenge,co-locatedwiththe36thAssociationfortheAdvance-
mentofArtificialIntelligence(AAAI)conference,andpresentsanoverviewofthe
issues,challenges,andpotentialsinthefield,alongwithnewresearchresults.This
book provides information for researchers, students, industry professionals, clini-
cians,andpublichealthagenciesinterestedintheapplicationsofAIinpublichealth
andmedicine.
Memphis,USA ArashShaban-Nejad
Minneapolis,USA MartinMichalowski
SanJose,USA SimoneBianco
v
Contents
Multimodal Artificial Intelligence: Next Wave of Innovation
inHealthcareandMedicine ........................................ 1
ArashShaban-Nejad,MartinMichalowski,andSimoneBianco
Unsupervised Numerical Reasoning to Extract Phenotypes
fromClinicalTextbyLeveragingExternalKnowledge ................ 11
AshwaniTanwar,JingqingZhang,JuliaIve,VibhorGupta,andYikeGuo
Domain-specific Language Pre-training for Dialogue
ComprehensiononClinicalInquiry-AnsweringConversations ......... 29
ZhengyuanLiu,PavitraKrishnaswamy,andNancyF.Chen
ClinicalDialogueTranscriptionErrorCorrectionUsingSeq2Seq
Models ........................................................... 41
GayaniNanayakkara,NirmalieWiratunga,DavidCorsar,KyleMartin,
andAnjanaWijekoon
CustomizedTrainingofPretrainedLanguageModelstoDetect
PostIntentsinOnlineHealthSupportGroups ....................... 59
TootiyaGiyahchi,SameerSingh,IanHarris,andCorneliaPechmann
EXPECT-NLP: An Integrated Pipeline and User Interface
forExploringPatientPreferencesDirectlyfromPatient-Generated
Text .............................................................. 77
DavidJohnson, NickDragojlovic, NicolaKopac, YifuChen,
MarilynLenzen,SarahLeHuray,SamanthaPollard,DeanRegier,
MarkHarrison, AmyGeorge, GiuseppeCarenini, RaymondNg,
andLarryLynd
MedicationErrorDetectionUsingContextualLanguageModels ....... 91
YuJiangandChristianPoellabauer
vii
viii Contents
LatentRepresentationWeightsLearningoftheIndefiniteLength
ofViewsforConceptionDiagnosis .................................. 101
BoLi,MengzeSun,YuanYu,YuanyuanZhao,ZhongliangXiang,
andZhiyongAn
PhenotypingwithPositiveUnlabelledLearningforGenome-Wide
AssociationStudies ................................................ 117
AndreVauvelle,HamishTomlinson,AaronSim,andSpirosDenaxas
Out-of-Distribution Detection for Medical Applications:
GuidelinesforPracticalEvaluation ................................. 137
KarinaZadorozhny,PatrickThoral,PaulElbers,andGiovanniCinà
ARobustSystemtoDetectandExplainPublicMaskWearing
Behavior .......................................................... 155
AkshayGuptaandBiplavSrivastava
AFederatedCoxModelwithNon-proportionalHazards .............. 171
D.KaiZhang,FrancescaToni,andMatthewWilliams
AStepTowardsAutomatedFunctionalAssessmentofActivities
ofDailyLiving .................................................... 187
BappadityaDebnath, MaryO’brien, SwagatKumar,
andArdhenduBehera
TheInterpretationofDeepLearningBasedAnalysisofMedical
Images—An Examination of Methodological and Practical
ChallengesUsingChestX-rayData ................................. 203
SteinarValssonandOgnjenArandjelovic´
Predicting Drug Functions from Adverse Drug Reactions
byMulti-labelDeepNeuralNetwork ................................ 215
PranabDasandDilwarHussainMazumder
PatternDiscoveryinPhysiologicalDatawithBytePairEncoding ...... 227
NazgolTavabiandKristinaLerman
PredictingICUAdmissionsforHospitalizedCOVID-19Patients
withaFactorGraph-basedModel ................................... 245
YuruiCao, PhuongCao, HaotianChen, KarlM.Kochendorfer,
AndrewB.Trotter, WilliamL.Galanter, PaulM.Arnold,
andRavishankarK.Iyer
SemanticNetworkAnalysisofCOVID-19VaccineRelatedText
fromReddit ....................................................... 257
ChadA.Melton,JintaeBae,OlufuntoA.Olusanya,JonHaelBrenas,
EunKyongShin,andArashShaban-Nejad
Contents ix
TowardsProvidingClinicalInsightsonLongCovidfromTwitter
Data .............................................................. 267
RohanBhambhoria,JadSaab,SaraUppal,XinLi,ArturYakimovich,
JunaidBhatti, NirmaKhatriValdamudi, DianaMoyano,
MichaelBales,ElhamDolatabadi,andSedefAkinliKocak
PredictingInfectionsintheCovid-19Pandemic—LessonsLearned ..... 279
SharareZehtabian, SiavashKhodadadeh, DamlaTurgut,
andLadislauBölöni
ImprovingRadiologyReportGenerationwithAdaptiveAttention ..... 293
LinWangandJieChen
InstantaneousPhysiologicalEstimationUsingVideoTransformers ..... 307
AmbareeshRevanur, AnanyanandaDasari, ConradS.Tucker,
andLászlóA.Jeni
Automated Vision-Based Wellness Analysis for Elderly Care
Centers ........................................................... 321
XijieHuang,JeffryWicaksana,ShichaoLi,andKwang-TingCheng
Efficient Extraction of Pathologies from C-Spine Radiology
ReportsUsingMulti-taskLearning ................................. 335
ArijitSehanobish, NathanielBrown, IshitaDaga, JayashriPawar,
DanielleTorres, AnasuyaDas, MurrayBecker, RichardHerzog,
BenjaminOdry,andRonVianu
Benchmarking Uncertainty Quantification on Biosignal
ClassificationTasksUnderDatasetShift ............................. 347
TongXia,JingHan,andCeciliaMascolo
MiningAdverseDrugReactionsfromUnstructuredMediums
atScale ........................................................... 361
HashamUlHaq,VeyselKocaman,andDavidTalby
AGraph-basedImputationMethodforSparseMedicalRecords ....... 377
RamonViñas,XuZheng,andJerHayes
UsingNursingNotestoPredictLengthofStayinICUforCritically
IllPatients ........................................................ 387
SudeshnaJana,TirthankarDasgupta,andLipikaDey
AutomaticClassificationofDementiaUsingTextandSpeechData ..... 399
HeeJeongHan,SuhasB.N.,LingQiu,andSaeedAbdullah
UnifiedTensorNetworkforMultimodalDementiaDetection .......... 409
TruongHoang,Thuy-TrinhNguyen,andHoangD.Nguyen
Contributors
AbdullahSaeed College of Information Sciences and Technology, Pennsylvania
StateUniversity,UniversityPark,PA,USA
AnZhiyong SchoolofComputerScienceandTechnology,ShandongTechnology
andBusinessUniversity,Yantai,PR,China;
School of Statistics, Shandong Technology and Business University, Yantai, PR,
China
Arandjelovic´ Ognjen UniversityofStAndrews,StAndrews,Scotland,UK
ArnoldPaulM. CarleFoundationHospital,Urbana,IL,USA
B.N.Suhas CollegeofInformationSciencesandTechnology,PennsylvaniaState
University,UniversityPark,PA,USA
BaeJintae KoreaUniversity,Seoul,SouthKorea
BalesMichael Hoffmann-LaRocheLtd.,Mississauga,ON,Canada
BeckerMurray CoveraHealth,NYC,NewYork,USA
BeheraArdhendu EdgeHillUniversity,Ormskirk,UK
BhambhoriaRohan Queen’sUniversity,Kingston,ON,Canada
BhattiJunaid Manulife,Toronto,ON,Canada
BiancoSimone Altos Labs—Bay Area Institute of Science, BAI Computational
InnovationHub,RedwoodCity,CA,USA
BrenasJonHael SangerInstitute,Cambridge,UK
BrownNathaniel CoveraHealth,NYC,NewYork,USA
BölöniLadislau DepartmentofComputerScience,UniversityofCentralFlorida,
Orlando,FL,USA
CaoPhuong UniversityofIllinoisUrbana-Champaign,Champaign,IL,USA
xi
