Intelligent Agents in Data-intensive Computing

231 Pages·2016·5.21 MB·English

by  Joanna Kołodziej

Studies in Big Data 14
Joanna Kołodziej
Luís Correia
José Manuel Molina Editors
Intelligent
Agents in
Data-intensive
Computing
Studies in Big Data
Volume 14
Series editor
Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland
e-mail: [email protected]
About this Series
Theseries“StudiesinBigData”(SBD)publishesnewdevelopmentsandadvances
in the various areas of Big Data- quickly and with a high quality. The intent is to
coverthetheory,research,development,andapplicationsofBigData,asembedded
inthefieldsofengineering,computerscience,physics,economicsandlifesciences.
The books of the series refer to the analysis and understanding of large, complex,
and/or distributed data sets generated from recent digital sources coming from
sensorsorotherphysicalinstrumentsaswellassimulations,crowdsourcing,social
networks or other internet transactions, such as emails or video click streams and
other. The series contains monographs, lecture notes and edited volumes in Big
Data spanning the areas of computational intelligence incl. neural networks,
evolutionary computation, soft computing, fuzzy systems, as well as artificial
intelligence,datamining,modernstatisticsandOperationsresearch,aswellasself-
organizing systems. Of particular value to both the contributors and the readership
are the short publication timeframe and the world-wide distribution, which enable
both wide and rapid dissemination of research output.
More information about this series at http://www.springer.com/series/11970
ł í
Joanna Ko odziej Lu s Correia
(cid:129)
é
Jos Manuel Molina
Editors
Intelligent Agents
in Data-intensive Computing
123
Editors
JoannaKołodziej JoséManuel Molina
Department ofComputer Science Department ofComputer Science
CracowUniversity of Technology Universidad Carlos III deMadrid
Kraków Madrid
Poland Spain
Luís Correia
Faculty of Sciences
University of Lisbon
Lisboa
Portugal
ISSN 2197-6503 ISSN 2197-6511 (electronic)
Studies in BigData
ISBN978-3-319-23741-1 ISBN978-3-319-23742-8 (eBook)
DOI 10.1007/978-3-319-23742-8
LibraryofCongressControlNumber:2015948781
SpringerChamHeidelbergNewYorkDordrechtLondon
©SpringerInternationalPublishingSwitzerland2016
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
foranyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
SpringerInternationalPublishingAGSwitzerlandispartofSpringerScience+BusinessMedia
(www.springer.com)
To our Families and Friends with Love
and Gratitude
Preface
SincetheintroductionoftheInternet,wewitnessanexplosivegrowthinthevolume,
velocity, and variety of data created on a daily basis. These data originated from
numeroussourcesincludingmobiledevices,sensors,individualarchives,Internetof
Things,governmentdataholdings,softwarelogs,publicprofilesinsocialnetworks,
commercial datasets, etc. Everyday, more than 2.5 exabytes of data—as much as
90 % of the data in the world—are generated, and such data volumes are growing
fasterthantheycanbeanalyzed.Recently,largefederatedcomputingfacilitieshave
become prominent for enabling advanced scientific discoveries. For example, the
WorldwideLargeHadronColliderComputingGrid,aglobalcollaborationofmore
than 150 computing centers in nearly 40 countries, has evolved to provide global
computingresourcestostore,distribute,andanalyze25petabytesofdataannually.1
In geosciences, new remote sensing technologies provide an increasingly detailed
view of the world’s natural resources and environment. The data captured in geo-
sciencesdomain enable more sophisticated and accurateanalysisand prediction of
naturalevents(e.g.,climatechange)anddisasters(e.g.,earthquake).Hugeprogress
can also be observed in developments of sensor-based and mobile technologies
supporting thenew Internet ofThings(IoT) systems.
The issue so-called “Data Intensive Computing (DIC)”, or more generally
“Big Data” problem, requires a continuous increase of processing speed in data
servers and in the network infrastructure, and it becomes difficult for the analysis
and interpretation with on-hand data management applications. Hundreds of
petabytes of big data generated everyday need to be efficiently processed (stored,
distributed, and indexed with a schema and semantics) in a way that does not
compromise end-users’ Quality of Service (QoS)in terms of data availability,data
search delay, data analysis delay, data processing cost, and the like. The evolution
of big data processing technologies includes new generation scalable cloud com-
puting data centers, distributed message queue frameworks (e.g., Apache Kafka,2
1wlcg.web.cern.ch
2www.en.wikipedia.org/wiki/Apache_Kafka
vii
viii Preface
Amazon Kinesis3), data storage frameworks (e.g., MongoDB,4 Cassandra5),
parallelprocessingframeworks(e.g.,Hadoop,6Spark7),anddistributeddatamining
frameworks (e.g., Mahout8, GraphLab9). However, there is still a lack of effective
decisionmakinganddataminingsupporttoolsandtechniquesthatcansignificantly
reduce the data analysis delays, minimize data processing cost, and maximize data
availability.
Over the past decades, multi-agent systems (MAS) have been put forward as a
keymethodologytodevelopandtacklethecomplexityofself-managinglarge-scale
distributed systems and can be applied in solving the problems with massive data
processing and analytics and social components. The proactiveness of intelligent
agentsbecameavaluablekeyfeatureintherecentdataintensivecomputingandbig
data analytics. The architecture of the multi-agent systems in such approaches
provides agents with different competence levels that can adequately fit the com-
putingcapabilitiesofeachsystemcomponent,fromclientagentsinmobiledevices
to more heavy-duty ones as nodes/servers in networks of highly distributed envi-
ronments.SystemswithsuchMASsupportcanruninacooperativestance,withno
need for central coordination.
This compendium herewith presents novel concepts in the analysis, implemen-
tation, and evaluation of next generation agent-based and cloud-based models,
technologies,simulations,andimplementationsfordataintensiveapplications.The
general big data analytics problems are defined in “Bio-Inspired ICT for Big Data
Management in Healthcare”. Di Stefano et al. present the concept of bio-inspired
approachtoICTsystems,whichishelpfulinextractionofgeneralknowledgefrom
data, and make it available to those who have to design ICT interventions and
services, considering the multitude of resources, in terms of data and sources,
computational limits, and social dynamics. The authors demonstrate how to effi-
ciently deploy the multi-agent framework to implement such bio-inspired mecha-
nisms (social dynamics and biodiversity) for big data processing, storage
management, and analysis. The illustrating application area is patient-centered
healthcare, where big data analytics and storage techniques become crucial due to
the large amounts of personal data related to patient care. The presented health
mining paradigm allows to analyze correlations and causality relations between
diseases and patients, which is the background of today’s personalized medicine.
3www.aws.amazon.com/kinesis
4www.mongodb.org
5cassandra.apache.org
6hadoop.apache.org
7spark.apache.org
8mahout.apache.org
9Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J.M. Hellerstein, “Distributed
GraphLab:AFrameworkforMachineLearningintheCloud,”inVLDBEndow,2012,pp.716–727.
Preface ix
The remaining eight chapters are structured into two main parts:
1. Agents: A growing number of complex distributed systems are composed of
customisable knowledge-based building blocks (agents). “Control Aspects in
Multiagent Systems”–“Large-Scale Simulations with FLAME” discuss novel
agent-oriented concepts and techniques that can be useful in big data analytics,
from agents’ organization, interaction and evolution models, to large-scale
agents’ simulators and demonstrators.
2. Clouds:Collectivelyknownascomputationalresourcesorsimplyinfrastructure,
for which computing elements, storage, and services represent a crucial com-
ponent in the formulation of intelligent cloud systems. Consequently, “Cloud
Computing and Multiagent Systems, a Promising Relationship”–“Adaptive
Resource Allocation in Cloud Computing Based on Agreement Protocols”
showcase techniques and concepts for modernization of standard cloud mech-
anisms, to manage big data sets and to integrate them with multi-agent
frameworks.
Agents
The first three chapters in this part focus on MAS organization, evolution, and
management. In “Control Aspects in Multiagent Systems”, Cicirelli and Nigro
discuss the control methods in agent systems. They developed a scalable control
framework for modeling, analysis, and execution of parallel and distributed
time-dependentMASwithasynchronousagents’actionsandmessage-passing.That
frameworkhasbeenimplementedinpopularJADEMASenvironmentandtestedin
the following two practical use cases: (i) MAS-supported help desk offering ser-
vices to a company’s customers and (ii) the schedulability analysis of a real-time
taskingmodelunderfixedpriorityscheduling.Theprovidedexperimentsshowhigh
flexibility and practical usefulness of the proposed control methods, thanks to the
separation of the agents’ control structures from the business logic of MAS.
Theanalysisofagents’behaviorandinteractionsremainsachallengingresearch
task, especially in big data approaches. Recently, Markovian agents (MAs) have
been proposed as an effective MAS solution for data intensive problems, such as
wireless sensor networks deployed in environmental protection, situation aware-
ness, etc. This model is represented by a continuous time Markov chain (CTMC).
Additionally, for Markovian agents, the infinitesimal generator has a fixed local
component that may depend on the geographical position of the MA, and a com-
ponent that depends on the interactions with other MAs. Gribaudo and Iacono in
“ADifferentPerspectiveofAgent-BasedTechniques:MarkovianAgents”focuson
the MAS performance evaluation problem and exploit decoupling between the
MAS model representation with the software implementation and the performance
evaluation technique used to study the overall behavior of the system. In their
approach,MAispresentedasaveryvaluabletoolwhenthenumberofentitiesofa
x Preface
model is too big for a reasonable application of other state space-based techniques
or accurate simulation approaches.
The implementation of scalable efficient agents’ organization and interaction
models is a challenging task also for engineers and practitioners. Gath et al. in
“Autonomous, Adaptive, and Self-Organized Multiagent Systems for the
Optimization of Decentralized Industrial Processes” provide a comprehensive
criticalstate-of-the-artanalysisofagent-basedapproachesintransportlogisticsand
indicate the main limitations for their application in Industry 4.0 processes. The
authorsdevelopedthedispAgentmodelwithcoordinationandnegotiationprotocols
foragents’synchronization,andasolverfortheagents’individualdecisions.They
evaluated this model on two established benchmarks for the vehicle routing
problem with real-world big data. The experiments demonstrated the difficulties in
practical applications of theoretically developed agents’ behavioral models.
Potentially, the discussed difficulties in MAS research may be partially solved
throughdevelopmentsofnovelimplementationtools(newdedicatedprogramming
languages, ontologies) and scalable simulation platforms for large-scale MAS.
Subburaj and Urban in “Formal Specification Language and Agent Applications”
developed the extended Descartes specification language for the improvement of
MAS implementation dedicated to concrete applications. The domains analyzed
include information management, electronic commerce, and medical applications
where agents are used in patient monitoring and health care. Coackley et al. in
“Large-Scale Simulations with FLAME” present various implementations of
FLAMEsimulationenvironmentforexecutionofverycomplexdata-intensivetasks
in large-scale MAS. They justified the flexibility, efficiency, and usefulness of the
developed software environments in a comprehensive application analysis in dif-
ferent areas (economy, biology, and engineering). A formal verification of the
effectiveness of the proposed simulator was discussed for the case of a rule-based
system, which can be formally analyzed using model checking techniques.
Clouds
Cloud computing gives application developers the ability to marshal virtually
unlimited resources with an option to pay-per-use as needed, instead of requiring
upfront investments in resources that may never be optimally used. Once appli-
cationsarehostedincloudresources,usersareabletoaccessthemfromanywhere
at any time, using devices in a wide range of classes, from mobile devices
(smartphones, tablets) to large computers. The data center cloud provides virtual
centralization of application, computing, and data. While cloud computing opti-
mizestheuseofresources,itdoesnot(yet)provideaneffectivesolutionhostingbig
data applications.
Large-scale distributed data-intensive applications, e.g., 3D model reconstruc-
tion in medical imaging, medical body area networks, earth observation applica-
tions,distributedbloganalysis,andhighenergyphysicssimulation,needtoprocess