Partitional Clustering Algorithms

420 Pages·2016·8.13 MB·English

by  Celebi M. Embre (Ed.)

M. Emre Celebi Editor
Partitional
Clustering
Algorithms
Partitional Clustering Algorithms

M. Emre Celebi
Editor
Partitional Clustering
Algorithms
123
Editor
M.EmreCelebi
DepartmentofComputerScience
LouisianaStateUniversityShreveport
Shreveport,LA,USA
ISBN978-3-319-09258-4 ISBN978-3-319-09259-1(eBook)
DOI10.1007/978-3-319-09259-1
SpringerChamHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2014946973
©SpringerInternationalPublishingSwitzerland2015
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Clustering,theunsupervisedclassificationofpatternsintogroups,isoneofthemost
important tasks in exploratory data analysis. Primary goals of clustering include
gaining insight into, classifying, and compressing data. Clustering has a long and
rich history that spans a variety of scientific disciplines including anthropology,
biology, medicine, psychology, statistics, mathematics, engineering, and computer
science.Asaresult,numerousclusteringalgorithmshavebeenproposedsincethe
early1950s.Amongthesealgorithms,partitional(nonhierarchical)oneshavefound
manyapplications,especiallyinengineeringandcomputerscience.
The goal of this volume is to summarize the state of the art in partitional
clustering. The intended audience includes researchers and practitioners, who are
increasinglyusingpartitionalclusteringalgorithmstoanalyzetheirdata.
The volume opens with a chapter on model-based clustering entitled “Recent
Developments in Model-Based Clustering with Applications.” In this chapter,
Melnykov et al. review the latest developments in this field including semi-
supervised clustering, nonparametric mixture modeling, initialization strategies,
merging mixture components, and handling spurious solutions. In “Accelerating
Lloyd’sAlgorithmfork-MeansClustering,”HamerlyandDrakepresentasurveyof
triangleinequality-basedaccelerationtechniquesforthecelebratedk-meanscluster-
ingalgorithm.Basedonextensiveexperiments,theauthorsconcludethatasuitable
application of the triangle inequality can provide dramatic speedups of up to 40x
overanaiveimplementationofthestandardLloyd’salgorithm.Inanotherk-means
related chapter entitled “Linear, Deterministic, and Order-Invariant Initialization
Methods for the k-Means Clustering Algorithm,” Celebi and Kingravi investigate
theempiricalperformanceofsixlinear,deterministic,andorder-invariantk-means
initialization methods on a large and diverse collection of data sets from the
UCI Machine Learning Repository. Their results demonstrate that two relatively
unknownhierarchicalinitializationmethodsoutperformtheremainingfourmethods
withrespecttotwoobjectiveeffectivenesscriteria.Theyalsoshowthatoneofthe
most recent initialization methods performs surprisingly poorly. In “Nonsmooth
v
vi Preface
OptimizationBasedAlgorithmsinClusterAnalysis,”BagirovandMohebiapproach
theproblemofpartitionalclusteringusingnonsmoothandnonconvexoptimization.
Basedonthisformulation,theydesignanefficientincrementalalgorithmsimilarto
k-meansthatcanhandle` and` normsbesidesthecommonlyused` norm.
1 1 2
In“FuzzyClusteringAlgorithmsandValidityIndicesforDistributedData,”Ven-
draminetal.presentaframeworktogeneralizeseveralfuzzyclusteringalgorithms
tohandledistributeddatawithoutresortingtoapproximations.Thisframeworkalso
allows the exact calculation of a variety of relative validity indices to evaluate the
quality of fuzzy partitions. The authors also describe a procedure based on this
framework for the estimation of the number of clusters in parallel and distributed
settings.
In“DensityBasedClustering:AlternativestoDBSCAN,”Brauneetal.propose
two algorithms similar to the celebrated DBSCAN algorithm. Unlike DBSCAN,
both algorithms require only one input parameter. One of these algorithms gives
similar results to DBSCAN while being able to assign multiple cluster labels to a
datapoint,whereastheotheroneissignificantlyfasterthanDBSCAN.
In “Nonnegative Matrix Factorization for Interactive Topic Modeling and Doc-
ument Clustering,” Kuang et al. propose a novel formulation of the nonnegative
matrix factorization (NMF) problem based on the block coordinate descent algo-
rithm. The authors present a clustering algorithm based on this formulation and
proveitsconvergence.Inadditiontoextendingthisframeworktosparseandweakly
supervised clustering, the authors describe a method to determine the number
of clusters based on random sampling and consensus clustering. Experiments on
various real-world document data sets demonstrate the advantage of the proposed
NMF clustering algorithm in terms of clustering quality, convergence behavior,
sparseness,andconsistency.
In “Overview of Overlapping Partitional Clustering Methods,” Ben N’Cir et al.
review the fundamental concepts of partitional overlapping clustering and present
a survey of widely known partitional overlapping clustering algorithms as well
as techniques to evaluate the quality of non-disjoint partitions. Furthermore, the
authors investigate the ability of various clustering algorithms to generate overlap-
pingpartitionsfrommulti-labeledreal-worlddatasets.
In “On Semi-Supervised Clustering,” Bongini et al. present a survey of semi-
supervisedclustering(SSC).Theauthorsfirstgiveaconceptualoverviewofthefield
and then discuss some of the most important algorithms for SSC including COP-
COBWEB, COP-kMeans, HMRF k-means, seeded k-means, constrained k-means,
andactivefuzzyconstrainedclustering.Theauthorsconcludewithadiscussionof
futuredirectionsforthisrelativelynewfield.
In “Consensus of Clusterings Based on High-Order Dissimilarities,” Aidos
and Fred describe a novel dynamic clustering algorithm based on a recently
proposeddissimilaritymeasurecalled“dissimilarityincrements.”Startingfroman
initial partition, this algorithm incorporates a merging strategy based on either a
likelihood-ratiotestoratestbasedontheminimumdescriptionlengthprinciple.The
authors address the initialization dependence of their algorithm using a consensus
function-based combination strategy. Finally, the best partition is selected using a
Preface vii
criterion based on dissimilarity increments. Experimental results demonstrate the
effectiveness of the proposed algorithm on a variety of synthetic as well as real-
worlddatasets.
In “Hubness-Based Clustering of High-Dimensional Data,” Tomašev et al.
investigatethehubnessphenomenonobservedink-nearestneighborgraphsofhigh-
dimensional data. The authors demonstrate that hubness complicates the cluster
discoveryprocessbyreducingtheseparabilityofclusters.Theauthorsthenreview
some recent clustering algorithms that exploit the hubness phenomenon and then
introduceakernel-basedclusteringalgorithmthatdoesnotrestricttheshapeofthe
clusterstohyperspheres.
A chapter entitled “Clustering for Monitoring Distributed Data Streams” by
Barouti et al. completes the volume. The authors consider an application of
clustering to monitoring data streams in a distributed system. Unlike conventional
clustering algorithms that group similar data points into clusters, monitoring
requires that clusters with dissimilar points cancel each other as much as possible.
Theauthorsdeviseanovelclusteringalgorithmtotacklethisproblemanddemon-
stratethatityieldsareductionincommunicationload.
Wehopethatthisvolumefocusedonpartitionalclusteringwilldemonstratethe
significant progress that has occurred in this field in recent years. We also hope
thatthedevelopmentsreportedinthisvolumewillmotivatefurtherresearchinthis
excitingfield.
Shreveport,LA,USA M.EmreCelebi

Contents
1 RecentDevelopmentsinModel-BasedClustering
withApplications ........................................................... 1
VolodymyrMelnykov,SemharMichael,andIgorMelnykov
2 AcceleratingLloyd’sAlgorithmfork-MeansClustering .............. 41
GregHamerlyandJonathanDrake
3 Linear,Deterministic,andOrder-InvariantInitialization
MethodsfortheK-MeansClusteringAlgorithm ....................... 79
M.EmreCelebiandHassanA.Kingravi
4 NonsmoothOptimizationBasedAlgorithmsinClusterAnalysis..... 99
AdilM.BagirovandEhsanMohebi
5 FuzzyClusteringAlgorithmsandValidityIndices
forDistributedData........................................................ 147
L.Vendramin,M.C.Naldi,andR.J.G.B.Campello
6 DensityBasedClustering:AlternativestoDBSCAN................... 193
ChristianBraune,StephanBesecke,andRudolfKruse
7 NonnegativeMatrixFactorizationforInteractiveTopic
ModelingandDocumentClustering...................................... 215
DaKuang,JaegulChoo,andHaesunPark
8 OverviewofOverlappingPartitionalClusteringMethods ............ 245
Chiheb-EddineBenN’Cir,GuillaumeCleuziou,
andNadiaEssoussi
9 OnSemi-SupervisedClustering........................................... 277
MarcoBongini,FriedhelmSchwenker,andEdmondoTrentin
10 ConsensusofClusteringsBasedonHigh-OrderDissimilarities ...... 313
HelenaAidosandAnaFred
ix