Table Of ContentLecture Notes in Artificial Intelligence 1572
Subseries ofLectureNotesin ComputerScience
Editedby J.G. Carbonelland J. Siekmann
Lecture Notes in Computer Science
Editedby G.Goos,J. Hartmanisand J.van Leeuwen
3
Berlin
Heidelberg
NewYork
Barcelona
HongKong
London
Milan
Paris
Singapore
Tokyo
Paul Fischer Hans Ulrich Simon (Eds.)
Computational
Learning Theory
4th European Conference, EuroCOLT’99
Nordkirchen, Germany, March 29-31, 1999
Proceedings
1 3
SeriesEditors
JaimeG.Carbonell, CarnegieMellonUniversity,Pittsburgh,PA,USA
Jo¨rgSiekmann, UniversityofSaarland,Saarbru¨cken,Germany
VolumeEditors
PaulFischer
Lehrstuhlfu¨rInformatikII,Universita¨tDortmund
D-44221Dortmund,Germany
E-mail:[email protected]
HansUlrichSimon
Fakulta¨tfu¨rMathematik,RuhrUniversita¨tBochum
D-44780Bochum,Germany
E-mail:[email protected]
Cataloging-in-Publicationdataappliedfor
DieDeutscheBibliothek-CIP-Einheitsaufnahme
Computationallearningtheory:4thEuropeanconference;
proceedings/EuroCOLT’99,Nordkirchen,Germany,March29-31,
1999.PaulFischer;HansUlrichSimon(ed.).-Berlin;Heidelberg;
NewYork;Barcelona;HongKong;London;Milan;Paris;
Singapore;Tokyo:Springer,1999
(Lecturenotesincomputerscience;1572:Lecturenotesinartificial
intelligence)
ISBN3-540-65701-0
CRSubjectClassification(1998):I.2.6,I.2.3,F.4.1,F.1.1,F.2
ISBN3-540-65701-0Springer-VerlagBerlinHeidelbergNewYork
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting,
reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965,
initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer-Verlag.Violationsare
liableforprosecutionundertheGermanCopyrightLaw.
(cid:1)c Springer-VerlagBerlinHeidelberg1999
PrintedinGermany
Typesetting:Camera-readybyauthor
SPIN10703032 06/3142–543210 Printedonacid-freepaper
Preface
This volume contains papers presented at the Fourth European Conference on
ComputationalLearningTheory,whichwasheldatNordkirchenCastle,inNord-
kirchen, NRW, Germany, from March 29 to 31, 1999. This conference is the
fourth in a series of bi-annual conferences established in 1993.
TheEuroCOLTconferencesarefocusedontheanalysisoflearningalgorithms
and the theory of machine learning, and bring together researchersfrom a wide
varietyofrelated(cid:12)elds.Someofthe issuesandtopicsthatareaddressedinclude
the sample and computational complexity of learning speci(cid:12)c model classes,
frameworksmodeling the interaction between the learner, teacher and the envi-
ronment (such as learning with queries, learning control policies and inductive
inference),learningwithcomplexmodels(suchasdecisiontrees,neuralnetworks,
and support vector machines), learning with minimal prior assumptions (such
as mistake-bound models, universal prediction, and agnostic learning), and the
study of model selection techniques. We hope that these conferences stimulate
an interdisciplinary scienti(cid:12)c interaction that will be fruitful in all represented
(cid:12)elds.
Thirty-(cid:12)ve papers were submitted to the program committee for considera-
tion, and twenty-one of these were accepted for presentation at the conference
and publication in these proceedings. In addition, Robert Schapire (AT & T
Labs), and Richard Sutton (AT & T Labs) were invited to give lectures and
contributea writtenversionto theseproceedings.Therewereanumber ofother
joint events including a banquet and an excursion to Mu¨nster. The IFIP WG
1.4 Scholarship was awarded to Andra(cid:19)s Antos for his paper \Lower bounds on
the rate of convergence of nonparametric pattern recognition".
The EuroCOLT’99 conference was sponsored by the DFG (Deutsche For-
schungsgemeinschaft) and by IFIP through WG 1.4.
We want to thank everybody who helped to make this meeting possible: the
authors for submitting papers, the Program Committee and referees for their
e(cid:11)ort in composing the program, the Steering Committee, the sponsors, the
local organizers, and Springer-Verlag. Thanks to Norbert Klasner and Karsten
Tinnefeld for their help in preparing the proceedings, and thanks to Margret
Vaupelfor heradministrativesupport.The programcommittee wishesto thank
everybody who acted as subreferee.
January 1999 Paul Fischer and Hans Simon
Conference Chairs
EuroCOLT’99
Organization
EUROCOLT’99isorganizedbythedepartmentofComputerScience,University
of Dortmund.
Organizers
Paul Fischer (University of Dortmund, Germany)
Hans Simon (Ruhr-Universita¨t Bochum, Germany)
Steering Commitee
Shai Ben-David (Chair) Technion, Haifa
Nicol(cid:18)o Cesa-Bianchi,University of Milano
Paul Fischer, University of Dortmund
Gabor Lugosi, Pompeu Fabra University, Barcelona
Arun Sharma, University of New South Wales, Sydney
Hans Simon, Ruhr-Universita¨t Bochum
Program Commitee
Martin Anthony, London School of Economics
Peter Auer, Technical University of Graz
Shai Ben-David, Technion, Haifa
Nader Bshouty, University of Calgary
John Case, University of Delaware
Nicol(cid:18)o Cesa-Bianchi,University of Milano
Paul Fischer, University of Dortmund
Jyrki Kivinen, University of Helsinki
Nick Littlestone, NEC Research Institute, Princeton
Ga(cid:19)bor Lugosi, Pompeu Fabra University, Barcelona
Arun Sharma, University of New South Wales, Sydney
Hans Simon, Ruhr-Universita¨t Bochum (Chair)
Organization VII
Subreferees
Peter Bartlett Claudio Gentile Mark Pendrith
Gianluca Della Vedova Tibor Hegedu}s Dale Schuurmans
Nadav Eiron Christian Killin Frank Stephan
Tapio Elomaa Juha K¨arkka¨inen Henry Tirri
Ran El-Yaniv Eric Martin Jaak Vilo
Yoav Freund Petri Myllym¨aki
Sponsoring Institutions
Deutsche Forschungsgemeinsschaft(DFG) and IFIP WG 1.4.
Table of Contents
Invited Lectures
Theoretical Views of Boosting ::::::::::::::::::::::::::::::::::::::: 1
Robert E. Schapire
Open Theoretical Questions in Reinforcement Learning ::::::::::::::::: 11
Richard S. Sutton
Learning from Random Examples
A Geometric Approach to Leveraging Weak Learners ::::::::::::::::::: 18
Nigel Du(cid:11)y, David Helmbold
Query by Committee, Linear Separation and Random Walks :::::::::::: 34
Ran Bachrach, Shai Fine, Eli Shamir
Hardness Results for Neural Network Approximation Problems :::::::::: 50
Peter Bartlett, Shai Ben-David
Learning from Queries and Counterexamples
Learnability of Quanti(cid:12)ed Formulas :::::::::::::::::::::::::::::::::: 63
V(cid:19)(cid:16)ctor Dalmau, Peter Jeavons
Learning Multiplicity Automata from Smallest Counterexamples::::::::: 79
Ju¨rgen Forster
Exact Learning when Irrelevant Variables Abound ::::::::::::::::::::: 91
David Guijarro, V(cid:19)(cid:16)ctor Lav(cid:19)(cid:16)n, Vijay Raghavan
An Application of Codes to Attribute{E(cid:14)cient Learning :::::::::::::::: 101
Thomas Hofmeister
Learning Range Restricted Horn Expressions ::::::::::::::::::::::::: 111
Roni Khardon
Reinforcement Learning
On the Asymptotic Behavior of a Constant Stepsize Temporal-Di(cid:11)erence
Learning Algorithm :::::::::::::::::::::::::::::::::::::::::::::::: 126
Vladislav Tadi(cid:19)c
X Table of Contents
On-line Learning and Expert Advice
Direct and Indirect Algorithms for On-line Learning of Disjunctions ::::: 138
D. P. Helmbold, S. Panizza, M. K. Warmuth
Averaging Expert Predictions ::::::::::::::::::::::::::::::::::::::: 153
Jyrki Kivinen, Manfred K. Warmuth
Teaching and Learning
On Teaching and Learning Intersection-ClosedConcept Classes :::::::::: 168
Christian Kuhlmann
Inductive Inference
Avoiding Coding Tricks by Hyperrobust Learning :::::::::::::::::::::: 183
Matthias Ott, Frank Stephan
Mind Change Complexity of Learning Logic Programs:::::::::::::::::: 198
Sanjay Jain, Arun Sharma
Statistical Theory of Learning and Pattern Recognition
Regularized Principal Manifolds ::::::::::::::::::::::::::::::::::::: 214
Alex J. Smola, Robert C. Williamson, Sebastian Mika, Bernhard Scho¨lkopf
Distribution-Dependent Vapnik-Chervonenkis Bounds :::::::::::::::::: 230
Nicolas Vayatis, Robert Azencott
Lower Bounds on the Rate of Convergence of Nonparametric Pattern
Recognition :::::::::::::::::::::::::::::::::::::::::::::::::::::: 241
Andr(cid:19)as Antos
On Error Estimation for the Partitioning Classi(cid:12)cation Rule::::::::::::: 253
Ma(cid:19)rta Horva(cid:19)th
Margin Distribution Bounds on Generalization::::::::::::::::::::::::: 263
John Shawe-Taylor, Nello Cristianini
Generalization Performance of Classi(cid:12)ers in Terms of Observed Covering
Numbers:::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 274
John Shawe-Taylor, Robert C. Williamson
Entropy Numbers, Operators and Support Vector Kernels::::::::::::::: 285
Robert C. Williamson, Alex J. Smola, Bernhard Scho¨lkopf
Author Index ::::::::::::::::::::::::::::::::::::::::::::::::: 301
