Erik Siegel & Adam Retter

581 Pages·2014·11.73 MB·English

eXist
Get a head start with eXist, the open source NoSQL database and “ This book tells you e
application development platform built entirely around XML technologies. everything you need
With this hands-on guide, you’ll learn eXist from the ground up, from using X
to know to implement
this feature-rich database to work with millions of documents to building
eXist, all the way from
complex web applications that take advantage of eXist’s many extensions.
writing your first queries
If you’re familiar with XML—as a student, professor, publisher, or developer—
you’ll find that eXist is ideal for all kinds of documents. This book shows you to building sophisticated i
how to store, query, and search documents with XQuery and other XML web applications.” s
technologies, and how to construct applications on top of the database
—Priscilla Walmsley
with tools such as eXide and eXist’s built-in development environment.
XML consultant and author of XQuery
t
■ Manage both data-oriented and text-oriented markup “ Erik Siegel and Adam
documents securely
Retter have written a
■ Build a sample application that analyzes and searches technical tour de force
Shakespeare’s plays
about the database in
■ Go inside the architecture and learn how eXist processes
their eponymous book
documents
eXist, one that's both
■ Learn how to work with eXist’s internal development environment
eminently readable
■ Choose among various indexes, including a full-text index
based on Apache Lucene while still digging deep
■ Dive into eXist’s APIs for integrating or interacting with the into the inner workings
database of the database and how
eXist
■ Extend eXist by building your own Triggers, Scheduled Tasks, to use it.”
and XQuery extension modules
—Kurt Cagle
Principal Evangelist at Avalon Consulting
Erik Siegel is a content engineer and XML specialist who runs Xatapult consul-
tancy in The Netherlands. He specializes in content design and conversion, XML
S
schemas and transformations, and eXist and XProc applications. i
e
g
Adam Retter, Director of Evolved Binary in the UK and a cofounder of eXist
e
Solutions in Germany, has been a member of the eXist Core Development Team l &
since 2005. He is also a member of the XML Guild and an Invited Expert to the
R
W3C XML Query Working Group.
e
t A NOSQL DOCUMENT DATABASE AND APPLICATION PLATFORM
t
e
r
XML/DATABASES Twitter: @oreillymedia
facebook.com/oreilly
US $44.99 CAN $47.99
ISBN: 978-1-449-33710-0
Erik Siegel & Adam Retter
www.it-ebooks.info
eXist
Get a head start with eXist, the open source NoSQL database and application “ This book tells you e
development platform built entirely around XML technologies. With this everything you need
hands-on guide, you’ll learn eXist from the ground up, from using this feature- X
to know to implement
rich database to work with millions of documents to building complex web
eXist, all the way from
applications that take advantage of eXist’s many extensions.
writing your first queries
If you’re familiar with XML—as a student, professor, publisher, or developer—
you’ll find that eXist is ideal for all kinds of documents. This book shows you to building sophisticated i
how to store, query, and search documents with XQuery and other XML web applications.” s
technologies, and how to construct applications on top of the database
—Priscilla Walmsley
with tools such as eXide and eXist’s built-in development environment.
XML consultant and author of XQuery
t
■ Manage both data-oriented and text-oriented markup “ Erik Siegel and Adam
documents securely
Retter have written a
■ Build a sample application that analyzes and searches technical tour de force
Shakespeare’s plays
about the database in
■ Go inside the architecture and learn how eXist processes
their eponymous book
documents
eXist, one that's both
■ Learn how to work with eXist’s internal development environment
eminently readable
■ Choose among various indexes, including a full-text index
based on Apache Lucene while still digging deep
■ Dive into eXist’s APIs for integrating or interacting with the into the inner workings
database of the database and how
eXist
■ Extend eXist by building your own Triggers, Scheduled Tasks, to use it.”
and XQuery extension modules
—Kurt Cagle
Principal Evangelist at Avalon Consulting
Erik Siegel is a content engineer and XML specialist who runs Xatapult consul-
tancy in The Netherlands. He specializes in content design and conversion, XML
S
schemas and transformations, and eXist and XProc applications. i
e
g
Adam Retter, Director of Evolved Binary in the UK and a cofounder of eXist
e
Solutions in Germany, has been a member of the eXist Core Development Team l &
since 2005. He is also a member of the XML Guild and an Invited Expert to the
R
W3C XML Query Working Group.
e
t A NOSQL DOCUMENT DATABASE AND APPLICATION PLATFORM
t
e
r
XML/DATABASES Twitter: @oreillymedia
facebook.com/oreilly
US $44.99 CAN $47.99
ISBN: 978-1-449-33710-0
Erik Siegel & Adam Retter
www.it-ebooks.info
eXist
Erik Siegel and Adam Retter
www.it-ebooks.info
eXist
by Erik Siegel and Adam Retter
Copyright © 2015 Erik Siegel and Adam Retter. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/
institutional sales department: 800-998-9938 or [email protected] .
Editors: Simon St. Laurent and Allyson MacDonald Proofreader: Rachel Head
Production Editor: Matthew Hacker Indexer: Lucie Haskins
Copyeditor: Rachel Monaghan Interior Designer: David Futato
Cover Designer: Ellie Volckhausen
Illustrator: Rebecca Demarest
December 2014: First Edition
Revision History for the First Edition
2014-12-04: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781449337100 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. eXist, the cover image of a lettered
araçari, and related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and the authors have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use of
or reliance on this work. Use of the information and instructions contained in this work is at your own
risk. If any code samples or other technology this work contains or describes is subject to open source
licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
978-1-449-33710-0
[LSI]
www.it-ebooks.info
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
What Is eXist? 1
eXist Compared to Other Database Systems 3
History 5
Competitors 7
Open Source Competitors 8
Closed Source, Commercial Competitors 8
Who Is Using eXist, and for What? 9
Contributing to the Community 13
Individuals Using eXist 14
Organizations Using eXist 14
Authors Using eXist 15
Developers Using eXist 16
Additional Resources 16
2. Getting Started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Downloading and Installing eXist 19
Preconditions 19
Downloading eXist 20
Things to Decide Before Installing 20
Installing eXist 22
Post-Installation Checks 22
Starting and Stopping eXist with a GUI 23
Starting and Stopping eXist from the Command Line 24
A First Tour Around Town 24
The Dashboard 24
Playing Around 27
What’s in Your Database 27
What’s on Your Disk 28
iii
www.it-ebooks.info
The Java Admin Client 29
Getting Files into and out of the Database 30
Hello eXist! 31
Hello Data 31
Hello XQuery 32
Hello XSLT 33
Hello XInclude 35
Hello XForms 35
3. Using eXist 101. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Preparations and Basic Application Setup 39
eXist Terminology 40
Exporting Documents from eXist 40
Designing an Application’s Collection Structure and Importing Data 42
Viewing the Data 43
Listing the Plays (XML) 45
Listing with the collection Function 45
Listing with the xmldb Extension Module 48
Listing the Plays (HTML) 48
Analyzing the Plays 51
Linking the Analysis to the Play Overview 55
Searching the Plays 56
Searching Using Straight XQuery 56
Searching Using an Index 58
Creating a Log 60
What’s Next? 64
4. Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Deployment Architectures 67
Embedded Architecture 68
Client/Server Database Architecture 69
Web Application Platform Architecture 70
Storage Architecture 72
XML Document Storage and Indexing 72
Binary Document Storage 76
Efficient XML Processing Architecture 76
Collections 77
Documents 79
Dynamic Level Numbering of Nodes 80
Dynamic Level Numbering and Updates 83
Paging and Caching 85
iv | Table of Contents
www.it-ebooks.info
5. Working with the Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
The Database’s Content 87
Help: Where Is My XML? 87
Terminology 88
Properties of Collections and Resources 88
System Collections 90
Addressing Collections, Resources, and Files 91
The XPath Collection and Doc Functions in eXist 93
The collection Function 93
The doc Function 94
Querying the Database Using REST 94
Security 95
GET Requests 95
PUT Requests 97
DELETE Requests 97
POST Requests 98
Ad Hoc Querying 100
Updating Documents 101
eXist’s XQuery Update Extension 102
XUpdate 105
Controlling the Database from Code 107
Specifying Collections and Resources for the xmldb Extension Module 107
Getting Information 108
Creating Resources and Collections 109
Setting Permissions 110
Moving, Removing, and Renaming 110
6. XQuery for eXist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
eXist’s XQuery Implementation 111
XQuery 1.0 Support 111
XQuery 3.0 Support 112
Other XQuery Extras 116
XQuery Execution 118
Serialization 118
Controlling Serialization 119
Serialization Options 119
Controlling XQuery Execution 123
eXist XQuery Pragmas 123
Limiting Execution Time and Output Size 124
Other Options 124
XQuery Documentation with xqDoc 125
Table of Contents | v
www.it-ebooks.info
7. Extension Modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Types of Extension Modules 127
Extension Modules Written in Java 127
Extension Modules Written in XQuery 128
Enabling Extension Modules 128
Enabling Java Extension Modules 129
Enabling XQuery Extension Modules 130
8. Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Security Basics 134
Users 134
Groups 135
Permissions 135
Default Permissions 138
Managing Users and Groups 140
Group Managers 140
Tools for User and Group Management 141
User and Group Management with the Java Admin Client 145
Scenario 145
Creating a Group 145
Creating Users 147
Setting Group Managers 149
Managing Permissions 151
Tools for Permission Management 151
Permission Management with the Java Admin Client 154
Access Control Lists 156
Access Control Entries 157
ACLs by Example 158
Managing ACLs 164
Realms 166
LDAP Realm Module 166
Other Realm Modules 174
Hardening 174
Reducing Collateral Damage 175
Reducing the Attack Surface 177
User Authentication in XQuery 187
xmldb:authenticate 188
xmldb:login 188
Backups 189
9. Building Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Overview 191
vi | Table of Contents
www.it-ebooks.info
Which Technology to Use? 192
Application Aspects 192
Getting Started, Quickly? 193
Where to Store Your Application? 194
URL Mapping Using URL Rewriting 194
Anatomy of a URL Rewriting-Based Application 195
How eXist Finds the Controller 198
The URL Rewriting Controller’s Environment 199
The Controller’s Output XML Format 200
Advanced URL Control 203
Changing the URL for URL Rewriting 205
Changing Jetty Settings: Port Number and URL Prefix 205
The controller-config.xml Configuration File 206
Proxying eXist Behind a Web Server 207
Requests, Sessions, and Responses 209
The request Extension Module 209
The session Extension Module 211
The response Extension Module 211
Application Security 212
Running with Extra Permissions 214
Global Error Pages 215
Building Applications with RESTXQ 215
Configuring RESTXQ 216
RESTXQ Annotations 217
RESTXQ XQuery Extension Functions 227
Packaging 227
Examples 229
The Packaging Format 229
The Prepare and Finish Scripts 233
Creating Packages 234
Additional Remarks About Packages 234
10. Other XML Technologies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
XSLT 238
Embedding Stylesheets or Not 238
Invoking XSLT with the Transform Extension Module 240
Passing XSLT Parameters 241
Invoking XSLT by Processing Instruction 242
Stylesheet Details 243
XInclude 243
Including Documents 244
Including Query Results 245
Table of Contents | vii
www.it-ebooks.info
Error Handling and Fallback 245
Validation 246
Implicit Validation 246
Explicit Validation 248
Collations 251
Supported Collations 251
Specifying Collations 251
XSL-FO 252
XForms 254
XForms Instances 255
XForms Submissions 258
betterForm 263
XSLTForms 265
11. Basic Indexing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Indexing Example 272
Index Types 274
Structural Index 274
Range Indexes 274
NGram Indexes 275
Full-Text Indexes 275
Configuring Indexes 275
Configuring Range Indexes 276
Configuring NGram Indexes 278
Maintaining Indexes 278
Using Indexes 279
Using the Structural Index 279
Using the Range Indexes 279
Using the NGram Indexes 280
General Optimization Tips 281
Debugging Indexes 281
Checking Index Definitions 282
Checking Index Usage 282
Tracing the Optimizer 283
12. Text Indexing and Lookup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
Full-Text Index and KWIC Example 285
Configuring Full-Text Indexes 286
Configuring the Search Context 287
Handling Mixed Content 290
Maintaining the Full-Text Index 291
Searching with the Full-Text Index 292
viii | Table of Contents
www.it-ebooks.info