Table Of ContentPreface Page: vii
Contents Page: ix
About the Author Page: xix
List of Symbols and Abbreviations Page: xxi
Symbols Page: xxi
Greek Letters Page: xxiii
Latin Letters Page: xxiv
Abbreviations and Algorithms Page: xxviii
Part I: Algebraic Tensors Page: 1
Chapter 1: Introduction Page: 3
1.1 What are Tensors? Page: 3
1.1.1 Tensor Product of Vectors Page: 3
1.1.2 Tensor Product of Matrices, Kronecker Product Page: 5
1.1.3 Tensor Product of Functions Page: 7
1.2 Where do Tensors Appear? Page: 8
1.2.1 Tensors as Coefficients Page: 8
1.2.2 Tensor Decomposition for Inverse Problems Page: 9
1.2.3 Tensor Spaces in Functional Analysis Page: 10
1.2.4 Large-Sized Tensors in Analysis Applications Page: 10
1.2.5 Tensors in Quantum Chemistry Page: 13
1.3 Tensor Calculus Page: 13
1.4 Preview Page: 14
1.4.1 Part I: Algebraic Properties Page: 14
1.4.2 Part II: Functional Analysis of Tensors Page: 15
1.4.3 Part III: Numerical Treatment Page: 16
1.4.4 Topics Outside the Scope of the Monograph Page: 17
1.5 Software Page: 18
1.6 Comments about the Early History of Tensors Page: 18
1.7 Notations Page: 19
Chapter 2: Matrix Tools Page: 23
2.1 Matrix Notations Page: 23
2.2 Matrix Rank Page: 25
2.3 Matrix Norms Page: 27
2.4 Semidefinite Matrices Page: 29
2.5 Matrix Decompositions Page: 30
2.5.1 Cholesky Decomposition Page: 30
2.5.2 QR Decomposition Page: 31
2.5.3 Singular-Value Decomposition Page: 33
2.6 Low-Rank Approximation Page: 39
2.7 Linear Algebra Procedures Page: 41
2.8 Dominant Columns Page: 44
Chapter 3: Algebraic Foundations of Tensor Spaces Page: 49
3.1 Vector Spaces Page: 49
3.1.1 Basic Facts Page: 49
3.1.2 Free Vector Space over a Set Page: 50
3.1.3 Quotient Vector Space Page: 52
3.1.4 (Multi-)Linear Maps, Algebraic Dual, Basis Transformation Page: 53
3.2 Tensor Product Page: 54
3.2.1 Constructive Definition Page: 54
3.2.2 Characteristic Properties Page: 56
3.2.3 Isomorphism to Matrices for d = 2 Page: 58
3.2.4 Tensors of Order d > 3 Page: 60
3.2.5 Different Types of Isomorphisms Page: 63
3.2.6 Rr and Tensor Rank Page: 65
3.3 Linear and Multilinear Mappings Page: 76
3.3.1 Definition on the Set of Elementary Tensors Page: 76
3.3.2 Embeddings Page: 77
3.4 Tensor Spaces with Algebra Structure Page: 85
3.5 Symmetric and Antisymmetric Tensor Spaces Page: 88
3.5.1 Basic Definitions Page: 88
3.5.2 Quantics Page: 91
3.5.3 Determinants Page: 92
3.5.4 Application of Functionals Page: 93
Part II: Functional Analysis of Tensor Spaces Page: 95
Chapter 4: Banach Tensor Spaces Page: 97
4.1 Banach Spaces Page: 97
4.1.1 Norms Page: 97
4.1.2 Basic Facts about Banach Spaces Page: 98
4.1.3 Examples Page: 100
4.1.4 Operators Page: 101
4.1.5 Dual Spaces Page: 104
4.1.6 Examples Page: 106
4.1.7 Weak Convergence Page: 106
4.1.8 Continuous Multilinear Mappings Page: 108
4.2 Topological Tensor Spaces Page: 108
4.2.1 Notations Page: 108
4.2.2 Continuity of the Tensor Product, Crossnorms Page: 110
4.2.3 Projective Norm ||.||^(V;W) Page: 116
4.2.4 Duals and Injective Norm ||.|| (V;W) Page: 120
4.2.5 Embedding of V* into L(V W;W) Page: 127
4.2.6 Reasonable Crossnorms Page: 128
4.2.7 Reflexivity Page: 132
4.2.8 Uniform Crossnorms Page: 133
4.2.9 Nuclear and Compact Operators Page: 136
4.3 Tensor Spaces of Order d Page: 137
4.3.1 Continuity, Crossnorms Page: 137
4.3.2 Recursive Definition of the Topological Tensor Space Page: 140
4.3.3 Proofs Page: 143
4.3.4 Embedding into Embedding into L(V; Vj) and L(V;V�) Page: 147
4.3.5 Intersections of Banach Tensor Spaces Page: 151
4.3.6 Tensor Space of Operators Page: 154
4.4 Hilbert Spaces Page: 155
4.4.1 Scalar Product Page: 155
4.4.2 Basic Facts about Hilbert Spaces Page: 155
4.4.3 Operators on Hilbert Spaces Page: 157
4.4.4 Orthogonal Projections Page: 159
4.5 Tensor Products of Hilbert Spaces Page: 161
4.5.1 Induced Scalar Product Page: 161
4.5.2 Crossnorms Page: 163
4.5.3 Tensor Products of L(Vj ; Vj) Page: 164
4.5.4 Gagliardo–Nirenberg Inequality Page: 165
4.5.5 Partial Scalar Products Page: 170
4.6 Tensor Operations Page: 171
4.6.1 Vector Operations Page: 171
4.6.2 Matrix-Vector Multiplication Page: 172
4.6.3 Matrix-Matrix Operations Page: 172
4.6.4 Hadamard Multiplication Page: 174
4.6.5 Convolution Page: 174
4.6.6 Function of a Matrix Page: 176
4.7 Symmetric and Antisymmetric Tensor Spaces Page: 179
4.7.1 Hilbert Structure Page: 179
4.7.2 Banach Spaces and Dual Spaces Page: 180
Chapter 5: General Techniques Page: 183
5.1 Vectorisation Page: 183
5.1.1 Tensors as Vectors Page: 183
5.1.2 Kronecker Tensors Page: 185
5.2 Matricisation Page: 187
5.2.1 General Case Page: 187
5.2.2 Finite-Dimensional Case Page: 189
5.2.3 Hilbert Structure Page: 194
5.2.4 Matricisation of a Family of Tensors Page: 198
5.3 Tensorisation Page: 198
Chapter 6: Minimal Subspaces Page: 201
6.1 Statement of the Problem, Notations Page: 201
6.2 Tensors of Order Two Page: 202
6.2.1 Existence of Minimal Subspaces Page: 202
6.2.2 Use of the Singular-Value Decomposition Page: 205
6.2.3 Minimal Subspaces for a Family of Tensors Page: 206
6.3 Minimal Subspaces of Tensors of Higher Order Page: 207
6.4 Hierarchies of Minimal Subspaces and Page: 210
6.5 Sequences of Minimal Subspaces Page: 213
6.6 Minimal Subspaces of Topological Tensors Page: 218
6.6.1 Setting of the Problem Page: 218
6.6.2 First Approach Page: 218
6.6.3 Second Approach Page: 221
6.7 Minimal Subspaces for Intersection Spaces Page: 224
6.7.1 Algebraic Tensor Space Page: 224
6.7.2 Topological Tensor Space Page: 225
6.8 Linear Constraints and Regularity Properties Page: 226
6.9 Minimal Subspaces for (Anti-)Symmetric Tensors Page: 229
Part III: Numerical Treatment Page: 231
Chapter 7: r-Term Representation Page: 233
7.1 Representations in General Page: 234
7.1.1 Concept Page: 234
7.1.2 Computational and Memory Cost Page: 235
7.1.3 Tensor Representation versus Tensor Decomposition Page: 236
7.2 Full and Sparse Representation Page: 237
7.3 r-Term Representation Page: 238
7.4 Tangent Space and Sensitivity Page: 241
7.4.1 Tangent Space Page: 241
7.4.2 Sensitivity Page: 242
7.5 Representation of Vj Page: 244
7.6 Conversions between Formats Page: 247
7.6.1 From Full Representation into r-Term Format Page: 247
7.6.2 From r-Term Format into Full Representation Page: 248
7.6.3 From r-Term into N-Term Format for r>N Page: 248
7.6.4 Sparse-Grid Approach Page: 249
7.6.5 From Sparse Format into Page: 251
7.7 Representation of (Anti-)Symmetric Tensors Page: 253
7.7.1 Sums of Symmetric Rank-1 Tensors Page: 254
7.7.2 Indirect Representation Page: 254
7.8 Modifications Page: 256
Chapter 8: Tensor Subspace Representation Page: 257
8.1 The Set Tr Page: 257
8.2 Tensor Subspace Formats Page: 261
8.2.1 General Frame or Basis Page: 261
8.2.2 Transformations Page: 264
8.2.3 Tensors in KI Page: 265
8.2.4 Orthonormal Basis Page: 266
8.2.5 Summary of the Formats Page: 270
8.2.6 Hybrid Format Page: 271
8.3 Higher-Order Singular-Value Decomposition (HOSVD) Page: 273
8.3.1 Definitions Page: 273
8.3.2 Examples Page: 275
8.3.3 Computation and Computational Cost Page: 277
8.4 Tangent Space and Sensitivity Page: 283
8.4.1 Uniqueness Page: 283
8.4.2 Tangent Space Page: 284
8.4.3 Sensitivity Page: 285
8.5 Conversions between Different Formats Page: 287
8.5.1 Conversion from Full Representation into Tensor Subspace Format Page: 287
8.5.2 Conversion from Rr to Tr Page: 287
8.5.3 Conversion from Tr to Rr Page: 291
8.5.4 A Comparison of Both Representations Page: 292
8.5.5 r-Term Format for Large r > N Page: 293
8.6 Joining two Tensor Subspace Representation Systems Page: 293
8.6.1 Trivial Joining of Frames Page: 293
8.6.2 Common Bases Page: 294
Chapter 9: r-Term Approximation Page: 297
9.1 Approximation of a Tensor Page: 297
9.2 Discussion for r = 1 Page: 299
9.3 Discussion in the Matrix Case d = 2 Page: 301
9.4 Discussion in the Tensor Case d > 3 Page: 303
9.4.1 Nonclosedness of Rr Page: 303
9.4.2 Border Rank Page: 304
9.4.3 Stable and Unstable Sequences Page: 306
9.4.4 A Greedy Algorithm Page: 308
9.5 General Statements on Nonclosed Formats Page: 309
9.5.1 Definitions Page: 309
9.5.2 Nonclosed Formats Page: 311
9.5.3 Discussion of F = Rr Page: 312
9.5.4 General Case Page: 312
9.5.5 On the Strength of Divergence Page: 313
9.5.6 Uniform Strength of Divergence Page: 314
9.5.7 Extension to Vector Spaces of Larger Dimension Page: 317
9.6 Numerical Approaches for the r-Term Approximation Page: 318
9.6.1 Use of the Hybrid Format Page: 318
9.6.2 Alternating Least-Squares Method Page: 320
9.6.3 Stabilised Approximation Problem Page: 329
9.6.4 Newton’s Approach Page: 330
9.7 Generalisations Page: 332
9.8 Analytical Approaches for the r-Term Approximation Page: 333
9.8.1 Quadrature Page: 334
9.8.2 Approximation by Exponential Sums Page: 335
9.8.3 Sparse Grids Page: 346
Chapter 10: Tensor Subspace Approximation Page: 347
10.1 Truncation to Tr Page: 347
10.1.1 HOSVD Projection Page: 348
10.1.2 Successive HOSVD Projection Page: 350
10.1.3 Examples Page: 352
10.1.4 Other Truncations Page: 354
10.1.5 L Estimate of the Truncation Error Page: 355
10.2 Best Approximation in the Tensor Subspace Format Page: 358
10.2.1 General Setting Page: 358
10.2.2 Approximation with Fixed Format Page: 359
10.2.3 Properties Page: 361
10.3 Alternating Least-Squares Method (ALS) Page: 362
10.3.1 Algorithm Page: 362
10.3.2 ALS for Different Formats Page: 364
10.3.3 Approximation with Fixed Accuracy Page: 367
10.4 Analytical Approaches for the Tensor Subspace Approximation Page: 369
10.4.1 Linear Interpolation Techniques Page: 369
10.4.2 Polynomial Approximation Page: 372
10.4.3 Polynomial Interpolation Page: 374
10.4.4 Sinc Approximations Page: 376
10.5 Simultaneous Approximation Page: 383
10.6 Resume Page: 385
Chapter 11: Hierarchical Tensor Representation Page: 387
11.1 Introduction Page: 387
11.1.1 Hierarchical Structure Page: 387
11.1.2 Properties Page: 390
11.1.3 Historical Comments Page: 391
11.2 Basic Definitions Page: 392
11.2.1 Dimension Partition Tree Page: 392
11.2.2 Algebraic Characterisation, Hierarchical Subspace Family Page: 394
11.2.3 Minimal Subspaces Page: 395
11.2.4 Conversions Page: 398
11.3 Construction of Bases Page: 400
11.3.1 Hierarchical Basis Representation Page: 400
11.3.2 Orthonormal Bases Page: 410
11.3.3 HOSVD Bases Page: 415
11.3.4 Tangent Space and Sensitivity Page: 422
11.3.5 Sensitivity Page: 422
11.3.6 Conversion from Rr to Hr Revisited Page: 429
11.4 Approximations in Hr Page: 431
11.4.1 Best Approximation in Hr Page: 431
11.4.2 HOSVD Truncation to Hr Page: 433
11.5 Joining two Hierarchical Tensor Representation Systems Page: 446
11.5.1 Setting of the Problem Page: 446
11.5.2 Trivial Joining of Frames Page: 447
11.5.3 Common Bases Page: 447
Chapter 12: Matrix Product Systems Page: 453
12.1 Basic TT Representation Page: 453
12.2 Function Case Page: 456
12.3 TT Format as Hierarchical Format Page: 456
12.3.1 Related Subspaces Page: 456
12.3.2 From Subspaces to TT Coefficients Page: 457
12.3.3 From Hierarchical Format to TT Format Page: 458
12.3.4 Construction with Minimal pj Page: 460
12.3.5 Extended TT Representation Page: 460
12.3.6 Properties Page: 461
12.3.7 HOSVD Bases and Truncation Page: 462
12.4 Conversions Rr to Tp Page: 463
12.4.1 Conversion from Rr to Tp Page: 463
12.4.2 Conversion from Tp to Hr with a General Tree Page: 463
12.4.3 Conversion from Hr to Tp Page: 465
12.5 Cyclic Matrix Products and Tensor Network States Page: 467
12.5.1 Cyclic Matrix Product Representation Page: 467
12.5.2 Site-Independent Representation Page: 470
12.5.3 Tensor Network Page: 471
12.6 Representation of Symmetric and Antisymmetric Tensors Page: 472
Chapter 13: Tensor Operations Page: 473
13.1 Addition Page: 474
13.1.1 Full Representation Page: 474
13.1.2 r-Term Representation Page: 475
13.1.3 Tensor Subspace Representation Page: 475
13.1.4 Hierarchical Representation Page: 477
13.2 Entry-wise Evaluation Page: 477
13.2.1 r-Term Representation Page: 478
13.2.2 Tensor Subspace Representation Page: 478
13.2.3 Hierarchical Representation Page: 479
13.2.4 Matrix Product Representation Page: 480
13.3 Scalar Product Page: 480
13.3.1 Full Representation Page: 481
13.3.2 r-Term Representation Page: 481
13.3.3 Tensor Subspace Representation Page: 482
13.3.4 Hybrid Format Page: 484
13.3.5 Hierarchical Representation Page: 485
13.3.6 Orthonormalisation Page: 489
13.4 Change of Bases Page: 490
13.4.1 Full Representation Page: 490
13.4.2 Hybrid r-Term Representation Page: 490
13.4.3 Tensor Subspace Representation Page: 491
13.4.4 Hierarchical Representation Page: 491
13.5 General Binary Operation Page: 492
13.5.1 r-Term Representation Page: 493
13.5.2 Tensor Subspace Representation Page: 493
13.5.3 Hierarchical Representation Page: 494
13.6 Hadamard Product of Tensors Page: 495
13.7 Convolution of Tensors Page: 496
13.8 Matrix-Matrix Multiplication Page: 496
13.9 Matrix-Vector Multiplication Page: 497
13.9.1 Identical Formats Page: 498
13.9.2 Separable Form (13.25a) Page: 498
13.9.3 Elementary Kronecker Tensor (13.25b) Page: 499
13.9.4 Matrix in p-Term Format (13.25c) Page: 500
13.10 Functions of Tensors, Fixed-Point Iterations Page: 501
13.11 Example: Operations in Quantum Chemistry Applications Page: 503
Chapter 14: Tensorisation Page: 507
14.1 Basics Page: 507
14.1.1 Notations, Choice for TD Page: 507
14.1.2 Format Htens Page: 509
14.1.3 Operations with Tensorised Vectors Page: 510
14.1.4 Application to Representations by Other Formats Page: 512
14.1.5 Matricisation Page: 513
14.1.6 Generalisation to Matrices Page: 514
14.2 Approximation of Grid Functions Page: 515
14.2.1 Grid Functions Page: 515
14.2.2 Exponential Sums Page: 516
14.2.3 Polynomials Page: 516
14.2.4 Multiscale Feature and Conclusion Page: 520
14.2.5 Local Grid Refinement Page: 520
14.3 Convolution Page: 521
14.3.1 Notation Page: 521
14.3.2 Separable Operations Page: 523
14.3.3 Tensor Algebra A(l0) Page: 524
14.3.4 Algorithm Page: 531
14.4 Fast Fourier Transform Page: 534
14.4.1 FFT for Cn Vectors Page: 534
14.4.2 FFT for Tensorised Vectors Page: 535
14.5 Tensorisation of Functions Page: 537
14.5.1 Isomorphism Fn Page: 537
14.5.2 Scalar Products Page: 538
14.5.3 Convolution Page: 539
14.5.4 Continuous Functions Page: 539
Chapter 15: Multivariate Cross Approximation Page: 541
15.1 Approximation of General Tensors Page: 541
15.1.1 Approximation of Multivariate Functions Page: 542
15.1.2 Multiparametric Boundary-Value Problem and PDE with Stochastic Coefficients Page: 543
15.1.3 Function of a Tensor Page: 545
15.2 Notations Page: 546
15.3 Properties in the Matrix Case Page: 548
15.4 Case Page: 551
15.4.1 Matricisation Page: 551
15.4.2 Nestedness Page: 553
15.4.3 Algorithm Page: 555
Chapter 16: Applications to Elliptic Partial Differential Equations Page: 559
16.1 General Discretisation Strategy Page: 559
16.2 Solution of Elliptic Boundary-Value Problems Page: 560
16.2.1 Separable Differential Operator Page: 561
16.2.2 Discretisation Page: 561
16.2.3 Solution of the Linear System Page: 563
16.2.4 Accuracy Controlled Solution Page: 565
16.3 Solution of Elliptic Eigenvalue Problems Page: 566
16.3.1 Regularity of Eigensolutions Page: 566
16.3.2 Iterative Computation Page: 568
16.3.3 Alternative Approaches Page: 569
16.4 On Other Types of PDEs Page: 569
Chapter 17: Miscellaneous Topics Page: 571
17.1 Minimisation Problems on Page: 571
17.1.1 Algorithm Page: 571
17.1.2 Convergence Page: 572
17.2 Solution of Optimisation Problems Involving Tensor Formats Page: 573
17.2.1 Formulation of the Problem Page: 574
17.2.2 Reformulation, Derivatives, and Iterative Treatment Page: 575
17.3 Ordinary Differential Equations Page: 576
17.3.1 Tangent Space Page: 576
17.3.2 Dirac–Frenkel Discretisation Page: 576
17.3.3 Tensor Subspace Format Tr Page: 577
17.3.4 Hierarchical Format Hr Page: 579
17.4 ANOVA Page: 581
17.4.1 Definitions Page: 581
17.4.2 Properties Page: 582
17.4.3 Combination with Tensor Representations Page: 584
17.4.4 Symmetric Tensors Page: 584
References Page: 585
Index Page: 599
Description:Special numerical techniques are already needed to deal with n × n matrices for large n. Tensor data are of size n × n ×...× n=nd, where nd exceeds the computer memory by far. They appear for problems of high spatial dimensions. Since standard methods fail, a particular tensor calculus is needed to treat such problems. This monograph describes the methods by which tensors can be practically treated and shows how numerical operations can be performed. Applications include problems from quantum chemistry, approximation of multivariate functions, solution of partial differential equations, for example with stochastic coefficients, and more. In addition to containing corrections of the unavoidable misprints, this revised second edition includes new parts ranging from single additional statements to new subchapters. The book is mainly addressed to numerical mathematicians and researchers working with high-dimensional data. It also touches problems related to Geometric Algebra.