250085 VU Tensor methods for data science and scientific computing (2021W)

This course will cover the basics of low-rank tensor decompositions, a modern computational tool for large-scale problems. Possible applications, which will be discussed in the course, are associated with such areas as data science, quantitative neuroscience, spectroscopy, psychometrics, arithmetic complexity and data compression; however, some of the most illustrative applications belong to the field of scientific computing. The course will first cover the canonical polyadic, Tucker, block-term and tensor-train decompositions from a linear-algebraic perspective and then focus on the use of low-rank tensor decompositions in computational mathematics. In the second part, the course will focus on the tensor-train (TT) decomposition, originally developed under the name of matrix product states (MPS) in computational quantum physics. This tensor decomposition appears naturally as a representation of functions from low-rank refinement in the construction of finite-element approximations and will be presented in this way in the course. In particular, in the context second-order linear elliptic problems, the low-rank approximation of functions, depending on their regularity, will be analyzed and state-of-the-art methods for preconditioning and solving optimality equations (linear systems) will be covered (including the construction, implementation and numerical analysis of such methods).

***

This course spotlights the intersection of two areas of modern applied mathematics:
* low-rank approximation and analysis of abstract data represented by multi-dimensional arrays
and
* adaptive numerical methods for solving PDE problems.
For the mentioned two areas, however seemingly disjoint, the idea of exactly representing or approximating «data» in a suitable low-dimensional subspace of a large (possibly infinite-dimensional) space is equally natural. The notions of matrix rank and of low-rank matrix approximation, presented in basic courses of linear algebra, are central to one of many possible expressions of this idea.

In psychometrics, signal processing, image processing and (vaguely defined) data mining, low-rank tensor decompositions have been studied as a way of formally generalizing the notion of rank from matrices to higher-dimensional arrays (tensors). Several such generalizations have been proposed, including the canonical polyadic (CP) and Tucker decompositions and the tensor-SVD, with the primary motivation of analyzing, interpreting and compressing datasets. In this context, data are often thought of as parametrizations of images, video, social networks or collections of interconnected texts; on the other hand, data representing functions (which often occur in computational mathematics) are remarkable for the possibility of precise analysis.

The tensor-train (TT) and the more general hierarchical Tucker decompositions were developed in the community of numerical mathematics, more recently and with particular attention to PDE problems. In fact, exactly the same and very similar representations had long been used for the numerical simulation of many-body quantum systems by computational chemists and physicists under the names of «matrix-product states» (MPS) and «multilayer multi-configuration time-dependent Hartree». These low-rank tensor decompositions are based on subspace approximation, which can be performed adaptively and iteratively, in a multilevel fashion. In a broader context of PDE problems, this leads to numerical methods that are formally based on generic discretizations but effectively operate on adaptive, data-driven discretizations constructed «online», in the course of computation. In several settings, such methods achieve the accuracy of sophisticated problem-specific methods.

***

The goal of the course is to introduce students to the foundations of modern low-rank tensor methods.
The course is to provide students with ample opportunity for starting own research.