Learning Coupled Actions of Lie Groups: Kuramoto Models for Robotics and Hyperbolic Data

Table of Links

Abstract and 1. Introduction

2. Some recent trends in theoretical ML

   2.1 Deep Learning via continuous-time controlled dynamical system

   2.2 Probabilistic modeling and inference in DL

   2.3 Deep Learning in non-Euclidean spaces

   2.4 Physics Informed ML

3. Kuramoto model

   3.1 Kuramoto models from the geometric point of view

   3.2 Hyperbolic geometry of Kuramoto ensembles

   3.3 Kuramoto models with several globally coupled sub-ensembles

4. Kuramoto models on higher-dimensional manifolds

   4.1 Non-Abelian Kuramoto models on Lie groups

   4.2 Kuramoto models on spheres

   4.3 Kuramoto models on spheres with several globally coupled sub-ensembles

   4.4 Kuramoto models as gradient flows

   4.5 Consensus algorithms on other manifolds

5. Directional statistics and swarms on manifolds for probabilistic modeling and inference on Riemannian manifolds

   5.1 Statistical models over circles and tori

   5.2 Statistical models over spheres

   5.3 Statistical models over hyperbolic spaces

   5.4 Statistical models over orthogonal groups, Grassmannians, homogeneous spaces

6. Swarms on manifolds for DL

   6.1 Training swarms on manifolds for supervised ML

   6.2 Swarms on manifolds and directional statistics in RL

   6.3 Swarms on manifolds and directional statistics for unsupervised ML

   6.4 Statistical models for the latent space

   6.5 Kuramoto models for learning (coupled) actions of Lie groups

   6.6 Grassmannian shallow and deep learning

   6.7 Ensembles of coupled oscillators in ML: Beyond Kuramoto models

7. Examples

   7.1 Wahba’s problem

   7.2 Linked robot’s arm (planar rotations)

   7.3 Linked robot’s arm (spatial rotations)

   7.4 Embedding multilayer complex networks (Learning coupled actions of Lorentz groups)

8. Conclusion and References

6.5 Kuramoto models for learning (coupled) actions of Lie groups

One of the main points of the present study is that many important ML problems are naturally stated as learning coupled actions of certain Lie groups. These include special orthogonal and unitary groups, as well as actions of the Lorentz groups for learning in hyperbolic geometries. In the present subsection we briefly discuss problems of this kind and conceptual approaches to them.

6.5.1 Non-Abelian Kuramoto models for learning (coupled) actions of orthogonal and unitary groups

Problems of learning optimal (coupled) rotations are common in robotics, when modeling motions. Apparent examples of this kind are rotations of linked robot’s arm with k joints. If the arm moves in the plane, this yields k coupled actions of SO(2). In the three-dimensional space, these are coupled actions of SO(3).

The problem of learning actions of SO(d) for d > 3 and SU(n) for n > 2 is relevant in computational physics and some other fields.

The system (10) with k oscillators is a convenient model of k coupled rotations in the n-dimensional vector space. Parameters to be learned are coupling strengths Kij . This model depends on a small number of parameters and can be trained efficiently. The obvious drawback is the limited representative power. There are several ways of increasing representative power, but they inevitably entail increased number of parameters. For instance, one can include several oscillators for each rotation.

Alternative models for learning SO(3) rotations are provided by models (12) and (13) on S3.

6.5.2 Kuramoto models on spheres for learning (coupled) actions of Lorentz groups

Computational physics is one of many fields that are being revolutionized with the advent of DL. Since Minkowski spacetime and its symmetries are one of central concepts in mathematical physics, DL models involving Lorentz groups have been investigated within this field [128].

Kuramoto models with several sub-ensembles (8) and (19) provide a natural framework for encoding (sequential) coupled actions of hyperbolic isometries in the unit disc and unit balls, respectively. Hence, these models can be used for learning data in hyperbolic spaces. Having in mind omnipresence of hyperbolic data, we believe that this is potentially the most promising field of applications of Kuramoto models in DL.

The idea can be straightforwardly extended to learning on products of spheres or balls, by training the model (19). Underline that training of (19) must be based on optimization algorithms over Riemannian manifolds, because only coupling strengths are Euclidean. The remaining parameters belong to special orthogonal groups (phase shifts) and to spheres (initial points).

In general, models (8) and (19) are significant for learning in hyperbolic geometries. An advantage of this approach is that it avoids mathematical apparatus (involving exponential map, gyrovector spaces and Möbius addition) that is frequently used in ML algorithms over hyperbolic data [32, 38, 39, 129].

The idea explained here raises important issues regarding representative power of the proposed models (19). This can be mathematically stated as the controllability problem over the Lie groups SO(d, 1) × · · · × SO(d, 1), where couplings, phase shifts and initial positions are regarded as controls. Namely, the question is if we can obtain each Lorentz transformation by varying coupling strength, the phase shift and three initial points. This question is quite involved and is still to be investigated using the general control theory on Lie groups [130]. Local controllability results seem to follow from the general theory. However, results on global controllability demand a careful and demanding analysis.