Riemannian Geometry

Overview & Motivation

In the Smooth Manifolds topic, we built the language for doing calculus on curved spaces: charts, tangent vectors, the differential. But that machinery alone cannot answer the most basic geometric questions. How long is a curve on the sphere? What angle do two curves make when they cross? What is the area of a region on a surface? Smooth manifolds, by themselves, have no notion of length, angle, or volume — they are topological objects with a differentiable structure, nothing more.

The missing ingredient is a Riemannian metric: a smoothly varying inner product on each tangent space. This single piece of additional structure transforms a smooth manifold into a Riemannian manifold — a space where we can measure everything. Lengths of curves, distances between points, angles between tangent vectors, areas, volumes, curvature: all flow from the metric tensor $g$.

Why should this matter for machine learning?

1. Natural gradient descent. The parameter space of a statistical model $\{p_\theta : \theta \in \Theta\}$ is a smooth manifold. The Fisher information matrix $g_{ij}(\theta) = \mathbb{E}\!\left[\frac{\partial \log p_\theta}{\partial \theta^i}\frac{\partial \log p_\theta}{\partial \theta^j}\right]$ is a Riemannian metric on $\Theta$. Standard gradient descent ignores this geometry; natural gradient descent (Amari, 1998) uses the metric to compute steepest descent in the intrinsic sense, converging faster and more invariantly.

2. The Cramér–Rao bound. The inverse of the Fisher metric gives the minimum variance of any unbiased estimator. This is a statement about the Riemannian geometry of the parameter space.

3. KL divergence as Riemannian distance. For nearby distributions, the Kullback–Leibler divergence $\mathrm{KL}(p_\theta \| p_{\theta + d\theta}) \approx \frac{1}{2} g_{ij}\, d\theta^i\, d\theta^j$ is the squared Riemannian distance element. Information-theoretic quantities are geometric.

What we cover. We construct the Riemannian metric and prove that every smooth manifold admits one (§2). We define curve lengths and the Riemannian distance function (§3). The musical isomorphisms — flat $\flat$ and sharp $\sharp$ — bridge tangent and cotangent spaces, revealing that the gradient is metric-dependent (§4). The Fundamental Theorem of Riemannian Geometry establishes the Levi-Civita connection as the unique torsion-free, metric-compatible way to differentiate vector fields (§5). Parallel transport carries vectors along curves, and its path-dependence — holonomy — is the first shadow of curvature (§6). The Riemannian volume form enables coordinate-invariant integration (§7). Isometries and Killing vector fields capture the symmetries of a Riemannian manifold (§8). We close with computational tools (§9) and the direct connection to the Fisher information metric and natural gradient descent (§10).

Prerequisites. This topic assumes familiarity with Smooth Manifolds: charts, smooth atlases, tangent spaces $T_pM$, the differential $dF_p$, and partitions of unity. We reference the Spectral Theorem when discussing eigendecompositions of the metric tensor.

---

Riemannian Metrics

The central object in Riemannian geometry is a smoothly varying choice of inner product on each tangent space.

In local coordinates $(x^1, \ldots, x^n)$, the metric is represented by a matrix of smooth functions:

$$g_{ij}(p) = g_p\!\left(\frac{\partial}{\partial x^i}\bigg|_p,\, \frac{\partial}{\partial x^j}\bigg|_p\right)$$

The inner product of two tangent vectors $v = v^i \frac{\partial}{\partial x^i}$ and $w = w^j \frac{\partial}{\partial x^j}$ is:

$$g_p(v, w) = g_{ij}(p)\, v^i\, w^j$$

where we use the Einstein summation convention (repeated indices are summed). At each point, the matrix $(g_{ij})$ is symmetric and positive definite — which is precisely the setting of the Spectral Theorem. Its eigenvalues are the principal stretches of the metric, and its eigenvectors are the principal directions.

Three fundamental examples.

Example 1: The Euclidean metric on $\mathbb{R}^n$. In standard coordinates, $g_{ij} = \delta_{ij}$ (the identity matrix). The inner product is the ordinary dot product $g_p(v, w) = v^1 w^1 + \cdots + v^n w^n$. The metric is the same everywhere — flat space.

Example 2: The Poincaré disk $\mathbb{D}^2$. The open unit disk $\{(x, y) : x^2 + y^2 < 1\}$ with the metric:

$$g = \frac{4}{(1 - x^2 - y^2)^2}\left(dx^2 + dy^2\right)$$

This is a conformal metric — it is a scalar multiple $\lambda^2$ of the Euclidean metric, where $\lambda = 2/(1 - r^2)$ is the conformal factor. As a point approaches the boundary of the disk ($r \to 1$), $\lambda \to \infty$: distances blow up. The Poincaré disk is a model of the hyperbolic plane $\mathbb{H}^2$ — a space of constant negative curvature.

Example 3: The round metric on $S^2$. In spherical coordinates $(\theta, \varphi)$ with $\theta \in (0, \pi)$ (colatitude) and $\varphi \in [0, 2\pi)$ (azimuth):

$$g = d\theta^2 + \sin^2\!\theta\; d\varphi^2 \qquad \Longleftrightarrow \qquad g_{ij} = \begin{pmatrix} 1 & 0 \\ 0 & \sin^2\!\theta \end{pmatrix}$$

The metric is diagonal: the $\theta$-direction has unit stretching everywhere, while the $\varphi$-direction shrinks as $\sin\theta \to 0$ near the poles. At the equator ($\theta = \pi/2$), the metric is locally Euclidean; at the poles, the $\varphi$-circles collapse to points.

Metric Tensor Explorer

Metric tensor g_ij at (θ=1.200, φ=0.800)

(

1.000

0

0

0.869

)

det(g) = 0.869

Eigenvalues (principal stretches)

λ₁ = 0.869, λ₂ = 1.000

Eigenvectors (principal directions)

e₁ = (0, 1.000)

e₂ = (1.000, 0)

Ellipse flattens near the poles where sin²θ → 0.

Manifold:Sphere S²Poincaré Disk 𝔻²EllipsoidShow metric ellipsesShow eigenvectorsθ:1.200φ:0.800

The natural question is: does every smooth manifold admit a Riemannian metric? The answer is yes, and the proof uses the partitions of unity from Smooth Manifolds.

The proof is non-constructive — it tells us that metrics exist, but the particular metric we get depends on the choice of atlas and partition of unity. In practice, the metrics we work with (Euclidean, round sphere, Poincaré, Fisher) come from the geometry of the problem, not from this existence argument.

---

Lengths of Curves and Riemannian Distance

The metric lets us measure the length of a curve, and from curve lengths we build a distance function.

In local coordinates, if $\gamma(t) = (\gamma^1(t), \ldots, \gamma^n(t))$, this becomes:

$$L(\gamma) = \int_a^b \sqrt{g_{ij}(\gamma(t))\, \dot\gamma^i(t)\, \dot\gamma^j(t)}\; dt$$

Example. On $S^2$ with the round metric, a curve $\gamma(t) = (\theta(t), \varphi(t))$ has length:

$$L(\gamma) = \int_a^b \sqrt{\dot\theta^2 + \sin^2\!\theta\; \dot\varphi^2}\; dt$$

For the equator parametrized by $\gamma(t) = (\pi/2, t)$ with $t \in [0, 2\pi]$: $\dot\theta = 0$, $\dot\varphi = 1$, $\sin(\pi/2) = 1$, so $L = 2\pi$, the circumference of a great circle.

Example: great-circle distance on $S^2$. For two points $p, q$ on the unit sphere with position vectors $\mathbf{p}, \mathbf{q} \in \mathbb{R}^3$, the Riemannian distance is $d(p, q) = \arccos(\mathbf{p} \cdot \mathbf{q})$ — the angle between the vectors, which is the length of the shorter great-circle arc connecting them.

---

Musical Isomorphisms

The Riemannian metric provides a canonical identification between tangent vectors and cotangent vectors — between “arrows” and “linear measurements.” This identification is called the musical isomorphisms, named for the flat ($\flat$) and sharp ($\sharp$) symbols borrowed from musical notation.

The flat map “lowers an index”: it takes a vector with an upper index $v^i$ and produces a covector with a lower index $v_j = g_{ij} v^i$.

The gradient depends on the metric. Given a smooth function $f : M \to \mathbb{R}$, the differential $df \in T_p^*M$ is a covector — it does not depend on any metric. But the gradient $\mathrm{grad}\, f = (df)^{\sharp}$ is a vector, and converting a covector to a vector requires the metric. In local coordinates:

$$(\mathrm{grad}\, f)^i = g^{ij} \frac{\partial f}{\partial x^j}$$

In flat Euclidean space with $g^{ij} = \delta^{ij}$, this recovers the familiar gradient $\nabla f = \left(\frac{\partial f}{\partial x^1}, \ldots, \frac{\partial f}{\partial x^n}\right)$. But on a curved manifold, the gradient depends on $g^{ij}$ — change the metric, and the gradient changes direction. This is precisely why the natural gradient (which uses the Fisher information metric $g^{ij}$) differs from the ordinary gradient (which implicitly uses the Euclidean metric).

---

The Levi-Civita Connection

On $\mathbb{R}^n$, differentiating a vector field is straightforward: differentiate each component. On a manifold, this does not work — the component functions live in different coordinate systems at different points, and the tangent spaces at different points are different vector spaces. To differentiate vector fields on a manifold, we need a connection: a rule for comparing tangent vectors at nearby points.

The covariant derivative $\nabla_X Y$ measures “how $Y$ changes as we move in the direction $X$,” in a way that is well-defined on a manifold.

There are infinitely many possible connections on any manifold. Two additional conditions — metric compatibility and torsion-freeness — single out a unique one.

The torsion-free condition says that infinitesimal parallelograms close: if we transport $X$ along $Y$ and $Y$ along $X$, we arrive at the same point (up to the Lie bracket correction).

Worked example: Christoffel symbols on $S^2$. With the round metric $g = d\theta^2 + \sin^2\!\theta\; d\varphi^2$, the metric is diagonal with $g_{\theta\theta} = 1$, $g_{\varphi\varphi} = \sin^2\!\theta$, and $g_{\theta\varphi} = 0$. Since the metric components depend only on $\theta$, the Christoffel symbol formula yields exactly two independent nonzero symbols (plus one symmetry partner):

$$\Gamma^\theta_{\varphi\varphi} = -\sin\theta\cos\theta, \qquad \Gamma^\varphi_{\theta\varphi} = \Gamma^\varphi_{\varphi\theta} = \cot\theta$$

All other $\Gamma^k_{ij} = 0$. The first says that moving in the $\varphi$-direction on the sphere generates an apparent acceleration in the $\theta$-direction (toward the equator). The second says that moving in the $\theta$-direction while pointing in the $\varphi$-direction requires a correction proportional to $\cot\theta$.

Connection Explorer

Christoffel symbols Γ^k_ij at θ = 1.000

Γ^θ_ij

	θ	φ
θ	0	0
φ	0	-0.455

Γ^φ_ij

	θ	φ
θ	0	0.642
φ	0.642	0

Only Γᶿφφ and Γᵠθφ are nonzero on S². Heatmap shows |Γ| magnitude.

Metric:Sphere S²Poincaré DiskFlat ℝ²θ:1.000Show torsion-free parallelogram

---

Parallel Transport

On flat $\mathbb{R}^n$, we can compare tangent vectors at different points by simply translating them. On a curved manifold, there is no canonical way to do this — the tangent spaces at different points are different vector spaces. The Levi-Civita connection provides the next best thing: we can “carry” a tangent vector along a curve while keeping it “as constant as possible.” This is parallel transport.

Path dependence and holonomy. On flat $\mathbb{R}^n$, parallel transport is path-independent — the result depends only on the endpoints. On a curved manifold, parallel transport depends on the path. If we transport a vector around a closed loop back to the starting point, it generally returns rotated. The rotation angle is called the holonomy of the loop.

Example: holonomy on $S^2$. Consider transporting a tangent vector around a spherical triangle with vertices at the north pole, the equator at longitude $0°$, and the equator at longitude $90°$. Each side is a geodesic (great circle arc), and the vector stays tangent to the geodesic along each side. After traversing all three sides, the vector has rotated by $90°$ — exactly the solid angle subtended by the triangle ($\frac{1}{8}$ of $4\pi$ steradians = $\pi/2$ steradians). This is a special case of the Gauss–Bonnet theorem: on a surface of constant curvature $K$, the holonomy of a loop enclosing area $A$ is $KA$.

Parallel Transport Explorer

Curve:Great circle (equator)Latitude circle θ = π/3Latitude circle θ = π/4Spherical triangleInitial angle:0°Show holonomy

---

Riemannian Volume Form and Integration

The Riemannian metric determines a canonical way to measure volumes on $M$.

The factor $\sqrt{\det g}$ is the Jacobian that corrects for the distortion introduced by the coordinate system. It ensures that the volume form is intrinsic — independent of the choice of coordinates.

Worked example: area of $S^2$. With the round metric $g = \mathrm{diag}(1, \sin^2\!\theta)$, we have $\det g = \sin^2\!\theta$, so $dV_g = \sin\theta\; d\theta \wedge d\varphi$. The total area is:

$$\mathrm{Area}(S^2) = \int_0^{2\pi}\!\int_0^{\pi} \sin\theta\; d\theta\; d\varphi = 2\pi \cdot [-\cos\theta]_0^{\pi} = 2\pi \cdot 2 = 4\pi$$

For the Poincaré disk with $\det g = \lambda^4 = \frac{16}{(1-r^2)^4}$, the volume form is $dV_g = \frac{4}{(1-r^2)^2}\, dx \wedge dy$. The “area” of the Poincaré disk is infinite — reflecting the fact that the hyperbolic plane has infinite extent, even though the disk looks bounded in Euclidean coordinates.

---

Isometries and Killing Vector Fields

Symmetries of a Riemannian manifold are the diffeomorphisms that preserve the metric.

Isometries preserve everything that depends on the metric: lengths, angles, areas, geodesics, curvature. The set of all isometries of $(M, g)$ forms a group under composition, denoted $\mathrm{Isom}(M, g)$.

Examples.

• $\mathrm{Isom}(\mathbb{R}^n, g_{\mathrm{Euc}}) = E(n)$, the Euclidean group of rotations, reflections, and translations. $\dim E(n) = n(n+1)/2$.
• $\mathrm{Isom}(S^n, g_{\mathrm{round}}) = \mathrm{O}(n+1)$, the orthogonal group. $\dim \mathrm{O}(n+1) = n(n+1)/2$.
• $\mathrm{Isom}(\mathbb{H}^n, g_{\mathrm{hyp}}) = \mathrm{O}^+(1, n)$, the proper Lorentz group. Also $\dim = n(n+1)/2$.

Killing vector fields are the infinitesimal generators of isometries: each Killing field generates a one-parameter family of isometries. On $S^2$, there are exactly three independent Killing fields — the infinitesimal rotations about the three coordinate axes — corresponding to $\dim \mathrm{SO}(3) = 3$.

The bound $n(n+1)/2$ decomposes as $n$ translations (or their curved analogues) plus $n(n-1)/2$ rotations — the most symmetry any $n$-dimensional geometry can have.

---

Computational Notes

The formulas in this topic are explicit enough for symbolic and numerical computation. Here we illustrate two core calculations.

Symbolic Christoffel symbols with SymPy. We can derive the Christoffel symbols for any metric directly from the formula $\Gamma^k_{ij} = \frac{1}{2} g^{k\ell}(\partial_i g_{j\ell} + \partial_j g_{i\ell} - \partial_\ell g_{ij})$.

import sympy as sp

theta, phi = sp.symbols('theta phi', positive=True)

# Round metric on S^2
g = sp.Matrix([[1, 0], [0, sp.sin(theta)**2]])
g_inv = g.inv()
coords = [theta, phi]

# Christoffel symbols Gamma^k_{ij}
n = 2
Gamma = [[[sp.Rational(0)] * n for _ in range(n)] for _ in range(n)]
for k in range(n):
    for i in range(n):
        for j in range(n):
            Gamma[k][i][j] = sp.Rational(1, 2) * sum(
                g_inv[k, l] * (
                    sp.diff(g[j, l], coords[i])
                    + sp.diff(g[i, l], coords[j])
                    - sp.diff(g[i, j], coords[l])
                )
                for l in range(n)
            )
            Gamma[k][i][j] = sp.simplify(Gamma[k][i][j])

# Print nonzero Christoffel symbols
for k in range(n):
    for i in range(n):
        for j in range(i, n):
            if Gamma[k][i][j] != 0:
                print(f"Gamma^{coords[k]}_{{{coords[i]},{coords[j]}}} = {Gamma[k][i][j]}")
# Output:
#   Gamma^theta_{phi,phi} = -sin(theta)*cos(theta)
#   Gamma^phi_{theta,phi} = cos(theta)/sin(theta)

Numerical parallel transport ODE. We solve $dV^k/dt + \Gamma^k_{ij}\, \dot\gamma^i\, V^j = 0$ numerically with forward Euler:

import numpy as np

def parallel_transport_s2(curve, curve_dot, V0, n_steps=500):
    """Parallel transport on S^2 via forward Euler."""
    dt = 1.0 / n_steps
    V = np.array(V0, dtype=float)
    trajectory = [V.copy()]

    for step in range(n_steps):
        t = step * dt
        theta, _ = curve(t)
        dgamma = np.array(curve_dot(t))
        sin_th, cos_th = np.sin(theta), np.cos(theta)

        # Christoffel symbols for S^2
        # Gamma^0_{11} = -sin(theta)*cos(theta)
        # Gamma^1_{01} = Gamma^1_{10} = cos(theta)/sin(theta)
        dV = np.zeros(2)
        dV[0] = sin_th * cos_th * dgamma[1] * V[1]  # -Gamma^0_{11} * dphi * V^phi
        dV[1] = -(cos_th / max(sin_th, 1e-10)) * (
            dgamma[0] * V[1] + dgamma[1] * V[0]
        )
        V = V + dV * dt
        trajectory.append(V.copy())

    return np.array(trajectory)

# Transport along latitude theta = pi/3, phi from 0 to pi/2
theta0 = np.pi / 3
curve = lambda t: (theta0, t * np.pi / 2)
curve_dot = lambda t: (0.0, np.pi / 2)
V0 = (1.0, 0.0)  # Initially pointing in theta-direction

result = parallel_transport_s2(curve, curve_dot, V0)

# Verify norm preservation: |V|_g should be constant
sin_th = np.sin(theta0)
norms = np.sqrt(result[:, 0]**2 + sin_th**2 * result[:, 1]**2)
print(f"Initial norm: {norms[0]:.6f}")
print(f"Final norm:   {norms[-1]:.6f}")
print(f"Max deviation: {np.max(np.abs(norms - norms[0])):.2e}")
# Output (typical):
#   Initial norm: 1.000000
#   Final norm:   0.999998
#   Max deviation: 2.14e-06

The norm is preserved to within the forward Euler truncation error — confirming metric compatibility numerically.

---

Connections to Machine Learning

The Fisher information metric turns the machinery of this topic into a tool for optimization and statistics.

The Fisher information metric. Let $\{p_\theta : \theta \in \Theta\}$ be a parametric family of probability distributions, with $\Theta \subseteq \mathbb{R}^n$ an open parameter space. The Fisher information matrix at $\theta$ is:

$$g_{ij}(\theta) = \mathbb{E}_{x \sim p_\theta}\!\left[\frac{\partial \log p_\theta(x)}{\partial \theta^i}\,\frac{\partial \log p_\theta(x)}{\partial \theta^j}\right]$$

When the model is identifiable, $g_{ij}(\theta)$ is positive definite for all $\theta$ — it is a Riemannian metric on $\Theta$. The parameter space becomes a Riemannian manifold $(\Theta, g)$.

Example: Gaussian family. For $p_\theta = \mathcal{N}(\mu, \sigma^2)$ with $\theta = (\mu, \sigma)$ and $\sigma > 0$:

$$g = \frac{1}{\sigma^2}\, d\mu^2 + \frac{2}{\sigma^2}\, d\sigma^2 \qquad \Longleftrightarrow \qquad g_{ij} = \begin{pmatrix} 1/\sigma^2 & 0 \\ 0 & 2/\sigma^2 \end{pmatrix}$$

The $\sigma$-direction is “steeper” than the $\mu$-direction by a factor of $\sqrt{2}$ — moving $\sigma$ changes the distribution more (in the KL sense) than moving $\mu$ by the same Euclidean amount.

Natural gradient descent. Standard gradient descent updates $\theta_{t+1} = \theta_t - \eta\, \nabla_{\mathrm{Euc}} L(\theta_t)$ using the Euclidean gradient — but this implicitly assumes the parameter space is flat with the Euclidean metric. When the parameter space is curved (which it always is for statistical models), the Euclidean gradient points in the wrong direction.

The natural gradient (Amari, 1998) uses the Fisher metric to compute the steepest descent direction in the Riemannian sense:

$$\tilde{\nabla} L = g^{-1}(\theta)\, \nabla_{\mathrm{Euc}} L(\theta)$$

This is exactly the sharp map applied to the Euclidean gradient: $\tilde{\nabla} L = (\nabla_{\mathrm{Euc}} L)^{\sharp}$. The natural gradient is invariant under reparametrization — it does not depend on the coordinates we use for $\Theta$.

KL divergence as Riemannian distance. For nearby parameters $\theta$ and $\theta + d\theta$:

$$\mathrm{KL}(p_\theta \| p_{\theta + d\theta}) \approx \frac{1}{2}\, g_{ij}(\theta)\, d\theta^i\, d\theta^j$$

The KL divergence is the squared infinitesimal Riemannian distance. This is why the Fisher metric is natural: it is the unique Riemannian metric (up to scale) for which KL divergence is the distance.

The Cramér–Rao bound. For any unbiased estimator $\hat\theta$ of $\theta$:

$$\mathrm{Cov}(\hat\theta) \succeq g^{-1}(\theta)$$

The inverse Fisher metric is the lower bound on estimation variance. The metric tells us how hard it is to distinguish nearby parameters — directions where $g$ is large are “easy” to estimate (the distributions are very different); directions where $g$ is small are “hard.”

Natural Gradient Explorer

Click anywhere on the plot to add a new starting point.

Target:N(0, 1)N(1, 0.5)N(-1, 2)Learning rate:0.15Show metric ellipsesReset points

---

Connections and Further Reading

Cross-topic connections.

Topic	Connection
Smooth Manifolds	The prerequisite: charts, tangent spaces, and the differential are the raw inputs for Riemannian geometry. The metric is the additional structure that enables measurement.
The Spectral Theorem	The metric tensor $g_{ij}$ at each point is symmetric positive definite — its eigendecomposition reveals the principal directions and magnitudes of the metric.
Singular Value Decomposition	The differential of a map between Riemannian manifolds decomposes via SVD into rotations and stretches. The singular values measure metric distortion.
PCA & Low-Rank Approximation	Local PCA on data near a manifold estimates the tangent space metric. The Riemannian metric is the theoretical foundation for manifold learning.

Where this leads.

• Geodesics & Curvature — The Levi-Civita connection defines geodesics as curves with zero acceleration ($\nabla_{\gamma'}\gamma' = 0$). The Riemann curvature tensor $R^l_{ijk}$ measures the failure of parallel transport to be path-independent. Sectional curvature, Ricci curvature, and scalar curvature each capture different aspects of how the manifold curves.

• Information Geometry & Fisher Metric — The Fisher information metric on statistical manifolds, natural gradient methods for neural network optimization, $\alpha$-connections, and the geometry of exponential families. This topic provides the complete Riemannian foundation; Information Geometry builds the statistical superstructure.