arXiv Artificial Intelligence

Riemannian Geometry for Pre-trained Language Model Embeddings

2 hours ago 23.09.2026 07:00 👁 1 views
Riemannian Geometry for Pre-trained Language Model Embeddings

Quick summary

arXiv:2607.07047v3 Announce Type: replace-cross Abstract: Understanding the geometric structure of pre-trained language model embeddings matters for interpretability and safety. We ask whether sentence-level classification signal lives in the Riemannian geometry of contextual token embeddings, and probe it by extracting per-token pullback metrics from a learned encoder's analytical Jacobian and aggregating them with the Fr\'echet mean on the symmetric positive definite (SPD) manifold; we call this procedure Riemannian Mean Pooling (RMP). Across three datasets with non-trivial linguistic struct

Key takeaways

  • arXiv:2607.07047v3 Announce Type: replace-cross Abstract: Understanding the geometric structure of pre-trained language model embeddings matters for interpretability and safety.
  • We ask whether sentence-level classification signal lives in the Riemannian geometry of contextual token embeddings, and probe it by extracting per-token pullback metrics from a learned encoder's analytical Jacobian and aggregating them with the Fr\'echet mean on the symmetric positive definite (SPD) manifold; we call this procedure Riemannian Mean Pooling (RMP).
  • Across three datasets with non-trivial linguistic struct

Why it matters

This model development creates a new option for users and a new testing obligation for developers. A fixed evaluation set comparing quality, cost and failure behavior is more useful than launch claims.

Kaynak sitede devamını oku: arXiv Artificial Intelligence ↗