arXiv Artificial Intelligence

Mid-Training Language Models on Raw Video

1 hour ago 09.10.2026 07:00 👁 1 views
Mid-Training Language Models on Raw Video

Quick summary

arXiv:2610.11019v1 Announce Type: cross Abstract: Multimodal large language models learn mostly from paired image-text data or annotated video, and raw web video is rarely used to further train an existing language model. We study whether raw video, with no captions and no text loss, can serve as mid-training data for a pretrained language model. Frames are encoded into continuous visual tokens, and the language model learns to predict the next visual token. We mid-train Qwen3-1.7B on raw clips from YT-Temporal-1B and then apply the same image-text instruction tuning to it and to the model wit

Key takeaways

  • arXiv:2610.11019v1 Announce Type: cross Abstract: Multimodal large language models learn mostly from paired image-text data or annotated video, and raw web video is rarely used to further train an existing language model.
  • We study whether raw video, with no captions and no text loss, can serve as mid-training data for a pretrained language model.
  • Frames are encoded into continuous visual tokens, and the language model learns to predict the next visual token.

Why it matters

“Mid-Training Language Models on Raw Video” highlights the need for repeatable measurement rather than a single impressive demonstration. Independent validation across datasets and clearly stated limitations determine whether a result can guide product decisions.

Kaynak sitede devamını oku: arXiv Artificial Intelligence ↗