Skip to main navigation Skip to search Skip to main content

S3-Prune: Stability-Aware Token Budgeting for Long-Form Video-Language Models

  • Seoul National University of Science and Technology (SNUST)
  • Korea Electronics Technology Institute

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

The deployment of long-form video-language models (Video-VLMs) is fundamentally constrained by the prohibitive scaling of attention complexity and KV-cache overhead as temporal and spatial resolutions increase. Existing pruning methods rely on local frame-level heuristics, which fail to capture shifting information density and often result in cumulative budget misallocation over long horizons. To address these challenges, we propose S3-prune, a stability-aware token budgeting framework that jointly models spatio-temporal demand while ensuring consistent allocation over time. S3-prune characterizes token demand by integrating spatial uncertainty (SU) from patch-level embedding shifts and segment transition (ST) from inter-segment semantic variations. To mitigate temporal instability, we introduce stability accumulation (SA) via a Kalman-filtered latent estimator, which smoothens stochastic spikes and provides uncertainty-aware margins for adaptive budgeting. Guided by these stabilized budgets, S3-prune executes a hierarchical two-stage selection process. Extensive evaluations across seven benchmarks and various backbones demonstrate that S3-prune significantly advances the efficiency-accuracy frontier, reducing inference latency while remaining robust even under aggressive token retention settings.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2026 - 19th European Conference, 2026, Proceedings
EditorsPaolo Favaro, Zuzana Kukelova, Atsuto Maki, Anna Rohrbach, Konrad Schindler, Federico Tombari
PublisherSpringer Science and Business Media Deutschland GmbH
Pages484-501
Number of pages18
ISBN (Print)9783032372512
DOIs
StatePublished - 2026
Event19th European Conference on Computer Vision, ECCV 2026 - Malmö, Sweden
Duration: 8 Sep 2026 → 12 Sep 2026

Publication series

NameLecture Notes in Computer Science
Volume17017 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference19th European Conference on Computer Vision, ECCV 2026
Country/TerritorySweden
CityMalmö
Period8/09/26 → 12/09/26

Keywords

  • Efficient Inference
  • Temporal Stabilization
  • Token Pruning
  • Video Language Model

Cite this