Tether the Subject, Release the Scene: Query-Aware Memory Routing for Long-Horizon Autoregressive Video Generation

HUST×

MiLM Plus

Long-horizon autoregressive video generation

Tether the Subject,
Release the Scene.

Query-aware memory routing preserves identity-bearing history while allowing backgrounds, viewpoints, and scene states to keep evolving.

Chen Li^1,2Peng Zhang²Hanyu Zhou¹Jialong Zuo¹ Fei Wang²Daiguo Zhou²Nong Sang¹Changxin Gao^1,*

¹Huazhong University of Science and Technology²MiLM Plus, Xiaomi Inc.^*Corresponding author

arXiv ↗Code · GitHub ↗

Training-free · frozen video generator · inference-time intervention

2,400blinded pairwise judgments

76.9%scene-progression expected preference

Training-freeinference-time routing on a frozen generator

See the effect first

Watch the trajectory—not only the last frame.

Four matched 30-second comparisons expose distinct long-horizon behaviors. Each case is pre-aligned into one side-by-side video, so a single playback control moves both methods through the same moment.

30 seconds · P04 cross-stack

TetherMem transfers to the Deep-Forcing attention stack.

This matched comparison isolates the routing intervention on Deep-Forcing. The outputs remain close overall, demonstrating cross-stack compatibility.

Deep-ForcingDeep-Forcing + TetherMem

Left · Deep-Forcing baseline, seed 639693304Right · same prompt, seed, and 30-second duration

Table 1 · Main comparison

See the full trade-off, not one convenient metric.

Human expected preference measures the complete 30-second experience. Short-clip quality, motion, and VBench diagnostics remain visible beside it, so identity retention cannot be mistaken for scene progression.

Highest overall preference0.780 EP#1 among nine long-video methods

Highest scene progression0.769 EP+0.161 over the strongest baseline

Identity remains competitive0.600 EPSecond-highest identity estimate

Table 1 · full results

Best valueSecond bestAll metrics ↑

Evaluation windowComplete 30-second videos

Human evidence2,400 blinded judgments

Annotators10 independent raters

Preference scaleTie-aware Davidson EP

Table 1. Main comparison with human preference and VBench diagnostics.
Method	5s	30s human EP ↑				Diag.	VBench ↑
Method	Img5 ↑	Ovr.	Prog.	ID	L1	nT ↑	Subj.	Back.	Smooth.	Dyn.
Base model · non-autoregressive
Wan2.1-T2V-1.3B	65.6	—	—	—	—	0.322	—	—	—	—
Long-memory / retrieval
LongLive-RAG	71.0	0.615	0.608	0.507	0.587	0.868	0.861	0.897	0.973	0.867
Forcing-based generation
Self Forcing	70.5	0.320	0.356	0.437	0.273	0.192	0.885	0.907	0.983	0.767
Rolling Forcing	72.2	0.553	0.429	0.633	0.651	0.040	0.933	0.935	0.985	0.433
Deep Forcing	70.5	0.526	0.495	0.531	0.508	0.350	0.899	0.916	0.983	0.800
Causal Forcing	70.2	0.210	0.440	0.285	0.158	1.226	0.824	0.873	0.973	0.867
Reward Forcing	69.3	0.573	0.530	0.487	0.620	0.366	0.903	0.927	0.985	0.733
Memory / positional methods
MemRoPE	72.1	0.640	0.578	0.559	0.666	0.551	0.892	0.908	0.986	0.700
CausVid	65.9	0.283	0.295	0.461	0.328	0.190	0.879	0.896	0.982	0.767
Spatiotemporal memory routing
TetherMem Ours	71.1	0.780	0.769	0.600	0.708	1.289	0.855	0.895	0.973	0.867

Human EP: Ovr. = overall preference, Prog. = scene progression, ID = subject identity, L1 = visual integrity. Baseline–baseline preferences are model-inferred; every TetherMem–baseline edge is directly observed.Bold cards = best · underline = second best

The problem

One memory policy cannot serve two opposite needs.

Subject identity needs long-range access. Scene progression needs recency. Applying one frame-level memory policy to both creates the familiar identity–progression trade-off.

The real choice is not “remember or forget.” It is which query should read which part of memory.

Rolling-Forcing

A short rolling context can preserve local continuity while repeatedly returning to the opening view.

Failure · stagnation

Deep-Forcing

Long-range motion can continue even as spatial detail and trajectory coherence deteriorate.

Failure · incoherent motion

RAG

LongLive-RAG

Frame-level retrieval gives subject and scene queries the same historical context, limiting progression.

Failure · shared policy

TetherMem

Query-aware routing keeps identity-bearing history available while scene queries favor recent states.

Route · subject ≠ scene

Figure 1: four 120-second video trajectories comparing Rolling Forcing, Deep Forcing, LongLive-RAG, and TetherMem — Figure 1. A shared 120-second timeline exposes distinct failure modes: stagnation, incoherent motion, limited progression, and sustained scene evolution.

Method

Route memory at the query level.

TetherMem places a positive spatiotemporal routing prior inside attention normalization. Switch the query below to see how the same retrieved memory is read differently.

Interactive routing view

Current querySubject query“Is this still the same person?”

Retrieved memoryIdentity-bearing history stays open

t−5

t−4

t−3

t−2

t−1

ReadoutTether the subjectLong-range identity evidence remains accessible.

t−5 older memoryBar height = relative routing weightt−1 recent memory

Normalized routing

Ã_q,i = softmax(s_q,i + log π_n(q,i))

The prior changes selection while attention remains normalized.

Region prior

π_reg = 1
or γ_n

Same-region paths stay open; cross-region reads are softened.

Recency prior

ρ_i = max(τ_i, ρ_min)

Background queries favor recent scene states.

02A

End-to-end routing

Current queries, retrieved history, and a time-aligned subject track meet at normalized memory routing. The subject is tethered; the scene is released.

Figure 2a: end-to-end TetherMem architecture and qualitative scene outcomes

02B

From frame selection to query routing

Frame-level selection decides which memories survive. TetherMem instead routes subject and scene queries differently inside the same retrieved memory budget.

Figure 2b: comparison between prior frame-level memory selection and TetherMem query-conditioned routing

Evidence

Identity and progression can coexist.

Human preference distinguishes coherent scene development from raw motion. TetherMem occupies the high-identity, high-progression region, while strong motion alone is not sufficient.

76.9%scene-progression expected preference

Scatter plots comparing identity, scene progression, and normalized translation across video methods

Qualitative trajectories

Progression appears across scenes.

Matched trajectories sample the same backbone pair across multiple prompts and seven timestamps over the 30-second evaluation window.

Matched LongLive-RAG and TetherMem trajectories across four prompts and seven timestamps

Citation

Reference TetherMem.

The preprint is available on arXiv ↗.

BIBTEX

@article{li2026tethermem,
  title  = {Tether the Subject, Release the Scene:
            Query-Aware Memory Routing for Long-Horizon
            Autoregressive Video Generation},
  author = {Li, Chen and Zhang, Peng and Zhou, Hanyu and
            Zuo, Jialong and Wang, Fei and Zhou, Daiguo and
            Sang, Nong and Gao, Changxin},
  year   = {2026},
  eprint = {2608.26902},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url    = {https://arxiv.org/abs/2608.26902}
}