Sliding-Window Attention

An attention variant that restricts each query to a fixed local window of key positions instead of the full sequence.

Sliding-window attention is an attention variant that limits each token to a fixed local neighborhood, trading full-sequence reach for bounded compute and memory on long contexts.

At a glance

Released

April 2020

Authors

Iz Beltagy, Matthew E. Peters, Arman Cohan

Optimizes

  • Attention Compute
  • Memory Bandwidth
  • Local Context Inference

What It Is

Sliding-window attention is an attention variant that limits each query position to keys within a fixed neighborhood. Instead of a full n-by-n attention matrix, only entries inside the window receive non-zero weight.

Why It Exists

Sliding-window attention targets attention compute, memory bandwidth for score materialization, and the cost of serving long sequences when dense all-pairs attention would dominate runtime.

How It Works

Each query position attends only to keys within W tokens to its left (and optionally right, depending on the pattern). Keys outside the window are zeroed before softmax normalization, so the model spends compute on local context rather than the full token grid.

Math Or Compute Schema

With sequence length n and window size W, sliding-window attention applies a band mask before softmax. The formulas below contrast dense multi-head attention against window-limited attention that restricts which keys each query can reach.
Multi-head attention (MHA)
Attention(Qi,Ki,Vi)=softmax ⁣(QiKi⊤dk)Vi\text{Attention}(Q_i, K_i, V_i) = \mathrm{softmax}\!\left(\frac{Q_i K_i^{\top}}{\sqrt{d_k}}\right) V_i
QQ
Query vectors for head i.
KK
Key vectors for head i.
VV
Value vectors for head i.
HH
Number of query heads.
dkd_k
Key dimension per head.
ii
Query head index.
Sliding-window attention
Attention(Qi,Ki,Vi)=softmax ⁣(BW⊙QiKi⊤dk)Vi\text{Attention}(Q_i, K_i, V_i) = \mathrm{softmax}\!\left(B_W \odot \frac{Q_i K_i^{\top}}{\sqrt{d_k}}\right) V_i
QQ
Query vectors for head i.
KK
Key vectors for head i.
VV
Value vectors for head i.
HH
Number of query heads.
WW
Window size: maximum token distance each query may attend to.
dkd_k
Key dimension per head.
ii
Query head index.
BWB_W
Band mask that zeroes key positions outside the sliding window.

Compared To Nearby Modules

Compared with multi-head attention, sliding-window attention limits reach to a local neighborhood rather than scoring all positions. Multi-query and grouped-query attention remain dense over allowed positions but reduce key-value head count. Sliding-window attention instead changes how far each query can look.
Comparison dimensionSliding-Window AttentionMulti-Head AttentionMulti-Query AttentionGrouped-Query Attention
Attention localityFixed local window of W tokens per queryFull sequence; every query sees every keyFull sequence with shared key-value headsFull sequence with grouped key-value heads
Compute scaling with sequence lengthLinear O(n·W) per head when W is fixedQuadratic O(n²) per headQuadratic O(n²) with fewer key-value projectionsQuadratic O(n²) with reduced key-value head count
Global token reachDistant tokens require stacked layers or companion patternsAny position can attend to any other positionAny position can attend to any other positionAny position can attend to any other position

Example Architectures

Sliding-window attention appears in long-context decoder models that combine local windows with periodic global layers or other patterns to recover distant dependencies.

Limitations And Tradeoffs

A fixed window cannot directly connect distant tokens in a single layer. Models that rely only on local windows may need stacked layers or companion patterns to move information across the full sequence.

Why It Still Matters

As sequence lengths grow, quadratic attention cost motivates locality patterns that preserve useful nearby signal while avoiding a full n-by-n score matrix on every layer.

Tags

References

  1. Beltagy, Iz, Matthew E. Peters, and Arman Cohan. "Longformer: The Long-Document Transformer." arXiv, 2020, https://arxiv.org/abs/2004.05150.