Local Attention

A broad attention pattern that lets each query look only at a nearby neighborhood instead of the full sequence.

Local attention is a broad attention pattern that limits each token to nearby tokens, lowering long-context attention cost while making distant connections less direct.

At a glance

Released

April 2020

Authors

Iz Beltagy, Matthew E. Peters, Arman Cohan

Optimizes

  • Attention Compute
  • Memory Bandwidth
  • Long Context Inference

What It Is

Local attention is a broad attention pattern where each query attends only to a nearby neighborhood instead of the full sequence. It is the umbrella idea behind window-limited attention layouts. Sliding-window attention is one common concrete version, while sparse attention is a broader family that may include local links plus other selective long-range links.

Why It Exists

Dense attention compares every token with every other token, so score computation and score storage grow quadratically as context gets longer. Local attention keeps most of the useful nearby context while avoiding a full all-pairs score matrix on every layer.

How It Works

For each query position, the model keeps only keys inside a local neighborhood and masks the rest before softmax. The neighborhood can be a fixed band, a block, or another nearby-only rule, but the core idea stays the same: spend attention compute on close tokens first instead of on the entire sequence.
Local attention neighborhood reach. The dense baseline lets each query reach the full sequence, while the local pattern keeps each query inside a nearby band.

Math Or Compute Schema

With sequence length n and local radius r, local attention applies a locality mask before softmax. The formulas below contrast dense multi-head attention against a masked local pattern that keeps only nearby token interactions.
Multi-head attention (MHA)
Attention(Qi,Ki,Vi)=softmax ⁣(QiKi⊤dk)Vi\text{Attention}(Q_i, K_i, V_i) = \mathrm{softmax}\!\left(\frac{Q_i K_i^{\top}}{\sqrt{d_k}}\right) V_i
QQ
Query vectors for head i.
KK
Key vectors for head i.
VV
Value vectors for head i.
HH
Number of query heads.
dkd_k
Key dimension per head.
ii
Query head index.
Local attention
Attention(Qi,Ki,Vi)=softmax ⁣(Lr⊙QiKi⊤dk)Vi\text{Attention}(Q_i, K_i, V_i) = \mathrm{softmax}\!\left(L_r \odot \frac{Q_i K_i^{\top}}{\sqrt{d_k}}\right) V_i
QQ
Query vectors for head i.
KK
Key vectors for head i.
VV
Value vectors for head i.
HH
Number of query heads.
rr
Local radius or neighborhood size around each query.
dkd_k
Key dimension per head.
ii
Query head index.
LrL_r
Locality mask that keeps only nearby key positions for each query.

Compared To Nearby Modules

Compared with baseline attention, local attention changes where each query is allowed to look. Sliding-window attention is a specific fixed-width local-attention layout. Sparse attention is broader: it also removes many pairs, but it can keep arbitrary selected long-range links instead of only a neighborhood.
Comparison dimensionLocal AttentionAttentionSliding-Window AttentionSparse Attention
Reach patternBroad umbrella for nearby-only neighborhoodsFull sequence; every query can score every keyFixed-width band around each query positionSelected subset of token pairs, which may be local, blockwise, or mixed
Compute scaling with sequence lengthOften near-linear O(n·r) when the local radius stays fixedQuadratic O(n²) per headLinear O(n·W) per head when W is fixedSubquadratic when the allowed-pair fraction stays fixed
Distant-token pathUsually indirect; far information moves through layers or companion linksDirect one-hop access to any tokenIndirect unless another pattern adds longer-range accessDepends on the sparsity pattern; some designs keep explicit long-range links

Example Architectures

Long-context transformers often use local attention as one building block, then add global tokens, occasional dense layers, or retrieval-style paths when they need stronger distant-token communication.

Limitations And Tradeoffs

Local attention makes nearby interactions cheap, but it weakens direct one-hop access to distant tokens. Information that starts far away may need several layers, companion global links, or other routing patterns before it can influence the current token strongly.

Why It Still Matters

As models push toward longer context windows, local attention remains a practical first move because nearby context is often the highest-value signal and because the pattern composes well with broader long-context systems.

Tags

References

  1. Beltagy, Iz, Matthew E. Peters, and Arman Cohan. "Longformer: The Long-Document Transformer." arXiv, 2020, https://arxiv.org/abs/2004.05150.