Skip to content

KV Cache

Definition
AI-generated

A KV cache (key–value cache) stores, for each decoder layer and attention head, the key and value vectors already computed for past tokens during autoregressive generation.

Topics
LLM and Agents
Synonyms
Key–value cache
Related
Context Window Large Language Model (LLM) Multi-Head Attention

Why it matters in GWAS

KV caching is what makes interactive LLM use (drafting methods text, parsing trait strings, agent tool calling) practical at scale: it cuts latency and memory bandwidth during long outputs or multi-turn chats. Teams benchmarking on-prem or regulated deployments should treat cache behavior (length, batching, quantization of cached tensors) as part of the reproducibility story alongside model weights and prompts.

Example usage

"We profiled wall-clock time per locus summary with KV cache enabled versus disabled on the same GPU."

Context Window Large Language Model (LLM) Multi-Head Attention Self-Attention Token Transformer
Related-term network

Click any node to open that term page.

Node order. The current term is first. Next come every page linked from Related terms, in the same top-to-bottom order as that list. Then new terms appear in two-hop layers: for each of those direct neighbors (in that same order), their related terms are appended if not already listed. A third pass does the same for two-hop nodes. The walk stops when a fixed maximum number of nodes is reached.

References

  • Vaswani A, et al. (2017). Attention is all you need. NeurIPS.
← Kurtosis Large Language Model (LLM) →

Last updated (UTC · Git history)