Skip to content

Positional Encoding

Definition
AI-generated

Positional encoding injects information about token order into transformer inputs because self-attention itself is permutation-invariant over positions.

Topics
LLM and Agents Machine learning concepts
Synonyms
Positional embedding (when learned and added like an embedding)
Related
Embedding Self-Attention Token

Why it matters in GWAS

Sequence models over DNA, RNA, or protein must represent genomic distance and strand context; the choice of absolute versus relative encoding affects extrapolation to longer contexts and alignment with biological spacing. Comparing architectures across papers requires noting whether positions are learned, rotary, or ALiBi-style.

Example usage

"We replaced learned absolute positional embeddings with rotary positional encoding so the regulatory model could score variable-length windows without retraining the position table."

Embedding Self-Attention Token Tokenization Transformer
Related-term network

Click any node to open that term page.

Node order. The current term is first. Next come every page linked from Related terms, in the same top-to-bottom order as that list. Then new terms appear in two-hop layers: for each of those direct neighbors (in that same order), their related terms are appended if not already listed. A third pass does the same for two-hop nodes. The walk stops when a fixed maximum number of nodes is reached.

References

  • Vaswani A, et al. (2017). Attention is all you need. NeurIPS.
← Population Stratification Positive Selection →

Last updated (UTC · Git history)