Skip to content

Transformer

Definition
AI-generated

A transformer is a neural network architecture built from stacked blocks of multi-head self-attention and position-wise feedforward layers, usually with residual connections and layer normalization; positional encodings supply order information without recurrence.

Topics
LLM and Agents
Related
Attention Mechanism BERT (Bidirectional Encoder Representations from Transformers) Decoder

Why it matters in GWAS

Transformers power large DNA and protein language models, biomedical text LLMs, and some single-cell foundation models used for annotation and in silico screens. Pretraining corpora and tokenization (k-mers, codons) shape inductive biases; genetic validity still requires independent functional or cohort benchmarks.

Example usage

"The sequence embedding came from a base-pair resolution transformer fine-tuned on human regulatory regions."

Attention Mechanism BERT (Bidirectional Encoder Representations from Transformers) Decoder Encoder Feedforward Network (FFN) Fine-Tuning Convolutional Neural Network (CNN) Cross-Attention Deep Learning Foundation Model KV Cache Large Language Model (LLM) Masked Attention Multi-Head Attention Neural Network Positional Encoding Pretraining Recurrent Neural Network (RNN) Self-Attention Tokenization
Related-term network

Click any node to open that term page.

Node order. The current term is first. Next come every page linked from Related terms, in the same top-to-bottom order as that list. Then new terms appear in two-hop layers: for each of those direct neighbors (in that same order), their related terms are appended if not already listed. A third pass does the same for two-hop nodes. The walk stops when a fixed maximum number of nodes is reached.

References

  • Vaswani A, et al. (2017). Attention is all you need. NeurIPS.
← Transfer RNA (tRNA) Transition/transversion ratio (Ti/Tv) →

Last updated (UTC · Git history)