Skip to content

Cross-Attention

Definition
AI-generated

Cross-attention lets one sequence (queries) attend to another (keys and values)—unlike self-attention, where queries, keys, and values come from the same sequence.

Topics
LLM and Agents Machine learning concepts
Related
Attention Mechanism Decoder Encoder

Why it matters in GWAS

Cross-attention connects heterogeneous inputs—e.g. variant context attending over gene annotations, or a clinical LLM attending over retrieved abstracts—so evaluation must check whether retrieved evidence is faithfully grounded or confounded by attention biases in the training data.

Example usage

"The report generator used cross-attention so each locus summary could attend over the top-k PubMed chunks returned by RAG."

Attention Mechanism Decoder Encoder Multi-Head Attention Retrieval-Augmented Generation (RAG) Self-Attention Transformer
Related-term network

Click any node to open that term page.

Node order. The current term is first. Next come every page linked from Related terms, in the same top-to-bottom order as that list. Then new terms appear in two-hop layers: for each of those direct neighbors (in that same order), their related terms are appended if not already listed. A third pass does the same for two-hop nodes. The walk stops when a fixed maximum number of nodes is reached.

References

  • Vaswani A, et al. (2017). Attention is all you need. NeurIPS.
← Cross-Ancestry Genetic Correlation Cross-Population Effect Heterogeneity →

Last updated (UTC · Git history)