Skip to content

BERT (Bidirectional Encoder Representations from Transformers)

Definition
AI-generated

BERT is a transformer encoder stack pretrained on large text with masked language modeling (predicting masked tokens in context) and, in the original recipe, next-sentence prediction; release checkpoints are then fine-tuned on downstream tasks with task-specific heads.

Topics
LLM and Agents Machine learning concepts
Synonyms
BERT
Related
Encoder Fine-Tuning Large Language Model (LLM)

Why it matters in GWAS

PubMed- or clinical-notes–adapted BERT-style models are common baselines for biomedical named-entity recognition, relation extraction, and document classification used to mine phenotypes, drugs, and genes from literature or EHR text; like all LLM-family tools, outputs need verification against curated databases and study-specific ontology rules.

Example usage

"We compared zero-shot prompts against a BERT baseline fine-tuned on 10k manually labeled GWAS abstract sentences."

Encoder Fine-Tuning Large Language Model (LLM) Masked Attention Pretraining Token Tokenization Transformer
Related-term network

Click any node to open that term page.

Node order. The current term is first. Next come every page linked from Related terms, in the same top-to-bottom order as that list. Then new terms appear in two-hop layers: for each of those direct neighbors (in that same order), their related terms are appended if not already listed. A third pass does the same for two-hop nodes. The walk stops when a fixed maximum number of nodes is reached.

References

  • Devlin J, et al. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. NAACL.
← Bernoulli distribution Beta-binomial distribution →

Last updated (UTC · Git history)