Download reference data

Example

import gwaslab as gl

Configuration and default paths

Downloaded files go under ~/.gwaslab/ (data_directory). The local registry (config.json, mapping keywords → paths) also defaults to ~/.gwaslab/config.json. On first import, GWASLab migrates a legacy registry from {package}/data/config.json if present.

Key	Default	Override
`data_directory`	`~/.gwaslab/`	`gl.set_default_directory()`, `gwaslab init --directory DIR`, `gwaslab config set data_directory PATH`, or `GWASLAB_DATA_DIR`
`config`	`~/.gwaslab/config.json`	`gl.options.set_option("config", path)`, `gwaslab config set config PATH`, or `GWASLAB_CONFIG`
`reference`	package `data/reference.json`	`gl.update_available_ref()` to refresh from GitHub

User overrides for data_directory and config are persisted in ~/.gwaslab/settings.json.

Example

gl.set_default_directory("/data/refs/")
gl.options.paths["config"]  # ~/.gwaslab/config.json by default
gl.get_path("1kg_eas_hg19")  # resolves from registry; does not download

CLI:

gwaslab init                          # create dirs, migrate config, scan default data_directory
gwaslab init --directory /data/refs   # set data_directory (persisted) and scan
gwaslab config set data_directory /data/refs
gwaslab path config                   # show registry JSON path

Cache and storage layout

Location	Purpose
`~/.gwaslab/` (`data_directory`)	Downloaded genomic refs, sumstats subdirs (`GCST…/`), recombination extracts
`~/.gwaslab/config.json`	Registry: keyword → `{local_path, kind, source, …}`
`~/.gwaslab/settings.json`	Persisted path overrides
`~/.gwaslab/lookup/`	Harmonize rsID sweep cache (per workflow)
`~/.gwaslab/recombination/hg19\|hg38/`	Lazy-downloaded recombination maps
Package `data/`	Read-only catalog (`reference.json`), built-in HapMap3/chains
`platformdirs` user cache (`gwaspipe`)	infer_strand HDF5 cache (outside `data_directory`)

Registry entries include kind: ref (catalog/local genomic refs) or sumstats (GWAS Catalog GCST). Filter with gwaslab list ref --downloaded --kind ref.

Check available reference data

Processed 1KG VCF, rsID tables, HapMap3 EAF, and recombination maps are hosted on Hugging Face (Cloufield/gwaslab-reference). Other files (dbSNP, UCSC FASTA, Ensembl/RefSeq GTF, liftOver chains) are downloaded from their original source.

The catalog is nested JSON in reference.json (one keyword per entry with url, description, suggested_use, optional tbi, md5sum).

Example

gl.check_available_ref()

stdout (abbreviated):

Start to check available reference files...
 - Available keywords: 1kg_eas_hg19 1kg_eur_hg19 ucsc_genome_hg19 ...
Finished checking available reference files...

Each keyword maps to metadata such as:

{
  "url": "https://huggingface.co/datasets/Cloufield/gwaslab-reference/resolve/main/1kg/hg19/EAS.ALL.split_norm_af.1kgp3v5.hg19.vcf.gz",
  "description": "1000 Genomes Project East Asian (1KG EAS) VCF on hg19 …",
  "suggested_use": "LD reference panel for region plot; infer strand for EAS",
  "tbi": {"url": "https://huggingface.co/datasets/Cloufield/gwaslab-reference/resolve/main/1kg/hg19/EAS.ALL.split_norm_af.1kgp3v5.hg19.vcf.gz.tbi"},
  "md5sum": "6162a93cb80935168c0bfa519748b054"
}