Sumstats — Core

Loading, metadata, orchestration pipelines, harmonization helpers, and I/O.

init

__init__(sumstats: Union[str, DataFrame, NoneType] = None, fmt: Optional[str] = None, tab_fmt: str = 'tsv', snpid: Optional[str] = None, rsid: Optional[str] = None, chrom: Optional[str] = None, pos: Optional[str] = None, ea: Optional[str] = None, nea: Optional[str] = None, ref: Optional[str] = None, alt: Optional[str] = None, eaf: Optional[str] = None, neaf: Optional[str] = None, maf: Optional[str] = None, n: Optional[str] = None, beta: Optional[str] = None, se: Optional[str] = None, chisq: Optional[str] = None, z: Optional[str] = None, f: Optional[str] = None, t: Optional[str] = None, p: Optional[str] = None, q: Optional[str] = None, mlog10p: Optional[str] = None, test: Optional[str] = None, info: Optional[str] = None, OR: Optional[str] = None, OR_95L: Optional[str] = None, OR_95U: Optional[str] = None, beta_95L: Optional[str] = None, beta_95U: Optional[str] = None, HR: Optional[str] = None, HR_95L: Optional[str] = None, HR_95U: Optional[str] = None, ncase: Optional[str] = None, ncontrol: Optional[str] = None, neff: Optional[str] = None, i2: Optional[str] = None, phet: Optional[str] = None, dof: Optional[str] = None, snpr2: Optional[str] = None, status: Optional[str] = None, other: Optional[List[str]] = None, exclude: Optional[List[str]] = None, include: Optional[List[str]] = None, chrom_pat: Optional[str] = None, snpid_pat: Optional[str] = None, direction: Optional[str] = None, verbose: bool = True, study: str = 'Study_1', trait: str = 'Trait_1', build: str = '99', species: str = 'homo sapiens', readargs: Optional[Dict[str, Any]] = None, **kwreadargs: Any) -> None

Load and preformat summary statistics data into standardized GWASLab format.

Workflow and Priority

The function follows a strict 9-step workflow where each step builds upon previous ones:

**Phase 1: Configuration (Steps 1-3)**
1. Initialize parameters - Set up basic data structures and validate inputs
2. Handle parquet format - Special handling for parquet files (early exit)
3. Load format configuration - Load predefined format mappings from formatbook if fmt specified

**Phase 2: Mapping (Steps 4-7)**
4. Check path and header - Discover available columns in input data
5. Build column mappings - Map user-specified column names to GWASLab standard names
6. Handle VCF format special case - VCF requires special column handling
7. Apply include/exclude filters - Apply user-specified column inclusion/exclusion filters

**Phase 3: Data (Steps 8-9)**
8. Load data - Actually load data from file(s) or DataFrame
9. Post-process data - Transform loaded data into final GWASLab format

**Priority Order for Column Mappings:**
1. Formatbook (fmt parameter) - Base mappings (Step 3)
2. User-specified headers - Override/extend formatbook (Step 5)
3. Include filter - Subset to specified columns (Step 7)
4. Exclude filter - Remove excluded columns (Step 7)

**Design Principles:**
- Configuration before data: All setup happens before data loading
- User overrides formatbook: User parameters take precedence over formatbook defaults
- Transform after load: All data transformations happen after data is loaded
- Early validation: Column existence and type validation happens during discovery phase

Parameters:

Name	Type	Description	Default
`sumstats`	`str or DataFrame`	Input summary statistics, provided either as a file path or a DataFrame. When summary statistics are split by chromosome, a single path pattern may be supplied using the `@` symbol as a placeholder for the chromosome number. For example: "gwas/chr@.sumstats.gz" will load "gwas/chr1.sumstats.gz", "gwas/chr2.sumstats.gz", ... automatically.	`None`
`fmt`	`str`	Format name to get predefined mapping if provided. (e.g., 'gwaslab', 'vcf').	`None`
`tab_fmt`	`str`	Table format ('tsv', 'parquet').	`'tsv'`
`snpid`	`str`	Column name for SNP identifiers in the input data. Expected format is CHR:POS:NEA:EA (e.g., 1:123:A:G), although the delimiter may vary depending on the source.	`None`
`rsid`	`str`	Column name for rsID in the input data. Values should follow the standard "rs" prefix (e.g., rs12345).	`None`
`chrom`	`str`	Column name for chromosome in input data.	`None`
`pos`	`str`	Column name for position in input data.	`None`
`ea`	`str`	Column name for effect allele in input data. (assuming alternative allele)	`None`
`nea`	`str`	Column name for non-effect allele in input data. (assuming reference allele)	`None`
`eaf`	`str`	Column name for effect allele frequency in input data.	`None`
`neaf`	`str`	Column name for non-effect allele frequency in input data.	`None`
`maf`	`str`	Column name for minor allele frequency in input data.	`None`
`n`	`str or int`	Column name or constant value for sample size.	`None`
`beta`	`str`	Column name for beta in input data.	`None`
`se`	`str`	Column name for standard error in input data.	`None`
`chisq`	`str`	Column name for chi-square in input data.	`None`
`z`	`str`	Column name for z-score in input data.	`None`
`f`	`str`	Column name for F-statistic in input data.	`None`
`t`	`str`	Column name for T-statistic in input data.	`None`
`p`	`str`	Column name for p-value in input data.	`None`
`q`	`str`	Column name for Q-statistic in input data.	`None`
`mlog10p`	`str`	Column name for -log10(p) in input data.	`None`
`test`	`str`	Column name for test type in input data.	`None`
`info`	`str`	Column name for imputation info in input data.	`None`
`OR`	`str`	Column name for odds ratio in input data.	`None`
`OR_95L`	`str`	Column name for lower 95% CI of OR in input data.	`None`
`OR_95U`	`str`	Column name for upper 95% CI of OR in input data.	`None`
`beta_95L`	`str`	Column name for lower 95% CI of beta in input data.	`None`
`beta_95U`	`str`	Column name for upper 95% CI of beta in input data.	`None`
`HR`	`str`	Column name for hazard ratio in input data.	`None`
`HR_95L`	`str`	Column name for lower 95% CI of HR in input data.	`None`
`HR_95U`	`str`	Column name for upper 95% CI of HR in input data.	`None`
`i2`	`str`	Column name for I2 statistic in input data.	`None`
`snpr2`	`str`	Column name for SNP R2 in input data.	`None`
`phet`	`str`	Column name for p-heterogeneity in input data.	`None`
`dof`	`str`	Column name for degrees of freedom in input data.	`None`
`ncase`	`str or int`	Column name or constant value for case count.	`None`
`ncontrol`	`str or int`	Column name or constant value for control count.	`None`
`neff`	`str or int`	Column name or constant value for effective sample size.	`None`
`direction`	`str`	Column name for meta-analysis effect-direction strings, where each character ("+", "-", "0") represents the direction of effect for one cohort (e.g., "++-0+").	`None`
`status`	`str`	Column name for status in input data.	`None`
`study`	`str`	Column name for study ID in input data.	`"Study_1"`
`trait`	`str`	Column name for trait in input data.	`"Trait_1"`
`build`	`str`	Genome build version (e.g., '19' and '38').	`'99'`
`species`	`str`	species	`"homo sapiens"`
`other`	`list`	Additional columns in the raw file to load.	`None`
`chrom_pat`	`str`	Regex pattern to filter chromosomes like'chrX'	`None`
`snpid_pat`	`str`	Regex pattern to filter SNPs based on snpid like'chrX:'.	`None`
`verbose`	`bool`	Enable verbose output.	`False`
`readargs`	`dict`	Additional arguments for reading files using pd.read_csv() like `nrows`, `comment`. Example, {"nrows": 1000} means to load first 1000 rows.	`None`

Returns:

Type	Description
`DataFrame`	Formatted summary statistics with standardized column names.
`dict`	Updated readargs dictionary.

Raises:

Type Description

ValueError

If input is not a path or DataFrame, or if columns are missing.

Less used parameters

exclude : list, optional Columns to exclude explicitly. Columns should be passed as GWASLab built-in HEADER keywords in uppercase like BETA, DIRECTION. Not original headers. include : list, optional Columns to include explicitly. Columns should be passed as GWASLab built-in HEADER keywords in uppercase like BETA, DIRECTION. Not original headers.

copy

copy() -> Sumstats

Return a deep copy of the Sumstats object.

summary

summary() -> typing.Any

Generate a structured quality-control summary for GWAS summary statistics.

This function computes descriptive metrics across several categories:
    - META: Basic dataset information
    - CHR: Chromosome counts and distribution
    - MISSING: Missing value counts across relevant fields
    - MAF: Minor allele frequency distribution
    - P: P-value significance levels
    - BETA: Effect-size magnitude among genome-wide significant variants
    - STATUS: Variant QC status summary (see `lookup_status`)

Returns:

Type Description

dict

A hierarchical dictionary containing comprehensive quality-control summary statistics. The dictionary is organized into the following sections:

META (key: "overview") Basic dataset metadata including: - "Row_num": Total number of variants (rows) in the dataset - "Column_num": Total number of columns in the dataset - "Column_names": Comma-separated list of all column names - "Last_checked_time": Timestamp of when the summary was generated - "QC and Harmonization": Status of QC and harmonization checks

CHR (key: "chromosomes") Chromosome distribution information: - "Chromosomes_notations": Sorted list of unique chromosome identifiers - "Chromosomes_numbers": Count of unique chromosomes - "chr{chr_id}": Count of variants per chromosome (one key per chromosome)

MISSING (key: "missing_values") Missing value statistics: - "Missing_total": Total number of variants with at least one missing value - "Missing_{col_name}": Count of missing values per column (only included if > 0) - Each count also includes a corresponding "{key} percentage" entry

MAF (key: "MAF", only if EAF column exists) Minor allele frequency distribution: - "Common (MAF>=0.05)": Count of common variants - "Low_frequency (0.01<MAF<=0.05)": Count of low-frequency variants - "Rare (0.001<MAF<=0.01)": Count of rare variants - "Ultra Rare (MAF<=0.001)": Count of ultra-rare variants - Each count also includes a corresponding "{key} percentage" entry

P (key: "p_values", only if P column exists) P-value significance statistics: - "Minimum": Minimum p-value in the dataset - "P<5e-8": Count of genome-wide significant variants (p < 5e-8) - "P<5e-6": Count of suggestive significant variants (p < 5e-6) - Each count also includes a corresponding "{key} percentage" entry

BETA (key: "beta_for_significant_variants", only if both P and BETA columns exist) Effect size magnitude for genome-wide significant variants (p < 5e-8): - "P<5e-8 with ABS(BETA)>{threshold}": Count of significant variants with absolute beta exceeding threshold, for thresholds: 10, 3, 1, 0.5, 0.3, 0.2, 0.1, 0.05 - Each count also includes a corresponding "{key} percentage" entry

STATUS (key: "variant_status", only if STATUS column exists) Variant QC status summary, organized by status code: - Each status code (as string key) contains: - "count": Number of variants with this status code - "explanation": Dictionary explaining each digit of the status code: - "Genome_Build": Reference genome build (CHM13/hg19/hg38/Unmapped/Unknown/Unchecked) - "rsID&SNPID": Validation status of rsID and SNPID fields - "CHR&POS": Validation status of chromosome and position fields - "Stadardize&Normalize": Standardization and normalization status - "Align": Alignment status to reference genome - "Panlidromic_SNP&Indel": Palindromic SNP and indel status - Each count also includes a corresponding "{key} percentage" entry

variants (key: "variants") Additional variant-level metadata: - "variant_number": Total number of variants - "min_P": Minimum p-value (only if P column exists and has values) - "min_minor_allele_freq": Minimum minor allele frequency (only if EAF column exists)

samples (key: "samples", only if N column exists) Sample size statistics: - "sample_size": Maximum sample size - "sample_size_median": Median sample size - "sample_size_min": Minimum sample size

All numeric values are converted to native Python types (int, float, str, list, dict) via the to_python function. Percentages are calculated as fractions of the total variant count and are included for all count-based statistics.

lookup_status

lookup_status(status: str = 'STATUS') -> typing.Any

Decode and analyze variant status codes.

Processes status codes that encode multiple layers of variant validation information 
using a string-based format. Returns a structured DataFrame with detailed status 
breakdown and frequency statistics.

Status Code Structure:
Each status code string contains 7+ digits encoding:
- 1st 2 digits: Genome build mapping (CHM13/hg19/hg38)
- 3rd digit: rsID and SNPID validation status
- 4th digit: Chromosome and position validation
- 5th digit: Standardization and normalization status
- 6th digit: Alignment to reference genome
- 7th digit: Palindromic SNP/indel status

Returns:

Type Description

DataFrame

A DataFrame with one row per unique status code, sorted by status code index. All columns are of string dtype. The DataFrame contains the following columns:

Index: Status codes (as strings), sorted in ascending order. Each status code is a 7-digit string where each digit position encodes different validation information.

Genome_Build (str): Reference genome build mapping status. Possible values: - "CHM13": Mapped to CHM13 reference - "hg19": Mapped to hg19/GRCh37 reference - "hg38": Mapped to hg38/GRCh38 reference - "Unmapped": Variant could not be mapped to any reference - "Unknown": Genome build status unknown - "Unchecked": Genome build not yet checked Encoded in the 1st-2nd digits of the status code.

rsID&SNPID (str): Validation status of rsID and SNPID identifier fields. Possible values: - "rsid valid & SNPID valid": Both identifiers are valid - "rsid valid & SNPID invalid": rsID valid, SNPID invalid - "rsid invalid & SNPID valid": rsID invalid, SNPID valid - "rsid invalid & SNPID invalid": Both identifiers invalid - "rsid valid & SNPID valid": Both valid (alternative encoding) - "rsid valid & SNPID unknown": rsID valid, SNPID status unknown - "rsid unknown & SNPID valid": rsID status unknown, SNPID valid - "rsid invalid & SNPID unknown": rsID invalid, SNPID status unknown - "rsid unknown & SNPID invalid": rsID status unknown, SNPID invalid - "Unchecked": Validation not yet performed Encoded in the 3rd digit of the status code.

CHR&POS (str): Validation status of chromosome and position fields. Possible values: - "CHR valid & POS valid": Both chromosome and position are valid - "CHR invalid & POS invalid": Both chromosome and position invalid - "CHR invalid & POS valid": Chromosome invalid, position valid - "CHR valid & POS invalid": Chromosome valid, position invalid - "CHR valid & POS unknown": Chromosome valid, position status unknown - "CHR unknown & POS valid": Chromosome status unknown, position valid - "CHR invalid & POS unknown": Chromosome invalid, position status unknown - "CHR unknown & POS invalid": Chromosome status unknown, position invalid - "Unchecked": Validation not yet performed Encoded in the 4th digit of the status code.

Stadardize&Normalize (str): Standardization and normalization status of variant alleles. Possible values: - "standardized SNP": Variant is a standardized SNP - "standardized & normalized insertion": Standardized insertion variant - "standardized & normalized deletion": Standardized deletion variant - "standardized & normalized indel": Standardized indel variant - "standardized indel": Standardized but not normalized indel - "indistinguishable or not normalized allele": Allele cannot be distinguished or normalized - "invalid allele notation": Allele notation is invalid - "Unknown": Standardization status unknown - "Unchecked": Standardization not yet performed Encoded in the 5th digit of the status code.

Align (str): Alignment status to reference genome. Possible values: - "Match: NEA=REF": Non-effect allele matches reference - "Flipped_fixed": Alleles were flipped and fixed - "Reverse_complementary_fixed": Reverse complement applied and fixed - "Flipped": Alleles need to be flipped - "Reverse_complementary": Reverse complement needed - "Reverse_complementary+Flipped": Both reverse complement and flip needed - "Both_alleles_on_ref+indistinguishable": Both alleles on reference, indistinguishable - "Not_on_reference_genome": Variant not found on reference genome - "Unchecked": Alignment not yet checked Encoded in the 6th digit of the status code.

Panlidromic_SNP&Indel (str): Palindromic SNP and indel status. Possible values: - "Not_palindromic_SNPs": Variant is not a palindromic SNP - "Palindromic+strand": Palindromic SNP on positive strand - "Palindromic-strand_fixed": Palindromic SNP on negative strand, fixed - "Indel_match": Indel matches reference - "Indel_flipped_fixed": Indel flipped and fixed - "Palindromic-strand": Palindromic SNP on negative strand - "Indel_flipped": Indel needs to be flipped - "Indistinguishable": Variant type indistinguishable - "No_matching_or_no_info": No matching information available - "Unchecked": Status not yet checked Encoded in the 7th digit of the status code.

Count (str): Absolute count of variants with this status code. Represented as a string but contains numeric count values.

Percentage(%) (str): Relative percentage of variants with this status code, calculated as (count / total_variants) * 100. Rounded to 2 decimal places. Represented as a string but contains numeric percentage values.

The DataFrame is sorted by the status code index in ascending order. Only status codes that appear in the input Series are included in the output.

update_meta

update_meta(verbose: bool = True, **kwargs: Any) -> None

Update Sumstats Object meta info based on the statistics of the current sumstats. Including information on variants, samples.

validate_meta

validate_meta() -> typing.Dict[str, typing.Any]

Validate internal metadata consistency (build fields, QC flags).

Returns:

Type	Description
`dict`	Structured report with keys `valid` (bool), `issues` (list), `build` (dict).

check_sumstats_qc_status

check_sumstats_qc_status() -> typing.Any

Check the QC and harmonization status of the sumstats.

Returns:

Type	Description
`dict`	Dictionary containing QC and harmonization status information with keys: - "basic_check": Basic QC check status - "harmonize": Harmonization status - "qc_and_harmonization_status": Overall QC and harmonization status

basic_check

basic_check(remove=False, remove_dup=False, threads=1, fix_id_kwargs={}, remove_dup_kwargs={}, fix_chr_kwargs={}, fix_pos_kwargs={}, fix_allele_kwargs={}, sanity_check_stats_kwargs={}, consistency_check_kwargs={}, normalize=True, normalize_allele_kwargs={}, verbose=True)

All-in-one function for Sumstats quality control (QC), which is a wrapper of separate functions including: fix_id for SNPID and rsID check, fix_chr for chromosome notation (CHR) check, fix_pos for basepair position (POS) check, fix_allele for allele notation (EA and NEA) check, check_sanity for statistics sanity check and datatype check (BETA, SE, P and so forth), check_data_consistency for checking if convitable data are consistent (e.g., if the calculated BETA/SE is close to the original Z), normalize_allele for indel normalization, remove_dup for removal of multi-allelic variants, indels, and duplicated variants, sort_coordinate for sorting the genomic coordinates, and sort_column for sorting the order of columns in the dataframe.

    For a detailed description of all checks performed, see the documentation in docs/QC&Filtering.md.

Parameters:

Name	Type	Description	Default
`remove`	`bool`	Whether to remove bad quality variants detected in _fix_chr, _fix_pos, and _fix_allele.	`False`
`remove_dup`	`bool`	Whether to remove duplicated or multi-allelic variants using remove_dup.	`False`
`threads`	`int`	Number of threads to use for parallel processing.	`1`
`fix_id_kwargs`	`dict`	Keyword arguments passed to `fix_id`.	`{}`
`remove_dup_kwargs`	`dict`	Keyword arguments passed to `remove_dup`.	`{}`
`fix_chr_kwargs`	`dict`	Keyword arguments passed to `fix_chr`.	`{}`
`fix_pos_kwargs`	`dict`	Keyword arguments passed to `fix_pos`.	`{}`
`fix_allele_kwargs`	`dict`	Keyword arguments passed to `fix_allele`.	`{}`
`sanity_check_stats_kwargs`	`dict`	Keyword arguments passed to `check_sanity`.	`{}`
`consistency_check_kwargs`	`dict`	Keyword arguments passed to `check_data_consistency`.	`{}`
`normalize`	`bool`	Whether to perform indel normalization.	`True`
`normalize_allele_kwargs`	`dict`	Keyword arguments passed to `normalize_allele`.	`{}`
`verbose`	`bool`	Whether to print progress information.	`True`

Returns:

Type	Description
`Sumstats`	`self` after QC.

Examples:

>>> mysumstats = gl.Sumstats("sumstats.txt.gz", fmt="plink2")
>>> mysumstats.basic_check()

harmonize

harmonize(basic_check=True, ref_seq=None, ref_rsid_tsv=None, ref_rsid_vcf=None, ref_infer=None, ref_alt_freq=None, ref_maf_threshold=0.4, maf_threshold=0.4, threads=1, extract_threads=None, remove=False, check_ref_kwargs={}, remove_dup_kwargs={}, assign_rsid_kwargs={}, infer_strand_kwargs={}, flip_allele_stats_kwargs={}, liftover_kwargs={}, fix_id_kwargs={}, fix_chr_kwargs={}, fix_pos_kwargs={}, fix_allele_kwargs={}, sanity_check_stats_kwargs={}, normalize_allele_kwargs={}, verbose=True, sweep_mode=False)

Standard pipeline for harmonizing sumstats including: 1. Basic check and standardization using fix_id, fix_chr, fix_pos, fix_allele, check_sanity, normalize_allele, sort_coordinate, and sort_column. 2. Reference-based annotation and flipping using check_ref, flip_allele_stats, infer_strand, and assign_rsid. 3. Optional duplicate removal with removedup and final sorting via sort_coordinate and sort_column. For infer_strand, and assign_rsid, check_ref with ref_seq is required.

Parameters:

Name	Type	Description	Default
`basic_check`	`bool`	Whether to run basic QC pipeline (`fix_id`, `fix_chr`, `fix_pos`, etc.).	`True`
`ref_seq`	`str`	Full path to reference sequence file in FASTA format for allele flipping.	`None`
`ref_rsid_tsv`	`str`	Full path to rsID TSV reference file.	`None`
`ref_rsid_vcf`	`str`	Full path to rsID VCF/BCF reference file.	`None`
`ref_infer`	`str`	Full path to reference VCF/BCF file for strand inference.	`None`
`ref_alt_freq`	`str`	Allele frequency field name in VCF/BCF INFO for strand inference.	`None`
`ref_maf_threshold`	`float`	MAF threshold (reference VCF/BCF) for strand inference.	`0.4`
`maf_threshold`	`float`	MAF threshold (sumstats) for strand inference.	`0.4`
`threads`	`int`	Number of threads for parallel processing.	`1`
`extract_threads`	`int`	Worker count for VCF/BCF lookup extraction during `infer_strand2` and `assign_rsid2` sweep mode. Defaults to 1 inside extract (memory-safe).	`None`
`remove`	`bool`	Whether to remove bad variants during QC.	`False`
`check_ref_kwargs`	`dict`	Arguments passed to `check_ref`.	`{}`
`remove_dup_kwargs`	`dict`	Arguments passed to `remove_dup`.	`{}`
`assign_rsid_kwargs`	`dict`	Arguments passed to `assign_rsid`.	`{}`
`infer_strand_kwargs`	`dict`	Arguments passed to `infer_strand`.	`{}`
`flip_allele_stats_kwargs`	`dict`	Arguments passed to `flip_allele_stats`.	`{}`
`liftover_kwargs`	`dict`	Reserved for liftover integration; not applied in the current harmonize pipeline.	`{}`
`fix_id_kwargs`	`dict`	Arguments passed to `fix_id`.	`{}`
`fix_chr_kwargs`	`dict`	Arguments passed to `fix_chr`.	`{}`
`fix_pos_kwargs`	`dict`	Arguments passed to `fix_pos`.	`{}`
`fix_allele_kwargs`	`dict`	Arguments passed to `fix_allele`.	`{}`
`sanity_check_stats_kwargs`	`dict`	Arguments passed to `check_sanity`.	`{}`
`normalize_allele_kwargs`	`dict`	Arguments passed to `normalize_allele`.	`{}`
`verbose`	`bool`	Whether to print progress information.	`True`
`sweep_mode`	`bool`	If False, use lookup (per-variant) mode. If True, use sweep mode (fast for large datasets).	`False`

Returns:

Type	Description
`Sumstats`	The Sumstats object (`self`).

Examples:

>>> mysumstats.harmonize(ref_seq=gl.get_path("fasta38"), ref_infer=gl.get_path("1kg_eur_vcf"))

align_with_template

align_with_template(template, **kwargs)

Align summary statistics to a template mold by CHR/POS.

Merges template with self, aligns columns to the mold schema, and flips allele statistics when orientations disagree.

Parameters:

Name	Type	Description	Default
`template`	`Sumstats or DataFrame`	Reference mold with target column layout and allele orientation.	required

Returns:

Type	Description
`None`	Updates `self.data` in place.

set_build

set_build(build: Union[str, int], verbose: bool = True) -> None

Set genome build in sumstats status column.

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`pd.DataFrame or Sumstats object`	Sumstats data or object	required
`build`	`str`	Genome build identifier	`"99"`
`status`	`str`	Status column name	`"STATUS"`
`verbose`	`bool`	Whether to print log messages	`True`
`log`	`Log`	Logging object	required
`species`	`str`	Species name. If None, will try to extract from Sumstats object metadata.	required

Returns:

Type	Description
`tuple`	(sumstats DataFrame, processed_build) When called via :meth:`Sumstats.set_build()`, updates the Sumstats object in place (modifies `self.data` and `self.build`) and the method returns `None`.

infer_build

infer_build(verbose: bool = True, **kwargs: Any) -> None

Infer genome build version using Hapmap3 SNPs.

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with inferred build version. When called via :meth:`Sumstats.infer_build()`, updates the Sumstats object in place (modifies `self.data`, `self.build`, and `self.meta["gwaslab"]["genome_build"]`) and the method returns `None`.

liftover

liftover(to_build: Optional[str] = None, from_build: Optional[str] = None, chain_path: Optional[str] = None, **kwargs: Any) -> None

Perform liftover of variants to a new genome build.

Can be called with either:
- from_build and to_build (chain file will be automatically found)
- chain_path (direct path to chain file)

This is a fast chain-based liftover implementation that directly parses
UCSC chain files and performs vectorized coordinate conversion. It converts
genomic coordinates from one genome build (e.g., hg19/GRCh37) to another
(e.g., hg38/GRCh38) using UCSC chain files.

Chain files are automatically obtained from built-in data (for hg19<->hg38)
or downloaded from UCSC if not available. Built-in chain files are preferred
for better performance and reliability.

Parameters:

Name	Type	Description	Default
`chain_path`	`str`	Path to UCSC chain file. If provided, from_build and to_build are optional.	`None`
`from_build`	`str`	Source genome build (e.g., "19"). If None, uses sumstats_obj.build if available.	`None`
`to_build`	`str`	Target genome build (e.g., "38"). Required if chain_path is not provided.	`None`
`remove`	`bool`	Whether to remove unmapped variants	`True`
`verbose`	`bool`	Whether to print progress messages	`True`

Returns:

Type Description

DataFrame

DataFrame with lifted coordinates (or updated sumstats_obj.data if Sumstats object). The returned DataFrame contains: - Updated CHR and POS columns with lifted coordinates (from the target build) - Updated STATUS column with new status codes reflecting the liftover results - Unmapped variants are either removed (if remove=True) or kept with NA values for CHR and POS (if remove=False) When called via :meth:Sumstats.liftover(), updates the Sumstats object in place (modifies self.data) and the method returns None.

Notes

If called with a Sumstats object, the function will update the object's data
and metadata in place. If called with a DataFrame, it returns a new DataFrame.

sort_coordinate

sort_coordinate(**sort_kwargs)

Sort variants by genomic coordinates (chromosome, then position).

Sorts the dataframe first by chromosome number, then by position in ascending order.
The index is reset to sequential integers after sorting.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats or DataFrame`	Sumstats object or DataFrame containing the data to sort.	required
`verbose`	`bool`	If True, print progress messages.	`False`

Returns:

Type	Description
`DataFrame`	DataFrame with sorted genomic coordinates. When called via :meth:`Sumstats.sort_coordinate()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

sort_column

sort_column(**kwargs)

Reorder columns according to a specified order.

Reorders the dataframe columns to match a predefined standard order, placing standard
GWAS columns first (SNPID, rsID, CHR, POS, EA, NEA, statistics, etc.) followed by
any additional columns not in the standard list.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats or DataFrame`	Sumstats object or DataFrame containing the data to reorder.	required
`verbose`	`bool`	Whether to print progress. Default is True.	required

Returns:

Type	Description
`DataFrame`	Modified sumstats with reordered columns. When called via :meth:`Sumstats.sort_column()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

fill_data

fill_data(verbose: bool = True, **kwargs: Any) -> None

Fill missing statistical values in genetic summary statistics from available columns.

This function systematically derives missing statistical values using relationships
between different statistical measures (e.g., converting beta/SE to Z-scores, or ORs
to betas). It handles multiple conversion pathways and maintains data consistency.

Parameters:

Name	Type	Description	Default
`insumstats`	`DataFrame or Sumstats`	Summary statistics DataFrame or Sumstats object.	required
`to_fill`	`str or list of str`	Column name(s) to fill. Common values include "OR","OR_95L","OR_95U","BETA","SE","P","Z","CHISQ","MLOG10P","MAF", etc.	required
`overwrite`	`bool`	If True, overwrite existing values in target columns.	`False`
`verbose`	`bool`	Whether to display progress messages.	`True`
`extreme`	`bool`	If True, use extreme value calculations for -log10(P). Helpful when P<1e-300 (float64 datatype limits).	`False`

Returns:

Type	Description
`DataFrame`	Modified summary statistics DataFrame with filled values. When called via :meth:`Sumstats.fill_data()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `None`.

Less used parameters

df : str, optional Column name containing degrees of freedom for chi-square tests. Only used when CHISQ only_sig : bool, optional, default False If True, only update values for significant variants. sig_level : float, optional, default 5e-8 Significance threshold for P-value filtering.

check_ref

check_ref(ref_seq: Any, **kwargs: Any) -> Sumstats

Check if non-effect allele (NEA) is aligned with reference genome.

This function checks whether the non-effect allele (NEA) in the summary statistics
matches the reference genome sequence. It updates the status codes in the summary
statistics DataFrame to reflect the alignment status.

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame or Sumstats`	Summary statistics DataFrame or Sumstats object containing variant data.	required
`ref_seq`	`str`	Path to the reference genome FASTA file.	required
`chrom`	`str`	Column name for chromosome information in sumstats.	`"CHR"`
`pos`	`str`	Column name for position information in sumstats.	`"POS"`
`ea`	`str`	Column name for effect allele in sumstats.	`"EA"`
`nea`	`str`	Column name for non-effect allele in sumstats.	`"NEA"`
`status`	`str`	Column name for status codes in sumstats.	`"STATUS"`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper.	required
`remove`	`bool`	If True, remove variants not on the reference genome.	`False`
`verbose`	`bool`	If True, print progress messages.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with status codes reflecting alignment with reference genome. When called via :meth:`Sumstats.check_ref()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

Notes

The function uses the following status codes (6th character of status string):
0: Alleles match reference (NEA == ref)
3: Flipped (EA == ref but NEA != ref)
4: Reverse_complementary (rev_NEA == ref)
5: Reverse_complementary + Flipped
6: Both alleles on genome but indistinguishable (indel)
8: Not on reference genome
9: Unchecked

infer_strand

infer_strand(ref_infer: Any, **kwargs: Any) -> Sumstats

Parameters cache_options : A dictionary with the following keys: - cache_manager: CacheManager object or None. If any between cache_loader and cache_process is not None, or use_cache is True, a CacheManager object will be created automatically. - trust_cache: bool (optional, default: True). Whether to completely trust the cache or not. Trusting the cache means that any key not found inside the cache will be considered as a missing value even in the VCF file. - cache_loader: Object with a get_cache() method or None. - cache_process: Object with an apply_fn() method or None. - use_cache: bool (optional, default: False). If any of the cache_manager, cache_loader or cache_process is not None, this will be set to True automatically. If set to True and all between cache_manager, cache_loader and cache_process are None, the cache will be loaded (or built) on the spot.

    The usefulness of a cache_loader or cache_process object is to pass a custom object which already has the cache loaded. This can be useful if the cache is loaded in background in another thread/process while other operations are performed.
    The cache_manager is a CacheManager object is used to expose the API to interact with the cache.

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with inferred strand information. When called via :meth:`Sumstats.infer_strand()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

infer_strand2

infer_strand2(**kwargs)

Annotate summary statistics with reference allele frequency and infer strand orientation.

This function performs a two-step process:
1. **Annotation**: Annotates summary statistics with reference allele frequency (RAF) from a 
   reference VCF/BCF or TSV file using `_annotate_sumstats`. The RAF is stored in a column 
   (default: "RAF") for use in strand inference.
2. **Strand Inference**: Infers strand orientation for palindromic SNPs and indels using the 
   annotated RAF and effect allele frequency (EAF) from the sumstats using `_infer_strand`.

The function determines strand orientation by comparing EAF (from sumstats) with RAF (from 
reference) to identify whether variants are on the forward (+) or reverse (-) strand. This 
is particularly important for palindromic SNPs (A/T, G/C) and indels where strand orientation 
cannot be determined from alleles alone.

**STATUS Code Updates:**
- Non-palindromic SNPs: STATUS 7th digit = `0`
- Palindromic SNPs (forward): STATUS 7th digit = `1`
- Palindromic SNPs (reverse): STATUS 7th digit = `2`
- Indels (forward): STATUS 7th digit = `3`
- Indels (ambiguous): STATUS 7th digit = `4`
- Indels (reverse): STATUS 7th digit = `6`
- Palindromic SNPs (MAF > threshold): STATUS 7th digit = `7`
- Variants not found in reference: STATUS 7th digit = `8`
- Variants not checked: STATUS 7th digit = `9`

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame or Sumstats`	Summary statistics DataFrame or Sumstats object. Must contain columns: CHR, POS, EA, NEA, EAF, STATUS.	required
`path`	`str or None`	Path to reference file (VCF/BCF or TSV). If provided, overrides `tsv_path`. The function will automatically detect the file format.	required
`vcf_path`	`str or None`	Path to VCF/BCF file. If provided, overrides both `path` and `tsv_path`.	required
`tsv_path`	`str or None`	Path to precomputed lookup TSV file. If not provided and `vcf_path` is given, a lookup table will be generated from the VCF/BCF file.	required
`assign_cols`	`tuple or str`	Column names to extract from reference file during annotation. The first column will be renamed to `raf` (default: "RAF") for use in strand inference.	`("AF",)`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper with automatic format detection.	required
`threads`	`int`	Number of threads for parallel processing during annotation and lookup table generation.	`6`
`reuse_lookup`	`bool`	If True, reuse existing lookup table TSV file if found. If False, regenerate from VCF/BCF.	`True`
`convert_to_bcf`	`bool`	If True, convert VCF to BCF format for faster processing. Note: `strip_info` will be set to False if converting to BCF (INFO fields are needed for AF extraction).	`False`
`strip_info`	`bool`	If True, strip INFO fields from VCF during lookup table generation to reduce file size. Set to False if INFO fields are needed for other purposes.	`True`
`chrom`	`str`	Column name for chromosome in sumstats.	`"CHR"`
`pos`	`str`	Column name for position in sumstats.	`"POS"`
`ea`	`str`	Column name for effect allele in sumstats.	`"EA"`
`nea`	`str`	Column name for non-effect allele in sumstats.	`"NEA"`
`eaf`	`str`	Column name for effect allele frequency in sumstats.	`"EAF"`
`raf`	`str`	Column name to store reference allele frequency (from reference file). This column will be created/updated during annotation.	`"RAF"`
`flipped_col`	`str`	Column name indicating if alleles were flipped during harmonization (True = reverse strand). This column is used as a hint for strand inference but is not required.	`"ALLELE_FLIPPED"`
`strand_col`	`str`	Column name to store strand orientation ('+' for forward, '-' for reverse, '?' for unknown). This column will be created if it doesn't exist.	`"STRAND"`
`maf_threshold`	`float`	Maximum minor allele frequency threshold for palindromic SNPs. Palindromic SNPs with MAF > threshold in either EAF or RAF will be excluded from strand determination and assigned STATUS 7th digit = `7`. Higher values allow more variants to be processed, but may reduce accuracy for ambiguous cases.	`0.40`
`ref_maf_threshold`	`float`	Maximum minor allele frequency threshold for reference allele frequency (RAF). Used as an additional filter for palindromic SNPs. Variants with MAF(RAF) > threshold will be excluded from strand determination.	`0.40`
`daf_tolerance`	`float`	Difference in allele frequency tolerance for indels. For indels, the function compares \|EAF - RAF\| (forward) and \|EAF - (1 - RAF)\| (reverse). If the difference is within `daf_tolerance`, the strand is assigned. If both differences are within tolerance or both exceed tolerance, the indel is marked as ambiguous (STATUS 7th digit = `4`).	`0.20`
`verbose`	`bool`	If True, print progress messages and warnings.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame or Sumstats`	If input is a DataFrame, returns updated DataFrame with RAF and STRAND columns. If input is a Sumstats object, returns the Sumstats object with updated data.

Notes

- This function requires the sumstats to have been harmonized (STATUS codes present) and 
  to have EAF values for variants of interest.
- The function uses optimized bulk lookup methods for faster processing compared to 
  per-variant VCF queries.
- For palindromic SNPs, strand inference is only performed if MAF can be reliably 
  determined from EAF alone (EAF < maf_threshold OR EAF > 1 - maf_threshold).
- The function automatically handles chromosome name mapping using ChromosomeMapper.
- Lookup tables are cached as TSV files for faster subsequent runs when `reuse_lookup=True`.

See Also

_annotate_sumstats : Function that performs the annotation step.
_infer_strand : Function that performs the strand inference step.

flip_allele_stats

flip_allele_stats(**kwargs: Any) -> Sumstats

Adjust statistics when allele direction has changed based on STATUS codes.

This function adjusts effect sizes and allele-specific statistics when variants have been
flipped or converted to reverse complement. It handles multiple scenarios: reverse
complement conversion for SNPs, allele swapping for REF/ALT mismatches, flipping for
standardized indels, and strand flipping for palindromic variants. Run after checking
with reference sequence.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats or DataFrame`	Sumstats object or DataFrame containing the data to process.	required
`verbose`	`bool`	If True, print progress messages during processing.	`False`

Returns:

Type	Description
`DataFrame`	Summary statistics with effect sizes and alleles flipped where required. When called via :meth:`Sumstats.flip_allele_stats()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

assign_rsid

assign_rsid(ref_rsid_tsv=None, ref_rsid_vcf=None, **kwargs)

Assign rsID to variants by matching with reference file.

This function assigns rsID to variants in the summary statistics by matching
chromosome position and alleles with a reference file (VCF or TSV format).
It supports different overwrite modes and can process data in parallel.

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame`	Summary statistics DataFrame containing variant data.	required
`path`	`str`	Path to the reference file (VCF or TSV).	required
`ref_mode`	`str`	Reference file format ("vcf" for VCF files, "tsv" for TSV files).	`"vcf"`
`snpid`	`str`	Column name for SNP IDs in the summary statistics.	`"SNPID"`
`rsid`	`str`	Column name for rsIDs in the summary statistics.	`"rsID"`
`chr`	`str`	Column name for chromosome information.	`"CHR"`
`pos`	`str`	Column name for position information.	`"POS"`
`ref`	`str`	Column name for reference allele (non-effect allele).	`"NEA"`
`alt`	`str`	Column name for alternative allele (effect allele).	`"EA"`
`status`	`str`	Column name for status codes.	`"STATUS"`
`threads`	`int`	Number of CPU cores to use for parallel processing.	`1`
`chunksize`	`int`	Size of chunks for processing large reference files.	`5000000`
`ref_snpid`	`str`	Column name for SNP IDs in the reference TSV file.	`"SNPID"`
`ref_rsid`	`str`	Column name for rsIDs in the reference TSV file.	`"rsID"`
`overwrite`	`str`	Overwrite mode for rsID assignment: - "all": overwrite rsID for all available rsID - "invalid": only assign rsID for variants with invalid rsID - "empty": only assign rsID for variants with NA rsID	`"empty"`
`verbose`	`bool`	If True, print progress messages.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper.	required

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with rsID assignments. When called via :meth:`Sumstats.assign_rsid()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

Notes

The function first checks if the required columns are present in the summary
statistics. For VCF reference files, it matches variants based on
CHR:POS:REF:ALT/ALT:REF. For TSV reference files, it matches based on SNPID.
The function handles parallel processing for large datasets and provides
detailed logging of the assignment process.

assign_rsid2

assign_rsid2(**kwargs)

Assign rsIDs to GWAS summary statistics using reference data with allele matching and STATUS filtering.

This function assigns rsIDs to a GWAS summary statistics DataFrame by matching variants against a reference
VCF or TSV file. It performs allele-aware matching and applies STATUS-based filtering to determine which
variants should be assigned rsIDs. The function handles various reference formats and allows control over
overwrite behavior for existing rsID values.

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame or Sumstats`	Summary statistics DataFrame or Sumstats object.	required
`path`	`str or None`	Path to reference file (VCF/BCF or TSV). Overrides `tsv_path`.	required
`vcf_path`	`str or None`	Path to VCF/BCF file. Overrides `path` and `tsv_path`.	required
`tsv_path`	`str or None`	Path to precomputed lookup TSV file. If not provided, generated from VCF.	required
`reuse_lookup`	`bool`	If True, reuse existing lookup TSV if available.	required
`convert_to_bcf`	`bool`	If True, convert VCF to BCF before processing.	required
`strip_info`	`bool`	If True, strip INFO fields when converting VCF to BCF.	required
`threads`	`int`	Number of threads for bcftools operations.	required
`overwrite`	`str`	Overwrite mode: "all", "invalid", or "empty". Determines which existing rsID values to overwrite. Default is "empty".	required
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper with automatic format detection.	required
`log`	`Log`	Log object for recording progress. Default is a new Log instance.	required
`verbose`	`bool`	If True, log detailed progress messages. Default is True.	required

Returns:

Type	Description
`DataFrame`	The input `sumstats` DataFrame with rsID column updated. When called via :meth:`Sumstats.assign_rsid2()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

Raises:

Type	Description
`ValueError`	If required columns are missing in `sumstats` or invalid `overwrite` value is provided. FileNotFoundError If specified reference file is not found.

Notes

- The function first checks for required columns in `sumstats`.
- STATUS filtering uses a regex pattern to identify variants eligible for rsID assignment.
- Overwrite modes:
  * "all": overwrite all rsIDs for eligible variants
  * "invalid": overwrite only non-rsID formatted values (e.g., not matching "rs[0-9]+")
  * "empty": only fill missing rsID values
- If `ref_mode` is "auto", the function determines whether to use VCF or TSV based on file extension.

rsid_to_chrpos

rsid_to_chrpos(**kwargs)

Assign CHR and POS using rsIDs (uses fast HDF5-based parallel processing).

This function uses optimized HDF5-based parallel processing which is much faster than
the old TSV-based approach. It requires an HDF5 reference file (generated from VCF using
`process_vcf_to_hfd5()`) or will auto-generate the path from a VCF reference file.

This function assigns CHR and POS values to summary statistics by matching rsIDs against
reference HDF5 files (one per chromosome) containing rsID to POS mappings. The HDF5 files
are typically generated from a VCF file using `process_vcf_to_hfd5()` and contain
precomputed POS values grouped by modulo 10 (rsID % 10). CHR is extracted from the filename.
The function processes data in parallel using multiple CPU cores for improved performance
on large datasets.

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame`	Input summary statistics DataFrame containing variant data.	required
`rsid`	`str`	Column name containing rsID values in `sumstats`.	`"rsID"`
`chrom`	`str`	Column name for chromosome values to be updated.	`"CHR"`
`pos`	`str`	Column name for position values to be updated.	`"POS"`
`path`	`str`	Path to the HDF5 reference file. If not provided, must specify either `ref_rsid_to_chrpos_vcf` or `ref_rsid_to_chrpos_hdf5`.	required
`ref_rsid_to_chrpos_vcf`	`str`	Path to VCF file containing rsID to CHR:POS mappings. If provided, the corresponding HDF5 file path will be automatically generated using the mod10 naming convention.	required
`ref_rsid_to_chrpos_hdf5`	`str`	Path to pre-generated HDF5 file containing rsID to CHR:POS mappings. Takes precedence over `ref_rsid_to_chrpos_vcf`.	required
`build`	`str`	Reference genome build identifier. "99" indicates unknown or unspecified build.	`"99"`
`status`	`str`	Column name for status codes in `sumstats`.	`"STATUS"`
`threads`	`int`	Number of threads to use for parallel processing.	`4`
`block_size`	`int`	Deprecated: This parameter is ignored. The function now uses modulo 10 grouping (rsID % 10) to match the HDF5 file structure created by `process_vcf_to_hfd5()`.	required
`verbose`	`bool`	If True, print progress messages.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with CHR and POS values assigned based on rsID matches. Variants without valid rsIDs or matches will retain original values (or NaN if new columns were created). When called via :meth:`Sumstats.rsid_to_chrpos()` or :meth:`Sumstats.rsid_to_chrpos2()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

Notes

- The HDF5 reference files are organized as one file per chromosome with groups named
  "group_0" through "group_9" (based on rsID % 10), each containing a DataFrame with
  columns "rsn" (int64) and "POS" (int32). CHR is extracted from the filename.
- This function uses modulo 10 grouping to match the HDF5 structure created by
  `process_vcf_to_hfd5()`.
- The HDF5 files follow the naming convention:
  `{vcf_file_name}.chr{chr_num}.rsID_CHR_POS_mod10.h5`
- Non-valid rsIDs (containing non-numeric characters after "rs") are processed separately.
- Optimized with vectorized operations and efficient HDF5 access using pandas index operations.

rsid_to_chrpos2

rsid_to_chrpos2(**kwargs)

Assign CHR and POS using rsIDs (uses fast HDF5-based parallel processing).

This function uses optimized HDF5-based parallel processing which is much faster than
the old TSV-based approach. It requires an HDF5 reference file (generated from VCF using
`process_vcf_to_hfd5()`) or will auto-generate the path from a VCF reference file.

This function assigns CHR and POS values to summary statistics by matching rsIDs against
reference HDF5 files (one per chromosome) containing rsID to POS mappings. The HDF5 files
are typically generated from a VCF file using `process_vcf_to_hfd5()` and contain
precomputed POS values grouped by modulo 10 (rsID % 10). CHR is extracted from the filename.
The function processes data in parallel using multiple CPU cores for improved performance
on large datasets.

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame`	Input summary statistics DataFrame containing variant data.	required
`rsid`	`str`	Column name containing rsID values in `sumstats`.	`"rsID"`
`chrom`	`str`	Column name for chromosome values to be updated.	`"CHR"`
`pos`	`str`	Column name for position values to be updated.	`"POS"`
`path`	`str`	Path to the HDF5 reference file. If not provided, must specify either `ref_rsid_to_chrpos_vcf` or `ref_rsid_to_chrpos_hdf5`.	required
`ref_rsid_to_chrpos_vcf`	`str`	Path to VCF file containing rsID to CHR:POS mappings. If provided, the corresponding HDF5 file path will be automatically generated using the mod10 naming convention.	required
`ref_rsid_to_chrpos_hdf5`	`str`	Path to pre-generated HDF5 file containing rsID to CHR:POS mappings. Takes precedence over `ref_rsid_to_chrpos_vcf`.	required
`build`	`str`	Reference genome build identifier. "99" indicates unknown or unspecified build.	`"99"`
`status`	`str`	Column name for status codes in `sumstats`.	`"STATUS"`
`threads`	`int`	Number of threads to use for parallel processing.	`4`
`block_size`	`int`	Deprecated: This parameter is ignored. The function now uses modulo 10 grouping (rsID % 10) to match the HDF5 file structure created by `process_vcf_to_hfd5()`.	required
`verbose`	`bool`	If True, print progress messages.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with CHR and POS values assigned based on rsID matches. Variants without valid rsIDs or matches will retain original values (or NaN if new columns were created). When called via :meth:`Sumstats.rsid_to_chrpos()` or :meth:`Sumstats.rsid_to_chrpos2()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

Notes

- The HDF5 reference files are organized as one file per chromosome with groups named
  "group_0" through "group_9" (based on rsID % 10), each containing a DataFrame with
  columns "rsn" (int64) and "POS" (int32). CHR is extracted from the filename.
- This function uses modulo 10 grouping to match the HDF5 structure created by
  `process_vcf_to_hfd5()`.
- The HDF5 files follow the naming convention:
  `{vcf_file_name}.chr{chr_num}.rsID_CHR_POS_mod10.h5`
- Non-valid rsIDs (containing non-numeric characters after "rs") are processed separately.
- Optimized with vectorized operations and efficient HDF5 access using pandas index operations.

annotate_sumstats

annotate_sumstats(**kwargs: Any) -> None

Annotate GWAS summary statistics by assigning fields (e.g., rsID, AF) from a lookup table extracted from a VCF/BCF.

Two modes:
  (1) If tsv_path exists and reuse_lookup=True → skip extraction.
  (2) Otherwise extract from VCF → create tsv_path → annotate.

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame or Sumstats`	Summary statistics DataFrame or Sumstats object. Must contain CHR, POS, EA, NEA.	required
`vcf_path`	`str or None`	Required if lookup table needs to be generated.	required
`tsv_path`	`str`	Lookup table file (tsv or tsv.gz).	required
`assign_cols`	`tuple[str]`	Columns to assign (e.g., ("rsID","AF")).	required
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper with automatic format detection.	required
`threads`	`int`	bcftools threads. chrom, pos, ea, nea : str Column names in sumstats.	required
`reuse_lookup`	`bool`	If True, reuse lookup if exists.	required

Returns:

Name	Type	Description
`sumstats`	`DataFrame`
`tsv_path`	`str`

check_af

check_af(ref_infer, **kwargs)

Check the difference between effect allele frequency (EAF) in summary statistics and alternative allele frequency in reference VCF.

This function calculates the difference between the effect allele frequency (EAF) in the summary 
statistics and the alternative allele frequency (ALT_AF) in the reference VCF file. The difference 
(DAF) is stored in a new column in the summary statistics DataFrame.

**Purpose:**
- Quality control: Identify variants with large differences in allele frequency between sumstats 
  and reference, which may indicate:
  - Population differences
  - Allele mismatches or strand flips
  - Data quality issues
- Validation: Verify that EAF values in sumstats are consistent with reference population frequencies.

**DAF Calculation:**
- DAF = EAF (sumstats) - ALT_AF (reference VCF)
- Positive DAF: EAF in sumstats is higher than reference
- Negative DAF: EAF in sumstats is lower than reference
- Large |DAF| values (> 0.2) may indicate issues requiring investigation

**When to use:**
- After `infer_af()` to validate inferred EAF values
- Before harmonization to identify potential allele mismatches
- For quality control to flag variants with unusual frequency differences

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`Sumstats or DataFrame`	Sumstats object or DataFrame containing variant data. Must have columns: CHR, POS, EA, NEA, EAF.	required
`ref_infer`	`str`	Path to the reference VCF file. Must be indexed (tabix) for efficient querying.	required
`ref_alt_freq`	`str`	Field name for alternative allele frequency in VCF INFO section (e.g., "AF", "AF_popmax", "gnomAD_AF"). If None, the function will attempt to auto-detect common AF field names.	required
`maf_threshold`	`float`	Minor allele frequency threshold for filtering variants. Only variants with MAF ≤ threshold in both sumstats and reference are included in DAF statistics. This helps focus on common variants where frequency differences are more meaningful.	`0.4`
`column_name`	`str`	Name of the column to store the difference values. The final column name will be `column_name + suffix`.	`"DAF"`
`suffix`	`str`	Suffix to append to the column name (e.g., "_pop1", "_gnomad"). Useful when comparing multiple reference populations.	`""`
`threads`	`int`	Number of CPU cores to use for parallel processing. Set to 1 if processing < 10,000 variants.	`1`
`chr`	`str`	Column name for chromosome in sumstats.	`"CHR"`
`pos`	`str`	Column name for position in sumstats.	`"POS"`
`ref`	`str`	Column name for reference/non-effect allele in sumstats.	`"NEA"`
`alt`	`str`	Column name for alternative/effect allele in sumstats.	`"EA"`
`eaf`	`str`	Column name for effect allele frequency in sumstats.	`"EAF"`
`status`	`str`	Column name for status codes. By default, only processes variants with STATUS digit 4 = 0 (standardized and normalized), unless `force=True`.	`"STATUS"`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper with automatic format detection.	required
`force`	`bool`	If True, check all variants regardless of STATUS codes. If False, only processes variants with valid harmonization status (STATUS digit 4 = 0).	`False`
`verbose`	`bool`	If True, print progress messages and DAF statistics (max, min, mean, std, abs statistics).	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with a new column (`column_name + suffix`) containing the difference between EAF (sumstats) and ALT_AF (reference VCF). Variants that could not be matched in the reference VCF will have DAF = NaN.

Notes

- The difference in allele frequency (DAF) is calculated as: DAF = EAF (sumstats) - ALT_AF (reference VCF)
- **Important**: This DAF is NOT the derived allele frequency. It is simply the difference in 
  allele frequencies between two datasets.
- The function requires the reference VCF to be indexed (tabix) for efficient chromosome-based 
  querying. Use `tabix -p vcf reference.vcf.gz` to create an index.
- By default, only processes variants with valid harmonization status (STATUS digit 4 = 0) to 
  ensure alleles are standardized and normalized.
- The function provides comprehensive statistics about DAF values including:
  - Maximum and minimum DAF
  - Mean and standard deviation of DAF
  - Statistics for absolute DAF values
  - Count of variants with |DAF| > 0.2 (potential issues)
- Only variants with valid chromosome position and allele information are checked by default.
- For small datasets (< 10,000 variants), the function automatically sets `threads=1` to avoid overhead.
- Large |DAF| values (> 0.2) may indicate:
  - Population differences (expected for population-specific variants)
  - Allele mismatches (check EA/NEA alignment)
  - Strand flips (use `infer_strand()` to resolve)
  - Data quality issues (verify EAF calculation in sumstats)

See Also

infer_af : Infer EAF from reference VCF before checking differences.
plot_daf : Visualize DAF distribution to identify outliers.
infer_strand : Resolve strand orientation issues that may cause large DAF values.

check_af2

check_af2(**kwargs)

Check the difference between effect allele frequency (EAF) in summary statistics and alternative allele frequency in reference VCF using sweep mode.

This function performs a two-step process:
1. **Annotation**: Annotates summary statistics with reference allele frequency (AF) from a 
   reference VCF/BCF or TSV file using `_annotate_sumstats`. The AF is stored in a column 
   (default: same as `ref_alt_freq`) for use in DAF calculation.
2. **DAF Calculation**: Calculates the difference between EAF (sumstats) and ALT_AF (reference) 
   with proper allele matching. The difference (DAF) is stored in a new column.

**DAF Calculation:**
- DAF = EAF (sumstats) - ALT_AF (reference VCF)
- Positive DAF: EAF in sumstats is higher than reference
- Negative DAF: EAF in sumstats is lower than reference
- Large |DAF| values (> 0.2) may indicate issues requiring investigation

**When to use:**
- After `infer_af2()` to validate inferred EAF values
- Before harmonization to identify potential allele mismatches
- For quality control to flag variants with unusual frequency differences

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame or Sumstats`	Summary statistics DataFrame or Sumstats object. Must contain columns: CHR, POS, EA, NEA, EAF.	required
`path`	`str or None`	Path to reference file (VCF/BCF or TSV). If provided, overrides `tsv_path`. The function will automatically detect the file format.	required
`vcf_path`	`str or None`	Path to VCF/BCF file. If provided, overrides both `path` and `tsv_path`.	required
`tsv_path`	`str or None`	Path to precomputed lookup TSV file. If not provided and `vcf_path` is given, a lookup table will be generated from the VCF/BCF file.	required
`assign_cols`	`tuple or str`	Column names to extract from reference file during annotation. The first column will be used as the reference AF for DAF calculation.	`("AF",)`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper.	required
`threads`	`int`	Number of threads for parallel processing during annotation and lookup table generation.	`6`
`reuse_lookup`	`bool`	If True, reuse existing lookup table TSV file if found. If False, regenerate from VCF/BCF.	`True`
`convert_to_bcf`	`bool`	If True, convert VCF to BCF format for faster processing. Note: `strip_info` will be set to False if converting to BCF (INFO fields are needed for AF extraction).	`False`
`strip_info`	`bool`	If True, strip INFO fields from VCF during lookup table generation to reduce file size. Set to False if INFO fields are needed for other purposes.	`True`
`chrom`	`str`	Column name for chromosome in sumstats.	`"CHR"`
`pos`	`str`	Column name for position in sumstats.	`"POS"`
`ea`	`str`	Column name for effect allele in sumstats.	`"EA"`
`nea`	`str`	Column name for non-effect allele in sumstats.	`"NEA"`
`eaf`	`str`	Column name for effect allele frequency in sumstats.	`"EAF"`
`ref_alt_freq`	`str`	Field name for alternative allele frequency in VCF INFO section (e.g., "AF", "AF_popmax", "gnomAD_AF"). This should match the field name in the reference VCF.	`"AF"`
`column_name`	`str`	Name of the column to store the difference values. The final column name will be `column_name + suffix`.	`"DAF"`
`suffix`	`str`	Suffix to append to the column name (e.g., "_pop1", "_gnomad"). Useful when comparing multiple reference populations.	`""`
`status`	`str`	Column name for status codes. By default, only processes variants with STATUS digit 4 = 0 (standardized and normalized), unless `force=True`.	`"STATUS"`
`force`	`bool`	If True, check all variants regardless of STATUS codes. If False, only processes variants with valid harmonization status (STATUS digit 4 = 0).	`False`
`verbose`	`bool`	If True, print progress messages and DAF statistics.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame or Sumstats`	If input is a DataFrame, returns updated DataFrame with DAF column. If input is a Sumstats object, returns the Sumstats object with updated data. When called via :meth:`Sumstats.check_af2()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `None`.

Notes

- This function uses optimized bulk lookup methods for faster processing compared to 
  per-variant VCF queries.
- The function automatically handles chromosome name mapping using ChromosomeMapper.
- Lookup tables are cached as TSV files for faster subsequent runs when `reuse_lookup=True`.
- The difference in allele frequency (DAF) is calculated as: DAF = EAF (sumstats) - ALT_AF (reference VCF)
- **Important**: This DAF is NOT the derived allele frequency. It is simply the difference in 
  allele frequencies between two datasets.
- Large |DAF| values (> 0.2) may indicate:
  - Population differences (expected for population-specific variants)
  - Allele mismatches (check EA/NEA alignment)
  - Strand flips (use `infer_strand()` to resolve)
  - Data quality issues (verify EAF calculation in sumstats)

See Also

_infer_af_with_annotation : Function that infers EAF from reference VCF before checking differences.
_infer_strand_with_annotation : Function that infers strand orientation which may affect DAF values.

infer_af

infer_af(ref_infer, **kwargs)

Infer effect allele frequency (EAF) in summary statistics using reference VCF ALT frequency.

This function infers the effect allele frequency (EAF) in the summary statistics by matching
variants with a reference VCF file. It extracts the alternative allele frequency (ALT_AF) from 
the reference VCF INFO field and updates the EAF values in the summary statistics DataFrame.

**Workflow:**
1. Matches variants in sumstats with reference VCF by CHR:POS:EA:NEA.
2. Extracts ALT frequency from VCF INFO field (specified by `ref_alt_freq`).
3. Handles allele matching: If EA matches ALT in VCF, uses ALT_AF directly. If EA matches REF 
   in VCF, uses 1 - ALT_AF.
4. Updates EAF column in sumstats with inferred values.

**When to use:**
- When sumstats are missing EAF values but have valid CHR, POS, EA, NEA.
- When you want to fill in EAF from a reference population (e.g., 1000 Genomes, gnomAD).
- As a preprocessing step before strand inference or allele frequency comparison.

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`Sumstats or DataFrame`	Sumstats object or DataFrame containing variant data. Must have columns: CHR, POS, EA, NEA.	required
`ref_infer`	`str`	Path to the reference VCF file. Must be indexed (tabix) for efficient querying.	required
`ref_alt_freq`	`str`	Field name for alternative allele frequency in VCF INFO section (e.g., "AF", "AF_popmax", "gnomAD_AF"). If None, the function will attempt to auto-detect common AF field names.	required
`threads`	`int`	Number of CPU cores to use for parallel processing. Set to 1 if processing < 10,000 variants.	`1`
`chr`	`str`	Column name for chromosome in sumstats.	`"CHR"`
`pos`	`str`	Column name for position in sumstats.	`"POS"`
`ref`	`str`	Column name for reference/non-effect allele in sumstats.	`"NEA"`
`alt`	`str`	Column name for alternative/effect allele in sumstats.	`"EA"`
`eaf`	`str`	Column name for effect allele frequency. This column will be created if it doesn't exist, and existing values will be updated where inference is successful.	`"EAF"`
`status`	`str`	Column name for status codes. By default, only processes variants with STATUS digit 4 = 0 (standardized and normalized), unless `force=True`.	`"STATUS"`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper with automatic format detection.	required
`force`	`bool`	If True, infer EAF for all variants regardless of STATUS codes. If False, only processes variants with valid harmonization status (STATUS digit 4 = 0).	`False`
`verbose`	`bool`	If True, print progress messages and statistics about inference success rate.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with inferred EAF values. Variants that could not be matched in the reference VCF will have EAF = NaN.

Notes

- The function requires the reference VCF to be indexed (tabix) for efficient chromosome-based 
  querying. Use `tabix -p vcf reference.vcf.gz` to create an index.
- By default, only processes variants with valid harmonization status (STATUS digit 4 = 0) to 
  ensure alleles are standardized and normalized.
- The function uses parallel processing to improve performance on large datasets. For small 
  datasets (< 10,000 variants), it automatically sets `threads=1` to avoid overhead.
- After inference, the function reports statistics about:
  - Number of variants with EAF successfully inferred
  - Number of variants still missing EAF (not found in reference or missing ref_alt_freq field)
- The inferred EAF values are stored in the specified EAF column, overwriting existing values 
  where inference is successful.
- The function automatically handles chromosome name mapping using ChromosomeMapper. 
  for proper matching.

See Also

check_af : Calculate difference between EAF (sumstats) and ALT_AF (reference) after inference.
infer_eaf_from_maf : Infer EAF from MAF using reference VCF (alternative method).

infer_af2

infer_af2(**kwargs)

Infer effect allele frequency (EAF) in summary statistics using reference VCF ALT frequency in sweep mode.

This function performs a two-step process:
1. **Annotation**: Annotates summary statistics with reference allele frequency (AF) from a 
   reference VCF/BCF or TSV file using `_annotate_sumstats`. The AF is stored in a column 
   (default: same as `ref_alt_freq`) for use in EAF inference.
2. **EAF Inference**: Infers EAF values by matching variants and handling allele orientation. 
   If EA matches ALT in VCF, uses ALT_AF directly. If EA matches REF in VCF, uses 1 - ALT_AF.

**Workflow:**
1. Matches variants in sumstats with reference VCF by CHR:POS:EA:NEA.
2. Extracts ALT frequency from VCF INFO field (specified by `ref_alt_freq`).
3. Handles allele matching: If EA matches ALT in VCF, uses ALT_AF directly. If EA matches REF 
   in VCF, uses 1 - ALT_AF.
4. Updates EAF column in sumstats with inferred values.

**When to use:**
- When sumstats are missing EAF values but have valid CHR, POS, EA, NEA.
- When you want to fill in EAF from a reference population (e.g., 1000 Genomes, gnomAD).
- As a preprocessing step before strand inference or allele frequency comparison.

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame or Sumstats`	Summary statistics DataFrame or Sumstats object. Must contain columns: CHR, POS, EA, NEA.	required
`path`	`str or None`	Path to reference file (VCF/BCF or TSV). If provided, overrides `tsv_path`. The function will automatically detect the file format.	required
`vcf_path`	`str or None`	Path to VCF/BCF file. If provided, overrides both `path` and `tsv_path`.	required
`tsv_path`	`str or None`	Path to precomputed lookup TSV file. If not provided and `vcf_path` is given, a lookup table will be generated from the VCF/BCF file.	required
`assign_cols`	`tuple or str`	Column names to extract from reference file during annotation. The first column will be used as the reference AF for EAF inference.	`("AF",)`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper.	required
`threads`	`int`	Number of threads for parallel processing during annotation and lookup table generation.	`6`
`reuse_lookup`	`bool`	If True, reuse existing lookup table TSV file if found. If False, regenerate from VCF/BCF.	`True`
`convert_to_bcf`	`bool`	If True, convert VCF to BCF format for faster processing. Note: `strip_info` will be set to False if converting to BCF (INFO fields are needed for AF extraction).	`False`
`strip_info`	`bool`	If True, strip INFO fields from VCF during lookup table generation to reduce file size. Set to False if INFO fields are needed for other purposes.	`True`
`chrom`	`str`	Column name for chromosome in sumstats.	`"CHR"`
`pos`	`str`	Column name for position in sumstats.	`"POS"`
`ea`	`str`	Column name for effect allele in sumstats.	`"EA"`
`nea`	`str`	Column name for non-effect allele in sumstats.	`"NEA"`
`eaf`	`str`	Column name for effect allele frequency. This column will be created if it doesn't exist, and existing values will be updated where inference is successful.	`"EAF"`
`ref_alt_freq`	`str`	Field name for alternative allele frequency in VCF INFO section (e.g., "AF", "AF_popmax", "gnomAD_AF"). This should match the field name in the reference VCF.	`"AF"`
`status`	`str`	Column name for status codes. By default, only processes variants with STATUS digit 4 = 0 (standardized and normalized), unless `force=True`.	`"STATUS"`
`force`	`bool`	If True, infer EAF for all variants regardless of STATUS codes. If False, only processes variants with valid harmonization status (STATUS digit 4 = 0).	`False`
`verbose`	`bool`	If True, print progress messages and statistics about inference success rate.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame or Sumstats`	If input is a DataFrame, returns updated DataFrame with inferred EAF values. If input is a Sumstats object, returns the Sumstats object with updated data. When called via :meth:`Sumstats.infer_af2()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `None`.

Notes

- This function uses optimized bulk lookup methods for faster processing compared to 
  per-variant VCF queries.
- The function automatically handles chromosome name mapping using ChromosomeMapper.
- Lookup tables are cached as TSV files for faster subsequent runs when `reuse_lookup=True`.
- By default, only processes variants with valid harmonization status (STATUS digit 4 = 0) to 
  ensure alleles are standardized and normalized.
- After inference, the function reports statistics about:
  - Number of variants with EAF successfully inferred
  - Number of variants still missing EAF (not found in reference or missing ref_alt_freq field)
- The inferred EAF values are stored in the specified EAF column, overwriting existing values 
  where inference is successful.

See Also

_check_af_with_annotation : Calculate difference between EAF (sumstats) and ALT_AF (reference) after inference.
_infer_strand_with_annotation : Function that infers strand orientation which may affect EAF values.

infer_eaf_from_maf

infer_eaf_from_maf(ref_infer, **kwargs)

Infer effect allele frequency (EAF) in summary statistics from MAF using reference VCF ALT frequency.

This function infers the effect allele frequency (EAF) in the summary statistics by first
extracting the reference allele frequency from a VCF file, then using this information along
with the summary statistics MAF to calculate the correct EAF. It handles cases where the
effect allele might need to be flipped based on the reference data.

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`Sumstats or DataFrame`	Sumstats object or DataFrame containing variant data.	required
`ref_infer`	`str`	Path to the reference VCF file.	required
`ref_alt_freq`	`str`	Field name for alternative allele frequency in VCF INFO section.	required
`threads`	`int`	Number of CPU cores to use for parallel processing.	`1`
`chr`	`str`	Column name for chromosome information.	`"CHR"`
`pos`	`str`	Column name for position information.	`"POS"`
`ref`	`str`	Column name for reference allele (non-effect allele).	`"NEA"`
`alt`	`str`	Column name for alternative allele (effect allele).	`"EA"`
`eaf`	`str`	Column name for effect allele frequency.	`"EAF"`
`maf`	`str`	Column name for minor allele frequency.	`"MAF"`
`ref_eaf`	`str`	Temporary column name for storing reference allele frequency.	`"_REF_EAF"`
`status`	`str`	Column name for status codes.	`"STATUS"`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper.	required
`force`	`bool`	If True, infer EAF for all variants regardless of status.	`False`
`verbose`	`bool`	If True, print progress messages.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame`	Updated summary statistics DataFrame with inferred EAF values.

Notes

- The function first extracts reference allele frequencies from the VCF file.
- It then compares these reference frequencies with the MAF in the summary statistics
  to determine if flipping is needed.
- If the reference allele frequency and MAF suggest different major/minor alleles,
  the EAF is calculated as 1 - MAF (flipping the allele).
- The temporary reference EAF column (_REF_EAF) is dropped before returning the DataFrame.
- The function provides statistics about the number of variants for which EAF was
  successfully inferred and those still missing EAF values.

infer_eaf_from_maf2

infer_eaf_from_maf2(**kwargs)

Infer effect allele frequency (EAF) in summary statistics from MAF using reference VCF ALT frequency in sweep mode.

This function performs a two-step process:
1. **Annotation**: Annotates summary statistics with reference allele frequency (AF) from a 
   reference VCF/BCF or TSV file using `_annotate_sumstats`. The AF is stored in a temporary 
   column (default: "_REF_EAF") for use in EAF inference.
2. **EAF Inference**: Infers EAF values by comparing reference AF with MAF from sumstats. 
   If the reference allele frequency and MAF suggest different major/minor alleles, the EAF is 
   calculated as 1 - MAF (flipping the allele), otherwise MAF is used directly.

**Workflow:**
1. Matches variants in sumstats with reference VCF by CHR:POS:EA:NEA.
2. Extracts ALT frequency from VCF INFO field (specified by `ref_alt_freq`).
3. Compares reference AF with MAF to determine if flipping is needed:
   - If ref_AF >= 0.5 (ref allele is major) != MAF > 0.5 (MAF is minor), then flip (use 1 - MAF)
   - Otherwise use MAF directly
4. Updates EAF column in sumstats with inferred values.

**When to use:**
- When sumstats have MAF values but are missing EAF values.
- When you want to infer EAF from MAF using a reference population (e.g., 1000 Genomes, gnomAD).
- As an alternative to `infer_af2()` when you have MAF but not direct EAF information.

Parameters:

Name	Type	Description	Default
`sumstats`	`DataFrame or Sumstats`	Summary statistics DataFrame or Sumstats object. Must contain columns: CHR, POS, EA, NEA, MAF.	required
`path`	`str or None`	Path to reference file (VCF/BCF or TSV). If provided, overrides `tsv_path`. The function will automatically detect the file format.	required
`vcf_path`	`str or None`	Path to VCF/BCF file. If provided, overrides both `path` and `tsv_path`.	required
`tsv_path`	`str or None`	Path to precomputed lookup TSV file. If not provided and `vcf_path` is given, a lookup table will be generated from the VCF/BCF file.	required
`assign_cols`	`tuple or str`	Column names to extract from reference file during annotation. The first column will be used as the reference AF for EAF inference.	`("AF",)`
`mapper`	`ChromosomeMapper`	ChromosomeMapper instance to use for chromosome name conversion. If not provided and sumstats is a Sumstats object, uses sumstats.mapper. If not provided, creates a default mapper.	required
`threads`	`int`	Number of threads for parallel processing during annotation and lookup table generation.	`6`
`reuse_lookup`	`bool`	If True, reuse existing lookup table TSV file if found. If False, regenerate from VCF/BCF.	`True`
`convert_to_bcf`	`bool`	If True, convert VCF to BCF format for faster processing. Note: `strip_info` will be set to False if converting to BCF (INFO fields are needed for AF extraction).	`False`
`strip_info`	`bool`	If True, strip INFO fields from VCF during lookup table generation to reduce file size. Set to False if INFO fields are needed for other purposes.	`True`
`chrom`	`str`	Column name for chromosome in sumstats.	`"CHR"`
`pos`	`str`	Column name for position in sumstats.	`"POS"`
`ea`	`str`	Column name for effect allele in sumstats.	`"EA"`
`nea`	`str`	Column name for non-effect allele in sumstats.	`"NEA"`
`eaf`	`str`	Column name for effect allele frequency. This column will be created if it doesn't exist, and existing values will be updated where inference is successful.	`"EAF"`
`maf`	`str`	Column name for minor allele frequency in sumstats. This is required for EAF inference.	`"MAF"`
`ref_alt_freq`	`str`	Field name for alternative allele frequency in VCF INFO section (e.g., "AF", "AF_popmax", "gnomAD_AF"). This should match the field name in the reference VCF.	`"AF"`
`ref_eaf`	`str`	Temporary column name for storing reference allele frequency. This column will be created during annotation and dropped before returning.	`"_REF_EAF"`
`status`	`str`	Column name for status codes. By default, only processes variants with STATUS digit 4 = 0 (standardized and normalized), unless `force=True`.	`"STATUS"`
`force`	`bool`	If True, infer EAF for all variants regardless of STATUS codes. If False, only processes variants with valid harmonization status (STATUS digit 4 = 0).	`False`
`verbose`	`bool`	If True, print progress messages and statistics about inference success rate.	`True`
`log`	`Log`	Logging object for recording process information.	`Log()`

Returns:

Type	Description
`DataFrame or Sumstats`	If input is a DataFrame, returns updated DataFrame with inferred EAF values. If input is a Sumstats object, returns the Sumstats object with updated data. When called via :meth:`Sumstats.infer_af2()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `None`.

Notes

- This function uses optimized bulk lookup methods for faster processing compared to 
  per-variant VCF queries.
- The function automatically handles chromosome name mapping using ChromosomeMapper.
- Lookup tables are cached as TSV files for faster subsequent runs when `reuse_lookup=True`.
- By default, only processes variants with valid harmonization status (STATUS digit 4 = 0) to 
  ensure alleles are standardized and normalized.
- The function requires MAF values in sumstats to infer EAF.
- After inference, the function reports statistics about:
  - Number of variants with EAF successfully inferred
  - Number of variants still missing EAF (not found in reference or missing ref_alt_freq field)
- The inferred EAF values are stored in the specified EAF column, overwriting existing values 
  where inference is successful.
- The temporary reference EAF column (`ref_eaf`) is dropped before returning.

See Also

_infer_af_with_annotation : Function that infers EAF directly from reference ALT_AF.
_check_af_with_annotation : Calculate difference between EAF (sumstats) and ALT_AF (reference) after inference.

to_format

to_format(path, build=None, verbose=True, **kwargs)

Convert summary statistics to a tool-specific output format.

Supports VCF, VEP, BED, tabular (TSV/CSV/Parquet), and other presets
registered in the formatbook.

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`Sumstats or DataFrame`	Input summary statistics.	required
`path`	`str`	Output file path prefix.	`"./sumstats"`
`fmt`	`str`	Output format preset. See `get_formats_list()` for available names.	`"gwaslab"`
`tab_fmt`	`str`	Tabular format when `fmt` is not `vcf`, `bed`, or `annovar` (`tsv`, `csv`, or `parquet`). extract, exclude : list or str, optional Variants to keep or drop (rsID list or file path).	`"tsv"`
`cols`	`list`	Extra columns to include in tabular output.	required
`id_use`	`str`	Variant identifier column for extraction filters.	`"rsID"`
`hapmap3`	`bool`	Restrict output to HapMap3 SNPs.	`False`
`exclude_hla`	`bool`	Drop variants in the HLA region.	`False`
`hla_range`	`tuple`	HLA exclusion window in Mb on chromosome 6.	`(25, 34)`
`build`	`str`	Genome build label written to metadata.	`None`
`n`	`float`	Sample size value added as N when missing.	required
`no_status`	`bool`	Omit the STATUS column from output.	`False`
`output_log`	`bool`	Write a companion `.log` file.	`True`
`float_formats`	`dict`	Per-column float format strings for tabular export.	required
`xymt_number`	`bool`	Encode sex chromosomes and MT as numeric codes.	`False`
`xymt`	`list`	Sex/MT chromosome labels when `xymt_number` is False.	`["X", "Y", "MT"]`
`chr_prefix`	`str`	Prefix prepended to chromosome names.	`""`
`meta`	`dict`	Metadata merged into SSF-style sidecar files.	required
`ssfmeta`	`bool`	Emit GWAS-SSF metadata JSON alongside the sumstats file.	`False`
`md5sum`	`bool`	Write an MD5 checksum file. Hashes the compressed output file on disk (`.tsv.gz` / `.csv.gz`), not decompressed TSV content. The digest identifies the gwaslab-produced artifact; it will not match `md5sum` on the same TSV compressed manually with the `gzip` command, because Python and GNU gzip write different gzip headers even at the same level.	`False`
`gzip`	`bool`	Gzip-compress tabular output (`.tsv.gz` / `.csv.gz`). Uses gzip compression level 6 (same default as the `gzip` command), not pandas' inferred level 9. Decompressed content matches the same plain file compressed with `gzip -6`; compressed file size is equivalent within ~1%. Override via `to_csvargs={"compression": {"method": "gzip", "compresslevel": N}}`.	`True`
`bgzip`	`bool`	BGzip-compress output (requires `tabix` indexing).	`False`
`tabix`	`bool`	Build a Tabix index for compressed output.	`False`
`tabix_indexargs`	`dict`	Extra arguments passed to the Tabix indexer.	`{}`
`to_csvargs`	`dict`	Extra `DataFrame.to_csv` keyword arguments.	required
`to_tabular_kwargs`	`dict`	Extra tabular writer keyword arguments.	required
`validate`	`bool`	Validate SSF output via `gwas-ssf` CLI with pandas fallback.	`False`
`gwas_ssf_path`	`str`	Path to the `gwas-ssf` executable.	required
`verbose`	`bool`	Print progress messages.	`True`
`log`	`Log`	Logging object.	`Log()`

Returns:

Type	Description
`None`	Output is written to disk at `path`.

to_pickle

to_pickle(path='~/mysumstats.pickle', overwrite=False)

to_gsf

to_gsf(path, partition_cols=None, compression='zstd', verbose=True)

Save sumstats to GSF (GWASLab Standard Format) file.

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`Sumstats or DataFrame`	Sumstats object or DataFrame to process.	required
`path`	`str or Path`	Output path (.gsf file or directory for partitioned)	required
`meta`	`dict`	GWASLab metadata dictionary	required
`partition_cols`	`list of str`	Columns to partition by (e.g., ["CHR"])	`None`
`compression`	`str`	Compression codec: "snappy", "gzip", "brotli", "zstd", "lz4"	`"zstd"`
`log`	`Log`	Log object	required
`verbose`	`bool`	Print progress messages	`True`

Examples:

>>> write_gsf(sumstats, "sumstats.gsf")
>>> write_gsf(sumstats, "sumstats_partitioned", partition_cols=["CHR"])

Returns:

Type	Description
`str`	Path to written file

offload

offload()

view_sumstats

view_sumstats(expr=None)

View the sumstats dataframe, optionally filtering by an expression.

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`Sumstats or DataFrame`	Sumstats object or DataFrame to view.	required
`expr`	`str`	A query expression string to filter the dataframe (e.g., 'P < 5e-8'). If None, returns the original dataframe.	`None`

Returns:

Type	Description
`DataFrame`	The filtered or original dataframe.

reload

reload(delete_files=None)

Reload data from temporary pickle file.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to the pickle file to reload	required
`log`	`Log`	Logger instance	required
`delete_files`	`list of str`	Additional files to delete after successful reload	`None`

Returns:

Type	Description
`DataFrame`	Reloaded dataframe

report

report(output_path='gwas_qc_report.html', **kwargs)

Generate a comprehensive QC report including basic QC, harmonization (optional), lead variants, MQQ plots, regional plots, and output (optional).

This function performs:
1. Basic QC using basic_check()
2. Harmonization using harmonize() (optional, if harmonize_kwargs provided)
3. Lead variant extraction using get_lead()
4. MQQ plot generation
5. Regional plots for each lead variant locus
6. Output to specified format using to_format() (optional, if output_kwargs provided)
7. HTML/PDF report generation with all results

Parameters:

Name	Type	Description	Default
`sumstats`	`Sumstats`	Sumstats object to analyze.	required
`output_path`	`str`	Path to save the report (HTML or PDF when `weasyprint` is installed).	`"gwas_qc_report.html"`
`basic_check_kwargs`	`dict`	Keyword arguments passed to `basic_check()`.	required
`harmonize_kwargs`	`dict`	Keyword arguments passed to `harmonize()`. When provided, harmonization runs after basic QC.	required
`get_lead_kwargs`	`dict`	Keyword arguments passed to `get_lead()`.	required
`mqq_plot_kwargs`	`dict`	Keyword arguments passed to `plot_mqq()` for the genome-wide plot.	required
`regional_plot_kwargs`	`dict`	Keyword arguments passed to `plot_region()` for locus plots.	required
`output_kwargs`	`dict`	Keyword arguments passed to `to_format()` (must include `path`).	required
`report_title`	`str`	Title shown in the HTML report.	`"GWAS Quality Control Report"`
`verbose`	`bool`	Print progress messages.	`True`

Returns:

Type	Description
`str`	Path to the generated report file.

Notes

For PDF output, `weasyprint` must be installed.
If PDF format is requested but weasyprint is not available, the function will
generate HTML instead and issue a warning.

Examples:

>>> import gwaslab as gl
>>> mysumstats = gl.Sumstats("sumstats.txt.gz")
>>> # Basic report
>>> gl.generate_qc_report(
...     mysumstats,
...     output_path="my_report.html",
...     get_lead_kwargs={"sig_level": 5e-8, "windowsizekb": 500}
... )
>>> # With harmonization
>>> gl.generate_qc_report(
...     mysumstats,
...     output_path="my_report.html",
...     harmonize_kwargs={"ref_seq": "ref.fa", "ref_infer": "ref.vcf.gz"}
... )
>>> # With output
>>> gl.generate_qc_report(
...     mysumstats,
...     output_path="my_report.html",
...     output_kwargs={"path": "clean_sumstats", "fmt": "ldsc", "gzip": True}
... )

Panel

Panel(panel_type: str, **kwargs: Any) -> Panel

Create a Panel object with this Sumstats object's data.

    Works like the Panel class constructor, but automatically passes self.data
    to the sumstats parameter for panel types that require it (e.g., "region", "ld_block").

Parameters:

Name	Type	Description	Default
`panel_type`	`str`	Type of panel ("track", "arc", "ld_block", "region") **kwargs Panel-specific parameters	required

Returns:

Type	Description
`Panel`	Panel object that can be used with plot_panels

Examples:

    >>> mysumstats = gl.Sumstats("data.txt.gz")
    >>> panel1 = mysumstats.Panel("region", region=(1, 1000000, 2000000), vcf_path="ld.vcf.gz")
    >>> panel2 = mysumstats.Panel("track", track_path="genes.gtf", region=(1, 1000000, 2000000))
    >>> gl.plot_panels([panel1, panel2])