Sumstats — Fix

Standardization and per-step QC methods (the fix_* family and related checks).

fix_id

fix_id(**kwargs: Any) -> Sumstats

Fix various aspects of genomic data including SNPID, rsID, chromosome positions, and allele information.

This function performs multiple data quality checks and fixes: (1) validates and fixes SNPID format
(CHR:POS:NEA:EA pattern), (2) extracts and fixes chromosome and position from SNPID or rsID, (3) validates
rsID format (rsxxxxxx pattern), (4) extracts and fixes EA and NEA from SNPID, (5) standardizes separators
and removes prefixes in SNPID, and (6) generates new SNPID from available CHR, POS, EA, NEA data.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats`	Sumstats object containing the data to fix.	required
`fixprefix`	`bool`	Whether to remove 'chr' prefix in SNPID.	required
`fixchrpos`	`bool`	Whether to fix chromosome and position from SNPID.	required
`fixid`	`bool`	Whether to generate new SNPID from available data.	required
`fixeanea`	`bool`	Whether to fix EA and NEA from SNPID.	required
`fixeanea_flip`	`bool`	Whether to flip EA and NEA during fixing.	required
`fixsep`	`bool`	Whether to standardize separators in SNPID.	required
`reversea`	`bool`	Whether to reverse alleles in SNPID.	required
`overwrite`	`bool`	Whether to overwrite existing values.	required
`verbose`	`bool`	Whether to print progress.	required
`forcefixid`	`bool`	Whether to force fix even without status check.	required

Returns:

Type	Description
`DataFrame`	Modified sumstats.data with fixed data. When called via :meth:`Sumstats.fix_id()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

fix_chr

fix_chr(**kwargs: Any) -> Sumstats

Standardize chromosome notation and handle special chromosome cases (X, Y, MT).

All chromosome notations are converted to string type first. After fix, all chromosome
notations will be int. This function normalizes chromosome labels to a consistent format,
extracts chromosome numbers from various formats (e.g., "chr1", "1", "chrX"), maps special
chromosomes (X, Y, mitochondrial) to standardized numeric identifiers, and optionally
removes invalid chromosome values.

Chromosome mappings (x, y, mt), chrom_list, and minchr are automatically derived from
the Sumstats object's chromosomes attribute (Chromosomes instance).

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats`	Sumstats object containing the data to fix.	required
`add_prefix`	`str`	Prefix to prepend to chromosome labels (e.g., "chr").	`""`
`remove`	`bool`	If True, remove records with invalid or unrecognized chromosome labels.	`False`
`verbose`	`bool`	If True, print progress or diagnostic messages.	`False`
`skip_status`	`bool`	If True, skip updating the STATUS column. Useful when STATUS column has issues or when STATUS column is not needed.	`False`

Returns:

Type	Description
`DataFrame`	Summary statistics table with standardized chromosome identifiers. When called via :meth:`Sumstats.fix_chr()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

fix_pos

fix_pos(**kwargs: Any) -> Sumstats

Standardize and validate genomic base-pair positions.

This function checks that reported genomic positions fall within valid chromosomal bounds
and optionally removes invalid entries. It handles string-formatted positions with thousands
separators, converts positions to Int64 type, and filters out positions outside the specified
range. If explicit limits are not provided, a default maximum bound is applied.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats`	Sumstats object containing the data to fix.	required
`remove`	`bool`	If True, remove records with invalid or out-of-range positions.	`False`
`verbose`	`bool`	If True, print progress or diagnostic messages.	`False`
`lower_limit`	`int`	Minimum acceptable genomic position. Default is 0.	required
`upper_limit`	`int`	Maximum acceptable genomic position.	required
`limit`	`int`	Default upper limit applied when `upper_limit` is not provided.	`250000000`
`skip_status`	`bool`	If True, skip updating the STATUS column. Useful when STATUS column has issues or when STATUS column is not needed.	`False`

Returns:

Type	Description
`DataFrame`	Summary statistics with standardized and validated base-pair positions. When called via :meth:`Sumstats.fix_pos()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

fix_allele

fix_allele(**kwargs: Any) -> Sumstats

Validate and standardize allele representations.

This function checks allele fields for valid nucleotide characters (A, T, C, G), converts
all alleles to uppercase, standardizes their format using categorical data types, and
classifies variants as SNPs, indels, normalized, or not normalized. Optionally, rows with
invalid allele values can be removed.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats`	Sumstats object containing the data to fix.	required
`remove`	`bool`	If True, remove variants with invalid allele representations.	`False`
`verbose`	`bool`	If True, print progress or warning messages.	`False`

Returns:

Type	Description
`DataFrame`	Summary statistics table with validated and standardized allele values. When called via :meth:`Sumstats.fix_allele()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

flip_snpid

flip_snpid(**kwargs: Any) -> None

Flip alleles in SNPID values without changing status codes or statistics.

Converts SNPID from CHR:POS:EA:NEA format to CHR:POS:NEA:EA format by swapping
the last two allele components. This function only modifies the SNPID column and
does not affect EA, NEA, STATUS, or any statistics.

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`Sumstats or DataFrame`	Sumstats object or DataFrame to process.	required
`snpid`	`str`	Column name for SNPID.	required
`overwrite`	`bool`	Whether to overwrite existing values.	required
`verbose`	`bool`	Whether to print progress.	required

Returns:

Type	Description
`DataFrame`	Modified sumstats with flipped alleles. When called via :meth:`Sumstats.flip_snpid()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `None`.

strip_snpid

strip_snpid(**kwargs: Any) -> None

Strip non-standard characters from SNPID values to standardize format.

Removes leading and trailing non-standard characters from SNPID values that match
the pattern (xxx:)CHR:POS:ATCG_Allele:ATCG_Allele(:xxx), keeping only the core
CHR:POS:NEA:EA format.

Parameters:

Name	Type	Description	Default
`sumstats_or_dataframe`	`Sumstats or DataFrame`	Sumstats object or DataFrame to process.	required
`overwrite`	`bool`	Whether to overwrite existing values.	required
`verbose`	`bool`	Whether to print progress.	required

Returns:

Type	Description
`DataFrame`	Modified sumstats with stripped SNPIDs. When called via :meth:`Sumstats.strip_snpid()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `None`.

remove_dup

remove_dup(**kwargs: Any) -> Sumstats

Remove duplicate or multiallelic variants based on user-selected criteria.

Supports multiple duplicate-identification strategies depending on variant identifiers
(SNPID, rsID) or allele and coordinate combinations. Can also collapse multi-allelic
sites by retaining a single representative variant. Variants are sorted by a specified
column (e.g., P-value) before removal to ensure the best variant is kept.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats`	Sumstats object containing the data to process.	required
`mode`	`str`	String encoding the deduplication rules; may include one or more of: - 'ds' : Identify duplicates using SNPID. - 'dr' : Identify duplicates using rsID. - 'dc' : Identify duplicates using chromosome, position, effect allele, and non-effect allele. - 'm' : Identify multi-allelic variants (same chromosome + position).	required
`keep`	`(first, last, False)`	Which record to retain when duplicates are detected.	`'first'`
`keep_col`	`str`	Column to sort by prior to duplicate removal; used only when `keep` is not False.	required
`remove_na`	`bool`	If True, remove rows containing missing values in deduplication-relevant columns.	`False`
`keep_ascend`	`bool`	If True, sort in ascending order when determining which duplicate to keep.	`True`
`verbose`	`bool`	If True, print progress and summary information.	`False`

Returns:

Type	Description
`DataFrame`	Summary statistics with duplicates and multi-allelic variants removed according to the specified mode. When called via :meth:`Sumstats.remove_dup()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

check_sanity

check_sanity(**kwargs: Any) -> Sumstats

Check whether numerical summary statistics fall within valid ranges.

This function validates commonly used GWAS fields (sample sizes, allele
frequencies, effect sizes, test statistics, etc.) against expected numeric
ranges. Columns not present in the input are ignored.

Parameters:

Name	Type	Description	Default
`n`	`tuple of (float, float)`	Valid range for sample size (N). Default from qc_researved_header_json: (0, 2147483647).	required
`ncase`	`tuple of (float, float)`	Valid range for number of cases (N_CASE). Default from qc_researved_header_json: (0, 2147483647).	required
`ncontrol`	`tuple of (float, float)`	Valid range for number of controls (N_CONTROL). Default from qc_researved_header_json: (0, 2147483647).	required
`eaf`	`tuple of (float, float)`	Valid range for effect allele frequency (EAF). Default from qc_researved_header_json: (0, 1).	required
`mac`	`tuple of (float, float)`	Valid range for minor allele count (MAC). Default: (0, 2**31-1).	required
`maf`	`tuple of (float, float)`	Valid range for minor allele frequency (MAF). Default from qc_researved_header_json: (0, 0.5).	required
`chisq`	`tuple of (float, float)`	Valid range for chi-square statistics (CHISQ). Default from qc_researved_header_json: (0, inf).	required
`z`	`tuple of (float, float)`	Valid range for z-scores (Z). Default from qc_researved_header_json: (-9999, 9999).	required
`t`	`tuple of (float, float)`	Valid range for t-statistics (T). Default from qc_researved_header_json: (-99999, 99999).	required
`f`	`tuple of (float, float)`	Valid range for F-statistics (F). Default from qc_researved_header_json: (0, inf).	required
`p`	`tuple of (float, float)`	Valid range for p-values (P). Default from qc_researved_header_json: (0, 1).	required
`mlog10p`	`tuple of (float, float)`	Valid range for negative log10 p-values (MLOG10P). Default from qc_researved_header_json: (0, 99999).	required
`beta`	`tuple of (float, float)`	Valid range for effect size estimates (BETA). Default from qc_researved_header_json: (-100, 100).	required
`se`	`tuple of (float, float)`	Valid range for standard errors (SE). Default from qc_researved_header_json: (0, inf).	required
`OR`	`tuple of (float, float)`	Valid range for odds ratios (OR). Default from qc_researved_header_json: (0, 100).	required
`OR_95L`	`tuple of (float, float)`	Valid range for lower bound of 95% CI for OR. Default from qc_researved_header_json: (0, inf).	required
`OR_95U`	`tuple of (float, float)`	Valid range for upper bound of 95% CI for OR. Default from qc_researved_header_json: (0, inf).	required
`HR`	`tuple of (float, float)`	Valid range for hazard ratios (HR). Default from qc_researved_header_json: (0, 100).	required
`HR_95L`	`tuple of (float, float)`	Valid range for lower bound of 95% CI for HR. Default from qc_researved_header_json: (0, inf).	required
`HR_95U`	`tuple of (float, float)`	Valid range for upper bound of 95% CI for HR. Default from qc_researved_header_json: (0, inf).	required
`info`	`tuple of (float, float)`	Valid range for imputation info score (INFO). Default from qc_researved_header_json: (0, 2).	required
`float_tolerance`	`float`	Numerical tolerance applied when comparing floating-point values.	`1e-7`
`verbose`	`bool`	If True, print progress and warnings.	`True`

Returns:

Type	Description
`DataFrame`	Filtered sumstats with out-of-range variants removed.

Notes

Default sanity-check ranges load from ``qc_researved_header_json`` when a
range parameter is ``None``. Override any field by passing an explicit
``(low, high)`` tuple.

When called via :meth:`Sumstats.check_sanity`, updates ``self.data`` in
place and the method returns ``self``.

check_data_consistency

check_data_consistency(**kwargs: Any) -> Sumstats

Check consistency between related statistical values. Minor inconsistencies are likely due to rounding.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats`	Sumstats object containing the data to check.	required
`rtol`	`float`	Relative tolerance for comparisons (fractional).	`1e-05`
`atol`	`float`	Absolute tolerance for comparisons (absolute threshold).	`1e-08`
`equal_nan`	`bool`	If True, treat NaN values as equal during consistency checks.	`False`
`verbose`	`bool`	If True, print progress or warning messages.	`False`

Returns:

Type	Description
`DataFrame`	Summary statistics table including annotations for detected inconsistencies. When called via :meth:`Sumstats.check_data_consistency()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.

check_id

check_id(**kwargs: Any) -> None

Validate variant identifiers (not yet implemented).

normalize_allele

normalize_allele(**kwargs: Any) -> Sumstats

Normalize indels in parallel using left-alignment and parsimony principles.

This function standardizes allele representations for insertion/deletion variants by
left-aligning and trimming shared sequence context. It removes common suffixes and
prefixes from both alleles and adjusts positions accordingly, following the VCF
normalization standard.

Parameters:

Name	Type	Description	Default
`sumstats_obj`	`Sumstats`	Sumstats object containing the data to normalize.	required
`chunk`	`int`	Size of chunks for parallel processing.	`3000000`
`threads`	`int`	Number of threads used for parallel processing.	`1`

Returns:

Type	Description
`DataFrame`	Summary statistics with normalized indel allele representations. When called via :meth:`Sumstats.normalize_allele()`, updates the Sumstats object in place (modifies `self.data`) and the method returns `self`.