Insurance by BogdanovaThe Encyclopedia of Insurance and Risk Science

RESEARCH DOSSIER

Fairness Auditing of Claims Settlement Outcomes

Abstract. Insurance realizes its protective function when covered losses are converted into timely and appropriate compensation or services. Fairness auditing examines whether that conversion is consistent with applicable contractual entitlements, lawful distinctions, procedural obligations, and legitimate differences in loss circumstances across claimant populations. This monograph develops an integrated treatment of ex post claims fairness, addressing notification, investigation, initial offers, denials, payment amounts, waiting times, dispute resolution, and final remedies. It examines the comparability of claims, the construction of independent reference valuations, selection and censoring, conditional outcome contrasts, matched designs, multilevel models, missing demographic information, and the interpretation of causal evidence. Comparative supervisory materials and empirical studies illustrate the distinction between aggregate monitoring and independently validated case errors. Synthetic calculations demonstrate how composition effects can reverse apparent disparities, why payment among accepted claims can conceal unequal access, and how incomplete group data constrain identification. The study concludes that meaningful auditing requires jointly evaluating entitlement, procedural opportunity, settlement adequacy, delay, and response to correction. It is an analytical and methodological synthesis of published evidence, not an audit of a particular insurer or an estimate of global discriminatory conduct.

Keywords: Claims settlement; insurance fairness; market conduct; disparate outcomes; claims analytics; conditional standardization; causal inference; selection bias; claims duration; claims adjudication; redress; algorithmic accountability; privacy; consumer protection.

Introduction

The realization of insurance protection

A promise of compensation is a contingent obligation. Before a loss, the insurer assesses exposure, defines coverage and prices the transfer of risk. After a loss, that obligation becomes a sequence of factual, legal, operational and financial determinations. A claimant must gain access to the process, establish the relevant facts, receive a reasoned assessment and, where entitlement exists, obtain a payment or service. Fairness at this stage concerns the distribution of realized protection and the conditions under which the obligation is fulfilled.

A technically correct premium classification cannot establish that later claims decisions are fair. An insurer can price a portfolio consistently and still allocate adjuster time, requests for documentation or negotiated offers in ways that systematically disadvantage particular claimants. A separately justified pricing difference may also affect the composition of the audit population. Claims fairness therefore has its own scientific object: variation in the administration and fulfillment of claims conditional on contractual and factual circumstances.

The governing question is how a claim journey would differ under comparable circumstances across groups, and whether the differences that remain can be justified. Three judgments are required: a descriptive statement about observed outcomes, a methodological statement about the comparability and uncertainty of those outcomes, and a normative or legal statement about their permissibility. Each has a distinct evidentiary basis. A statistical disparity is an investigative signal. A verified departure from entitlement is a case-level finding. An adjudicated violation requires the governing legal standard and relevant evidence.

Method and scope of inquiry

The study is a structured narrative synthesis of insurance supervisory standards, public regulatory investigations, published social-science findings, and statistical literature. The sources were selected for their relevance to the claims process, fairness definitions, identification, missingness, and remedies. The research used official regulator and author/publisher materials accessible on November 19, 2026. Full-text access varied: claims taken from abstracts or official executive summaries are restricted to the findings explicitly available there. No systematic-review coverage, database meta-analysis or primary claimant-level empirical study is asserted.

The comparison concerns first-party property, motor, life, disability and health claims, with third-party liability discussed where negotiation and entitlement differ. The same analytic questions travel across lines, but the substantive entitlement and measurement units change. The study distinguishes indemnity payments from fixed benefits, provider reimbursement from beneficiary access, and health prior authorization from adjudication of incurred claims. Synthetic cases below are expressly invented to display mathematical implications. They contain no personal information and do not describe any existing insurer.

Fairness Concepts in Claim Payment

Contractual adequacy and procedural treatment

Contractual fairness concerns the fidelity of the outcome to the applicable agreement and mandatory law. In indemnity insurance the reference amount depends on covered damage, valuation provisions, deductible, limits, exclusions, prior payments, applicable taxes and other lawful adjustments. In fixed-benefit insurance it depends on the insured event, eligibility and scheduled benefit. In liability insurance the settlement value additionally depends on contested liability, damages and legal costs. The claims auditor must establish the relevant payment obligation or a defensible range of settlement values before comparing performance.

Procedural fairness concerns how people reach an outcome: access to information, language and disability accommodations, opportunities to submit evidence, impartial assessment, explained decisions and effective review. A correctly priced payment received after an avoidable delay may still constitute an inferior service outcome. A claimant who agrees to a discount under financial pressure may receive less than a comparably situated claimant who can wait. Equal averages can therefore coexist with different experiences or losses concentrated in vulnerable cases.

The IAIS (2024) sets out claims handling and dispute resolution in ICP 19.10, with attention to documentation, communication, explanations, competence and outsourcing oversight. The standard also treats complaints as a related but separate domain. It supplies a supervisory benchmark for the audit design, while national implementation establishes the enforceable duties in particular markets.

Distinct meanings of equality

Unconditional equality of denial rates asks whether the overall proportion of adverse decisions differs across groups. Conditional equality asks whether the proportions differ after both groups are compared on relevant characteristics. Error-rate equality compares outcomes with an independently established reference decision. Equal procedural opportunity asks whether comparable individuals can access and navigate the same evidence and correction process. None of these metrics alone captures every relevant dimension.

Mitchell et al. (2021) examine the normative and modeling choices underlying algorithmic fairness definitions. The claims application requires the auditor to distinguish a contract-relevant feature from a feature merely associated with the historical action. A claimant's ability to negotiate may predict settlement amount while having no legitimate role in the valuation of covered damage. Predictive usefulness cannot itself authorize a settlement distinction.

Hardt et al. (2016) formulate equality of opportunity for supervised learning using sensitive group membership, a target and a prediction. In claims, an analogous measurement is the chance that a genuinely eligible claim is recognized, independently of claimant group. This adaptation requires a reliable eligibility reference. Chouldechova (2017) demonstrates incompatibility among fairness criteria under unequal outcome prevalence in a different application. The mathematical warning travels across domains; the empirical recidivism findings do not. An insurance audit must state which metrics have priority and why.

The reference decision

Let A denote claimant group, X predecision claim characteristics, D the insurer's adverse decision, and E the independently assessed entitlement indicator. D equals one for an adverse coverage decision and E equals one where entitlement is established under the designated review rule. The false-denial rate is P(D = 1 | E = 1, A = a). The ordinary denial rate is P(D = 1 | A = a). An observed denial is an action; a false denial is an action assessed against evidence of entitlement.

For settlements rather than determinate benefits, independence can require two reviewers, documented market estimates and an adjudication procedure for disagreement. Denote the supportable interval by [L, U]. A payment below L triggers examination of possible shortfall; a payment within the range remains compatible with the reference assumptions. The interval must reflect genuine factual and legal ambiguity, rather than be widened to rationalize historical payments. The same principle applies to overpayment and potential fraud: a fair process protects the pool against unsupported claims and protects entitled claimants against erroneous rejection.

Comparable Claims and Legitimate Covariates

Constructing the case population

Begin with an inception cohort of notifications during a defined time interval. The data lineage should preserve policy, insured event, claim, coverage component, claimant and payment identifiers. Multiple claim lines generated by one encounter or one catastrophe event must not be mistaken for independent people. One person may hold several policies; one household may submit several related claims. A reopened claim must remain connected to its original history. Analysts should retain files that were abandoned, denied, withdrawn, unresolved, escalated or settled and classify each transition separately.

The scope should identify the risk population: insured exposure, potentially covered loss, notified claim, investigated claim, eligible claim, approved claim or paid claim. Different denominators answer different questions. Denial among notified claims concerns adjudication conditional on reporting. To estimate access to claims among insured losses, evidence about unreported covered events is necessary. That evidence may require independent event data, surveys or linked records, subject to lawful availability. An inception cohort still cannot represent people excluded from insurance before notification.

A lawful comparison set

Candidate covariates include policy form and effective date, insured peril, covered object, verified damage, limits, deductible, currency, location-specific repair inputs, independently documented liability circumstances, medical eligibility, and loss-date market conditions. Each requires a rationale tied to entitlement or genuine complexity. Geography may capture local labor prices but also segregated access and unequal service capacity. Adjusting for location can remove real variation in the allocation of adjusters. A robust analysis reports estimates with different justified adjustment sets and explains the substantive question for each.

A useful register separates four classes of variables. Baseline entitlement features establish the claim standard. Preexisting contextual variables capture case complexity or environmental conditions. Process-generated features describe what the insurer or claimant did after notification. Sensitive information supports monitoring where lawful. These categories are defined for a specific estimand: the time order of a variable alone does not make it appropriate.

For instance, an insurer's fraud referral is generated by its investigation and can be influenced by a biased screening rule. Treating referral as a legitimate baseline risk adjustment could subtract the very disparity under investigation. An independently validated discrepancy in evidence, available at the relevant decision time, has a different epistemic status. Likewise, number of documents submitted may reflect access to assistance or repetitive requests rather than inherent complexity. Distinguish information available before the decision from information created by the decision.

Case comparability and support

Exact matching is suitable for a small set of decisive features, such as policy form, peril and entitlement category. Coarsened strata can group continuous loss sizes or valuation dates. Propensity-score or distance matching can balance additional baseline characteristics, but neither guarantees balance on omitted features. For each match design report sample attrition, standardized covariate differences, overlap plots or tabulations, and the target population retained. Where one group has no comparable cases, the contrast is unsupported there. The resulting common-support estimand should be labeled accordingly and the excluded population quantified.

An audit of matched case pairs should distinguish differences in initial eligibility assessment from differences in bargaining. Two homes with the same covered repair cost may have different net entitlement solely because deductibles differ. Matching on gross physical damage while ignoring policy terms would overstate a disparity. Conversely, two claims with the same first offer may require different corrections because one has a higher independently assessed entitlement.

Data lineage and case states

A minimum audit extract records the policy version, notification timestamp, cause of loss, independently sourced valuation inputs, initial assignment and routing, evidence requests and receipt dates, decision dates and codes, offer history, payment transactions, reopenings, complaints and all review outcomes. A linked process history should record the model and rules version, vendor, override and justification, and whether automated recommendations were visible to the decision maker. Each monetary field must specify gross or net status, currency and valuation date. Dates require a time-zone convention and calendar versus business-day definition.

Preserve original values alongside corrected values. A reason code subsequently edited after an appeal cannot replace the reason that was initially communicated. Distinguish a zero amount, an unknown amount and an amount that cannot yet be determined. Record system migrations, backfills and changes in product definitions as potential measurement discontinuities. These safeguards enable replay: an auditor should reconstruct the information available at a specified decision time without accidentally granting the original decision maker knowledge of later events.

Measurement of Denial, Amount and Delay Disparities

Denial, abandonment and final adjudication

For group g with n_g notified claims and d_g denials by a specified cutoff, the observed risk is d_g/n_g. The absolute difference p_A - p_B expresses excess denials per claim; the risk ratio p_A/p_B expresses relative frequency when the denominator is nonzero. Both should be reported with counts, uncertainty and outcome definitions. Administrative closure, duplicate submissions, partial denials and coverage denials must be distinguishable. Where claims remain pending, a simple observed proportion at cutoff is not an ultimate denial probability.

Public reporting illustrates the importance of these definitions. CMS (2026) publishes Transparency in Coverage data at plan and issuer levels. Long, Lo and Pestaina (2026), in their Marketplace denial and appeal analysis, explain that reported final adjudication does not count an initial denial later paid as a final denial. They also describe reporting coverage limitations arising from returning issuers and plans. These aggregate sources can flag variation across insurers and products but cannot establish individual group disparity or correctly identify all initially adverse experiences.

Amount adequacy

Let B_i be an independently established payable amount for a determinate covered claim after legitimate adjustments, and P_i the cumulative net benefit paid at a stated horizon. For B_i > 0, the adequacy ratio is P_i/B_i and the shortfall is max(B_i - P_i, 0). Cases with B_i = 0 require separate eligibility and overpayment analysis. Avoid silently replacing a negative net payment or reversal with zero. Where B_i is an interval, analyze lower-bound shortfall max(L_i - P_i, 0) and report uncertainty in valuation rather than treating an estimate as immutable.

Two natural aggregate summaries are the mean of claim-level ratios and the ratio of total payments to total reference entitlements. The former weights each eligible claim equally; the latter weights by entitlement size. A few high-value commercial losses can dominate the total ratio. A distribution of ratios, median shortfall, tail shortfall and the share receiving less than a predetermined reference tolerance reveal harms hidden by the mean. The tolerance is a review trigger, not a legal permission to underpay.

Consider two synthetic groups, each with 100 claims independently assessed as eligible for $1,000 each. In group A, 80 are approved and receive $950 each; in B, 95 are approved and receive $900 each. Among approved claims A appears more adequately compensated: 95% versus 90%. Across all eligible claims at the horizon, A receives $76,000 of $100,000, or 76%, while B receives $85,500, or 85.5%. Their shortfalls are $24,000 and $14,500. The higher conditional payment adequacy in A therefore coexists with a 9.5-percentage-point lower overall realization of entitlement. These illustrative numbers assume no pending cases, partial payments or later remedies. Their purpose is to demonstrate the selection introduced by conditioning on approval.

Time and competing events

Claims time has several origins: loss date, notification, complete evidence, coverage decision, acceptance of offer, first payment, and final settlement. Each measures a different burden. Report the end-to-end time and appropriate stage-specific waiting times. A clock that starts only once the insurer labels the evidence complete excludes time spent obtaining repeated documentation and can mask process disparities. Likewise, time to closure is a poor substitute for time to meaningful compensation.

With right-censored files, estimate the probability of remaining unresolved at time t using survival methods under their assumptions. Treat denial, withdrawal and payment as competing outcomes when they preclude the specified first event. A Kaplan–Meier curve for payment that treats denial as ordinary independent censoring estimates a hypothetical net probability, not necessarily the observed cumulative incidence of payment. The Aalen–Johansen competing-risk estimator is appropriate for actual transition probabilities under the required observation assumptions. Reopenings can be represented through a multistate model rather than forced into a single absorbing closure category.

A practical horizon measure is the restricted mean time unpaid through tau: the integral over time from zero to tau of the probability that first payment has not occurred. That event definition keeps permanent nonpayment in the unpaid state until tau, which is appropriate when the estimand concerns realized access to payment, not the duration of administrative work. If entitlement has not been established independently, report eligible and all-notified versions separately. Compare quantiles only where the relevant cumulative incidence reaches those quantiles.

Service quality, offers and remedies

Record the offer/reference ratio at each negotiation stage, the number and nature of evidence requests, the completeness and accessibility of explanations, time to contact, and whether an independent review route was disclosed. An offer is not a final payment. A withdrawn claim must be investigated for reason and timing; a legitimate withdrawal, an inaccessible process and an unsupported claim have different meanings.

Complaints per thousand notifications measure recorded dissatisfaction, not all experienced harm. Appeals per denial depend on awareness, time, representation and accessibility. Reversal among appellants measures a selected population. To estimate wrongful denials per total claim, sample both appealed and unappealed cases and weight according to the design. Record remedy amount and elapsed time after the original error, including legally applicable interest or cost reimbursement. A corrected payment does not erase the earlier burden.

Empirical contexts

Baker and McElrath (1996) report an ethnic difference in payment timing after Hurricane Andrew in Whose Safety Net?. The publisher abstract supports the existence of the reported historical finding but does not support a current prevalence estimate or detailed reproduction. This historical case motivates attention to delay as a distributive outcome, particularly when access to liquidity differs.

ASIC's (2023) home insurance claims review documents variation in service quality and identifies communication, project management, vulnerability recognition, complaints and staffing as improvement areas. The research combines operational review with consumer experience in an Australian home insurance setting. It establishes the usefulness of examining process and outcome together; it is not an estimate of protected-group discrimination.

The HHS OIG (2022) Medicare Advantage denial investigation used separate stratified samples of 250 authorization denials and 250 payment denials from 15 major organizations during a week in 2019. Independent expert review found that some rejected requests met relevant coverage and billing requirements. The two samples distinguish prospective access to care from reimbursement. The report supports case-review validation as a method, not the conclusion that demographic discrimination caused those errors.

Causal and Statistical Auditing

Estimands and standardization

Set the adverse outcome to one for denial and zero otherwise within an explicitly defined decision horizon and cohort. The raw group difference is the contrast of the two observed group means. Let m_g(x) denote the conditional probability of denial for group g at baseline characteristics x. A standardized risk difference over a prespecified target distribution w(x) is the sum or integral of [m_A(x) - m_B(x)] weighted by w(x). This is a conditional descriptive disparity unless further causal assumptions hold. Its interpretation depends on which variables were included, which histories were represented, and whether both conditional risks are estimable at each x.

Consider a synthetic portfolio with two independently verified complexity categories. In group A, 200 simple claims have a 5% denial rate and 800 complex claims have a 20% rate. In B, 800 simple claims have a 7% rate and 200 complex claims have a 22% rate. A has 170 denials among 1,000 claims, or 17%. B has 100 among 1,000, or 10%. Under equal weights for simple and complex claims, A has 12.5% standardized denial and B 14.5%. The raw A-minus-B contrast is +7 percentage points, the standardized contrast is -2 percentage points.

The reversal does not establish that A is favored or B is treated unlawfully. It demonstrates that radically different case composition can drive the aggregate direction. If complexity were defined using insurer-generated investigation length, the adjustment would instead be suspect. The example makes the choice of reference weights and legitimation of strata central to interpretation.

Decomposition and uncertainty

For discrete strata, let p_g(x) represent the within-stratum adverse fraction and q_g(x) the group-specific stratum share. The raw gap can be decomposed exactly as the sum of q_A(x)[p_A(x) - p_B(x)] plus the sum of [q_A(x) - q_B(x)]p_B(x). The first term is a within-stratum contrast evaluated at A's case mix; the second is a composition contrast evaluated at B's rates. Using B's distribution and A's rates yields a different allocation of the interaction. A decomposition is therefore a descriptive accounting identity, not an automatic causal allocation of responsibility.

For the previous example, the within-stratum term is -0.02. The composition term is (-0.6)(0.07) + (0.6)(0.22) = 0.09. Their sum is 0.07. An audit should report both reference choices or a symmetric standardization when appropriate; neither transforms illegitimate differences into permissible factors.

Uncertainty follows the sampling design and dependence structure. The familiar binomial standard error sqrt[p(1-p)/n] presumes independent Bernoulli observations. Multiple payments from a claim, claims from one event, and decisions from one adjuster violate that assumption. Stratified survey estimators, cluster-robust inference or a resampling scheme at the appropriate independent unit are preferable as justified by the design. Small cluster counts can make conventional cluster-robust intervals unreliable. Report confidence intervals, sample and cluster counts, effective sample size and sensitivity to influential observations.

A large portfolio can make a negligible disparity statistically detectable; a small group can exhibit economically serious harm while its interval includes zero. Choose a materiality standard grounded in the outcome, consumer impact and governing duties, with urgent review of verified individual errors regardless of group significance. Prespecify primary contrasts and use a multiplicity strategy for exploratory scans. For example, false-discovery-rate procedures may help allocate investigative effort across many outcomes, but dependence and the possibility of severe isolated harm require judgment beyond a single adjusted p-value.

Regression, hierarchy and calibration

A logistic regression of denial on group and legitimate characteristics estimates conditional log-odds contrasts under its specification. Report standardized predicted probabilities and risk differences in addition to odds ratios; odds ratios do not equal risk ratios and can differ between marginal and conditional models even without confounding. Flexible severity functions and interactions may be necessary. Validate fit on later periods or held-out event clusters rather than random claim lines that leak shared events between samples.

Hierarchical models can represent adjuster, vendor, geography, product and time effects. Partial pooling stabilizes noisy rates for sparse groups, but shrinkage can hide concentrated harm. Display raw counts and model-based estimates side by side. Adjuster assignment may be part of the mechanism causing disparity; conditioning on adjuster answers a within-adjuster question and can remove disparities caused by routing. Estimate both questions when possible. Random effects require a defensible distribution and relation to predictors; fixed-effect specifications may be better for some comparisons. A model's predictive calibration against past paid amounts validates prediction of historical actions, not the correctness of those actions.

For reference valuation, separate model error from human processing error. The difference between model-predicted entitlement and independently reviewed entitlement diagnoses valuation performance. The difference between final payment and the same reference diagnoses the realized outcome. The recommendation-to-decision gap diagnoses the use of the model. Each can differ by group even when the others show little disparity.

Independent case review and selective labels

Historical decisions create selective information. If the insurer investigates high-scoring fraud referrals thoroughly but rarely investigates low-scoring referrals, confirmed fraud labels depend on the referral. If only appellants receive a detailed independent entitlement review, wrongfulness is observed selectively. The general selective labels problem studied by Lakkaraju et al. (2017) illustrates the problem of evaluating decision systems where actions determine which outcomes can be observed. The original work is not a claims dataset; its logic motivates independent review of unselected claims.

A defensible audit samples accepted, denied, withdrawn, pending and appealed files from an identified frame. Oversampling sparse groups and adverse outcomes may increase information about harms, provided inclusion probabilities are stored and weights restore the intended population. Reviewers should receive the relevant policy and contemporaneous factual evidence, and be blinded to protected group and insurer action to the extent compatible with the review task. Some facts may reveal identity or vulnerability; document what could not be blinded. Distinguish cases with sufficient evidence, cases requiring further evidence, and genuinely disputed entitlements. Record reviewer agreement, reconciliation and changes to the reference rule. No reviewer is assumed infallible.

If cases are sampled with probability pi_i, a design-based estimate of a total number of reference errors is the sum over the sample of the error indicator divided by pi_i. A rate can use a known population denominator or a corresponding estimated denominator. The same design must govern interval estimation. In a weighted review, a large weight attached to one adverse case can exert disproportionate influence; report weight ranges and sensitivity rather than deleting the case solely because it changes the result.

Causal questions and intervention design

The causal object is generally the effect of a manipulable practice: automatic referral, additional documentation, queue assignment, vendor selection, language support or independent review. A group attribute may identify a disparity, but the interpretation of an intervention on identity is not automatically meaningful. The standard potential-outcome approach discussed in the methodological literature, including Hernán and Robins' Causal Inference: What If (2020), distinguishes defined interventions from observed associations. A policy-relevant question might be whether removing a redundant evidence request would reduce time to settlement, and whether this improvement differs across groups.

For a binary practice Z and outcome Y, write Y(1) and Y(0) for the outcomes under the two well-defined practice regimes. Estimating E[Y(1)-Y(0) | A = g] from nonrandom assignment requires consistency, conditional exchangeability and positivity for the specified regime and target group. Interference can arise when faster service for one claim consumes shared staff capacity and delays another. Treat capacity allocation as a system intervention or define clusters accordingly. A before-after model change without a comparison group can confound remediation with seasonality, staff changes or event severity.

Randomly allocate a beneficial review or improved communication intervention only when genuine uncertainty, lawful practice and consumer protections permit it. Never withhold an established entitlement or mandated service to construct a control. Cluster assignment may be needed for branch-level training or queue changes. A stepped rollout can support estimation if timings are defensible and temporal confounding is analyzed. A difference-in-differences study of a rule change needs a credible parallel-trend argument, stable composition or explicit adjustments, and attention to anticipation and staggered treatment. Its interpretation concerns the implemented change, not a blanket finding about identity.

Missing group information and partial identification

Group membership may be absent for privacy reasons or because records were collected for a different purpose. Missingness can depend on claimant engagement, channel, location, adverse outcome or reluctance to disclose. Complete-case estimates require assumptions about selection. Multiple imputation requires a specified missingness model and uncertainty propagation; an imputed group is not a verified identity.

Imai and Khanna (2016) describe Bayesian ethnicity prediction from surnames and geographic data in voter records with validation against self-reported ethnicity. That population-specific method illustrates possibilities and limits of proxy classification. Transfer to insurance requires separate lawful authority, an appropriate validation sample and tests for differential error. Hard assignment to the most probable group can distort smaller groups and reproduce geographic disparities. Even probabilistic aggregation requires assumptions about how observed outcomes relate to unobserved group within proxy strata.

McCartan et al. (2025) examine disparity estimation with unobserved race and identify a limitation of conventional surname-geography imputation: prediction errors may correlate with the outcome being audited. Their alternative identification strategy requires conditional independence of surname and outcome given latent race, residence and other observed characteristics. For claims auditing, that requirement is substantive: a surname may affect interactions with claim staff, or be associated with language access even within the same group and area. Where this assumption is not defensible, an audit should report sensitivity and partial identification rather than presenting the imputed disparity as established.

Consider an illustrative eligible-claim cohort in which recorded group A has 80 known claims with 16 denials and B has 120 with 12 denials. Suppose 20 additional claims have missing group, including 8 denials and 12 approvals. Assume only A or B is possible. The highest A-minus-B denial gap is obtained by assigning all eight missing denials to A and all twelve missing approvals to B: 24/88 minus 12/132 = 18.18 percentage points. The lowest gap assigns all twelve approvals to A and all eight denials to B: 16/92 minus 20/128 = 1.77 percentage points. The complete-case gap is 10 points. Under these assumptions the sign remains positive while the magnitude is uncertain. With other group possibilities, different constraints, uncertain outcomes or sampling error, these bounds change. The calculation uses observed counts and explicit allocation assumptions, not inferred personal identities.

Sensitivity and alternative explanations

A serious audit challenges its preferred interpretation. Evaluate changes in coverage versions, loss severity, repair-market prices, catastrophe exposure, model revisions, staffing, vendor shifts and claims maturity. Repeat estimation with direct entitlement features alone, then with separately justified context; compare this with a process-conditioned contrast clearly labeled as such. Examine whether a pattern persists across time, geographies and data capture systems. Validate the direction of discrepancies in a blinded review and retain contradictory cases.

For unmeasured confounding, report how a hypothetical omitted feature associated with the practice and outcome would have to vary to alter a causal conclusion. Avoid attaching scientific certainty to one sensitivity parameter chosen for convenience. Where claims remain unresolved, conduct horizon and censoring sensitivity rather than imputing a final adverse state automatically. Where reference valuation is uncertain, vary its assumptions within approved bounds. Treat the resulting range as information about identification, not an opportunity to choose the most favorable estimate.

Privacy, Evidence and Jurisdiction-Specific Rights

Comparative supervisory architecture

Insurance supervision combines market conduct, contract law, equality law, privacy and operational governance. These sources differ in legal authority. IAIS standards are internationally oriented supervisory benchmarks; national legislation, regulator rules, contracts and enforceable decisions determine duties for a particular claim. An audit mandate should map the jurisdiction of the risk, insurer, insured event, claimant and applicable policy law, and record effective dates of the instruments used.

In the United Kingdom, the FCA's ICOBS 8.1 claims rules address timely and fair handling, claimant guidance, rejection and settlement following agreement. Audit tests can examine unexplained refusals, elapsed time and the availability of guidance, while accounting for the rule's scope, exceptions and the applicable insurance law.

In the United States, the NAIC's (1997) Unfair Claims Settlement Practices Act is a model instrument. Its Sections 3 and 4 distinguish specified problematic conduct, including insufficient investigation and inadequate good-faith settlement, from the conditions under which the model prohibits the practice. Actual state enactments may vary, and a model instrument does not establish a private cause of action or identical deadlines across states. Statutory discrimination rules, protected classifications and exemptions likewise vary by line and jurisdiction.

In Australia, ASIC's review documents the regulatory setting following the 2022 treatment of claims handling and settling as a regulated financial service. Its findings concern operational fulfillment in the domestic setting. Comparable audits should identify the local rules on conduct and vulnerable consumers before attempting cross-country measurement. A common international analytic language is compatible with locally differentiated legal judgments.

Automated decision systems

The NAIC's (2023) model AI bulletin addresses insurer use of AI in consumer-impacting decisions, including claims, in relation to applicable law and governance expectations. State adoption and implementation must be verified before treating the bulletin as binding. EIOPA's (2025) AI governance opinion addresses national supervisors and adopts a proportionate approach; its fairness discussion includes outcome monitoring, auditing where appropriate and access to redress. These materials inform review of claims systems while retaining the distinction between regulation, guidance and technical standards.

A claims AI inventory should include image-based estimates, document classifiers, fraud flags, triage systems, automated coverage rules, settlement recommendations and communication tools. Audit the whole deployment: inputs, features, model version, threshold, advice shown, decision taken, human override and feedback into retraining. A language tool that writes an apparently persuasive denial without evaluating entitlement can affect the practical ability to contest the decision. A fraud model with equal classification errors may still impose unequal burdens through different investigation queues. Vendor models and human decisions must therefore be evaluated as one connected process while retaining the ability to assign responsibility to individual components.

Data protection and access

Protected characteristics and health information can be subject to heightened restrictions. The EU's General Data Protection Regulation (2016) provides an illustrative legal context for special-category processing and certain solely automated decisions with legal or similarly significant effects. Access, lawful basis, special-category conditions, decision safeguards and exceptions require case-specific legal assessment. An audit objective alone is not blanket authorization to collect or infer sensitive attributes; applicable rules and lawful grounds must be established before processing.

Purpose limitation can be made operational through separation of auditing datasets from ordinary claims workflow. Use pseudonymous linkage identifiers, minimal necessary fields, role-based access, logging, retention limits and controlled access for independent review. Pseudonymization reduces disclosure risk but does not make linkable records anonymous. Public reporting should suppress or combine small cells where disclosure risk is material, with attention to differencing attacks through repeated tables. Suppression also affects inference and should be documented. Avoid introducing sensitive attributes collected for auditing into production claim decisions unless a distinct lawful and substantively justified purpose permits it.

Evidence and the limits of a fairness finding

A reproducible audit retains a frozen cohort definition, data dictionary, transformation logic, versioned rules, analysis code, reference-review instructions, inclusion probabilities, model diagnostics, and an issue-resolution record. Sensitive case files remain in the controlled environment; aggregate outputs can be shared at a level compatible with confidentiality. A record of which facts were available to the insurer at each decision time enables due process for both claimant and insurer.

Separate a signal requiring investigation, a verified deviation from the reference standard, substantiated process deficiencies, and a legal determination. An adjusted residual may reflect omitted legitimate circumstances; conversely, an apparent absence of residual disparity may result from adjusting for downstream consequences of unequal treatment. This dual possibility explains why independent review and transparent model specification are essential. A dashboard alone cannot establish individual responsibility or lawfulness. Evidence should support a proportionate response without postponing urgent protection where independent review confirms harm.

Remediation and Outcome Monitoring

From discrepancy to correction

Begin with a decision-level trace. Identify where two comparable claims diverged: intake, triage, documentation, evaluation, negotiation, communication or review. Verify that the observed divergence is reproducible and identify the specific rule, workflow, staffing condition or vendor action. Classify whether the problem is a data error, an incorrect coverage interpretation, an unjustified input, a model-performance issue, an inconsistent human practice or an access barrier. Multiple causes may coexist. Correct the error mechanism, not the group statistic in isolation.

Remediation for a determinate underpayment includes reassessment, payment of the difference and any further remedy required by law or contract. For poorly documented settlements, reconstruct contemporaneous evidence and allow claimants to provide additional facts. Where a defective rule affected a wider population, identify the complete affected cohort from version history and routing logs, rather than remedying only the audit sample or consumers who complained. Contact arrangements must accommodate accessibility and avoid pressuring claimants into unfavorable waiver terms.

Operational corrections may include revised evidence requirements, independent valuation protocols, improved translation, capacity planning, vendor oversight, clearer adverse-decision notices and better access to review. A model correction can involve feature removal, improved reference data, threshold redesign or limits on automation. Changing a threshold to reach equal group denial rates without independently checking entitlement could introduce new wrongful denials or approvals. The correction objective is the quality and lawful consistency of decisions and procedures.

Governance and accountability

Assign an accountable claims executive, legal and privacy reviewers, an independent audit function, and owners of each operational or model correction. Keep the people who designed the original system from exclusively validating their own remediation. The governing body should receive the material findings and uncertainty, affected population, correction plan, resources and completion evidence. Auditors need authority to access vendor documentation, replay decisions and escalate concerns. Where mandatory regulatory notice or reporting applies, follow the applicable requirements.

A monitoring dashboard should report counts and denominators alongside absolute and relative group contrasts, reference error rates, payment adequacy distribution, time-to-payment incidence, complaints, appeals and net remedies. Display confidence intervals or other uncertainty summaries, sample coverage and changes in assessment methods. Portfolio-level monitoring can be accompanied by product, event and intersectional strata, but very sparse or noncomparable cells should be identified. The dashboard must support drill-down in a controlled setting and maintain a separate narrative record of cases requiring immediate correction.

Post-remediation assessment

Define the intervention date, version, affected scope and outcome horizons before evaluating the correction. Compare pre- and postintervention cohorts with compatible follow-up and product mix. Where a credible comparison exists, incorporate it and specify its assumptions. Monitor both average performance and tail outcomes: a faster median can coexist with a growing number of long-unresolved claims. Analyze whether a reduction in denials is accompanied by increased withdrawals, partial payments or documentation burden. Such substitution can create apparent improvement in one metric while leaving protection unchanged.

The correction is complete only when implementation has been independently verified, individual remedies have been tracked to fulfillment, and repeated monitoring shows that the identified mechanism has been addressed within defined uncertainty. An unresolved case can still require separate treatment even when the average indicator returns to a target range. If evidence indicates that a correction shifted harm elsewhere, reopen the issue. Maintain the link between the original finding, corrected decisions and subsequent measurements.

Reproducible audit specification

An audit protocol should be established before examining outcomes. At a minimum it defines the population and reference date; the product and law applicable to each cohort; the decision stages and estimands; lawful group-data collection and missingness strategy; predecision covariate justification; sampling and independent review; statistical method and uncertainty; escalation, correction and monitoring. A preregistered primary analysis prevents post hoc covariate choices from being driven by the desired direction. Exploratory tests remain useful when clearly labeled and replicated.

For a practical sequence, first reconcile case counts against transaction and policy systems. Second, independently review entitlement in a designed sample and verify any reference standard. Third, estimate descriptive group contrasts and their uncertainty for the full case journey. Fourth, compare standardized populations and report excluded support. Fifth, examine selection, pending cases, missing groups, alternative denominators and relevant mechanisms. Sixth, investigate materially adverse differences in the original files and validate correction plans. Seventh, monitor new cohorts and complete individual remedies. This sequence organizes established methods into a claims-specific workflow; it does not assert a new scientific theory.

Scientific synthesis and external validity

The evidence surveyed converges on the importance of independently assessed entitlement, the inclusion of delays and procedures, and the recognition that recorded administrative outcomes are products of institutional decisions. The historic disaster study indicates why settlement timing matters. The Australian supervisory review illustrates operational and consumer-experience evidence. The U.S. health oversight investigation shows why independent case review can reveal incorrect payment or authorization decisions. Public Marketplace reporting demonstrates denominator and adjudication limitations. Their populations, periods, products and research purposes differ; combining them into one estimated discrimination rate would be scientifically invalid.

The author's analytical contribution is to articulate how the same audit population connects the probability of recognition, the adequacy of compensation, elapsed time and remedial access. A claim closed in the insurer ledger may remain unresolved in terms of economic protection. A single favorable conditional metric can coexist with inferior unconditional realization. The numerical examples establish these claims by elementary accounting and standardization, not by proprietary observations. The synthesis identifies a research agenda for primary data: compare independently eligible claims across equivalent obligations, using longitudinal records and a lawful group-measurement protocol, and evaluate changes to specific practices rather than drawing causal conclusions from demographic coefficients alone.

Conclusion

Claims fairness concerns the actual realization of insurance protection across cases and claimant populations. The appropriate unit of investigation is the complete claim journey, interpreted under its contractual and jurisdictional conditions. Contractual adequacy, quality of procedure, decision errors, waiting time, and effective remedy each reveal a distinct dimension of realized protection.

A scientifically defensible audit requires a transparent cohort, an independent reference standard, comparison on explicitly justified characteristics, appropriate treatment of unresolved and selectively observed outcomes, and accurate representation of uncertainty. It distinguishes conditional statistical patterns from causal findings about defined practices and from legal determinations. These distinctions preserve the legitimacy of genuine differences in coverage and loss while keeping the investigation open to unfair processes and access barriers.

Outcome monitoring must culminate in correction and verified fulfillment. An insurer that recognizes a defect in a decision rule must examine its population-wide effects and the individual remedies owed. A fair settlement process can be evaluated through consistent evidence, independently reviewable decisions and the protection actually delivered. The present study provides a methodological synthesis for that evaluation. Its empirical references establish the existence of relevant audit and measurement problems in specific settings; new estimates of discriminatory conduct require appropriately authorized primary data and validated designs.

References

  • Australian Securities and Investments Commission (ASIC). (2023). Navigating the Storm: ASIC’s Review of Home Insurance Claims. Report 768, August 16. Report 768

  • Baker, T., & McElrath, K. (1996). Whose Safety Net? Home Insurance and Inequality. Law & Social Inquiry, 21(2), 229–264. DOI: 10.1111/j.1747-4469.1996.tb00080.x. Whose Safety Net?

  • Centers for Medicare & Medicaid Services (CMS). (2026). Health Insurance Exchange Public Use Files (Exchange PUFs), Transparency in Coverage PUF. Exchange public use files

  • Chouldechova, A. (2017). Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data, 5(2), 153–163. Fair Prediction with Disparate Impact

  • European Insurance and Occupational Pensions Authority (EIOPA). (2025). Opinion on Artificial Intelligence Governance and Risk Management. August 6. AI governance opinion

  • European Parliament & Council of the European Union. (2016). Regulation (EU) 2016/679 (General Data Protection Regulation). Official Journal L 119, 4 May, 1–88. GDPR consolidated text

  • Financial Conduct Authority (FCA). (n.d.). FCA Handbook, ICOBS 8.1, Insurers: general. ICOBS 8.1

  • Hardt, M., Price, E., & Srebro, N. (2016). Equality of Opportunity in Supervised Learning. Advances in Neural Information Processing Systems, 29. Equality of Opportunity

  • Hernán, M. A., & Robins, J. M. (2020). Causal Inference: What If. Boca Raton: Chapman & Hall/CRC. Causal Inference: What If

  • Imai, K., & Khanna, K. (2016). Improving Ecological Inference by Predicting Individual Ethnicity from Voter Registration Records. Political Analysis, 24(2), 263–272. DOI: 10.1093/pan/mpw001. Improving Ecological Inference

  • International Association of Insurance Supervisors (IAIS). (2024, as maintained online). Insurance Core Principles and ComFrame, ICP 19, Conduct of Business, standards 19.10–19.11. ICP 19

  • Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017). The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables. Proceedings of KDD ’17, 275–284. DOI: 10.1145/3097983.3098066. Selective Labels Problem

  • Long, M., Lo, J., & Pestaina, K. (2026). Claims Denials and Appeals in ACA Marketplace Plans in 2024. KFF, March 24. Marketplace claims analysis

  • McCartan, C., Fisher, R., Goldin, J., Ho, D. E., & Imai, K. (2025). Estimating Racial Disparities When Race is Not Observed. Journal of the American Statistical Association, 120(552), 2140–2153. DOI: 10.1080/01621459.2025.2526695. Estimating Racial Disparities

  • Mitchell, S., Potash, E., Barocas, S., D’Amour, A., & Lum, K. (2021). Algorithmic Fairness: Choices, Assumptions, and Definitions. Annual Review of Statistics and Its Application, 8, 141–163. Algorithmic Fairness

  • National Association of Insurance Commissioners (NAIC). (1997). Unfair Claims Settlement Practices Act, Model 900. Model 900

  • National Association of Insurance Commissioners (NAIC). (2023). Model Bulletin: Use of Artificial Intelligence Systems by Insurers. Adopted December 4, 2023. Model AI Bulletin

  • U.S. Department of Health and Human Services, Office of Inspector General (HHS OIG). (2022). Some Medicare Advantage Organization Denials of Prior Authorization Requests Raise Concerns About Beneficiary Access to Medically Necessary Care. OEI-09-18-00260. OIG investigation