Insurance by BogdanovaThe Encyclopedia of Insurance and Risk Science

RESEARCH DOSSIER

Underwriting Proxy Discrimination and Disparate Impact

Abstract. Insurance underwriting transforms uncertain future losses into present decisions about access, price, contract terms and the allocation of risk. When predictors capture information about a legally protected characteristic, that transformation may reproduce discrimination through apparently neutral variables, even in systems that omit the protected characteristic itself. This monograph examines proxy discrimination and disparate impact as connected but distinct actuarial, statistical, legal and institutional problems. It combines a targeted review of actuarial scholarship, selected United States state frameworks, European Union and Great Britain law and international supervisory guidance with transparent analytical examples of conditional prediction, standardization, selective labels, threshold decisions and pricing redistribution. The central finding is that predictive validity, the mechanism transmitting protected information, the distribution of consequences and legal permissibility require separate evidentiary assessments. An observed disparity can originate in legitimate exposure differences, proxy effects, biased outcome measurement, selection, or several mechanisms simultaneously. Conversely, aggregate parity can coexist with unjustified differential decisions. An adequate underwriting audit therefore follows the full decision process, identifies the relevant insurance product and legal standard, tests credible alternatives, documents uncertainty and monitors actual access as well as financial sustainability. The numerical examples are hypothetical and exactly reproducible; they establish logical relationships rather than estimates of market prevalence. The contribution is an insurance-specific analytical synthesis of established research and regulatory approaches.

Keywords: Underwriting; proxy discrimination; disparate impact; actuarial risk classification; insurance pricing; group fairness; indirect discrimination; selective labels; model validation; insurance market conduct; alternative data; causal inference.

Introduction and disciplinary foundation

Underwriting determines the conditions under which an insurance institution will assume a risk. The determination involves more than forecasting expected losses. Decisions about whether to quote, which contract to offer, which exclusions to impose, whether to request further evidence, and whether an applicant is eligible all govern access to protection. A model that ranks risks is one component of this institutional process. It acquires its public consequences through the rules, human judgments, operational constraints and market conditions that convert scores into decisions.

Insurance has a distinctive relationship with classification. A premium is a conditional estimate of future costs combined with expenses, financing demands and contractual terms. The insurer ordinarily cannot know an individual's future realized loss. It instead forms classes or estimates expected outcomes from observable characteristics. Some distinctions are legally permitted and economically significant. Others are prohibited, restricted or subject to justification in the applicable jurisdiction. The existence of predictive information does not determine its legal or ethical admissibility. A valid actuarial classification must be considered within the institutional purpose for which it is used.

The specific scientific problem is the relationship between protected information and ostensibly permitted predictors. A residential location can encode physical exposure to flooding, infrastructure conditions, distance to emergency services and patterns of socioeconomic segregation. A financial history can contain information about repayment behavior and about economic hardship. An occupational category can convey physical hazard, working conditions, and unequal labor-market opportunities. A single observed variable can carry several kinds of information at once. The task of the auditor is to distinguish these mechanisms, identify the material pathways through which they affect a decision and assess their admissibility. Simply assigning a variable to a list of approved or disapproved names leaves these issues unresolved.

This study treats a protected characteristic as a characteristic given legally relevant protection in the domain under analysis. Its identity and treatment depend on the governing law, product, applicant and decision. Examples include sex or race in particular legal regimes. A proxy is a variable or combination of variables that transmits information about such a characteristic. Proxy discrimination concerns the consequential use of that information in a context in which the relevant form of differentiation is disallowed or otherwise objectionable under a specified normative criterion. Disparate impact, used here in the statistical sense unless expressly discussing a legal doctrine, denotes an adverse difference between groups in an outcome associated with a practice. It can establish a question for investigation without resolving its cause, legal status or justification.

The research addresses three questions. How does a predictor with plausible risk relevance become a vehicle for protected information? Which disparities persist when risk composition, application selection, measurement and thresholds are examined? Which corrective responses can reconcile actuarial purpose, legally applicable constraints and consumer protection? The disciplinary foundation is underwriting; the complementary perspectives are insurance ethics and fairness and insurance governance and market conduct. The text is concerned with classification of prospective insurance risk and its immediate consequences. Claims adjudication enters only where it supplies training outcomes or illustrates a mechanism relevant to underwriting.

The methodology is a structured, narrative synthesis rather than a systematic review with a registered search protocol. Sources comprise official professional standards, official legislative and supervisory texts or summaries, and scholarly publication records. Source-specific access boundaries are retained. The legal comparison uses specified US state instruments EU and Great Britain instruments as illustrative, separately identified regimes. The review does not purport to determine binding rules for every jurisdiction, product, insurer or contract. The examples use constructed populations with stipulated probabilities, so they can demonstrate statistical identities independently of proprietary insurer data. No real applicant has been classified and no outcome is inferred for an actual protected population.

Risk prediction and social proxies

The actuarial object and the information set

Let Y be the prospective contractual loss over a specified exposure period, X a vector of candidate risk attributes and A a protected group indicator. The unconstrained conditional mean is m(x,a) = E[Y | X=x, A=a]. This statistical estimand describes an expectation under a particular distribution of exposure and contractual conditions. It is not a universal characteristic of an applicant. Altering the coverage limit, deductible, benefit definition, exposure period or claims process may alter the expected insured loss even for the same person or property.

A technical risk premium often begins with estimated frequency multiplied by expected severity conditional on an event, provided the definitions and exposure treatment are coherent. The premium charged may also include acquisition and administrative expenses, reinsurance, cost of capital, profit or contingency margins and taxes. A risk score, loss estimate and final premium must therefore be stored as distinct objects. A disparity in premium can be produced after loss prediction by discretionary adjustments or class-level commercial rules. Equally, a fairer loss estimate can be undone by a threshold that determines who will receive coverage.

The Actuarial Standards Board's risk classification standard (2005), particularly section 3.2, provides a useful professional reference. An actuary considers the relationship of the characteristic to expected outcomes, objectivity, practical use and applicable law. The standard does not require establishing cause and effect before using every risk characteristic. This professional proposition has a specific implication: predictive relevance and the causal or legal justification required by an anti-discrimination rule are separate questions. An actuary can establish a statistically useful relationship without having established a permissible basis for the corresponding decision. The standard's professional scope cannot displace legislation or regulation.

How protected information enters a score

If a predictor Z has different distributions across groups, then Z contains statistical information about A. An underwriting model trained without A can still produce a group-associated output. Correlation can arise through shared causes, mediation, geography, institutional history, sampling or measurement. It can also emerge conditionally: two variables with little marginal association with A may identify group membership in combination. Interactions, incomplete records, fine geographic resolutions and embeddings enlarge the information set beyond the labels shown in a conventional rating table.

Prince and Schwarcz's (2020) analysis of proxy discrimination identifies the difficulty of restricting the channel by removing a few recognizable surrogates. Their argument is especially pertinent to underwriting systems with large external-data inventories. A substitute model can recover similar information through combinations of remaining features. Consequently a test should assess the fitted system and its decision consequences rather than treating a feature blacklist as exhaustive evidence. The importance of a protected characteristic may be reconstructed indirectly even where individual conditional associations appear modest.

There is also an important distinction between a proxy for a prohibited attribute and an exposure indicator that is correlated with that attribute. For property risk, roof construction and physical flood depth might be associated with neighborhood demographics while representing material differences in insured peril. A classification rule might permit a directly measured hazard yet prohibit differentiation based on ethnicity. These facts do not automatically authorize a particular territorial model. The relevant scientific challenge is whether the model extracts physical exposure information or captures other information with consequences beyond the justified insurance purpose. A geographic variable does not become intrinsically impermissible from correlation alone, nor does an explanation in terms of location establish its admissibility.

The analysis must identify multiple pathways. Consider a hypothetical causal graph with A representing a socially assigned characteristic, H a physical hazard, S socioeconomic constraints, Z a measured predictor and Y the loss. Suppose H affects Z and Y; S affects Z and also the probability of reporting a claim; A is associated with S through social institutions. A score based on Z combines hazard information with information connected to differential reporting and social history. The direction and magnitude of these paths cannot be inferred solely from the joint distribution of A, Z and Y. A causal judgment about which paths should be preserved is inseparable from a normative judgment about the insurance purpose. Conditioning on S may remove a statistical disparity while obscuring a mechanism that a legal rule treats as relevant.

Omission, averaging and unawareness

If A is omitted, the loss prediction becomes m_U(x) = E[Y | X=x] = sum over a of m(x,a) P(A=a | X=x). The equality is the law of total expectation. It illustrates that an otherwise accurate prediction trained without A can use its conditional distribution. For comparison consider an averaging operation m_Q(x) = sum over a of m(x,a) q(a), where q is a specified reference distribution independent of x. Such an operation removes variation in the averaging weights but retains the estimated conditional loss functions. It depends on coherent estimates of these functions and on a declared reference distribution.

Lindholm and colleagues (2022), in Discrimination-free insurance pricing, develop mathematical treatment of direct and indirect discrimination and a pricing construction intended to avoid both. The authors also identify privacy concerns. Inspection of Sections 3–4 and 6 confirms that fixed-reference averaging may require adjustment to preserve aggregate premium adequacy, and that unsupported combinations of attributes can prevent conditional-mean estimation. The simple averaging example below is an exposition of the conditional-expectation mechanism; it is not claimed as a new pricing construction, a full replication of that article, or a universal legal test.

Suppose A takes values zero and one, and X is a binary candidate feature. At x=0, let expected losses for A=0 and A=1 be 100 and 200 monetary units, respectively. At x=1, let the corresponding losses be 150 and 250. Suppose P(A=1 | X=0)=0.2 and P(A=1 | X=1)=0.8. The unawareness prices are 120 and 230. With q(0)=q(1)=0.5 the averaged prices are 150 and 200. The original between-feature difference is 110; the reference-averaged difference is 50. The discrepancy of 60 is generated by the change in conditional group composition under these assumptions. If P(X=0)=P(X=1)=0.5, both pricing schedules collect an expected 175 before expenses and behavioral responses. If selection changes the feature distribution, the portfolio revenue equality need not survive.

The example separates a risk-feature gradient of 50 from a composition-related gradient of 60 in a deliberately simple additive setting. In actual models, interaction between feature and protected characteristic can alter the decomposition; different choices of q can change the resulting gradient and revenue. A protected characteristic might be legally unavailable even during model training. The averaging method is therefore useful for clarifying the problem under specified assumptions. Its feasibility and normative appropriateness require separate inquiry.

Data lineage and the meaning of the response

Proxy assessment must extend to the target and training population. Suppose a health-related prediction uses paid claims as a surrogate for medical need. Paid claims depend on coverage, access to providers, cost sharing, billing and utilization. A model can accurately predict payments while failing to represent the intended clinical need. In disability underwriting, historical claim duration may also be influenced by contract definitions, access to rehabilitation and claims practices. A calibrated forecast of an administratively constructed target is valid for that target only.

This distinction is particularly important in data acquisition. Vendor scores described as lifestyle, stability, reliability or fraud propensity may combine data generated by processes that the insurer does not observe. The insurer must know the source population, target definition, time window, coverage of records, treatment of missingness and changes to methodology. Data acquired for an apparently unrelated objective cannot be assumed to represent the loss exposure relevant to an insurance contract. A score trained to predict credit default may require different justification from one trained to estimate the frequency of covered automobile collisions.

The scientific finding is that proxy discrimination can arise from an input, a target, an observation process or a post-model decision rule. These are distinct points of failure. They require tests directed at the particular mechanism and cannot be eliminated by a single operation on the feature matrix.

Comparative discrimination doctrines

Legal concepts and evidentiary burdens

A statistical group difference is a measured property of a decision distribution. A legal finding is a conclusion under a specific prohibition, domain, comparator, evidentiary burden and possible justification. A common vocabulary across jurisdictions is useful only if it preserves these differences. Some legal regimes emphasize the differential effect of a neutral practice and its justification; others define unfair insurance discrimination through prohibited classifications and actuarial or cost-related constraints. An ordinary contractual difference based on hazard can be permitted in one context and restricted in another. The applicable rule has to be established before a statistical test is interpreted as evidence of compliance.

Direct discrimination involves treating an individual differently on the basis of a protected characteristic in the legally relevant sense. Indirect discrimination or disparate-impact doctrines may address a practice that is facially neutral but disadvantages a protected group. These legal terms are not interchangeable across countries or sectors. In scientific reporting, an adverse selection-rate ratio can quantify an outcome; it cannot by itself establish that an insurer has violated a prohibition or that a practice has been justified. Equally, an acceptable ratio cannot establish that direct discrimination has not occurred.

United States: state supervisory approaches

The state insurance setting requires attention to the particular jurisdiction and line. The National Association of Insurance Commissioners' artificial intelligence topic page describes its December 2023 Model Bulletin on AI use by insurers as emphasizing existing applicable law, governance and supervisory information. A model bulletin is a template for regulatory communication, not a uniform federal law. The applicability of an adopted version depends on state action and the instrument actually in force. The summary is used here for the nature of that supervisory initiative, not as proof that all fifty states adopted identical obligations.

New York's Department of Financial Services issued Circular Letter No. 7 on July 11, 2024. Its addressed entities and scope are defined in the letter, which concerns underwriting and pricing using external consumer data and AI or other predictive models. Its fairness analysis expects evaluation of differential adverse effects, a legitimate lawful explanation where such an effect exists, and examination of alternatives with less discrimination that can reasonably satisfy business needs. Assessments are bounded by available or reasonably inferable protected-class information. The final letter expressly does not expect new individual data collection for those analyses. It states supervisory expectations linked to existing law, not a universal fixed numerical cutoff for proving discrimination.

This approach gives concrete scientific meaning to comparison of alternatives. If the current model's difference is attributable partly to external data, and a comparably useful model using more direct hazard information produces materially less unjustified impact, the alternative merits documented assessment. If a variable accounts for a difference through an insurance-relevant relationship, the investigation still considers whether another practicable method meets the legitimate objective with less discrimination. The result depends on the covered practice and legal context; analytical convenience is not a sufficient reason for rejecting an alternative.

Colorado's enacted SB21-169 (2021) addresses unfair discrimination in insurance practice, including the use of external consumer data sources, algorithms and predictive models under rules by insurance type and practice. The original signed act's section 10-3-1104.9(8)(e) links protected-class correlation to disproportionate disadvantage exceeding the reasonable relationship to the insurance practice, including underwriting losses and costs. It also specifies scope exclusions and limits on compelled demographic collection. These observations concern the 2021 enacted text, not a consolidated statement of subsequent amendments. It requires a regulatory process directed at evaluation, monitoring and remediation. The amended Regulation 10-1-1 notice (Colorado Division of Insurance, 2025) identifies individual life insurance, private passenger automobile insurance and health benefit plans within its governance scope, with an effective date of October 15, 2025.

The two instruments demonstrate why regulatory history matters. A general enabling prohibition and a sector-specific governance regulation perform different functions. A governance obligation to inventory external data, assess outcomes and remedy adverse effects does not make the same test appropriate for all products. A full compliance determination would require the consolidated statute, operative text and any applicable quantitative testing rules and amendments. This study does not specify their exhaustive requirements or reporting deadlines; the official adoption notice and enacted summary support the narrower comparison presented here.

European Union: equal treatment and AI governance

The Council of the European Union's Directive 2004/113/EC (2004) addresses sex discrimination in goods and services, including insurance. Article 2 defines indirect discrimination using disadvantage arising from an apparently neutral practice, subject to objective justification by a legitimate aim with appropriate and necessary means. Article 5 addresses premiums and benefits. Its original paragraph 2 permitted a derogation under conditions; the legal effect of that provision changed through judicial review.

In Test-Achats (2011), Case C-236/09, the Court of Justice declared Article 5(2) invalid with effect from December 21, 2012. The decision concerns individual sex-based premium and benefit differentiation under that instrument. Its significance for this study is that empirical differences in risk can coexist with binding legal restrictions on the use of a characteristic. The statistical task then changes from unconstrained prediction to estimation and pricing within the permissible information and decision space. The judgment does not itself establish a particular proxy metric or general prohibition of every correlated hazard factor.

Artificial intelligence governance introduces a further layer. The European Commission's essential-services AI Act guidance identifies individual life and health risk assessment and pricing as the use case corresponding to Annex III point 5(c), with distinct examples for claims management and product design outside that specific use case. The intended purpose of a system matters. This guidance is illustrative; the exact applicable classification and dates require the operative regulation, including exceptions and subsequent amendments. The study does not claim a verified consolidated transition timetable.

EIOPA's AI governance Opinion announcement (2025) explains that its sectoral supervisory interpretation concerns existing insurance requirements and excludes AI Act high-risk and prohibited systems from the Opinion's scope. It addresses fairness, data governance, records, explanation and oversight proportionately. The general insurance governance framework and the AI-specific regime therefore have distinct scopes. A system outside the particular high-risk underwriting use case can still generate material conduct risk.

Great Britain: indirect discrimination and insurance exceptions

The United Kingdom Parliament's Equality Act section 19 (2010) addresses generally applied provisions, criteria or practices that put a protected group and the individual at a comparative disadvantage, subject to a proportionality justification. The comparison is confined to the specified Great Britain provisions and does not describe Northern Ireland law.

The Act's insurance and financial-service exceptions, Schedule 3, distinguish age and disability. Paragraph 20A includes insurance among financial services and requires relevant, reasonably reliable information when risk assessment involves age. Paragraph 21 qualifies the prohibition under section 29 concerning disability in insurance business where the information is relevant to risk and reasonably reliable, and the action itself is reasonable. These exceptions concern particular protected characteristics and statutory conditions. A statistical association therefore cannot substitute for evidence of the decision's justification under the relevant provision.

Comparative implications

The New York example emphasizes documented evaluation of adverse effects, underlying rationale and alternatives. The Colorado example links unfair discrimination to the governance of particular data and models through state-specific instruments. The EU equal-treatment example demonstrates a substantive limit on the actuarial use of sex; AI governance adds obligations directed at system purpose and risk. These are complementary illustrations of regulation, not a harmonized set of identical statutory burdens. No country ranking or universal legal threshold is inferred from them.

For each proposed underwriting model, the unit of regulatory analysis should identify the insurer, insurance line, risk location, jurisdiction, decision, characteristic, authoritative instrument and effective version. For multinational groups, the same centrally developed score may be suitable for one business line and unsuitable elsewhere. A model inventory must represent these jurisdictional deployments and the actual decision rules they feed. Legal analysis is an input to model specification, not a certificate appended after fitting. The comparison is deliberately bounded: Canadian, Australian, Northern Irish and other domestic regimes require jurisdiction-specific source work beyond the instruments assessed in this monograph.

Selection and group measurement

Defining the denominator

An underwriting disparity is interpretable only with a defined population. A quote request, submitted application, completed application, offered policy, accepted policy, renewal and covered exposure are different observational units. A study restricted to bound policies cannot estimate application-stage access without additional information about nonbound applicants. In a distribution system with several channels, one insurer may receive a different customer population than another because brokers have already filtered applicants. A low offer rate in a narrowly selected channel might reflect the initial selection rather than the insurer's score.

For each group g and decision stage, let N_g denote the number eligible for the stage and O_g the number receiving the specified favorable outcome. The rate r_g = O_g/N_g requires a positive denominator and consistent exposure definition. For two groups the risk difference RD = r_1 - r_0 and rate ratio RR = r_1/r_0, provided r_0 is positive, capture distinct aspects of the same disparity. A ratio is scale-dependent and can appear large when absolute rates are low. For premiums, limits or delays, continuous outcome summaries and quantiles may be more meaningful than a binary rate.

As an illustration, group zero contains 1,000 applicants with 800 offers, while group one contains 500 with 300 offers. The rates are 0.80 and 0.60; RD = -0.20 or minus 20 percentage points; RR = 0.75. With independent Bernoulli observations and ignoring clustering, the estimated standard error of the difference is sqrt(0.80.2/1000 + 0.60.4/500), approximately 0.0253. An illustrative normal 95% interval is about -0.250 to -0.150. None of these quantities is a legal determination. The interval does not address selection, inaccurate group assignment, multiple testing, or the causal source of the difference. In insurer data, household, broker and territory clustering may require a different variance estimator.

Composition and standardization

An aggregate disparity may arise with equal decisions within risk classes. Consider two risk categories: low and high. In both groups, low-risk applicants have a 90% offer probability, and high-risk applicants a 50% offer probability. Group zero has 800 low-risk and 200 high-risk applicants. Group one has 200 low-risk and 800 high-risk applicants. The respective expected offer counts are 820 and 580, yielding rates 0.82 and 0.58, a difference of -0.24. Under equal reference weights across risk categories, both standardized rates are 0.70. The raw disparity is entirely compositional in this example.

Now consider a second case in which group one has probabilities of 0.85 and 0.45, while group zero retains 0.90 and 0.50. Its expected rate is 0.53, and the standardized rate is 0.65 compared with 0.70 for group zero. The raw difference is -0.29; the standardized difference is -0.05. A residual conditional disparity persists. It is consistent with an unexplained group effect but does not identify its cause. The comparison requires confidence that low and high are genuinely relevant, comparably measured, and legally permissible classifications.

Standardization is useful precisely because it makes the reference distribution explicit. A decomposition based on the group-zero distribution could give a different answer if group effects vary across strata. To avoid presenting a reference choice as an objective natural constant, the analyst should report both group-specific distributions and a declared pooled or policy-relevant reference. If important strata lack observations for one group, a standardized comparison may depend on extrapolation. The overlap problem must be disclosed before interpreting a small residual.

A further issue is that control variables may themselves reflect discriminatory processes. Conditioning on the output of a previously biased risk score can make a disparity disappear by definition. Conditioning on a mediator may answer a different question from the total effect of a practice. Good standardization therefore starts with an independent account of the comparability variables and the meaning of the estimate. It cannot be achieved simply by entering all available covariates into a regression.

Calibration, error balance and threshold effects

For binary adverse outcomes, let Y=1 indicate a loss event and S a predicted probability. Calibration within group g means that, at a given score s, the observed proportion experiencing the event equals s, within estimation tolerance: P(Y=1 | S=s,A=g)=s. A classification threshold t makes the adverse action D=1 when S exceeds t. The false-positive rate is P(D=1 | Y=0,A=g) and the false-negative rate is P(D=0 | Y=1,A=g). In underwriting, a false positive may refer to a loss-free risk receiving a restrictive decision; the concept requires the counterfactual outcome under the specified coverage. For severity or expected monetary loss, calibration of means and calibration of frequency are separate.

Chouldechova's (2017) fair prediction study establishes in a recidivism prediction setting that certain fairness criteria cannot generally be simultaneously satisfied where outcome prevalence differs. This is relevant as a mathematical analogy, not as insurance evidence. The corresponding elementary relationship is Bayes' rule. If sensitivity is TPR, false-positive rate is FPR, and prevalence is pi_g, the positive predictive value in group g is TPRpi_g divided by TPRpi_g + FPR*(1-pi_g). With the same TPR=0.8 and FPR=0.2 for both groups, prevalence 0.10 gives a positive predictive value of 0.08/(0.08+0.18)=0.3077; prevalence 0.30 gives 0.24/(0.24+0.14)=0.6316. Equal error rates here coexist with unequal predictive values. Degenerate cases and perfect prediction require separate treatment.

The lesson for insurance is not that fairness is mathematically impossible. It is that the choice of criterion encodes a choice of harm, comparator and use. Premium setting based on conditional loss requires different evaluation from automatic rejection. An artificial target of equal false-positive rates may be misleading if observed losses are affected by whether coverage was issued. Conversely a score that is calibrated in the insured portfolio can create systematically unequal barriers at application thresholds.

The working-paper record of Lindholm and colleagues (2023), Proxy discrimination vs. demographic disparities, distinguishes discrimination-free pricing from group-fairness criteria and discusses methods to combine them. This distinction supports a separation between the analysis of information usage and group outcomes. The review relies on the repository working-paper abstract for that narrow point. It does not assert independent verification of the later journal paper's detailed results.

Threshold effects merit explicit testing. Suppose two prospective groups have similar mean predicted loss, but group one has more scores immediately above the automatic-rejection cutoff owing to rounding or the tail of a vendor variable. Moving the threshold by a small amount could substantially alter its access rate while changing the mean loss of newly admitted risks only slightly. Evaluating a model by average predicted losses alone would miss this. For each operational cutoff, the audit should show the score distribution, referral and denial rates, expected loss and realized outcomes where observed, along with the sensitivity of each metric to small changes.

Selective labels and observation

Future claims are normally observed for insured exposures rather than for every applicant who could have been insured. Let B=1 indicate a bound policy. The insurer can often estimate P(Y=1 | X,A,B=1) from its claims experience. The applicant-population quantity P(Y=1 | X,A) requires additional identifying assumptions or data. If acceptance depends on unrecorded assessments or if take-up depends on unmeasured affordability and risk, the conditional distributions differ. This is the selective-labels problem illustrated more generally by Lakkaraju and colleagues (2017) in The selective labels problem. The full paper includes corporate insurance requests for which profit or loss is observed only after approval. Its evaluation approach uses variation among decision makers subject to assumptions about case allocation. This example establishes the relevance of selection to underwriting evaluation without making unobserved outcomes identifiable in every portfolio.

Inverse-probability weighting can help only under assumptions that need to be stated. If selection is conditionally independent of the target given observed W, and p(W)=P(B=1|W) is strictly positive on the target support, then E[B Y/p(W)] identifies E[Y] for an appropriately defined latent Y. If policies offered differ and hence change the target itself, potential outcomes under common contractual terms must first be defined. If p(W) is near zero, weights become unstable; if it is zero, the missing outcome cannot be recovered by weighting. Unrecorded underwriting criteria make conditional independence doubtful.

A simple bound demonstrates the extent of missing information. Among 100 applicants, 60 are insured and 12 experience an observed event. With no assumption about the remaining 40 hypothetical outcomes under identical coverage, the overall event proportion is bounded by 12/100 and (12+40)/100, or 0.12 to 0.52. A precisely estimated insured event rate of 0.20 cannot resolve this range. Better information may arise from comparable external exposure records, historical shifts in eligibility rules, randomized permissible pilot designs, or sensitivity analysis. No consumer should be deliberately exposed to an unlawful adverse decision merely to improve identification.

Finally, group membership itself may be absent. Probabilistic group estimates can sometimes support aggregate auditing where permitted. They are uncertain measurements that should be validated, separated from operational scoring, and propagated through uncertainty analyses. A hard assignment derived from a name or geography can misclassify multiracial and minority populations and can recreate the very mechanism being examined. No imputed attribute should be represented as a verified fact about an individual. Privacy restrictions, purpose limitations, data minimization and retention must shape the assessment.

Model audit and remedies

An evidence-centered audit sequence

A model audit begins by fixing the version of the model, the decision policy, the data snapshot and the candidate population. An inventory should differentiate loss predictions, rule engines, pricing optimizers, questionnaires, vendor scores and human overrides. The analyst must be able to reconstruct a decision from the data and rule versions used at the time. Without reconstruction capability, a fairness result may refer to a model that no longer governs consumers. Material changes to a vendor score, tariff table, distribution platform or cutoff require renewed evaluation.

The Actuarial Standards Board's modeling standard (2019) discusses output validation, including comparison against historical observations and holdout results as appropriate to model purpose. A fairness audit supplements this validation by asking how errors and adverse decisions are distributed and whether the model uses permitted information. A well-performing holdout can reproduce the same selection or target distortion as the training data. Validation should therefore use time-separated cohorts, check product and coverage comparability, and distinguish predictive performance among observed risks from access outcomes among applicants.

The first empirical layer is an access funnel. Tabulate eligible records, completed forms, vendor-data availability, referrals, quotes, offers, acceptance, bind and renewal. Stratify by group, product, distribution channel, permitted risk factors and material time period. Record the reason for each transition. A missing variable can be as consequential as a high predicted risk: a default refusal for applicants with little external-data history may affect groups unequally, even when the fitted score is accurate for applicants with rich histories.

The second layer is prediction and decision diagnostics. For frequency, compare observed versus predicted event counts by exposure and group; for severity, compare distributions and appropriately defined conditional means; for total loss, check aggregate predicted versus realized cost under consistent valuation. Compare the burden of premium, deductible, limit and restrictive terms separately. Report exact denominators, observation months, uncertainty intervals and missingness. AUC alone cannot measure calibration or identify proxy use. A favorable loss ratio alone cannot establish that excluded applicants were correctly excluded.

The third layer is proxy investigation. Assess the relationship between each candidate feature and A, including conditional associations and multivariate predictability. Explore interactions, missing-data patterns and transformations as well as raw variables. A model predicting A from X can be used as a diagnostic of available protected-group information, subject to legal permission and controlled access. Its accuracy does not reveal whether the underwriting decision actually relies on that information. High reconstructability may arise in a variable the underwriting model barely uses; modest reconstructability can still have a large consequence if the score is sensitive near the threshold. The association test must be linked to the actual output and decision distribution.

The fourth layer is a controlled comparison of alternatives. Refit a model without suspicious features, use more direct hazard measures, change the representation of missing values, or constrain the model to reduce inappropriate effects. Keep the outcome definition, validation population and time period comparable. Evaluate changes in predictive accuracy, calibration, pricing adequacy, group outcomes, availability and uncertainty. Removing one feature while leaving downstream scores computed from it does not constitute removal. It is also necessary to assess interaction effects and the retrained model's substitute pathways.

Counterfactual comparisons and explanations

A useful testing technique is to create matched or counterfactual input pairs whose legitimate risk characteristics remain fixed while the disputed information is varied in a coherent way. If a score moves substantially despite unchanged admissible risk information, the result warrants investigation. Such a test requires a feasible intervention: replacing a postal code while holding local flood depth, construction characteristics and all correlated territorial inputs fixed may create implausible combinations. The investigator should distinguish local sensitivity from causal evidence and test against realistic changes in variables.

Explanations such as feature importance, partial dependence or attribution scores can describe fitted behavior. They do not establish the legal status of a predictor. A large attribution to geography may justify further review; it cannot show whether the model uses geography for hazard, business cost or group information without contextual evidence. Correlated features can divide or transfer importance depending on the explanation method. A complete decision explanation should include the rule that translated the score into a quote or referral, plus the materially influential data and applicable underwriting rationale.

The audit should also test manual intervention. An ostensibly neutral automated score can be applied disproportionately strictly by a reviewer, or a human override can correct systematic model limitations. Compare decisions before and after override, review reason codes, and examine matched cases at the same score and legitimate risk strata. Human oversight is meaningful when the reviewer has competence, authority, sufficient information and an auditable escalation process. Merely inserting an approval click in an automated workflow is insufficient to identify or remedy a disputed outcome.

Remediation and operational accountability

Remediation depends on the diagnosed mechanism. If a variable lacks an adequate relationship to insured loss and carries a material adverse effect, removal or replacement is a plausible response. If an admissible hazard indicator is measured differently across groups because of poor data coverage, correcting measurement may improve both accuracy and fairness. If a biased target has been used, target reconstruction is required; post-processing the score leaves the target error intact. If the disparity arises from a cutoff rather than prediction, revising decision policy, manual referral or evidence requirements may be more relevant than retraining.

In line with the New York alternative-investigation approach cited above, a credible comparison considers the same legitimate business need and records rejected alternatives and reasons. A small deterioration in one predictive statistic should not automatically outweigh a large reduction in unjustified access barriers. Conversely a large unquantified portfolio effect should not be dismissed because a favorable social metric improves. The appropriate tradeoff must be identified with reference to the governing rule and grounded in calibrated financial and consumer outcomes. Protected-group-specific thresholds may themselves constitute differential treatment and require legal authority; an audit metric is not permission to implement such a rule.

Consumer recourse is part of the evaluation architecture. An applicant may discover that an adverse outcome was based on incorrect identity linkage, obsolete data or an unsupported inference. Clear explanations and practical correction channels make such errors observable and remediable. Audit complaints, corrected records, reversals, processing delay and repeated adverse decisions. Complaint data are selective because people differ in knowledge and ability to challenge outcomes; low complaint volume cannot prove the absence of harm.

Vendor contracts should enable inspection of relevant data lineage, model specifications, validation results, changes and audit information consistent with law and legitimate confidentiality. A third party's commercial secrecy cannot make an insurer's statutory responsibilities disappear. Governance should assign owners for regulatory mapping, data quality, actuarial performance, conduct monitoring, remediation and escalation, with independent challenge of the implementation. Internationally, the IAIS conduct and culture guidance identifies fair treatment of a wide range of consumers as an ICP 19 implementation concern. Its supervisory framework informs governance rather than directly creating the same domestic legal remedy in every jurisdiction.

Tradeoffs between access and risk classification

Risk differentiation and pooling

Insurance is an institution for financing uncertain losses across a population. A premium schedule determines how that financing burden is distributed. Differentiation can align contributions with expected insured losses and influence which risks enter the pool. A restriction on information can redistribute premiums between policyholders without changing the aggregate expected loss in a fixed portfolio. In a dynamic market, changes in take-up, substitution and lapse can alter the portfolio itself. These mechanisms explain why fairness analysis must include market access and solvency consequences without reducing normative obligations to premium equivalence.

Consider a hypothetical policy covering a one-year loss of 1,000 units on occurrence of an event. Two equally sized eligible risk classes have event probabilities of 0.10 and 0.30. Their pure expected loss premiums are 100 and 300; a pooled premium of 200 applied to all 200 risks collects 40,000 units, equal to expected total claims if each class has 100 policyholders. Each lower-risk insured pays 100 more than expected loss and each higher-risk insured pays 100 less. If 50 low-risk policyholders leave and all high-risk policyholders remain, expected claims become 35,000 across 150 policies while revenue at the original pooled premium is 30,000. The new break-even pure premium would be 233.33, assuming no other changes.

The calculation establishes a possibility of composition change, not an empirical estimate of what will happen under a given law. It presupposes that the insurer could otherwise differentiate by the risk classes, that the loss amount is fixed, and that take-up changes as stipulated. Demand elasticity, competing insurers, compulsory coverage, subsidies, risk adjustment, regulatory premium constraints, policy design and distribution access can all alter the result. The example cannot establish that a prohibited protected characteristic is necessary for solvency. A portfolio can respond through lawful classifications, products, financing arrangements, risk sharing and support mechanisms.

A historical empirical example

The Federal Trade Commission's (2007) credit-based insurance scores report official summary provides a direct insurance example. The Commission reported that scores predicted automobile insurance risk in its data and differed across racial and ethnic groups, with implications for premiums. It also reported being unable to construct an alternative score that retained effective prediction while reducing those group differences in its study. These are findings of the specific historical investigation, not an estimate of current rating effects or a proof that better alternatives are impossible. The full report tables, supplementary statements and dissent were not audited here.

Three conclusions follow from this evidence at its documented scope. First, actuarial predictive usefulness and unequal distributions can coexist. Second, a reported association between a score and group membership cannot tell us whether every application of the score is unfair. Third, inability to identify an alternative in a particular study cannot exclude improvements with new data, different targets or altered underwriting methods. Contemporary analysis would require current state legal rules, comparable cohorts, vendor-specific methodology, claim and exposure definitions, and separate evaluation of applicants excluded by the practice.

The historical example also demonstrates the distinction between within-portfolio risk prediction and consumer welfare. A score may have marginal predictive value while increasing premiums for households with restricted insurance options. Measuring benefit to the insurer through prediction does not quantify reduced coverage, changes in mobility, financial distress or switching costs. Conversely a model improvement that smooths premiums may increase retention or broaden the risk pool. Welfare evaluation needs behavioral and distributional evidence rather than an assumption that any direction of change has the same outcome across all markets.

Product-specific consequences

In automobile insurance, the consequences of restrictive underwriting depend on compulsory insurance provisions, territorial law and availability of residual or alternative markets. An applicant excluded from preferred coverage may face higher premiums, reduced limits or difficulty obtaining legally required protection. Physical and behavioral predictors may have different explanatory value depending on vehicle use, exposure miles and coverage. Telematics can provide more direct observations of driving, yet device access, work schedules and measurement practices can generate new proxy channels. A fairer variable cannot be assumed simply because it is more technologically sophisticated.

In property insurance, the admissible risk often depends on physical location. A broad territorial restriction can conceal large within-area differences in wind, flood, construction and mitigation. A useful test compares a coarse geographic score with more direct property-level peril variables and examines effects on both risk accuracy and neighborhood access. The cost of collecting granular hazard data and the quality of such data vary across properties. A missing sensor or inspection result should trigger an evidence-quality decision rather than silently converting data scarcity into a high-risk classification.

In life and disability insurance, the target may be long-horizon mortality, morbidity, benefit incidence or claim duration. Outcome uncertainty requires careful cohort and exposure definitions. Applicants whose medical evidence is incomplete may be referred rather than immediately declined. The privacy and legal status of personal information depends on the jurisdiction and the product. The central question in each case is whether the alternative data actually improve prediction of the covered risk or reproduce differences that are legally restricted or inadequately justified. Group outcome comparisons must account for ages, contract features and relevant permissible underwriting factors.

In commercial underwriting, the insured may be a business while the data include information about an individual owner, employee or guarantor. An aggregate corporate loss model does not automatically make every personal variable necessary for the business risk. Potential adverse effects on small enterprises must be evaluated through the actual underwriting and market structure. Commercial risk differentiation may also be influenced by negotiated terms and broker discretion; group comparisons based only on final premiums can miss earlier barriers and unequal endorsements.

These examples explain why the object of analysis cannot be an abstract model labeled fair or unfair across all insurance. The analysis has to follow the insured peril, contractual exposure, protected population and decision purpose. A method chosen for individual life underwriting may not address property catastrophe risk, and a measure appropriate for applications may be misleading for renewals.

Comparative synthesis, limitations and reproducibility

Four distinct evidentiary questions

The preceding chapters yield a practical analytical synthesis. For each disputed classification, the first question concerns prediction: what risk is being estimated, under what contract and with which evidence of validity? The second concerns the proxy mechanism: where does protected information enter the prediction or decision, and under which assumptions can its contribution be distinguished? The third concerns outcomes: who experiences denial, premium increases, restrictive terms, delay or loss of coverage, compared with which denominator and counterfactual? The fourth concerns legal and institutional permissibility: which rules are applicable, which rationales are recognized and which alternative practices can reasonably meet the insurance objective?

These four questions organize established concepts rather than constitute a newly proposed theory. They are analytically useful because each admits different kinds of evidence. A calibration plot speaks principally to prediction. A test of reconstructability speaks to information. A group rate ratio speaks to outcomes. A statutory interpretation or official supervisory instrument speaks to admissibility. The investigation becomes unreliable when an answer to one substitutes for an answer to another. High predictive performance does not establish admissibility; negligible aggregate disparities do not establish absence of direct or proxy discrimination; and a legally approved variable does not establish that the model has measured it accurately.

In the hypothetical examples, conditional averaging altered price differentiation while preserving expected portfolio revenue under stipulated composition; risk standardization distinguished aggregate from within-stratum disparities; Bayes' rule exposed the incompatibility of selected metrics; bounds showed how insured-only observation limits inference; and the pooling example illustrated portfolio response. Each is a logical comparison in a constructed population. Together they explain why remediation should be tied to the diagnosed mechanism and evaluated against both the contractual risk purpose and consumer consequences.

Reproducible study specification

A real empirical investigation would begin with an inventory of legally permitted sources and a predefined target population, product, decision and study horizon. Distinguish policy-years, applications and individuals. For repeat applications, specify whether each decision or each unique customer is the unit, and account for dependence. Link model versions to decisions through stable identifiers. Record exposure, contract terms, dates and maturity of outcomes. A severity model using immature claims needs development adjustments before group calibration can be evaluated.

Define comparison groups and the protected-data collection or estimation mechanism under applicable law. Obtain or estimate group shares and assess uncertainty without exposing individual attributes to operational decision makers where that use is impermissible. Define the candidate comparability set independently of the disputed predictor. Prespecify an adverse decision and meaningful direction: larger premium, no offer, less coverage, more exclusions or greater delay. If many endpoints are screened, disclose multiplicity and avoid using isolated significance as proof of discrimination.

Fit or reconstruct the historical model, and evaluate an out-of-time baseline. Compare with a transparent hazard-only alternative and additional documented candidate models. Use consistent training and test sets wherever possible; assess performance under plausible changes in the applicant mix and data quality. For insurance loss outcomes, report prediction error with exposure conventions, frequency and severity calibration and aggregate adequacy. For access, report stage-specific counts and standardized comparisons including uncertainty. For proxy diagnosis, record association, output sensitivity and the assumptions behind any causal interpretation. Where a plausible causal explanation remains unidentified, report competing explanations.

For every alternative, calculate the change in expected claims, expense and revenue at fixed portfolio, and model take-up scenarios separately. Report the constraints used and test the sensitivity of conclusions to them. For a model designed for one jurisdiction, repeat the legal mapping for each market in which the system is deployed. Review operational implementation, including override and missing-data rules, after a model is selected. A prospective monitoring plan then checks whether the actual distribution of access and losses resembles the validation assumptions, and whether vendor changes or emerging exclusions alter it.

The constructed examples in this manuscript are reproducible with elementary arithmetic. The conditional averaging example uses X probabilities 0.5 each, A=1 conditional probabilities 0.2 and 0.8, and the four conditional means 100, 200, 150 and 250. The offer-rate example uses counts 800/1000 and 300/500; composition examples use explicitly stated group counts and probabilities. Error-rate computation uses TPR 0.8, FPR 0.2 and prevalence 0.10 and 0.30. Selection bound uses 12 observed events, 60 observed exposures and 40 unobserved outcomes. Pooling uses two 100-person classes and fixed severity. These are all assumptions, not extracted market datasets. They permit verification without proprietary or protected personal information.

Evidentiary limitations and research agenda

The public evidence considered here includes official professional standards, specific legislative and supervisory texts and summaries, a historical official insurance study summary, law-review and actuarial article records, and methodological abstracts. Access to some publisher full texts and legal pages was restricted. Abstract-level sources are used only for findings stated in the accessible records, not their detailed formulas, estimates or proofs. The identified working paper is cited as its 2023 repository version. The legal comparison does not represent a consolidated November 2026 audit of the entire EU AI Act or of every US state regulation. The cutoff is the research session; cited instruments retain their own dates. Jurisdiction-specific compliance questions demand authoritative operative texts and application to facts.

This research does not contain insurer-level applicant, premium, decision, claims or protected-attribute microdata. It cannot estimate the incidence of proxy discrimination, the effect of a particular commercial system, the prevalence of discriminatory predictors, or the causal welfare consequences of specific restrictions. Historical findings do not establish current market conditions. The numerical examples verify relationships under declared assumptions; they provide no confidence intervals for actual insurance markets. The offer-rate interval is a theoretical sampling calculation for independent observations, not the result of a field study.

There is consequently a substantial empirical research agenda. It should examine the quality of proxy testing under incomplete demographic data, the relationship between underwriting exclusions and later access to alternative coverage, group-specific target measurement in long-duration contracts, performance effects of direct hazard variables versus external scores, and the stability of re-estimated underwriting models after distribution shifts. Legal and privacy constraints must be built into study designs. Where actual counterfactual claims remain missing, partial identification and explicit sensitivity analysis may be more defensible than imputation that hides its assumptions.

The most important research priority is better alignment between data about the decision funnel and data about insured outcomes. Underwriting institutions often need accurate claim forecasts and credible evidence about people who never enter the claim database. Linking these perspectives with suitable privacy protections would permit more informative investigation of financial sustainability and access. It would also reveal the stages at which a change can achieve the greatest improvement. An absence of such data should be treated as a limit on inference, not filled with unverified causal explanations.

Conclusion

Underwriting proxy discrimination is an information and decision problem embedded in the insurance institution. A prohibited characteristic can influence apparently neutral predictions through correlated data, interactions, data availability, target construction and decision thresholds. A predictor's statistical association with loss establishes a relevant scientific question; its admissibility and actual contribution to a discriminatory decision require independent evaluation.

Disparate impact is a consequential property of a defined distribution of underwriting outcomes. It must be measured at the appropriate decision stage and with a meaningful denominator. Risk composition, measurement error and selective observation can materially alter the interpretation. Standardized rates, threshold analysis and calibration provide complementary evidence. No single metric establishes all the properties sought by actuarial risk classification and anti-discrimination law.

The comparative analysis shows that insurance fairness is governed through different legal mechanisms. State-specific unfair-discrimination and model-governance requirements, EU equal-treatment law, AI system oversight and professional actuarial standards serve related but distinct functions. Their authority, product scope and evidentiary burdens must remain explicit. Scientific validation supports the application of these rules without itself determining the legal outcome.

A rigorous investigation considers alternatives directed at the material cause: more direct risk measures, corrected targets and data, amended rules or thresholds, operational oversight and meaningful consumer recourse. It assesses pricing and sustainability under stated portfolio assumptions and access outcomes under observable decision conditions. A technically attractive model cannot receive a sound fairness conclusion when the relevant outcomes or legal conditions remain undefined.

The independent analytical contribution of the monograph is the connection of conditional prediction, protected-information pathways, selection and legal comparisons to decisions that actually allocate insurance protection. The evidence supports a disciplined separation of questions concerning predictive accuracy, proxy transmission, unequal consequences and admissibility. Their integration supplies a more complete account of underwriting than any isolated model metric. This conclusion is a synthesis of the studied literature, mathematical relationships and identified regulatory sources, without claiming a new theoretical system or empirically established market-wide result.

References