Insurance by BogdanovaThe Encyclopedia of Insurance and Risk Science

RESEARCH DOSSIER

Underwriting Counterfactual Decision Evaluation

Abstract. Underwriting counterfactual decision evaluation investigates the consequences of replacing an insurance acceptance, referral, evidence-acquisition or contractual-offer policy with a specified alternative. Its fundamental problem is that the historical rule governs both the observation of risk outcomes and the composition of insured exposures. This monograph examines potential-outcome estimands, target-trial design, selective labels, overlap, direct and inverse-probability policy evaluation, doubly robust methods, lawful randomized process interventions and quasi-experimental identification. It integrates the actuarial consequences of marginal selection, consumer access, portfolio dependence and market response while separating a zero actual contractual payment from an unobserved hypothetical insured loss. Transparent hypothetical comparisons demonstrate how more coverage may reduce expected contribution and how partial identification can delimit unsupported extrapolation. International supervisory evidence and US accelerated-underwriting studies illustrate implementation and monitoring boundaries. The research establishes that a credible alternative-policy evaluation requires an explicitly defined population, decision process, contract, horizon, identification assumptions and portfolio objective, with uncertainty reported at each evidentiary level. It is a documentary and analytical monograph; it does not estimate a proprietary insurer's causal effects or claim a new theory.

Keywords: Underwriting; counterfactual evaluation; causal inference; potential outcomes; selective labels; off-policy evaluation; risk classification; randomized holdouts; policy learning; adverse selection; insurance demand; partial identification; model governance; consumer protection; portfolio risk.

Introduction and Conceptual Foundations

Underwriting is the organization of decisions through which an insurance institution determines the risks and contractual obligations it will assume. The counterfactual problem arises whenever an institution asks what would have happened had it used a different decision rule. The historical record reveals the consequences of the rule actually applied, together with applicant responses and external events under that rule. The scientific task is to identify consequences of a specific alternative, distinguish what the record identifies from what assumptions supply, and relate the comparison to protection, financial sustainability, and institutional accountability.

Consider a life insurer that replaces medical review with an expedited pathway for applicants meeting a defined eligibility rule. A reduction in processing time among policies issued after the change may reflect the operational intervention, changes in the applicant mix, changes in distribution, or exclusion of applicants whose cases took longer and terminated before issue. A stable observed mortality ratio may reflect favorable applicant composition even if classification accuracy deteriorated. A valid study must specify the denominator and the intervention before interpreting either observation.

The object encompasses acceptance, rejection, deferral, referral, requirements for further information, risk class, offer terms, and limits. Pricing, contract interpretation, claims adjudication and reserving enter where they influence the counterfactual comparison; their independent scientific objects remain outside this study. The unit of selection may be an application, an insured person, a property, a fleet or a commercial group, whereas the unit of financial loss may be a contract or portfolio. Counting applications alone may overrepresent repeated submissions; counting contracts omits applicants who never receive coverage. A coherent evaluation links these units without equating them.

The research is a targeted documentary synthesis of causal inference, policy evaluation, insurance economics, actuarial practice and supervisory evidence. Three questions govern the argument: what outcomes would another underwriting policy produce; when can observational records identify those outcomes; how do the answers change when applicants and insurers adapt? Mathematical derivations and numerical scenarios below are authorial illustrations. No insurer microdata were provided, and no population-specific effect size, industrywide causal estimate, or claim of theoretical priority is advanced.

The decision environment

At the beginning of an application, an insurer observes a set of admissible variables X available at the decision time. A rule maps X and possibly a recorded history of subsequent information into an action A. The action can be a defined bundle: offer a specified contract, request additional evidence, refer to a senior underwriter, or decline. Acceptance of the offer by the applicant is a distinct event, designated B. The observed outcome depends on the action, the choice of the applicant, the eventual exposure, and how losses are recorded.

This requires a distinction between the technical performance of a prediction and the value of a decision. Calibration asks whether estimated probabilities correspond to observed frequencies among a defined population. Discrimination asks whether a score orders cases by risk. Decision value asks whether a mapping from information to actions improves outcomes relative to another mapping. Good calibration alone provides no answer about changed prices, coverage, take-up, capital concentration or procedural fairness. Decision theory supplies the bridge through specified consequences and admissible constraints.

An underwriting policy is also an institutional commitment, not merely a software function. A deployed rule includes eligibility, authorized exceptions, escalation, documentation and execution. Two implementations with the same score threshold may create different outcomes if one has a human appeal pathway and the other does not. Evaluations must compare operationally defined policies rather than labels such as automated and manual.

The public US description of accelerated underwriting (NAIC, n.d.) illustrates this heterogeneity: an accelerated pathway may still direct an applicant to traditional review when available evidence is insufficient. An evaluation contrasting acceleration with full review must account for that referral structure. The same insight applies to property surveys, commercial referral authority and renewal screening, even though the risks and legal settings differ.

Counterfactual Estimands

Potential outcomes and reference populations

Let X contain baseline information, A belong to a finite set of actions, and Y(a) denote a well-defined outcome under intervention a. The potential-outcome notation follows the causal orientation of Hernán and Robins (2020), Causal Inference: What If. A stochastic policy pi(a|x) specifies action probabilities that sum to one for each x. For a fixed target applicant population, a scalar policy value can be written in conventional notation:

V(pi) = E_X[sum over a of pi(a|X) m(a,X)],

where m(a,x) = E[Y(a)|X=x]. Two policies are compared by V(pi1) - V(pi0). This expression presumes that each potential outcome is well defined, that interference is absent or captured in the target specification, and that the policy acts on the recorded baseline population. A policy with evolving application demand requires a different population-level estimand, developed below.

If Y is a loss to the insurer, the preferred value may be lower. If Y is access to insurance, higher may be preferred. If Y is net financial contribution, the sign has yet another interpretation. A single numerical direction cannot be assigned before naming the outcome. Often the appropriate result is a vector rather than a scalar: changes in bind rate, protection, claims, processing expense, time to decision, capital consumption and distributional effects.

The effect of accepting an application must be separated from the effect of offering coverage. Binding can be influenced by price, contract wording, broker advice and elapsed time. Conditioning a comparison on B = 1 generally selects a post-decision subset. The primary offer-policy estimand is therefore measured over the eligible applicant cohort, retaining both purchasers and nonpurchasers. A secondary analysis of contractual loss per insured exposure addresses pricing and solvency, but its denominator changes under the policy.

For rejected applicants an insurer's contractual payment is zero under rejection. A hypothetical contractual loss under acceptance is unknown. These statements are consistent. For an applicant who experiences an uninsured fire, the insurer's payout under rejection is zero even though social loss is positive. Assigning a zero insured-loss value to an unobserved hypothetical acceptance outcome would confuse the accounting consequence of decline with the underlying risk.

Financial value and protection

Over a horizon H, define D(a) as discounted premium actually earned less discounted claim obligations and attributable costs, including acquisition, administration and review. A decline action may incur processing cost even if it earns no premium. Claims may be censored at the end of H; a financial comparison needs development assumptions and consistent discounting. Capital requirements and reinsurance costs may be nonlinear at portfolio level and cannot always be allocated consistently to isolated applications.

For a contract with net retained loss L, a simple illustrative contribution per application is E[B(P - L - C)] - K, where P is conditional earned premium, C is other conditional costs and K is costs incurred regardless of bind. This is an accounting decomposition, not a pricing recommendation. The expectation must preserve dependence between bind and risk. Substituting E[B] times E[L] when binding is associated with loss propensity produces error.

Protection is independently meaningful. A contract may transfer substantial financial risk even when no claim occurs. Welfare evaluation requires the value of risk reduction, willingness to pay and distributional considerations in addition to the insurer's margin. Premiums are transfers between parties within an aggregate welfare account, although administrative and risk-bearing costs consume resources. Aggregating profit and consumer surplus requires careful handling of transfers and taxes.

Estimands by decision type

Intervention Target population Primary outcome Important secondary outcome
Waive an examination Eligible applicants at original screening Completion and bind probability Ultimate mortality and cost
Change referral threshold Applications before referral Time, bind and net contribution Overrides and access by group
Offer a different deductible Applicants eligible for both terms Take-up and retained loss Uninsured household or business exposure
Change business appetite Defined market and submission window Portfolio contribution and access Accumulation and reinsurance
Introduce renewed risk review Existing contracts eligible for renewal Retention and future coverage Customer burden and selection

These estimands illustrate distinct identification problems. The final comparison should always state the policy versions, evaluation horizon, population, decision point, outcomes and permissible alternatives. An effect estimated only among successful applicants cannot silently be transferred to all proposals.

Selection and Identification

Alignment of eligibility and observation

Hernán and Robins (2016) propose explicit target-trial emulation as a structure for observational causal inference. Translated to underwriting, a target trial first identifies eligible applications at a predecision time, defines competing decision policies, specifies assignment, follows each application for a common horizon and establishes a statistical analysis plan. Eligibility must use only evidence already available then. Data recorded after a medical review, claim or renewal cannot be treated as baseline information for both policy arms.

An insurer may migrate software in January and classify business written after April as exposed to the new policy. If inclusion requires remaining active until April, early cancellations and failures disappear from the new cohort. Historical version labels and effective timestamps are necessary to align assignment and start of observation. A grace period for supplemental evidence must be explicit and consistent with each strategy.

Conditions for observational identification

With consistency, the actual outcome equals the potential outcome for the actual intervention. Conditional exchangeability means Y(a) is independent of action given sufficient baseline X. Positivity requires a nonzero probability of each action used by the target policy in the relevant baseline strata. The familiar g-formula follows: m(a,x) = E[Y|A=a,X=x] when these conditions and outcome ascertainment hold.

In underwriting, consistency requires specifying what an action includes: requesting an exam may vary in waiting time, privacy burden, vendor, and criteria applied to results. Conditional exchangeability fails when an underwriter's undocumented inspection, broker conversation or intuitive assessment drives both decision and future losses. Positivity fails when an appetite rule always declines a class, a territory or an exposure size. Outcome ascertainment fails when the original policy determines who can produce an insured claim. None of these failures disappears through larger samples of historical contracts.

The relevant overlap is policy-specific. A challenger that differs only among borderline eligible cases may have empirical support even when universal acceptance has none. Define the support set of pairs (x,a) for which the logging policy mu(a|x) is positive. Policy evaluation outside this set requires external evidence, substantive assumptions, a bounded estimand, or new observation. A report must identify the fraction of target probability assigned outside support; dropping such cases changes the estimand.

Selective labels and missing outcomes

Lakkaraju and colleagues (2017) analyze the selective labels problem in evaluating predictive systems when decision processes determine observed outcomes, including insurance examples. The underwriting translation is exact in one respect: policy records supply outcomes only for transactions that occur. The difficulty becomes more complex when coverage terms influence claim incentives or exposure duration. A missing hypothetical loss cannot be imputed as if it were simply a clerical omission.

There are at least four distinct missingness processes. A submitted application may lack a baseline variable; a declined case lacks the insured outcome; a bound policy may have incomplete exposure due to lapse or cancellation; and a covered event may be unreported or still immature. Each mechanism has a different remedy. Data recovery, inverse censoring weights, reserving and counterfactual modeling are not interchangeable.

If a binary event Y(a) is defined for every applicant under acceptance and observed only for a subset, observe fraction q and event rate r within that subset. Without restrictions on missing outcomes the mean lies between qr and qr + 1 - q. The lower bound assumes every missing case would be event-free; the upper assumes every missing case would experience the event. This elementary author-derived bound assumes valid measurement within the observed subset and a binary endpoint. It makes no missing-at-random assumption. For unbounded severity, similarly useful upper bounds require a justified cap or moment restriction; even when the policy limit bounds contract payments, expenses and aggregate exposures may complicate the cap.

A reject-inference model that assigns pseudo-labels using accepted risks can stabilize a numerical prediction yet leave the missing counterfactual unverified. Validation on the same selectively observed population tests internal fit. Only actual supporting evidence or an explicit sensitivity assumption can narrow the set of plausible outcomes for people outside historical acceptance.

Confounding and post-treatment conditioning

For an expedited process, unrecorded health history may influence referral and mortality, while referral determines the evidence acquired, completion and insurance purchase. Conditioning on later medical findings removes the comparability of what was initially known across pathways. Conditioning on acceptance can create associations between underlying risk, affordability and process burden. The correct adjustment set is derived from the assumed data-generating structure rather than selected solely for predictive importance.

A diagnostic study should reconstruct a causal diagram and distinguish common causes of treatment and outcome from mediators, colliders and observations caused by treatment. For longitudinal decisions, intermediate medical or engineering evidence may both inform later actions and respond to earlier ones. Standard regression that treats all such covariates as baseline may answer a different question.

Selection bias has a practical audit signature: records contain detailed variables for referred cases but not for cases accelerated or declined. The difference is evidence about the process itself. Harmonizing schemas without preserving the provenance of missing values can obscure why the variable was unavailable. Analysts should maintain separate flags for not requested, requested but absent, inaccessible, and observed zero.

Competing Decision Policies

Direct modeling, weighting, and augmentation

Consider historical observations (X_i,A_i,Y_i) sampled from a logging policy mu. Under the identification conditions above, a direct estimator averages predicted action-specific outcomes:

V_DM(pi) = mean over i of sum over a pi(a|X_i) m_hat(a,X_i).

A correctly specified outcome model can be efficient, but extrapolation to novel contexts may be hidden inside a flexible model. Prediction accuracy on observed actions is not assurance of accuracy for all candidate actions.

An inverse-probability estimator uses the ratio of new and historical action probabilities:

V_IPW(pi) = mean over i of [pi(A_i|X_i) / mu_hat(A_i|X_i)] Y_i.

With known historical probabilities and appropriate sampling this reweights actions toward the target policy. Large weights amplify noise. Clipping or normalization can stabilize the estimator but changes its finite-sample behavior and potentially its target; the chosen procedure and residual bias must be disclosed.

The augmented estimator combines the above methods. Dudík, Langford and Li (2011), in doubly robust policy evaluation, analyze this family for historical contextual-bandit data. In standard notation,

V_DR(pi) = mean over i of {sum over a pi(a|X_i) m_hat(a,X_i) + [pi(A_i|X_i)/mu_hat(A_i|X_i)] [Y_i - m_hat(A_i,X_i)]}.

Its correction is the weighted difference between observed and modeled reward. Under the relevant causal and regularity conditions, one sufficiently correct nuisance component can support consistency even if the other is misspecified. The expression cannot recover never-observed actions, unmeasured confounding, a poorly defined outcome or market responses outside the target distribution.

A useful algebraic check takes the conditional expectation of the correction given X. If mu_hat equals the true mu, the correction equals the sum over actions pi(a|X)[m(a,X)-m_hat(a,X)]. Adding the modeled term yields the true conditional policy value. If the outcome regression equals m, the conditional residual has mean zero for each action. These two statements explain the robustness property without implying immunity to failure of both components.

For example, an insurer has a randomized triage rule with known 0.8 probability of accelerated review for an eligible stratum. A realized accelerated case contributes weight 1/0.8 = 1.25 to a policy always accelerating that stratum, while a traditionally reviewed case contributes zero. If the acceleration probability is 0.02, the corresponding weight is 50. The difference in stability is a function of design, not simply the accuracy of the outcome model.

Learning versus evaluating a rule

Athey and Wager (2021) study policy learning with observational data under constraints such as simplicity and budgets. The distinction matters because finding a policy with the largest estimated value and reporting its value on the same data produces optimistic selection. A candidate family might include several referral cutoffs, a rule requiring inspection in high-concentration areas and a stochastic rule that sends ambiguous cases to manual review. Training, rule selection and independent evaluation should be separated, or appropriately nested.

A defensible optimization specifies permissible actions, monetary and nonmonetary consequences, processing capacity and group-level monitoring in advance. Governing constraints might require coverage eligibility to follow applicable law, an exposure ceiling, a maximum review backlog and a minimum probability of protective appeal. These are institutional choices with scientific consequences. An optimization cannot declare a legally inadmissible action acceptable because its modeled return is high.

The cross-fitting and orthogonal-score approach of Chernozhukov et al. (2018) supplies methods for nuisance estimation with machine learning. In applications sharing applicants, producers or households across folds, splitting individual records can leak information. Folds should respect natural dependence, and time-separated validation should test changes in underwriting practice. Cross-fitting does not justify unrestricted prediction at a new site or under a new contract.

Randomized process interventions

Randomization can resolve confounding of an assigned process when lawful and ethically justified. Examples include random assignment between two authorized evidence-acquisition procedures, ordering of equivalent review queues, or supplemental audits within an eligible cohort. A trial must define the assignment probability, protocol deviations, consent and legally required protections. It must preserve appeal and prohibit experiments that withhold mandatory coverage or expose participants to unjustified harm.

Intention-to-treat compares all applications by assignment. If some assigned to acceleration are referred and others withdraw, both remain in their assigned groups for the primary comparison. Per-protocol analyses answer different questions and require additional assumptions. Randomization of examination requirements also does not necessarily randomize eventual coverage. The long-term insurance outcome must be interpreted jointly with binding and persistence.

Random holdout audits are especially valuable for comparing information pathways. Their role is to reveal discrepancies between expedited and more extensive underwriting evaluations. They do not, by themselves, establish how every rejected applicant would have fared under insurance, and a more intensive classification is an imperfect benchmark. Changes in testing expectation may alter disclosure behavior; the audit can have an effect of its own.

Quasi-experimental designs

A deterministic eligibility threshold presents a possible regression discontinuity design. Lee and Lemieux (2010) describe regression discontinuity designs and their interpretation conditions. A sharp cutoff identifies a local discontinuity in outcomes if potential-outcome means are continuous at the threshold and units cannot sort in a way that invalidates the comparison. With imperfect compliance, the ratio of outcome and treatment discontinuities can identify a local effect under additional instrumental-variable conditions including exclusion and monotonicity.

Age cutoffs, application score thresholds and face-amount boundaries may be plausible settings, but age simultaneously affects baseline mortality and premium, while coverage amount can be manipulated. The researcher should inspect the score density, continuity of predecision characteristics, alternative thresholds, simultaneous institutional changes and sensitivity to bandwidth and polynomial specification. An effect near a cutoff cannot establish the consequences of eliminating that cutoff for every applicant.

A phased roll-out across branches may permit difference-in-differences if trends would have evolved in parallel absent adoption, there is no anticipation or relevant spillover and composition is sufficiently comparable. A producer or region may receive the new workflow first precisely because of poor recent experience, threatening this assumption. Event-study plots can reveal some preexisting changes but cannot prove counterfactual parallel trends. Staggered adoption with heterogeneous effects warrants explicit cohort-time comparisons rather than indiscriminate pooling.

Assignment to different reviewers may generate an instrumental variable when routing is plausibly exogenous and reviewers differ in decision tendencies. Reviewer expertise may also affect applicant assistance or the contract offered, violating exclusion for a narrow acceptance effect. An estimate based on such an instrument is local to applicants whose decisions respond to reviewer assignment; it should not be advertised as the universal effect of policy replacement.

Uncertainty, Effect Heterogeneity, and Insurance Economics

Worked example: denominators and loss selection

Consider a purely hypothetical cohort of 1,000 eligible applications with two observed baseline risk strata. Seven hundred are classified in a lower expected-loss stratum and 300 in a higher one. Assume all offers bind, no other selection within strata, mutually exclusive groups and a fixed one-year horizon. A bound policy generates 600 monetary units of premium, 100 of expenses and expected claim cost of 300 in the lower stratum or 700 in the higher stratum. A declined proposal has zero insurer cash flow in this simplified example. The assumptions make arithmetic transparent; they are not empirical claims.

The incumbent offers coverage to 90% of lower-stratum and 30% of higher-stratum proposals. A challenger offers coverage to 95% and 60%, respectively.

Quantity Incumbent Challenger
Expected lower-stratum contracts 630 665
Expected higher-stratum contracts 90 180
Expected total contracts 720 845
Expected premium 432,000 507,000
Expected expense 72,000 84,500
Expected claims 252,000 325,500
Expected contribution 108,000 97,000

The protection proxy measured by the share obtaining coverage increases from 72% to 84.5%. Expected insurer contribution decreases by 11,000, or 11 per initial application. Claims per issued policy increase from 350 to approximately 385.21. The expected incremental 125 contracts consist of 35 lower-stratum and 90 higher-stratum exposures. Multiplying their contributions gives 35 times 200 plus 90 times negative 200, which equals negative 11,000. This agrees with the aggregate calculation.

An observer who examined only the lower stratum could report 7,000 incremental contribution and miss the portfolio deterioration. Another observer who compared only claims per contract might miss the protection improvement. The appropriate judgment depends on a specified objective, risk limits, and possible redesign of terms, not on one metric taken in isolation.

If under the challenger the incremental higher-stratum loss were 550 rather than 700, its contribution per contract would become negative 50. The incremental comparison would then be 7,000 minus 4,500 = 2,500, assuming incumbent cohort losses remain unchanged and only marginal newly covered cases have this different expected loss. The threshold at which the incremental comparison breaks even is 500 + 7,000/90, or about 577.78. This calculation demonstrates how unidentified marginal costs, rather than average historical costs alone, determine whether expansion is financially favorable.

Worked example: partial identification

Suppose 600 of 1,000 applicants were historically insured under the action being evaluated. Among them 60 experienced the defined one-year insured event. Under the earlier binary potential-outcome assumptions, q = 0.6 and r = 0.1. The possible mean event frequency if all 1,000 were covered ranges from 0.06 to 0.46. A conventional estimate of 10% simply extends the observed rate to the unobserved 400.

Suppose additional credible evidence supports an event-frequency interval of 0.15 to 0.25 among the unobserved cases. Then the overall frequency lies between 0.6 times 0.10 + 0.4 times 0.15 = 0.12 and 0.6 times 0.10 + 0.4 times 0.25 = 0.16. The narrower interval derives entirely from the additional assumption or evidence. It should be labeled separately from uncertainty due to sampling.

Sampling uncertainty, identification uncertainty, policy uncertainty, model uncertainty and future regime uncertainty represent different layers. Confidence intervals can characterize estimation variability under a model but cannot by themselves establish validity of exchangeability or resolve missing counterfactual outcomes. Bounds and alternative scenarios should accompany point estimates where warranted.

Distributional rather than average effects

A positive mean benefit may coexist with negative effects among applicants with sparse data, unusual occupations or limited access to examinations. The analyst should distinguish pre-specified scientific strata from exploratory segmentation. Small groups need uncertainty reporting and, where appropriate, hierarchical shrinkage. Extremely precise-looking subgroup results derived from few claims are unstable. Group comparisons require reliable denominators and attention to missing eligibility data.

Catastrophe and commercial insurance require an additional layer because events affect multiple policies simultaneously. A portfolio of individually profitable risks can be unattractive if it occupies a concentrated geographic zone or adds exposure to a common supplier. Expected total loss is additive, but tail measures and capital constraints depend on joint distributions. Compare policy-level accumulation, scenario losses and reinsurance response using the same exposure and event definitions.

Selection, demand and market equilibrium

The fixed-population value expression is an appropriate short-run estimand only when the application distribution and decision context remain stable. Announcing a new accelerated process, expanding property appetite or changing prices can change who applies, who purchases, and which competitors respond. To distinguish these channels, index the applicant distribution by policy, F_pi. Market-level value is then an integral of conditional outcomes with respect to F_pi, and conditional outcomes themselves may depend on aggregate demand, contract choices and market conditions.

Einav, Finkelstein and Cullen (2010) show in their insurance pricing and welfare study how identifying price variation together with demand and cost information supports welfare analysis in employer-provided health insurance. Their setting does not establish underwriting effects for life or property insurers. Its methodological importance here is the distinction between average cost and the expected cost of the marginal purchasers affected by a policy.

Einav and Finkelstein (2011) discuss selection in insurance markets and the possibility of advantageous as well as adverse selection. An underwriting change can attract people whose willingness to purchase arises from risk aversion, anticipated need, reduced process burden or changes in broker routing. The direction cannot be inferred exclusively from the change in coverage rate. Behavioral responses and the meaning of the target population require independent study.

A useful decomposition of a change in expected contribution separates a rule effect at the original distribution from a composition effect at the new rule. Where feedback changes the conditional outcome mechanism itself, add a separate environment effect relative to a specified reference. This decomposition is accounting for modeled distributions, not causal identification by itself. Each term needs evidence; interactions and the order of decomposition matter.

For example, reducing delay might raise overall demand while disproportionately attracting applicants who would otherwise be unwilling to undergo medical examination. A competing insurer might lower its prices, altering both the number of applicants and take-up. A static backtest leaves those responses outside its scope. A market-level analysis may use randomized price or process variation where lawful, longitudinal market data and a structural demand-and-cost model, with out-of-sample validation of behavioral assumptions.

Implementation Feedback and Ethics

The actuarial and supervisory boundary

United States actuarial guidance in the adopted 2005 ASOP No. 12 addresses risk classification, including assessment of effects of changes and adverse selection. Its professional scope is relevant when an actuary designs, changes or reviews classification systems. The ASOP No. 56 adoption announcement (ASB, 2020) describes model-related actuarial services and the October 1, 2020 effective date, with holdout data included among possible validation approaches. These professional documents supply governance context, not a causal estimator or universal law. The adopted No. 12 text was inspected; the No. 56 discussion here rests on the official announcement rather than an independently read complete final standard.

The NAIC artificial intelligence topic page describes the model bulletin adopted in December 2023 and the continuing obligation to comply with applicable insurance laws. A model bulletin is not automatically an identical binding rule across every US jurisdiction. A real trial should record state, product, applicable text, adoption or effective dates, and consumer safeguards. The current article does not make a state-by-state adoption claim.

The European supervisory context differs. EIOPA's August 6, 2025 AI governance and risk management opinion is directed to national supervisory authorities and describes a proportionate risk-based approach under insurance-sector legislation. Its existence and described scope were checked on the official page; this study does not infer specific binding requirements from a publication summary. The applicable EU instrument and domestic implementation must be analyzed separately for a particular action, product and date.

Internationally, ICP 16 underwriting policy provisions (IAIS, 2024 online framework) address macroeconomic correlations, emerging risks and interactions with reinsurance. An alternative acceptance rule may therefore affect governance beyond its average loss contribution. ICP language is a supervisory reference rather than a directly enforceable uniform national statute. Cross-country comparisons must distinguish supervisory standards, professional standards, national law and insurer procedures.

Fairness and the meaning of counterfactuals

The counterfactual evaluation of a policy asks about consequences when the insurer changes its actions. Counterfactual fairness asks a different question concerning predictions or decisions when a protected characteristic is varied within a specified causal representation. Kusner et al. (2017) formulate counterfactual fairness using causal modeling. A literal change of a recorded attribute while leaving its descendants fixed is not necessarily the intervention described by such a model.

An analysis of decision fairness should distinguish disparities in observed offers, causal effects of a policy change on groups, individual consistency, permissible risk factors and legally prohibited discrimination. Similar average approval rates do not establish fairness, and different rates do not by themselves diagnose the mechanism. The meaning of each measure depends on the relevant population, classification, legal regime, and the data and causal assumptions.

A trial that improves average profitability by reducing access for an already disadvantaged group may be unacceptable even when estimated margins are precise. Likewise, suppressing all group indicators from an outcome model does not guarantee absence of proxy discrimination. Data use, secure evaluation of sensitive attributes where lawful, and decision-level appeal are separate governance questions. Statistical audit of disparities is an evidentiary process; it does not replace legal or contextual review.

Industry evidence and audit design

The 2024 SOA newsletter article mortality slippage study reports analysis of random holdout and post-issue audit monitoring in US accelerated life underwriting. The evidence concerns risk-class discrepancies translated into estimated future mortality experience. Swiss Re's accelerated underwriting study reports over 37,000 audited cases and variation in modeled slippage across programs. The two accounts show why prospective evidence acquisition is valuable, but their proprietary samples and modeling assumptions are not independently replicated here. They must not be pooled as if they were independent, randomized observations of ultimate mortality.

A random holdout chosen after eligibility screening identifies audit comparisons within the eligibility population, subject to compliance and outcome availability. Post-issue audits observe a population already filtered by issuance and take-up. These designs may answer different questions even with identical risk-class targets. If a reviewer treats every disagreement with full underwriting as truth, the analysis ignores benchmark error, inter-rater reliability and unobserved differences in future outcomes.

An insurer can improve the monitoring value of these designs by storing randomization probabilities, audit eligibility criteria, reasons for noncompliance, versions of both classification systems, and changes made after an audit. Mortality assumptions translating class differences into future losses should be reported separately from observed death claims. Comparison with mature experience requires attention to exposure, attained age, product duration, jurisdiction and claims ascertainment.

Feedback after deployment

An underwriting rule changes the data from which its successor learns. Expanding acceptance creates new observed loss data among formerly excluded classes; restricting acceptance removes such information. A rule that is repeatedly retrained on decisions made by its predecessor can reproduce gaps in observation. A favorable trend in measured calibration may reflect changing eligibility rather than better predictions.

A monitored roll-out should preserve the policy version and a fixed reference cohort definition. Compare actual incoming submissions, referral, offer and bind rates against expected values and against a contemporaneous comparator when feasible. Record changes in pricing and distribution that could modify demand. Retain audit capability and update cost estimates as experience matures. Make changes to capacity and referral procedures explicit rather than silently modifying the intervention.

Escalation conditions should be set before deployment. A rise in unresolved appeals, extreme increases in exposure concentration, deterioration in risk-adjusted outcomes, or material source-data changes can warrant pausing expansion and initiating review. The thresholds must be determined by the institution's lawful obligations and risk appetite, not by a universal numeric cutoff. The rollback plan needs a preserved prior policy, clear responsibility, notice where required, and a means to reconsider affected cases.

For sequential policies, the analyst must distinguish the history at each decision from the actions and information generated earlier. A dynamic policy assigns an action using available history at each stage. Longitudinal causal inference requires sequential exchangeability and positivity relative to each action history, or experimental design supplying probabilities. Multiplying importance weights over many decisions can create very high variance; this provides an operational reason to focus evaluation on well-defined decision points and feasible interventions.

Analytical Synthesis, Evidence Gaps, and Reproducibility

An integrated interpretation of the evidence

The literature supports a distinction between predictive assessment, causal identification and institutional policy evaluation. Predictive assessment estimates associations and conditional distributions under observed arrangements. Causal identification addresses the outcomes of specified alternative decisions under explicit assumptions. Institutional evaluation places those outcomes within the insurer's financial obligations, consumer protection and the behavior of the market. These are established analytical tasks brought together for underwriting; the synthesis is not proposed as a separate named theory.

The first research question concerns the outcomes another decision policy would yield. The answer depends on the target: a fixed set of applicants, a future applicant population or an evolving insurance market. A comparative assessment should report access, insurer contribution and portfolio consequences with common denominators, alongside subgroup outcomes and process costs. The examples establish that broader coverage can coexist with lower expected margin under the same illustrative inputs. Neither direction determines welfare alone.

The second question concerns observational identification. Records can support a policy comparison when the relevant decision alternatives occur within comparable baseline strata, when material confounders are measured, when actions and outcomes are consistently defined and when missingness is treated appropriately. When relevant actions never occur, or labels are absent by design, the record alone does not identify the desired counterfactual. An explicit interval, a different target or a prospective study is then scientifically preferable to apparently precise model output.

The third question concerns selection. Insurance applicants and producers respond to contract terms, requirements, speed, expected acceptance and price. Historical applicant distributions are conditional on past arrangements. Changes in participation, selection and portfolio dependence require additional evidence beyond the static policy-value estimator. A credible deployment therefore tests not only the accuracy of a challenger score but also the mechanisms determining who encounters and accepts its decisions.

Reproducible evaluation protocol

An independently inspectable research package for a real insurer study should contain the following information. This proposed operational sequence collects familiar requirements from the methods and governance discussion rather than introducing a novel scientific system.

Record Minimum information Scientific purpose
Cohort definition Eligibility, time zero, unit, horizon, period and jurisdiction Prevent denominator drift
Policy versions Action set, rule, effective times, valid overrides Establish consistency
Assignment Logged propensity or identified comparison design Establish support and comparability
Features Time of availability, provenance, missingness codes Control leakage and confounding
Outcomes Contractual definition, claim development, censoring, exposure Establish measurement validity
Analysis Predefined primary estimand, nuisance models, folds, weights Enable replication
Sensitivity Unmeasured-confounding scenarios, overlap, bounds, alternative outcomes Expose dependence on assumptions
Governance Permissions, lawful scope, restrictions, appeals, limits and monitoring Establish admissibility
Economic response Take-up model, demand shifts, competitive and capacity assumptions Distinguish short-run and market effects

The data dictionary should keep a versioned field for policy offer, actual bind, effective coverage, loss event and financial payment. Preserving those separations avoids the most common semantic error addressed here: treating absence of an insurer claim as evidence that the underlying applicant was safe. The auditor should be able to reconstruct both the operational decision and why the action was available at that moment.

For observational estimation, the analysis should report distributions of action propensities within the target cohort, the amount of unsupported target probability, weight summaries and effective sample size. For nonnegative weights w_i, a descriptive effective sample size is (sum w_i)^2 / sum w_i^2 when the denominator is positive. It describes weight concentration, not an automatic guarantee of independent observations or valid inference. Cluster structure, time dependence and outcome sparsity remain relevant.

Reporting should compare direct, weighted and augmented estimates where each is defensible. Disagreement can reveal sensitivity to nuisance modeling but does not identify which estimator is correct. A negative-control outcome or exposure may help detect bias when a credible no-effect relation exists; it cannot establish the absence of all confounding. Sensitivity parameters should be tied to plausible underwriting mechanisms, such as undocumented review information or unobserved take-up costs.

For a randomized comparison, preserve the assignment record, independently reconcile enrollment counts, show deviations and loss to follow-up, and report uncertainty with the appropriate assignment unit. For a threshold-based design, document the score construction, cutoff history, support on both sides and tests of manipulation. For a rollout comparison, establish timing and identify cointerventions. For a dynamic rule, version decision histories and assess sequential support.

Boundaries of empirical knowledge

The documentary research supports methodological and institutional conclusions, not a numerical assessment of an actual insurer. Conference abstracts, publisher abstracts, official summaries and industry reports were used only for the claims they establish. Some full texts were not retrievable directly; the source ledger records that boundary. Current state-level and country-level legal application, actual policy wording, proprietary application files, ultimate loss development, and independent validation are necessary for a concrete insurer evaluation.

The modeled calculations are fully specified and can be reproduced by arithmetic. They are illustrations of identification, denominator choice and marginal selection. No inference is made that the example's event frequencies, costs, premiums, or acceptance probabilities are realistic calibrations in any jurisdiction.

Future empirical work should prioritize ethically admissible evidence collection among currently unsupported actions, explicit logging of assignment probabilities, maturity-adjusted outcome follow-up and market demand information. A more complex model cannot compensate for the absence of informative observations. The most valuable new data are often those gathered where alternative decisions differ, where uncertainty affects the sign of the decision comparison, and where lawful access to evidence can be improved.

The original analytical contribution is the integration of these source-supported methods with the distinct underwriting units of offer, bind, exposure, claims and portfolio capacity; the distinction between the missing hypothetical insured loss and a zero actual insurer payment; the explicit numerical decomposition of marginal expansion; and the distinction between fixed-cohort and market-responsive estimands. These are interpretations and applications of existing causal, actuarial and economic methods. No scientific priority or named authorial doctrine is asserted.

Conclusion

The scientific evaluation of an underwriting policy begins by specifying the alternative and identifying the consequences that matter. A comparison of observed loss rates under past decisions is insufficient when the intervention changes who is offered coverage, who binds, what contract is obtained and what outcomes become observable. Causal inference provides methods for resolving parts of that problem under explicit assumptions. Insurance economics supplies a complementary account of changing demand and costs. Actuarial and supervisory practice identifies the institutional constraints within which the comparison has meaning.

The most defensible conclusion is conditional: within a defined applicant population and a supported decision set, a policy comparison can estimate changes in financial and protection outcomes using an appropriate experimental or observational design. Unsupported actions and missing potential losses require bounded conclusions or additional evidence. When the policy affects the applicant pool or portfolio dependence, the target changes and short-run estimates must be supplemented by demand, accumulation and behavioral analysis.

Evaluation quality depends on the correspondence between a scientific question and its exposure denominator, evidence structure and assumptions. An insurer can measure access and profitability simultaneously without confusing them, examine selection without attributing causality to correlations, and preserve uncertainty without abandoning a well-defined comparison. The practical scientific consequence is that underwriting decisions become assessable at the level at which they are actually made: the choice of an admissible policy and its consequences for applicants, contracts, the insurer and the insurance market.

References

  • Hernán, M. A., & Robins, J. M. (2020). Causal Inference: What If. Chapman & Hall/CRC. Author-maintained online edition. Source record

  • Hernán, M. A., & Robins, J. M. (2016). Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available. American Journal of Epidemiology, 183(8), 758–764. doi:10.1093/aje/kwv254. Source record

  • Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017). The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. doi:10.1145/3097983.3098066. Source record

  • Dudík, M., Langford, J., & Li, L. (2011). Doubly Robust Policy Evaluation and Learning. Proceedings of the 28th International Conference on Machine Learning. Author preprint arXiv:1103.4601. Source record

  • Athey, S., & Wager, S. (2021). Policy Learning With Observational Data. Econometrica, 89(1), 133–161. doi:10.3982/ECTA15732. Source record

  • Einav, L., Finkelstein, A., & Cullen, M. R. (2010). Estimating Welfare in Insurance Markets Using Variation in Prices. Quarterly Journal of Economics, 125(3), 877–921. doi:10.1162/qjec.2010.125.3.877. Source record

  • Einav, L., & Finkelstein, A. (2011). Selection in Insurance Markets: Theory and Empirics in Pictures. Journal of Economic Perspectives, 25(1), 115–138. doi:10.1257/jep.25.1.115. Source record

  • Lee, D. S., & Lemieux, T. (2010). Regression Discontinuity Designs in Economics. Journal of Economic Literature, 48(2), 281–355. doi:10.1257/jel.48.2.281. Source record

  • Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. Econometrics Journal, 21(1), C1–C68. doi:10.1111/ectj.12097. Source record

  • Actuarial Standards Board. (2005). Actuarial Standard of Practice No. 12: Risk Classification (for All Practice Areas). Adopted December 2005. Source record

  • Actuarial Standards Board. (2020). ASB Adopts ASOP No. 56. Announcement of Modeling standard, effective October 1, 2020. Source record

  • National Association of Insurance Commissioners. (n.d.). Accelerated Underwriting. Insurance Topics; describes guidance adopted August 14, 2024. Source record

  • National Association of Insurance Commissioners. (n.d.). Artificial Intelligence. Insurance Topics; describes December 2023 model bulletin. Source record

  • European Insurance and Occupational Pensions Authority. (2025, August 6). Opinion on Artificial Intelligence Governance and Risk Management. Official publication page. Source record

  • International Association of Insurance Supervisors. (2024). Insurance Core Principles and ComFrame. Online tool, ICP 16, underwriting policy provisions 16.7.5–16.7.7. Source record

  • Seeman, L., & Herzog, K. (2024, August). Accelerated Underwriting: Mortality Slippage Study and Monitoring Best Practices. Product Matters!, Society of Actuaries. Source record

  • Swiss Re. (2024). Accelerated Underwriting in Focus. Life and Health Risk Trends. Source record

  • Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017). Counterfactual Fairness. Advances in Neural Information Processing Systems, 30. Source record