RESEARCH DOSSIER
Predictive Uncertainty Quantification and Insurance Prediction Intervals
Abstract. Prediction in insurance has two inseparable objects: a specified future insured outcome and a statement about the uncertainty surrounding that outcome. A model that estimates expected claim frequency, conditional claim severity, loss cost, reserve development, or aggregate portfolio payment supplies only a partial answer unless the corresponding future predictive distribution or coverage set is examined. This monograph develops the mathematical meaning, construction, validation, and institutional interpretation of insurance prediction intervals. It begins with the distinction among parameter confidence regions, predictive distributions, individual-outcome sets, and financial risk margins. It then compares parametric, posterior-predictive, bootstrap, quantile-regression, split-conformal, and adaptive uncertainty procedures under their respective assumptions. Original derivations establish the rank basis of finite-sample conformal coverage, the distinction between exact discreteness and approximate nominal levels, and the effect of parameter mixing on count tails. Insurance-specific analysis treats exposure scaling, unreported and censored losses, clustered catastrophe claims, policyholder heterogeneity, delayed labels, contractual limits, portfolio reweighting, and nonstationary observation processes. An explicitly reproducible synthetic comparison demonstrates how two count distributions sharing a mean can imply materially different upper prediction bounds. The central conclusion is that an insurance interval must be identified by its target, horizon, sampling unit, information set, coverage proposition, evaluation population, and observed validation record. Statistical coverage is a property of a stated procedure and data environment; it cannot be inferred from the nominal percentage printed beside an estimated range. This work contributes an integrated, critically assessed synthesis of established methods and transparent original analytical comparisons. No new theoretical system, estimator, or empirical industry performance claim is introduced.
Keywords: Insurance prediction intervals; predictive uncertainty; conformal prediction; marginal coverage; conditional coverage; exchangeability; parametric bootstrap; posterior predictive distribution; predictive distributions; count models; claim severity; claims reserving; covariate shift; temporal dependence; actuarial data quality; calibration; interval score; uncertainty quantification.
Introduction
An insurer may know the fitted mean of a claim-count model and still be unable to determine how frequently the next policy will produce zero, three, or ten claims. The same distinction applies to bodily-injury severity, the ultimate cost of an open claim, the insured payment on a commercial property, and the annual total of a reinsurance layer. A point prediction condenses information; an uncertainty statement must describe the distribution of possible outcomes and the process by which that distribution was estimated. The scientific object of this dossier is the validity and utility of prospective intervals and prediction sets for insurance responses.
A prediction interval is operational only after the predicted object has been defined. A 90% interval for one randomly selected claim is not interchangeable with a 90% interval for one policy-year's total payments, a portfolio's aggregate loss, a calendar year's incurred claims, or an estimate of a reserve parameter. Their sampling units and randomness differ. The scope belongs primarily to the published Insurance Statistics and Predictive Modeling Science and intersects actuarial inference, pricing, reserving, risk modeling, and claims science while retaining a single Dossier identity.
Classical statistical treatments of probabilistic forecasting emphasize predictive distributions and coherent evaluation. Gneiting and Raftery (2007) analyze calibration, sharpness, and proper scoring rules; their work provides a foundation for explaining why an interval must be assessed both for coverage and for discriminatory precision. Lei et al. (2018) establish regression predictive inference procedures with finite-sample guarantees under exchangeability-related assumptions, while the conformal prediction synthesis by Angelopoulos and Bates (2023) details algorithmic procedures and limitations. A guarantee about exchangeable observations is scientifically meaningful, but insurance data frequently arise through selection, contract features, delayed reporting, calendar trends, and dependence that must be investigated before invoking it.
The present research uses a formal target-definition approach, analytical derivations, a source-by-claim review of statistical literature and authoritative actuarial guidance, and synthetic numerical examples whose assumptions and arithmetic are displayed. It is a methods monograph, not a performance evaluation on proprietary insurer observations. Claims about regulatory practice are limited to the issuing jurisdictions and documents reviewed. The initial uncertainty problem can be written as follows: from training observations and information , construct a data-dependent set intended to contain a future observed response . The meaning of "intended" becomes precise only after specifying the probability statement and the relevant stochastic mechanism.
Forecast Uncertainty and Interval Meaning
The insurance prediction target
Represent a policy-period record by , where contains risk characteristics available at prediction time, is the target response, indexes calendar or development time, measures exposure, and denotes contractual features such as deductible, occurrence limit, attachment, or aggregate cap. This is a bookkeeping representation, not an assertion that these variables are independent. For claim counts, takes values in the nonnegative integers. For ground-up severity, ; for paid severity, the relevant response may be , where is a deductible and is a payment cap. A prediction interval that ignores which variable is being predicted changes its subject even when its numerical endpoints look plausible.
The prediction target may be a claim observed during the next twelve months, an ultimate incurred amount on a claim open today, a loss incurred this accident year but reported later, or an insurer payment after contract modification. The censoring and observation rule is essential: reported-only claim severity follows a different selection mechanism from all events including zero-payment or unreported occurrences. Conditioning on a claim having been reported selects a different population from predicting a policyholder's future insurance payment before any claim occurs. A statistically valid interval for the reported-claim subpopulation need not have target-population coverage for all policyholders.
For nominal coverage, the ordinary marginal statement is
The probability includes randomness in the training or calibration sample and in the future case, under the stated sampling regime. A conditional statement at a fixed risk profile is stronger:
Further conditioning on the realized training data asks an even different question. Reporting "90% coverage" without identifying which probability was evaluated discards the central inferential information.
Confidence, prediction, tolerance, and capital quantities
A confidence interval for , an unknown parameter of a claim model, aims to satisfy over repeated samples. A prediction interval concerns an as-yet-unobserved response, and must account for process variation even when were known. In a Gaussian location model with known variance , a nominal central interval for the next has width governed by ; when the mean is estimated from independent observations, the classical predictive standard deviation is , which incorporates estimation of the mean. Replacing it with would silently substitute uncertainty about the mean for uncertainty about the next outcome.
A tolerance interval seeks to cover at least a stated fraction of an underlying population with a specified confidence level; its two probability levels must both be reported. A predictive distribution is richer than any single interval because it permits alternative quantiles, tail thresholds, event probabilities, and decisions. For discrete counts, there may be several defensible sets with the same or greater coverage; an interval can be contiguous in integers, while a general prediction set need not be.
Statistical prediction intervals also differ from accounting and economic risk margins. In IFRS 17 terminology, a risk adjustment for non-financial risk is the compensation an insurer requires for bearing insurance risk. That accounting concept addresses the entity's risk-bearing compensation within an applicable reporting framework. A 95% predictive quantile and a risk adjustment are constructed for different estimands and cannot be equated merely because both respond to uncertainty. The former may inform an assessment; it does not mechanically define the latter.
Sources of predictive variation
Process uncertainty refers to variation in future events given a data-generating law: the next claim may differ from the model expectation even if every parameter is known. Estimation uncertainty arises from inferring parameters, regression functions, distributional shapes, or model choices from a finite sample. Structural uncertainty concerns selection among models with differing assumptions, aggregation structures, or tail behavior. Measurement and reporting uncertainty concerns what the records reveal about actual insured events. Distribution shift concerns the mapping from historical to future populations. Dependence concerns the joint probability law: repeated observations from one policyholder, multiple claims in one catastrophe, or claims sharing a development period do not automatically contribute independent calibration information.
These categories are analytically useful but rarely separable by empirical fiat. A Bayesian posterior predictive distribution integrates over parameter uncertainty under a specified prior and likelihood; a frequentist bootstrap approximates repeated-sample variation under a resampling design; conformal calibration uses ranks of suitable scores to obtain certain coverage guarantees. None is a universal substitute for defining the observable response and its mechanism.
Prediction decisions and interval asymmetry
For a severity distribution with heavy right tail, a symmetric interval around a conditional mean can assign negative monetary endpoints, omit plausible extreme payments, and consume width inefficiently. One-sided upper prediction bounds are often more relevant for liquidity planning or claim triage; two-sided intervals serve other questions. For a zero-inflated response, a set containing the mass at zero and a disjoint interval of positive values may be operationally interpretable, even when a contiguous numerical interval seems more familiar. Contractual caps can create a point mass at the cap; it should remain visible in the predictive law.
The scientific sequence is target selection, probability statement, admissible prediction set, loss or decision criterion, fitting, calibration, and verification. This order prevents model convenience from silently defining the insurer's question.
Parametric and Bootstrap Prediction Intervals
Parametric predictive laws for counts and payments
Suppose a future policy exposure is and a frequency model assumes . If is treated as known, the exact predictive probability is for . A central quantile interval is selected from the count cumulative distribution and generally has attained coverage above its nominal minimum because probabilities are discrete. A continuous normal approximation can be poor at low frequencies, particularly for the probability of zero claims. When a fitted mean replaces the true mean, the plug-in Poisson predictive law does not automatically incorporate estimation variability; the effect depends on the inferential method and training sample.
A negative-binomial specification can represent count overdispersion relative to Poisson, but its tail behavior depends on parameterization and the empirical mechanism. It is insufficient to regard every observed variance-to-mean ratio above one as identifying a particular negative-binomial law; portfolio heterogeneity, catastrophe clustering, exposure variation, and reporting inconsistencies can produce related symptoms. A structural claim-count model must be tested at the sampling unit for which the prediction is issued.
Severity models may use lognormal, gamma, Pareto-type, mixture, or semiparametric distributions. Transforming a ground-up loss to insurer-paid loss requires applying the contractual function before constructing or evaluating payment intervals; in general, quantiles of a nonlinear transformed aggregate cannot be recovered by casually transforming quantiles of an unmodified individual claim distribution. If is nondecreasing, quantile relationships may simplify with appropriate definitions and continuity conditions, but aggregate caps and sums still require joint modeling.
An exact illustration of tail-model dependence
Consider two fully specified synthetic count models with the same mean of two claims per fixed exposure period. Model A is Poisson with mean . Model B is negative binomial with probability mass , corresponding to shape two and success probability one-half in a common convention. Both have mean two; Model B has variance four while Model A has variance two.
Direct summation gives the following independently calculable values:
| Quantity | Poisson(2) | Negative binomial, shape 2 |
|---|---|---|
| Mean | 2 | 2 |
| Variance | 2 | 4 |
| 0.135335 | 0.250000 | |
| 0.052653 | 0.109375 | |
| 0.947347 | 0.890625 | |
| Smallest integer with | 4 | 5 |
| Smallest integer with | 5 | 6 |
The two models share their expected claim count, yet a reported upper 90% integer prediction limit differs by one claim and their probability of at least five claims differs by more than a factor of two. These numbers come from exact probability sums, not observational insurer data or fitted model comparisons. The implication is conditional: a well-calibrated mean alone cannot adjudicate uncertainty bands. Additional calibration and validation of dispersion and tails are needed.
Parameter uncertainty and posterior prediction
Let be independent Poisson counts with a common rate and comparable exposure, and suppose a gamma prior on has shape and rate . Given total count , the posterior is gamma with shape and rate . For one further equal-exposure count, integrating its Poisson law over that posterior yields a negative-binomial posterior predictive distribution with mean and variance
The first term reflects conditional Poisson variation and the second reflects posterior variation in the rate. For a future exposure factor , the corresponding terms become and . The derivation is a direct application of total expectation and total variance; the resulting interval is valid as a posterior predictive statement under the assumed model and prior, rather than a distribution-free guarantee for every alternative data-generating process.
This example also shows the need to preserve parameterization conventions. Changing from a gamma rate to scale parameter without transforming the predictive formula introduces a systematic error. In practice, a policy-specific rate may change over calendar time, and posterior integration over an inadequate stationary likelihood does not repair the omitted trend.
Parametric simulation and prediction intervals
A parametric predictive simulation repeatedly draws model parameters under an explicitly specified inference law if parameter uncertainty is included, then draws future outcomes conditional on those parameters, applying the actual exposure and insurance contract. A one-sided upper bound can be obtained from the empirical predictive quantile of the simulated responses. Monte Carlo error must be distinguished from the larger modeling error: increasing simulation replications reduces computational quantile noise without validating the assumed frequency, severity, dependence, or reporting processes.
A plug-in parametric interval uses only a fitted parameter value. A Bayesian posterior predictive interval integrates parameter uncertainty using a posterior law; its interpretation is Bayesian conditional on prior and likelihood. A frequentist predictive interval may instead be constructed with pivots, exact predictive conditioning in special models, or calibration of a repeated-sampling procedure. Merely simulating many future claims at a fixed fitted parameter does not make the interval account for parameter estimation.
Bootstrap procedures and their target
Efron's original bootstrap paper (1979) introduces resampling to approximate sampling distributions; Davison and Hinkley (1997) systematically develop bootstrap analysis and its conditions. For insurance predictions, a bootstrap design must preserve the object that is sampled. Resampling individual claim rows from a portfolio with multiple claims per policy can destroy policy-level correlation. Resampling annual aggregate observations as if they were interchangeable can erase secular trends. A cluster or block bootstrap may be warranted when clusters or periods, rather than claim rows, approximate the independent units.
A computational bootstrap prediction procedure can refit the model to each resample and generate one synthetic future observation from the refitted law. Its empirical percentiles are then candidates for approximate prediction intervals. The procedure relies on the bootstrap's approximation to the sampling and predictive mechanism, and should be checked for small counts, boundary parameters, rare claims, dependent events, and tail estimators with unstable resampling laws. A nonparametric bootstrap that reproduces only observed maximum severities cannot, without a suitable tail model, learn beyond the observed empirical support; this matters acutely for rare large insurance losses.
For a count response, rounding a continuous upper bootstrap quantile changes the achieved discrete coverage. For insurance payments, simulated deductible, limit, and reporting transformations must occur for each bootstrap draw at the correct unit. Repeated tuning of the model and uncertainty construction on the final holdout data makes its apparent coverage optimistic.
Claims reserving as a boundary case
A reserving procedure often predicts aggregate unpaid development, rather than one additional individual claim. Historical triangles contain linked accident-year and development-year information, and the proper unit of replication is not a randomly selected triangle cell. Mack's reserving uncertainty paper (1993) derives standard errors for chain-ladder reserve estimates under specified distribution-free moment assumptions. A standard error for a reserve estimator is not by itself a calibrated interval for the future ultimate aggregate; such an interval additionally requires an appropriate predictive construction and coverage statement. The distinct dossier on stochastic reserving can examine the full triangle models; the present treatment retains this as an inferential boundary.
Conformal Prediction Sets
Exchangeable ranks as the source of finite-sample coverage
The core advantage of split conformal prediction is an exact finite-sample rank argument that applies to many fitted predictive models. Partition data into a model-training set and a calibration set of cases; fit a predictive model using only the former. Let be a real-valued nonconformity score evaluated on each calibration case, with larger values representing a poorer fit. For the future case, define using the same pre-fixed score and fitted model. If the calibration-and-test observations are exchangeable conditional on the training stage, their score vector is exchangeable. With distinct scores, the rank of the test score is uniform on the integers from one through .
Choose . When , let be the -th smallest calibration score, and construct
When , use , yielding an uninformative all-outcomes set unless another construction is specified. Exchangeable rank symmetry gives
subject to tie handling and the convention for randomized versus conservative sets. With distinct scores, actual marginal coverage for the threshold construction is ; ties commonly make deterministic inclusion conservative. This conclusion is derived from rank symmetry and does not require a correctly specified conditional regression function, provided the exchangeability and independence-of-calibration-from-fitting conditions are satisfied.
For calibration observations and target 90% coverage, . Under a continuous exchangeable score law, the rank argument yields exactly marginal coverage. For target 99% coverage at the same sample size, , and the unrandomized split-conformal construction must use an infinite threshold. This finite-calibration resolution limit is practically significant for thin insurance portfolios and rare coverage classes.
The literature's foundational algorithmic development is described in Vovk, Gammerman, and Shafer (2005). The rigorous regression treatment in Lei et al. (2018) and the accessible Angelopoulos–Bates introduction provide the conceptual basis for these standard derivations. A data-dependent calibration score chosen after observing the same calibration labels can invalidate the simple rank guarantee unless selection itself is covered by a valid procedure.
Residual conformal prediction
For a point predictor , a common score is , giving . The guarantee concerns coverage of the response under exchangeability. There is no automatic guarantee that the interval is short, that it is conditionally valid at every , or that its endpoints respect nonnegative or integer outcome constraints. For nonnegative claim amounts, intersecting with the known outcome support preserves coverage because the true target lies in that support; it also improves presentation. For integer counts, report the corresponding admissible integer set, with careful rounding conventions.
A weighted or normalized residual score such as can adapt interval width to estimated heteroscedasticity when the scale function is fitted without looking at the calibration labels used for final ranking. A poorly estimated scale harms efficiency while the marginal exchangeable rank guarantee can survive if the score construction respects the split and symmetry conditions. This illustrates the distinction between formal coverage validity and empirical interval usefulness.
Conformalized quantile regression
Romano, Patterson, and Candès (2019) combine lower and upper conditional quantile regressions with split conformal calibration. Fit and on a proper training set, and calculate the calibration score
With the conformal score threshold , the interval is
The conformal threshold can in some implementations be negative, shrinking conservative initial quantile bands. Its use does not convert marginal coverage into universal feature-conditional coverage. Quantile regression itself traces to Koenker and Bassett (1978), whose regression-quantile formulation permits asymmetric conditional outcome distributions.
For an insurance claim with many exact zeros and positive long-tailed payments, a conditional quantile model may be better shaped than a symmetric residual band. Nevertheless, quantile crossing, excessive smoothing over contract limits, and insufficient high-quantile observations can harm efficient coverage. Conformalization calibrates the empirical score distribution under its assumptions; it cannot create information about a future catastrophe mechanism absent from the calibration sample.
Two-stage frequency–severity prediction in contemporary actuarial research
Recent insurance-specific research advances beyond generic continuous-response conformity. Graziadei et al. (2026) extend split conformal prediction to two-stage frequency–severity insurance models, reporting evaluations using simulated and real datasets and proposing an out-of-bag adaptation for random-forest severity prediction. The work was first available online in 2025 and appears in the 2026 journal volume. Its contribution is especially relevant to compound insurance outcomes where the frequency process includes a zero mass and the positive severity process has a distinct conditional law. This published empirical-methodological contribution should be distinguished from the general distribution-free rank derivation above: specialized modeling and selection rules may create further requirements for the construction and interpretation of insurance prediction sets.
Three methodological questions emerge for applying two-stage insurance conformal procedures. The conformity score must be defined for the entire declared future payment, including zero frequency where appropriate, rather than only for claims already observed with positive severity. Training, out-of-bag fitting, and any remaining calibration mechanism must preserve the conditional symmetry or other assumptions required by the stated coverage guarantee. Finally, empirical predictive coverage should be evaluated separately for policy-year totals, conditional positive severity, and shifted or catastrophe-affected cohorts. These are analytical evaluation requirements inferred from the statistical object, not claims that the cited paper experimentally resolved every deployment scenario.
Discrete, censored, and structured outputs
An ordinary set construction may output several count values or a noncontiguous subset when using likelihood-style or cumulative-probability scores. If operational requirements demand a contiguous integer interval, a hull of the set can be reported; its coverage is at least that of the underlying set when the hull includes the original set, but its increased width must be recorded. A censored claim outcome cannot be treated as an observed exact severity without violating the score definition. One must either redefine the target as the observable censored variable or employ methods justified for incomplete labels and the censoring process.
Prediction of several future claims simultaneously requires a joint coverage statement if the intended set must contain every response at once. Independent marginal 90% intervals for ten claims do not imply a 90% chance that all ten claims lie inside their intervals. A portfolio-wide simultaneous set needs multiplicity-aware calibration or a suitable joint nonconformity score. A marginal interval for a randomly selected claim and a simultaneous interval for all claims are different statistical artifacts.
What conformal validity does and does not say
The distribution-free phrase is conditional on the sampling conditions; it means the finite-sample guarantee does not depend on specifying a particular response distribution when its structural assumptions hold. It does not mean validity for arbitrary shifts, unobserved contract changes, selective labels, or adversarial time trends. A conformal interval also need not coincide with a highest-density predictive region, a Bayesian credible set, an accounting confidence level, or a solvency-capital measure.
The distinction becomes especially important when management wants "the upper 99.5% claim amount" for a rare loss: a marginal distribution-free coverage mechanism may yield an extremely wide or uninformative set at sparse calibration sizes. That result is evidence about the available inferential resolution, not a failure to produce a desired business number.
Exchangeability and Coverage Conditions
Sampling assumptions as part of the conclusion
Exchangeability means that the joint law of the observations remains invariant under permutation. Independent and identically distributed samples are exchangeable, but exchangeability is the weaker mathematical condition used in standard conformal reasoning. Insurance data become nonexchangeable when calendar period, underwriting acceptance, geographical risk, contract wording, data collection method, or claims-reporting rules systematically change the joint distribution.
The most direct insurance mistake is to split a database randomly at the claim-row level when the intended prediction is for new policyholders in a future year. If the same policyholder contributes past and future claims across training and calibration records, dependence can make the effective test population unlike the randomly sampled held-out rows. Likewise, multiple storm claims sharing one event date and spatial footprint may form a dependent cluster rather than independent calibration points. Exchangeability should therefore be evaluated at the unit of the future decision: claim, policy, event, accident year, or portfolio.
A genuinely prospective evaluation should withhold later calendar periods and all information that becomes available only after the forecast origin. A record indicating final settlement, ultimate reserve development, post-loss inspection, or recovery from a reinsurer may leak the future label into the model. Such leakage can make intervals narrow and apparently well calibrated on a reconstructed historical test while failing at real issuance time.
Marginal and conditional coverage have different limits
Marginal coverage averages over the tested feature distribution. A 90% marginal method can achieve a high rate for common low-severity risks while giving materially lower coverage to rare, high-limit commercial risks. Group or conditional coverage must be assessed separately. The theoretical limits of distribution-free conditional inference established by Barber, Candès, Ramdas, and Tibshirani (2021) explain why finite-sample, assumption-free, nontrivial conditional coverage at every feature value is generally unattainable. A purported guarantee at all continuous feature values should therefore be examined for extra assumptions, restrictions on the class of subsets, or concessions in efficiency.
Group-conditional calibration may be pursued within pre-defined risk classes when each group has enough approximately exchangeable calibration observations. The same rank construction can then apply separately within a group, conditional on the training procedure, but small groups pay with wide or degenerate intervals and high sampling variability. Post hoc identification of groups that happened to underperform on the same evaluation data introduces multiple-comparison and selection complications; exploratory diagnostics should be followed by fresh prospective validation.
Covariate shift and weighted calibration
Under covariate shift, the feature distribution changes between historical training and future deployment, while the conditional outcome law is maintained. Tibshirani et al. (2019) construct weighted conformal methods under suitable likelihood-ratio information. In insurance, this might represent a forecast for a renewal population whose geographic or vehicle composition changes while the risk mechanism within each feature class remains stable. Correct transport requires credible overlap: a future risk class absent from historical data cannot be calibrated purely by reweighting existing observations.
Estimated weights introduce their own uncertainty. If a new exclusion, climate exposure, safety regime, or reporting rule changes , the covariate-shift assumption has failed even when observed feature weights are known exactly. No importance-weight correction of the covariate margin repairs a changed conditional claims mechanism without further modeling assumptions.
The beyond-exchangeability analysis of Barber, Candès, Ramdas, and Tibshirani (2023) develops robustness bounds for certain weighted and nonsymmetric conformal procedures. Such bounds quantify departures from ideal exchangeability under specified constructions; they should not be cited as an unrestricted guarantee for all dependence, unmeasured shift, or missing-label processes.
Time adaptation and its correct claim
In sequential prediction, calibration scores arrive over time and the underlying insurance portfolio may evolve. Gibbs and Candès (2021) develop adaptive conformal inference with a long-run frequency notion under distribution shift, while Angelopoulos, Barber, and Bates (2024) study online procedures with decaying step sizes. Their online coverage objects must be stated precisely. A long-run average of inclusion indicators can converge to a target while individual late-period forecasts experience temporary undercoverage. Therefore, retrospective average coverage under arbitrary sequences is not interchangeable with conditional protection for the next particular insurance contract.
Insurance response labels may arrive months or years after the forecast is issued. Updating calibration on only rapidly settled claims can select lighter and simpler losses while large open liabilities remain unresolved. A sequential calibration scheme must define when a label becomes known, how pending outcomes are handled, and how revisions are recorded. Delayed feedback is part of the statistical learning problem.
Grouped and Shifted Insurance Data
Exposure normalization and the unit of exchangeability
A policy with three months of risk exposure is not naturally comparable to one with twelve months if the response is annual claim count. An exposure offset in a conditional Poisson or related count model may address the expected count, but simple absolute residual scores can remain heteroscedastic across exposures. Calibration must reflect the intended future exposure distribution and the outcome unit. If the future target is a per-policy paid amount, calibrating on claims conditional on occurrence misses the zero-payment mass. If the target is a policy-year total, multiple claims of one policy-year should be aggregated before applying a policy-year score.
A synthetic unit-invariance check is straightforward: if exposure is doubled while all risk intensities remain unchanged in a Poisson model, the expected count doubles from to , and the variance also doubles. An interval designed only by multiplying the endpoints of a count interval by two does not generally produce the correct Poisson predictive quantiles, because variance scales with exposure rather than with its square. This basic result identifies a common source of misuse in portfolio conversions.
Clusters, catastrophe events, and dependence
A catastrophe can generate hundreds of claim rows with one latent event shock. Counting each row as an independent calibration case exaggerates the empirical sample size and conceals a shared tail driver. For predicting a new catastrophe event, event-level calibration scores or a model for event-conditioned aggregates may be more appropriate. For predicting the number of claims from a new exposed portfolio, the target requires a joint count/exposure/event specification. Dependence is part of the insurance phenomenon rather than a nuisance that can be removed by changing a standard error formula.
Different dependence structures can yield identical marginal claim distributions and very different aggregate probabilities. If two claim indicators each have probability , then regardless of dependence, but
Hence a procedure calibrated separately on individual claim indicators gives no automatic interval coverage for their sum. For regionally correlated property losses, reinsurance accumulation and catastrophe concentration are central to the predictive target.
Underwriting selection and selective labels
Training data generally contain only risks that an insurer accepted, and claims data contain events recorded under a particular claims-handling system. A model trained only on bound policies cannot automatically predict rejected applicants; a severity model trained only on closed claims can underrepresent slowly developing severe claims. Sampling propensity, survival and reporting censoring, missingness patterns, inflation adjustment, and policy wording changes determine what can be learned from historical records. When those mechanisms are identified and modeled, inverse-probability weighting, multiple imputation, survival methods, or joint models may be justified, each with its own assumptions. They do not confer unconditional distribution-free predictive coverage.
The U.S. ASOP No. 23 on Data Quality provides professional guidance on assessing data appropriateness and disclosing limitations; ASOP No. 56 on Modeling addresses model purpose, assumptions, structure, and validation. They are U.S. actuarial standards for applicable professional engagements, not universal international statutes. Their relevance to this scientific problem is methodological: any claimed interval reliability depends on the suitability of the observations and validation procedure for the declared purpose.
Contractual change and economic nonstationarity
Claim inflation, legal settlements, social inflation, repair technology, climate exposure, health care cost trends, and evolving policy terms can change the future distribution. Inflation-indexing the monetary target may improve comparability when a defensible index and valuation basis are chosen; it cannot eliminate shifts in tail shape, covered causes, or claims reporting. A policy-limit cap truncates the insurer payment while possibly leaving underlying economic loss extremely heavy tailed. Prediction sets for covered payment and gross loss should be calibrated and interpreted separately.
International comparability requires identifying currency, claims definition, accounting basis, underwriting year, accident year, reporting year, valuation date, legal environment, and inflation conventions. A nationally estimated historical interval cannot be transferred to another jurisdiction solely because two policies share an English product label.
Worked diagnostic design for a hypothetical insurer
A credible prospective validation of an individual policy-year count interval could proceed through defined observation years. Historical policies through year are used for fitting; a later, clearly separated calibration period supplies scores; year supplies the locked evaluation set. Policyholder identities do not cross the calibration–evaluation boundary where new-policyholder generalization is the target, and catastrophe-event clusters are not split across periods. Features available only after policy inception are excluded. Intervals are constructed from the locked procedure; future claim reports are evaluated after their maturity horizon has been declared.
The study would report overall empirical coverage, number of independent evaluation units, a confidence interval for coverage under a justified binomial approximation when observations are independent, median and upper-tail interval width, region and product group coverage, exceedance magnitude, and interval score. If year is a different distribution, the report would explicitly describe the resulting empirical performance as an external test, not an exchangeable conformal theorem. This is a proposed reproducible design; no such insurer evaluation is represented as having been conducted here.
Coverage Diagnostics and Limits
Coverage is a measurable empirical claim
For held-out cases , define and empirical coverage . This statistic estimates the inclusion frequency on the tested population. Under independent Bernoulli inclusion indicators with common inclusion probability , its variance is . A confidence interval for can be formed using Wilson or exact binomial methods as appropriate; a plug-in standard error alone can mislead at small samples or near the boundary. In clustered or temporally dependent evaluations, the binomial approximation must be replaced or justified at the independent-cluster level.
A nominal 90% interval achieving 88% coverage in 100 independent evaluation cases gives 88 included responses. It does not by itself establish that the underlying procedure is meaningfully miscalibrated, because sampling uncertainty remains; conversely, 90% over an aggregate test set does not establish coverage for every material risk group. The reporting obligation is to present point estimate, uncertainty, sample composition, dependence structure, and the decision-relevant error pattern.
Interval width, sharpness, and proper scores
Unconditionally widening a prediction interval can improve empirical coverage while rendering it useless for underwriting or claims operations. Under the standard two-sided interval score for lower bound , upper bound , observation , and tail probability , define
Smaller values are better, reflecting the tradeoff between width and misses. This form is discussed in the proper scoring-rules study of Gneiting and Raftery (2007). Reporting interval width alone rewards overly narrow intervals, while reporting coverage alone rewards uninformatively wide ones. The score is not a substitute for checking model transportability, regulatory relevance, or tail events that dominate a specific insurer's loss.
For count predictions, interval length can be defined by numerical span or cardinality of admissible integer values, and the choice should be stated. For monetary severity, a $1,000 band and a $1 million band are incomparable across insured limits without context; normalized widths may be useful, but division by small predicted means can be unstable. The score's scale and the choice of transformation or stratification must match the intended operational comparison.
Calibration by frequency band, risk group, and horizon
A complete diagnostic suite includes empirical coverage by claim frequency level, estimated severity band, exposure, policy limit, distribution channel, jurisdiction, and time horizon, subject to sufficient sample sizes. Evaluate one-sided upper misses separately when the operational objective is avoiding underestimation of large claims. Check scores against elapsed development time for reserves, and against event type for catastrophe risks. Repeatedly scanning many cuts of the same holdout data can overstate apparent discoveries; use pre-specified tests and subsequent evaluation periods to distinguish exploratory pattern detection from confirmed undercoverage.
The choice of nominal alpha should be explicit. An upper 99% bound, a two-sided 90% interval, and a simultaneous 95% set address different protection levels. The probability guarantee must reflect whether the future test unit is drawn randomly from the historical population or strategically selected as a high-risk insured. Applying marginal coverage to a purposively selected portfolio of unusually risky cases is a transport problem.
Failure modes and corresponding repairs
Five principal failure categories organize the practical audit. First, target mismatch occurs when claim-level data are used to support policy-level conclusions; repair requires reconstructing the correct observed unit. Second, calibration contamination occurs when fitted models, hyperparameters, or prediction scores are selected using the final calibration or evaluation labels without a valid adjusted procedure; repair requires a genuinely separated workflow. Third, structural nonexchangeability occurs through time shift, claims clustering, or underwriting selection; repair depends on defensible weighted, grouped, sequential, or model-based assumptions and must be evaluated prospectively. Fourth, thin data produce discretized, unstable or infinite prediction bounds; repair may require pooling under explicit assumptions, accepting weaker resolution, or collecting more data. Fifth, model uncertainty can remain even after perfect computation; repair requires scenario comparisons, sensitivity analysis, explicit tail models, and honest limitations.
Prediction uncertainty in insurance is also a governance problem of measurable accountability. A prediction system should preserve the model version, training cutoff, feature timing, calibration-sample definition, risk horizon, covered response, confidence level, realized outcomes, issue date, and revision trail. ASOP No. 56 supplies relevant U.S. professional principles for documenting assumptions and validating models; the statistical methods in this chapter provide the direct inferential content. Neither governance documentation nor a compliant workflow substitutes for the observed coverage record.
Research reproducibility and the exact numerical tests
The synthetic count comparison above is independently reproducible with the following equations and no insurer data:
Compute cumulative sums ; . The reported Poisson , whereas the negative-binomial tail is . The threshold values are and , respectively. The conformal rank example follows at and has no stochastic simulation error. All calculations are illustrations conditional on deliberately stipulated probability laws; none estimates the observed insurance market.
A reader can further check the beta-binomial or gamma-Poisson predictive distinction by the law of total variance, and can check interval-score penalties by inserting observations below, inside, and above the interval. These are mathematical replication tasks rather than claimed validation on operational claims data.
Boundaries of the evidence
The cited statistical papers and books establish the corresponding methods, asymptotic or finite-sample statements under their assumptions, and selected evaluation criteria. The actuarial standard pages establish their stated U.S. professional scope; the IFRS educational glossary establishes the named accounting concept. Several primary papers were examined through publisher abstracts and bibliographic records rather than a complete independent line-by-line full-text analysis; the original derivations and synthetic arithmetic here are presented as self-contained calculations. This monograph makes no claim of new method invention, new insurance data collection, external peer review, actuarial signoff on a live insurer model, or universal enforceability of a national standard.
Conclusion
Insurance prediction intervals are scientifically meaningful when the future insured response and the conditions of validity are explicit. A parameter interval, an individual payment interval, a prospective claim-count set, an aggregate reserve distribution, and an accounting risk adjustment all address distinct quantities. The difference persists even when each is expressed as a single monetary range or percentage.
Parametric and bootstrap procedures can incorporate predictive model structure, exposure, dependence, and parameter estimation when these elements are specified and investigated. Their coverage relies on the adequacy of the generating and sampling assumptions. Conformal methods provide an important complementary route: under suitable exchangeability and data-splitting conditions, ranks establish a transparent finite-sample marginal coverage guarantee independent of a correct regression model. The guarantee is defined for the tested sampling regime, and its substantive usefulness depends on interval efficiency, target comparability, group composition, finite calibration resolution, and stability of the future observation process.
Insurance data present recurrent threats to these conditions through selected policy populations, delayed or censored claims, catastrophe clusters, varying contract terms, exposure heterogeneity, and time trends. Weighted and sequential conformal procedures extend the applicable methods under their own conditions; they do not transfer an unconditional promise to arbitrary future risks. Groupwise and out-of-time evaluation, proper scores, documented modeling choices, and exact source-to-claim evidence are therefore necessary companions to formal coverage mathematics.
The synthetic mean-preserving count comparison establishes a simple but consequential analytical conclusion: identical fitted expected claim counts can produce materially different predictive tails and distinct quantile limits. Forecast uncertainty must be evaluated as a probabilistic target, with measured coverage and sharpness on the population for which the prediction will be used. These conclusions synthesize established statistical and actuarial methods and specify a rigorous basis for future comparative insurance studies without introducing a new theoretical doctrine.
References
Actuarial Standards Board. (2016). Actuarial Standard of Practice No. 23: Data Quality (effective April 30, 2017). https://www.actuarialstandardsboard.org/asops/data-quality/
Actuarial Standards Board. (2019). Actuarial Standard of Practice No. 56: Modeling (effective October 1, 2020). https://www.actuarialstandardsboard.org/asops/modeling-3/
Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4), 494–591. https://doi.org/10.1561/2200000101
Angelopoulos, A. N., Barber, R. F., & Bates, S. (2024). Online conformal prediction with decaying step sizes. Proceedings of Machine Learning Research, 235, 1616–1630. https://proceedings.mlr.press/v235/angelopoulos24a.html
Barber, R. F., Candès, E. J., Ramdas, A., & Tibshirani, R. J. (2021). The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA, 10(2), 455–482. https://doi.org/10.1093/imaiai/iaaa017
Barber, R. F., Candès, E. J., Ramdas, A., & Tibshirani, R. J. (2023). Conformal prediction beyond exchangeability. The Annals of Statistics, 51(2), 816–845. https://doi.org/10.1214/23-AOS2276
Davison, A. C., & Hinkley, D. V. (1997). Bootstrap Methods and Their Application. Cambridge University Press. https://doi.org/10.1017/CBO9780511802843
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7(1), 1–26. https://doi.org/10.1214/aos/1176344552
Gibbs, I., & Candès, E. (2021). Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34. https://proceedings.neurips.cc/paper_files/paper/2021/hash/0d441de75945e5acbc865406fc9a2559-Abstract.html
Graziadei, H., Marques F., P. C., de Melo, E. F. L., & Targino, R. S. (2026). Conformal prediction for frequency-severity modeling. Journal of Applied Statistics, 53(9), 1628–1647. https://doi.org/10.1080/02664763.2025.2567988
Gneiting, T., & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477), 359–378. https://doi.org/10.1198/016214506000001437
IFRS Foundation. (n.d.). IFRS 17 Insurance Contracts—Key terms (educational glossary). https://www.ifrs.org/supporting-implementation/supporting-materials-by-ifrs-standards/ifrs-17/key-terms/
Koenker, R., & Bassett, G., Jr. (1978). Regression quantiles. Econometrica, 46(1), 33–50. https://doi.org/10.2307/1913643
Lei, J., G'Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018). Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523), 1094–1111. https://doi.org/10.1080/01621459.2017.1307116
Mack, T. (1993). Distribution-free calculation of the standard error of chain ladder reserve estimates. ASTIN Bulletin, 23(2), 213–225. https://www.casact.org/abstract/distribution-free-calculation-standard-error-chain-ladder-reserve-estimates
Romano, Y., Patterson, E., & Candès, E. J. (2019). Conformalized quantile regression. Advances in Neural Information Processing Systems, 32. https://proceedings.neurips.cc/paper_files/paper/2019/hash/5103c3584b063c431bd1268e9b5e76fb-Abstract.html
Tibshirani, R. J., Barber, R. F., Candès, E. J., & Ramdas, A. (2019). Conformal prediction under covariate shift. Advances in Neural Information Processing Systems, 32, 2530–2540. https://proceedings.neurips.cc/paper/2019/hash/8fb21ee7a2207526da55a679f0332de2-Abstract.html
Vovk, V., Gammerman, A., & Shafer, G. (2005). Algorithmic Learning in a Random World. Springer. https://doi.org/10.1007/b106715