Insurance by BogdanovaThe Encyclopedia of Insurance and Risk Science

RESEARCH DOSSIER

Missing Data, Truncation, and Censoring in Insurance Analytics

Abstract. Insurance datasets represent a sequence of contractual, behavioral, administrative, and statistical selection events. A missing underwriting field, an unreported claim, a claim observed only above a deductible, a payment capped by a policy limit, and an open claim whose final settlement is unknown contain different information about the underlying risk. Treating them as interchangeable incomplete records changes the estimand, the likelihood, and frequently the economic interpretation of insurance outcomes. This monograph develops an integrated analysis of item and unit nonresponse, event-time censoring, left and right truncation, contractual payment transformations, delayed reporting, incomplete claim development, and the assumptions required to identify frequency, severity, duration, and aggregate loss characteristics. The analysis derives observation-specific likelihood contributions; distinguishes complete-case, weighting, imputation, survival, selection-adjusted, and partially identified approaches; and evaluates concrete consequences for underwriting, pricing, claims operations, and reserving. Two transparent numerical examples demonstrate the direction of selection bias in recorded insurance costs and the effect of deductibles and limits on observable severities. The central finding is that recoverability depends on the observation mechanism, the population at risk, information available outside the recorded sample, and the support of the observed data. Statistical sophistication cannot recover absent information without defensible identifying assumptions. The monograph provides a reproducible research and audit protocol, separates mathematical claims from source-specific empirical findings, and maps the results to established actuarial data-quality responsibilities. It presents original interpretive synthesis of existing statistical and actuarial literature without asserting a newly established theory, empirical discovery, or proprietary insurance dataset.

Keywords: Insurance analytics; missing data; MCAR; MAR; MNAR; sample selection; truncation; censoring; right censoring; left truncation; policy limits; deductibles; claim severity; reporting delays; IBNR; multiple imputation; inverse probability weighting; survival analysis; actuarial data quality; identification; sensitivity analysis.

Introduction

The observation process as part of insurance risk measurement

Insurance data are generated through a succession of institutions and events. An application is accepted or rejected; coverage is bound or lapses; an exposure generates a loss; the loss is recognized, reported, adjusted, paid, reopened, or litigated; a contractual deductible or limit transforms the gross loss into an insurer payment; and an extract records the state of these processes at a specified valuation date. The analytical record is consequently a function of both economic risk and the procedure by which information enters the insurer's systems. A field that is blank in a policy table has a different observation history from a claim whose settlement will occur after the extraction date.

The foundational distinction follows Rubin's (1976) inference analysis for missing data, the product-limit treatment of incomplete event times in Kaplan and Meier (1958), and the censored and truncated distribution framework of Turnbull (1976). The actuarial meaning of these distinctions becomes especially concrete when payments are restricted by deductibles and maximum coverage, as analyzed in Gatti and Wüthrich (2025). The unifying question is how the information available to the investigator relates to the complete economic process being estimated.

This monograph uses four levels of description. The latent process is the risk or event that could exist independently of a dataset. The contractual process maps that event into coverage and payment obligations. The observation process determines which parts of those events are recorded and at what time. The inferential process links recorded data to a precisely stated target. Each level can introduce restrictions, and conflating the levels can change the scientific object under examination.

Population, estimand, and notation

Let ii index insured exposures or claims and tt index time. Write XiX_i for an underlying gross loss, ZiZ_i for observed covariates, RiR_i for an indicator that a value is observed, TiT_i for an event time, CiC_i for a censoring time, and OiO_i for inclusion in a sample or claims extract. Let did_i denote a monetary deductible and MiM_i the available payment limit above that deductible. For a simplified single-loss contract, insurer payment is

Yi=min⁡{(Xi−di)+,Mi},(a)+=max⁡(a,0).Y_i=\min\{(X_i-d_i)_+,M_i\}, \qquad (a)_+=\max(a,0).

This is a contractual transformation, not the same random variable as the gross loss. The aggregate expected gross loss, expected insurer payment, expected payment conditional on a positive payment, and estimated ultimate settlement are distinct estimands. A conditional sample of positive claims supports different inferences from a complete policy-level exposure panel.

In event-time analysis the observed duration and event indicator are, for ordinary right censoring,

T~i=min⁡(Ti,Ci),Δi=1(Ti≤Ci).\widetilde T_i=\min(T_i,C_i), \qquad \Delta_i=\mathbf1(T_i\le C_i).

An administrative database can contain both this form of censoring and missing covariates. Their joint treatment requires an explicit joint or sequential observation model.

Research method and evidence boundaries

The study combines likelihood derivation, conditional-expectation identities, transparent hypothetical calculations, and critical synthesis of primary statistical papers, publisher records, and actuarial standards. It does not estimate coefficients on individual insurer records. The numerical examples are algebraic illustrations, and their parameters are stated rather than inferred. Source verification distinguishes accessible article text from bibliographic and abstract-level publisher evidence. The classical missing-data analysis of Little and Rubin (2002) and the actuarial loss-reserving treatment of Wüthrich and Merz (2008) supply methodological context rather than proprietary insurance observations.

The organization follows the research-design passport for this individual dossier. Its subject is the relation between incomplete data and valid estimation. General survival analysis, the full theory of claims reserving, and insurance model governance have broader independent scientific treatments. Here they are examined to the extent that incomplete observation affects identification, estimation, and interpretation.

Missing Data Mechanisms

Units, variables, and the meaning of absence

Unit nonresponse occurs when a target unit is never represented in the usable sample. Item nonresponse occurs when a recorded unit has one or more missing variables. These are analytically different. An insurer may possess complete premium amounts for issued policies but lack income information for a large subset of applicants. A claims database can omit declined policies entirely; estimating loss rates from that database then creates a selection problem even if each recorded claim has complete fields.

Let W=(Wobs,Wmis)W=(W_{\mathrm{obs}},W_{\mathrm{mis}}) represent complete data and RR its observation pattern. A generic observed-data likelihood can be written

L(θ,ψ;Wobs,R)=∫fθ(Wobs,Wmis) gψ(R∣Wobs,Wmis) dWmis.L(\theta,\psi;W_{\mathrm{obs}},R) =\int f_\theta(W_{\mathrm{obs}},W_{\mathrm{mis}}) \,g_\psi(R\mid W_{\mathrm{obs}},W_{\mathrm{mis}}) \,dW_{\mathrm{mis}}.

The factor fθf_\theta describes substantive insurance data and gψg_\psi describes response or inclusion. Writing both exposes the assumptions otherwise hidden by automatic row deletion.

Rubin (1976) distinguished missing completely at random (MCAR), where the response pattern does not depend on observed or missing values; missing at random (MAR), where dependence on missing values disappears after conditioning on observed data; and missing not at random (MNAR), where dependence on unobserved values remains. Under appropriate MAR and parameter-distinctness conditions the observation mechanism can be ignorable for likelihood inference. These precise qualification conditions appear in Rubin's original 1976 argument. The acronym MAR does not mean that response occurs by simple random sampling, and missingness alone cannot establish the mechanism's category.

Insurance examples clarify the distinctions. A technical failure that omits randomly selected policy records independently of their characteristics approximates MCAR. A survey's income field missing preferentially among younger customers, with age completely observed and appropriately included in a correctly specified model, may satisfy MAR conditional on age and other fully observed predictors. Underwriting loss-history information missing because applicants with unusually severe prior losses decline to disclose those losses may remain MNAR even after conditioning on recorded characteristics. The examples describe possible mechanisms, not assertions that any particular insurer's data follow them.

Why complete cases can change the target

For a response indicator RR, the complete-case mean converges, under conventional regularity conditions, to E(Y∣R=1)E(Y\mid R=1). The target population mean is E(Y)E(Y). Their difference reflects the covariance between outcome and response:

E(Y∣R=1)−E(Y)=Cov⁡(Y,R)Pr⁡(R=1).E(Y\mid R=1)-E(Y) =\frac{\operatorname{Cov}(Y,R)}{\Pr(R=1)}.

The identity is exact when the expectations exist and Pr⁡(R=1)>0\Pr(R=1)>0. It shows that the fraction of missing records is an incomplete description of bias; the relation between response and the desired outcome matters. Even a small missing fraction can be consequential if unusually large losses are systematically absent.

Consider a hypothetical portfolio with 80% low-loss exposures whose average loss is 2 thousand monetary units and 20% high-loss exposures averaging 10 thousand. Its exposure-weighted average is 0.8(2)+0.2(10)=3.60.8(2)+0.2(10)=3.6 thousand. Suppose recorded response rates are 90% and 30%, respectively. Among observed records the low-risk and high-risk masses are 0.720.72 and 0.060.06. The observed-record average is

0.72(2)+0.06(10)0.72+0.06=2.61538 thousand,\frac{0.72(2)+0.06(10)}{0.72+0.06} =2.61538\text{ thousand},

which understates the stipulated population mean by approximately 27.35%. These numbers are entirely constructed; no statistical significance, frequency estimate, or external empirical claim attaches to them. If the two strata and their inclusion rates are genuinely known, inverse probability weighting restores the population mixture. If missing strata are themselves unidentified, the correction cannot be computed from these records alone.

Missing exposures, missing outcomes, and unavailable counterfactuals

An omitted field in a policy row can sometimes be recovered from verified transaction records. A loss that has occurred but has not yet been reported is a latent event whose count and severity must be estimated. A rejected risk that never entered the insurer's exposure population represents a different target-population problem. A counterfactual outcome under a contract never purchased is a causal quantity requiring assumptions beyond ordinary imputation. These distinctions prevent a software routine from assigning the same remedy to all empty cells.

Administrative missingness is often patterned by product, geography, intermediary, distribution channel, claim type, legal reporting requirements, system migration, and valuation date. Analysts should reconstruct the workflow generating each field and record whether the field was never collected, became unavailable, was suppressed by a business rule, was pending, or was censored by a contract. The variable indicating why information is absent may be as analytically valuable as the imputed value itself.

Diagnostics and the limits of mechanism tests

A missingness matrix, response rates by observed risk strata, comparison of source systems, and time-of-extract analyses can reveal systematic nonresponse. They cannot generally distinguish an MNAR mechanism from an MAR mechanism that yields the same observed distribution. Little and Rubin (2002) emphasize the necessity of modeling the missing-data process; the practical consequences of ad hoc deletion and single replacement are explained in van Buuren's imputation study (2018).

Temporal audits are especially useful. A field observed only for policies issued after a software release may be structurally unavailable in older policy years. Coding these historical blanks as if values were selectively concealed by the policyholder invents an observation mechanism. Conversely, treating nonresponse that depends on hidden prior claims as a harmless historical coding issue can conceal selection. The diagnostic report must state both the empirical patterns observed and the assumptions that remain untestable.

Right and Left Censoring

Observation sets rather than invented event values

Right censoring records that an event time exceeds a known threshold. If a policy has not lapsed by the valuation date, its lapse time is greater than elapsed observation time; it is not equal to that time. Left censoring records that an event happened before an observation boundary without its exact time. Interval censoring records that the event occurred within a bounded interval, as when a condition's onset is detected between two scheduled assessments. An open claim's eventual closure date can be right-censored, while its paid-to-date amount is an evolving process and should not automatically be treated as a right-censored ultimate amount.

For a continuous duration TT with density fθf_\theta and survival Sθ(t)=Pr⁡(T>t)S_\theta(t)=\Pr(T>t), an exact observed event at tt contributes fθ(t)f_\theta(t). A right-censored observation at cc contributes Sθ(c)S_\theta(c). Left censoring at cc contributes Fθ(c)F_\theta(c); an event known to occur within (a,b](a,b] contributes Fθ(b)−Fθ(a)F_\theta(b)-F_\theta(a). These are information-set likelihood contributions under an appropriate observation model, and independence/ignorability of the censoring mechanism must be established or modeled.

The analysis of mixed observation intervals is part of Turnbull's (1976) nonparametric incomplete-data estimation. This source derives a self-consistency approach for grouping, censoring, and truncation. In any empirical application, identifiability outside observed intervals remains a separate question.

Survival estimation and its risk sets

For ordinary independent right censoring, the Kaplan–Meier product-limit estimator is

S^(t)=∏tj≤t(1−djnj),\widehat S(t)=\prod_{t_j\le t}\left(1-\frac{d_j}{n_j}\right),

where tjt_j are distinct event times, djd_j events at tjt_j, and njn_j individuals genuinely at risk immediately before tjt_j. This estimator uses the survival information contained in censored records rather than pretending censored times are completed event times. The original Kaplan–Meier article (1958) explicitly recognizes the need to scrutinize assumptions about censoring.

A critical issue in insurance is whether administrative follow-up is independent of the event time conditional on recorded covariates and enrollment. Loss of observation due to product migration, account termination, or transfer of a high-severity claim to another administrator may depend on the same latent risk that drives the modeled event. The resulting survivor curve need not describe the intended population without adjustment.

Delayed entry is an additional risk-set restriction. If coverage enters observation only at duration LiL_i, an individual contributes to the risk set at event time tt only when Li≤t≤T~iL_i\le t\le\widetilde T_i. An analyst who allows a late-entering policy to contribute risk time before it became observable introduces immortal-time or selection distortions. Left truncation requires conditioning on survival to entry, not merely assigning a later event timestamp.

Regression, hazards, and competing outcomes

Cox's (1972) regression models and life-tables develop proportional-hazards inference with an unspecified baseline hazard. In schematic form h(t∣Z)=h0(t)exp⁡(β⊤Z)h(t\mid Z)=h_0(t)\exp(\beta^\top Z); a risk set must account for delayed entry and event/censoring timing. The extended Cox-model treatment of Therneau and Grambsch (2000) discusses model diagnostics, repeated events, and additional structure.

A settlement-time analysis must define the event being modeled. Payment, administrative closure, legal finality, policy lapse, death, and recovery can compete or occur in sequence. Coding a recovery or withdrawal as ordinary independent censoring for settlement may induce bias if its occurrence is related to settlement risk. Multi-state descriptions and cause-specific hazards are sometimes needed, but the selected approach must follow the actual business process and scientific question.

Censoring and missingness together

A life insurer may observe the death date for all reported death claims while lacking a beneficiary characteristic for some claims. The death-time likelihood concerns censoring; the beneficiary-variable model concerns missing covariates. Applying an ordinary complete-case Cox model to beneficiary-complete cases introduces an additional response-selection process. Consequently a method valid for independent censoring alone need not remain valid after complete-case filtering.

It is useful to write the observation information as Oi=(T~i,Δi,Ri,Zobs,i,Li)\mathcal O_i=(\widetilde T_i,\Delta_i,R_i,Z_{\mathrm{obs},i},L_i). A joint likelihood or coherent sequential estimation can then specify which pieces are assumed independent conditional on which variables. The distinction matters more than the brand of software: each implemented risk set and weight corresponds to a substantive assertion about how insurance events become visible.

Truncation and Policy Limits

Sample truncation versus censored measurement

Truncation changes which units enter the dataset. Under fixed left truncation at dd, observations with X≤dX\le d are absent, and a retained exact value x>dx>d has conditional density

fθ(x∣X>d)=fθ(x)Sθ(d).f_\theta(x\mid X>d)=\frac{f_\theta(x)}{S_\theta(d)}.

The denominator adjusts for the probability of inclusion. Censoring, by contrast, retains a unit but records only a bound or interval for its value. The likelihood contribution for an above-limit gross loss known only to exceed bb is Sθ(b)S_\theta(b), subject to the precise data-generating design. Deleting records known to exceed the limit converts an upper-censoring problem into truncation and changes the likelihood.

Insurance contracts make these differences especially important. A per-loss deductible may lead to zero payments being omitted from a claims-payment table; the positive-payment extract is then left-truncated in gross severity and may be conditioned on claim reporting as well. A policy maximum can cause several different gross losses to produce exactly the same capped insurer payment. The observed cap is ordinarily a censored gross-loss amount when the analyst knows that the cap was reached, while the paid amount itself is an exact realization of the contractual payment variable.

This distinction is developed directly in the actuarial lower-truncated and right-censored claims study by Gatti and Wüthrich (2025) and in loss-model uncertainty research by Zhao, Upretee, and Yu (2023). Both discuss modeling insurance severities under deductibles and policy limits; the particular likelihood must still match whether the dataset contains payment-per-payment or payment-per-loss observations.

The full observation likelihood

Suppose a gross loss XX is continuously distributed with density fθf_\theta, a deductible dd, and a fixed gross-loss boundary b=d+Mb=d+M. Conditional on a claim producing a positive payment, an uncapped observed gross loss x∈(d,b)x\in(d,b) contributes

Li(θ)=fθ(x)Sθ(d).L_i(\theta)=\frac{f_\theta(x)}{S_\theta(d)}.

A retained loss reported as exceeding the cap contributes

Li(θ)=Sθ(b)Sθ(d).L_i(\theta)=\frac{S_\theta(b)}{S_\theta(d)}.

These equations assume every underlying loss above dd in the defined sample frame is reported and that an above-cap flag is retained. If the data consist solely of uncapped observations, conditional on d<X<bd<X<b, the appropriate density denominator becomes Fθ(b)−Fθ(d)F_\theta(b)-F_\theta(d). These three sample designs cannot be represented by the same likelihood merely because their payment columns have the same name.

If zero-payment losses are recorded at the exposure level, there is also a mass Pr⁡(X≤d)=Fθ(d)\Pr(X\le d)=F_\theta(d). An aggregate policy-level likelihood may require a frequency distribution as well as a conditional positive-severity model. A claim extract containing only paid events cannot, by itself, reveal the rate of losses never crossing the deductible. Frequencies calculated per recorded positive claim and frequencies calculated per policy exposure answer different questions.

A fully specified exponential severity illustration

Assume, solely for an algebraic illustration, that gross XX is exponentially distributed with mean μ=10,000\mu=10{,}000, that d=2,000d=2{,}000, and that the insurer pays at most M=18,000M=18{,}000 above the deductible. The gross-loss threshold corresponding to the payment cap is b=20,000b=20{,}000. Then

Pr⁡(X>d)=e−0.2=0.81873075,Pr⁡(X>b∣X>d)=e−1.8=0.16529889.\Pr(X>d)=e^{-0.2}=0.81873075, \quad \Pr(X>b\mid X>d)=e^{-1.8}=0.16529889.

If an analyst averages only observed gross losses above dd, exponential memorylessness gives E(X∣X>d)=d+μ=12,000E(X\mid X>d)=d+\mu=12{,}000. Treating this positive-claim average as the unconditional gross mean would overstate the stipulated unconditional mean of 10,00010{,}000 by 20%. The result does not imply the same sign of bias for insurer payment, which is transformed by both the deductible and cap.

For this contract,

E(Y)=∫dd+MPr⁡(X>x) dx=μ(e−d/μ−e−(d+M)/μ)≈6,833.95.E(Y)=\int_d^{d+M}\Pr(X>x)\,dx =\mu\left(e^{-d/\mu}-e^{-(d+M)/\mu}\right) \approx6{,}833.95.

Among positive payments, E(Y∣Y>0)≈8,347.01E(Y\mid Y>0)\approx8{,}347.01. These are exact consequences of the stipulated probability model up to rounding. They show why gross-loss means, payment-per-loss means, and payment-per-payment means must be labeled separately. They make no claim about an actual insurer, pricing sufficiency, or the adequacy of an exponential tail model for extreme losses.

The general expected-payment identity E[min⁡{(X−d)+,M}]=∫dd+MSX(x) dxE[\min\{(X-d)_+,M\}]=\int_d^{d+M}S_X(x)\,dx follows by integrating the tail of the positive transformed payment. It also reveals what changes when a deductible or limit changes: the integration region changes, and any projection beyond the observed gross-loss support requires a severity model or externally justified extrapolation.

Nonparametric recovery and its support limits

The statistical treatment of random truncation precedes modern insurance datasets. Lynden-Bell (1971) developed a selection-adjusted distribution method for observations subject to known selection, while Woodroofe (1985) analyzed distribution estimation with truncation. Wang, Jewell, and Tsai (1986) established product-limit asymptotics under truncation. The transfer to insurance requires an accurate observation rule and sufficient overlap across entry thresholds.

A fixed deductible shared by every observed positive claim leaves the distribution strictly below dd unobserved. Nonparametric positive-claim data alone do not determine its probability mass or shape. An observed frequency denominator, a lower-deductible portfolio, or an explicit parametric or semiparametric assumption may supply additional identification. Similarly, a fixed gross-loss cap observed only as a cap flag does not nonparametrically identify the shape of the tail above the cap. A risk measure such as an extreme high quantile or expected shortfall can therefore be profoundly sensitive to the tail assumption even when the fit to payments below the cap looks excellent.

Model risk and changing contract design

Policy versions matter. A dataset with multiple deductibles, coinsurance factors, layers, attachment points, and reinstatement terms cannot always be analyzed through one pooled cap transformation. Coverage triggers, aggregate limits, sublimits, salvage and subrogation, and inflation adjustment must be represented before a gross-to-net inference is trusted.

Zhao, Upretee, and Yu (2023) analyze multiple parametric severity models under censoring, emphasizing model selection and uncertainty. The implications are practical: models that fit the uncapped range similarly can imply materially different gross tails; choosing the family by an ordinary uncensored goodness-of-fit criterion can ignore the observation mechanism. Gatti and Wüthrich (2025) also discuss contract changes and extrapolation when deductibles or maximum cover change. Such extrapolation must be disclosed as model-based rather than an empirical recovery of information never observed.

Reporting and Observation Windows

Occurrence, reporting, settlement, and valuation

For a claim let AA be occurrence time, D≥0D\ge0 the reporting delay, and B=A+DB=A+D the reporting time. With an extract taken at calendar time τ\tau, the reporting indicator is 1(B≤τ)\mathbf1(B\le\tau). A loss incurred before τ\tau can remain absent because its reporting date occurs later. This is an incomplete event-count problem; it need not be missing data in a policy field, and it is not an ordinary right-censored event time if unreported event occurrences themselves are unenumerated.

The quantity traditionally termed incurred but not reported (IBNR) depends on the occurrence population, coverage criteria, report definition, and valuation time. A reporting-process model should distinguish IBNR from reported-but-not-settled claims and from changes in reserve adequacy on already known claims. These separations matter for both statistical likelihoods and audit documentation.

Verbelen, Antonio, Claeskens, and Crevecoeur (2022) provide a directly relevant event-occurrence and reporting-delay model using an expectation-maximization framework and an application to a European general liability portfolio. Their research shows that the occurrence mechanism and delay mechanism can be estimated jointly under a specified model. It does not imply that every triangle-based reserve is statistically identifiable without stable reporting assumptions.

Deriving the observed reporting intensity

Let insured losses occur according to a nonhomogeneous Poisson process with intensity λ(a)\lambda(a). Suppose delays have conditional distribution FD(u∣a)F_D(u\mid a). Then the expected number of claims incurred by τ\tau and reported by τ\tau is

E[Nreported(τ)]=∫a0τλ(a)FD(τ−a∣a) da.E[N_{\mathrm{reported}}(\tau)] =\int_{a_0}^{\tau}\lambda(a)F_D(\tau-a\mid a)\,da.

The expected count of pre-τ\tau claims still unreported at τ\tau is

E[NIBNR(τ)]=∫a0τλ(a){1−FD(τ−a∣a)} da.E[N_{\mathrm{IBNR}}(\tau)] =\int_{a_0}^{\tau}\lambda(a)\{1-F_D(\tau-a\mid a)\}\,da.

These expressions follow from independent marking or suitable conditional delay assumptions and clearly require an occurrence exposure window beginning at a0a_0. If delays depend on unobserved claim severity or if occurrence rates change during a catastrophe, the model must permit this dependence. A delay distribution estimated only from historically closed claims may systematically undersample slowly developing losses.

Reporting triangles aggregate by accident year and development lag. They can describe an observed development process efficiently, but aggregating to triangles discards information about individual contract terms, timing, geographic concentration, claim handling, and heterogeneity. The classic chain-ladder variability derivation by Mack (1993) and the stochastic reserving survey by England and Verrall (2002) show why reserving uncertainty requires explicit statistical treatment. The distinction between IBNR counts and unpaid severity remains essential even when both ultimately affect the reserve.

Calendar effects and nonstationary reporting

Administrative systems and economic conditions change. A new online claim portal may shorten reporting delay without altering underlying occurrence intensity. A natural catastrophe may generate both a spike in losses and a temporary backlog in reporting. A legal change may alter notification requirements or settlement procedures. Inflation may alter claim size and the threshold at which a claim is reported. Pooling these periods under an unchanged delay curve confounds insurance risk with changes in visibility.

A valid development analysis aligns accident, report, valuation, and settlement dates. It records claims reopened after prior closure and maintains the historical sequence of case reserves and payments. Reported estimates must not be evaluated using information recorded only after the intended forecast date. Such leakage produces apparently accurate backtests that cannot be replicated prospectively.

Exposure denominators and open portfolios

Claim counts require a denominator: policy-years, insured lives, vehicle-years, sums insured, or another contract-appropriate exposure measure. If exposure records disappear when policies terminate, frequency estimates may be distorted even when claim reporting is complete. Likewise, midterm additions and cancellations can make annual policy counts differ substantially from earned exposure. Missing denominators cannot be repaired simply by imputing claim severity.

Valuation-date completeness is a property of a snapshot, not a permanent attribute of the claim. A second extract after six months may reveal newly reported and reopened claims. Therefore the data dictionary should contain an as-of timestamp for each measured status and a controlled procedure for reconstructing past information states.

Statistical Corrections and Imputation

Correction begins by defining the target

Three questions precede method choice: Which population is the denominator? Which latent or contractual variable is the target? Which observation rule generated the data? If the target is average payment per positive claim, a conditional paid-claim model may suffice. If the target is total expected covered loss per exposure, the calculation also needs exposure and payment frequency. If it is the gross severity tail above coverage limits, an observed capped-payment sample usually supplies only partial information.

Statistical correction can be organized around observed cases, response probabilities, conditional outcome models, sampling likelihoods, survival risk sets, and sensitivity parameters. A method's accuracy depends on whether its assumptions match the data-generating and administrative process.

Weighting and its identification assumptions

For unit ii, suppose Ri=1R_i=1 indicates observation of YiY_i and π(Zi)=Pr⁡(Ri=1∣Zi)\pi(Z_i)=\Pr(R_i=1\mid Z_i) is known or consistently estimated. Under conditional response independence R⊥Y∣ZR\perp Y\mid Z, positivity 0<π(Z)≤10<\pi(Z)\le1, and appropriate sampling conditions, a Horvitz–Thompson-type estimate of the mean is

μ^IPW=1n∑i=1nRiYiπ^(Zi).\widehat\mu_{\mathrm{IPW}} =\frac1n\sum_{i=1}^n\frac{R_iY_i}{\widehat\pi(Z_i)}.

The normalized weighted mean uses the sum of observation weights in the denominator. The simple expression assumes an enumerated target population of size nn; changes are necessary for unknown population size, unequal base sampling weights, and complex portfolios. The classic incomplete-regressor estimation paper of Robins, Rotnitzky, and Zhao (1994) provides a foundation for weighting approaches to missing observations.

In the earlier constructed two-stratum example, correct response probabilities are 0.90.9 and 0.30.3. Dividing the observed stratum masses 0.720.72 and 0.060.06 by these probabilities recovers the underlying masses 0.80.8 and 0.20.2, and hence the stipulated 3.6-thousand mean. If the high-risk stratum cannot be identified among nonrespondents, or if some stratum never responds, the positivity and identification assumptions fail. Extremely small observation probabilities produce unstable weights; weight diagnostics and sensitivity analyses must accompany any point estimate.

Multiple imputation and uncertainty propagation

Multiple imputation constructs several completed datasets under a stated conditional model, analyzes each, and combines the estimates with between-imputation as well as within-imputation variability. It aims to reflect uncertainty about unobserved values rather than simply replace each blank cell with a single plausible number. Van Buuren's (2018) multiple-imputation framework and Carpenter and Kenward's (2013) practical imputation analysis present major approaches and applications.

For mm completed datasets with parameter estimates Q^j\widehat Q_j and estimated variances UjU_j, Rubin's pooling relationships are

Q‾=1m∑j=1mQ^j,U‾=1m∑j=1mUj,B=1m−1∑j=1m(Q^j−Q‾)2,\overline Q=\frac1m\sum_{j=1}^m\widehat Q_j,\quad \overline U=\frac1m\sum_{j=1}^mU_j,\quad B=\frac1{m-1}\sum_{j=1}^m(\widehat Q_j-\overline Q)^2,
T=U‾+(1+1m)B.T=\overline U+\left(1+\frac1m\right)B.

The total variance expression separates complete-data sampling uncertainty from additional imputation uncertainty. Its validity also depends on a compatible imputation and analysis procedure and the relevant missingness assumptions. A completed table is not evidence that previously absent observations actually occurred.

Insurance imputation models should respect structural restrictions. An impossible negative exposure duration, a death date preceding policy inception, a paid claim exceeding a valid contractual cap, or a claim report date preceding its occurrence can be generated by unconstrained generic imputation. Predictive distributions must reflect claim types, product cohorts, contractual relationships, calendar time, and administrative history. Imputation of gross losses above caps is especially assumption-sensitive because observed payment values alone may not determine the hidden severity tail.

Doubly robust estimation and its limits

Augmented inverse probability weighting combines a response model and an outcome model. Let m(Z)=E(Y∣Z)m(Z)=E(Y\mid Z). A representative estimator is

μ^AIPW=1n∑i=1n[m^(Zi)+Riπ^(Zi){Yi−m^(Zi)}].\widehat\mu_{\mathrm{AIPW}} =\frac1n\sum_{i=1}^n\left[ \widehat m(Z_i)+\frac{R_i}{\widehat\pi(Z_i)} \{Y_i-\widehat m(Z_i)\}\right].

Under the appropriate identification and regularity assumptions, it can remain consistent if either the response model or the conditional outcome model is correctly specified. This is the precise sense of double robustness discussed in Bang and Robins (2005). When both models are misspecified, when missingness depends on unobserved outcomes after conditioning, or when observation probabilities are zero, the robustness guarantee does not apply.

Complex prediction algorithms can be used to estimate nuisance functions, but out-of-sample tuning, cross-fitting when relevant, and dependency-aware validation remain necessary. A highly accurate prediction model for observed claims can still be biased for the original portfolio because prediction quality within the selected sample is not selection correction.

Likelihood and model-based correction for insurance losses

For deductibles and policy limits, an appropriately conditioned likelihood incorporates the probability of crossing the deductible and the survivor mass above the cap. For reporting delay, it incorporates the probability an event is observable by the extract date. For survival duration, it uses the event or survival contribution associated with the actual censoring information. These constructions can coexist in one analysis, but their combination requires care about dependence among frequency, severity, delay, and claim management.

A joint frequency–severity specification may be attractive when severe claims tend to be reported later. Modeling delay independently of severity can create an apparent decline in severity in recent accident periods even if the underlying distribution has not changed. An alternative is a conditional delay model in which reporting probability varies with observed or latent loss type; the additional complexity creates new assumptions whose identifiability must be evaluated.

Practical data-quality responsibilities

The U.S. Actuarial Standards Board's ASOP No. 23, Data Quality (2016 revision, effective for covered work products based on data provided or developed on or after April 30, 2017) addresses data selection, review, use, reliance, and disclosure in actuarial services. Its review obligation is distinct from a comprehensive audit. This is a professional standard within its specified applicability, not a universal legal rule for every insurer or jurisdiction.

A research-grade data-quality record should identify fields, collection channels, inclusion rules, completeness patterns, exposure denominators, valuation dates, contract transformations, known data alterations, adjustment assumptions, and consequential limitations. The standard's data-review and disclosure guidance supports careful documentation, while statistical validity ultimately depends on the actual assumptions and evidence for the individual study.

Identification and Sensitivity

What the observed distribution determines

A statistical model is identifiable for a target when distinct target-relevant parameter values imply distinct observed-data distributions, subject to the model assumptions. When every positive claim is retained only above one deductible, the fraction and severity of losses below that threshold are not nonparametrically determined by those positive records. When every gross claim exceeding a limit becomes the same capped payment, the hidden tail cannot be reconstructed uniquely from the capped amounts. When claim existence is itself unreported, observed payments alone cannot identify the full occurrence rate.

This is a property of information, not a failure of computational power. Rubin (1976) shows why explicit missingness assumptions matter for inference; the truncation-identification literature of Woodroofe (1985) similarly requires conditions on observable support. Modern censored-loss model comparisons illustrate the practical model dependence of actuarial conclusions.

The strongest defensible statement may be a range rather than a point estimate. Suppose a nonnegative outcome lies in [0,U][0,U], fraction qq is missing, and observed cases have mean μobs\mu_{\mathrm{obs}}. Without further identifying restrictions, a simple bound for the population mean is

(1−q)μobs≤E(Y)≤(1−q)μobs+qU,(1-q)\mu_{\mathrm{obs}} \le E(Y)\le (1-q)\mu_{\mathrm{obs}}+qU,

provided the observed-case mean is defined over the observed fraction of the target population and all values obey the assumed bound. This inequality is a direct algebraic consequence of splitting the mean by response. For unbounded severity, the upper bound may be infinite. A policy payment cap can bound an insurer-payment variable without bounding the corresponding gross loss.

Sensitivity analyses that answer substantive questions

Sensitivity analysis varies assumptions that observed data cannot settle. Under MNAR, one can specify how the missing outcome distribution differs from that of observed outcomes conditional on recorded covariates, then report how pricing or reserve estimates change across defensible departures. For severity tails, compare justified families or tail indices while preserving exactly the observed censoring and truncation rules. For delayed reporting, allow delay distributions and calendar effects to vary by claim class and year.

A credible sensitivity analysis specifies which assumptions are varied, why those ranges are relevant, which data support them, and which conclusions are stable. Changing algorithm hyperparameters without varying selection, tail, or delay assumptions is not a substitute. Model selection uncertainty is especially material for above-limit gross losses, as illustrated by Zhao, Upretee, and Yu (2023).

Validation without future information

A reproducible validation program should freeze a valuation date, identify the fields and claims actually visible at that date, fit the model using that information only, and compare predictions with genuinely later developments. The design should respect accident-year and calendar-year structure; ordinary random record splitting can leak information when related payments or claims from one underlying event appear in both partitions.

For reported-claim counts, backtests can compare projected incremental reports to newly observed claims, with separate evaluation during catastrophic or administrative disruptions. For severity, evaluate uncapped observations and the model's predicted frequency of cap hits, and separately label untestable extrapolation beyond the cap. For durations, verify time origin, follow-up, censoring pattern, delayed entry, and proportional-hazards assumptions where relevant. For missing covariates, use available verified auxiliary records to test imputation calibration rather than claiming that artificial masking alone reproduces actual MNAR behavior.

Evaluation needs both prediction and inference criteria. A model may forecast aggregate reported payments accurately while estimating the underlying loss-frequency mechanism incorrectly, because opposing errors in reporting rate and severity cancel. Good aggregate fit therefore does not establish identification of each component.

An integrated source-to-estimand audit

The research findings support a practical, six-question audit, formulated here as an analytical synthesis rather than a new named methodology. First, state the population and estimand, including whether it concerns gross losses or contractual insurer payments. Second, map every administrative and contractual step from exposure to final observable record. Third, classify each incomplete element as missingness, censoring, truncation, delayed entry, delayed reporting, or an explicit contractual transformation. Fourth, derive the likelihood or estimating equation matching the recorded information and articulate its identification assumptions. Fifth, test what the available data can test and perform sensitivity analysis for what they cannot. Sixth, document the exact valuation-date dataset, versioned transformations, source evidence, diagnostics, uncertainty and limits of interpretation.

The six questions have different answers by line of business. Life and health coverages frequently require duration, competing event, and longitudinal covariate analysis. Property and casualty severity requires attention to deductibles, sublimits, claim reopening, and settlement delay. Catastrophe and liability portfolios can combine a sudden shift in occurrence intensity with long or unstable reporting delays. The methodological goal is consistent scientific attribution of what is observed and what is inferred, rather than the universal application of a single preferred algorithm.

Research limitations and next questions

The source program includes foundational original statistics articles, specialized actuarial publications, and a U.S. professional standard. Some older works and books were checked through publisher bibliographic records or abstracts, not independently reproduced page by page. No proprietary policy data were used, no real-world coefficient was estimated, and no jurisdiction-specific legal compliance conclusion is made. The derivations here are theoretical conditional on their stated designs; the numerical examples are constructed rather than empirical findings.

Further empirical work should investigate the dependence between claim severity and reporting delay; joint nonresponse and delayed entry; transportability of severity models across changing deductibles and limits; selection of rejected underwriting applications; uncertainty propagation from incomplete exposure records to capital estimates; and auditing of missing characteristics when data availability varies systematically across demographic groups. Such studies require lawful, properly documented data access and predeclared target populations.

Conclusion

Missingness, censoring, truncation, delayed reporting, and contractual loss transformations are different observation mechanisms. Each determines what a recorded insurance dataset contains, which likelihood contributions are legitimate, and which estimands can be recovered. Once the observation process is defined, the foundational statistical results of Rubin, Kaplan–Meier, Cox, Turnbull, Lynden-Bell, and later missing-data researchers can be applied in their proper domains, while actuarial applications make clear that contract terms and administrative timing are part of the measurement model.

The mathematical results presented here establish several concrete conclusions. Complete-case means target selected observations and can diverge from population means; the direction and magnitude of the divergence follow the dependence between response and outcome. Product-limit methods preserve information in censoring when their risk sets and independence conditions are valid. A deductible can remove losses from a positive-claim sample, while a limit can conceal the gross severity of retained claims. Reporting delay couples the occurrence process to calendar-time visibility, creating a distinct source of incomplete information. Multiple imputation, weighting, augmented estimation, and joint likelihood methods help only to the extent that their identifying assumptions are justified. When support or positivity fails, sensitivity intervals and transparent bounds may be the scientifically appropriate result.

The contribution of this monograph is an original integrated interpretation of established methods centered on the insurance observation process and the identification of precise economic and statistical targets. It offers derivations, constructed examples, and a source-to-estimand audit that can be reproduced and challenged. It establishes no new named theory and claims no novelty for the underlying classical estimators. The governing conclusion is operational and scientific: an insurer's recorded data must be interpreted through the exact contractual and administrative mechanisms that made those data observable.

References