RESEARCH DOSSIER
Rare Event Learning and Class Imbalance in Insurance
Abstract. Rare-event prediction is a central inferential problem in insurance because the event of greatest decision relevance may account for only a small fraction of observed policies, claims, transactions, locations, or coverage periods. A classifier can distinguish insured events well while providing misleading probabilities, and a model trained on an artificially balanced sample can appear effective even as its portfolio-level deployment produces an unacceptable volume of false alerts. This monograph studies the statistical and operational foundations of learning from rare outcomes: the definition of a risk set and event horizon; ascertainment, label delay and informative selection; sampling, class reweighting and synthetic observations; prior-probability correction and calibration; cost-sensitive decision thresholds; precision–recall and proper-score evaluation; temporally faithful validation; and transfer across insurance portfolios. It establishes mathematical relationships among event prevalence, sensitivity, false-positive rate, positive predictive value, sampling odds, and decision loss, and works through reproducible numerical illustrations. The insurance-specific analysis distinguishes rare claim incidence, high-severity indicators, suspected fraud, cyber incidents, catastrophic loss flags, and delayed claims development. Original synthesis locates a reliable rare-event learning system in the complete sequence from event definition and exposure through ascertainment, estimation, probability restoration, operational decision and monitoring. The contribution is a source-grounded integration and set of analytical conclusions; it does not introduce a new named theory, framework or actuarial doctrine. External datasets and proprietary insurer results are not represented as original empirical findings.
Keywords: rare-event learning; class imbalance; insurance prediction; infrequent claims; claims severity indicator; fraud detection; event prevalence; case-control sampling; undersampling; SMOTE; logistic regression; Bayesian prior correction; probability calibration; cost-sensitive classification; precision; recall; positive predictive value; ROC; proper scoring rules; temporal validation; data leakage; portfolio shift; underwriting; actuarial modeling; operational risk.
Introduction
Insurance records usually contain many periods without a claim, many claims without a suspicious feature, and many policies without a large insured loss. The observations of interest are therefore rare relative to the administrative events that surround them. Their scarcity is material to insurance pricing, risk selection, post-loss investigation, customer service, catastrophe exposure analysis and loss prevention. Rarity also complicates inference: the information needed to describe a positive outcome is concentrated in few records, its measurement may arrive late, and costs are seldom symmetric across incorrect decisions.
The first methodological distinction is between an outcome being rare in the target portfolio and a training table having unequal class counts. The two facts can coincide, but they answer different questions. A portfolio prevalence of one in a thousand is an attribute of a defined population and horizon. A training prevalence of one in five may be an artifact of deliberately retaining all positives and only a fraction of negatives. Scientific conclusions require both quantities to be identified. King and Zeng's rare-events logistic regression study demonstrates the significance of rare outcome bias and event-enriched sampling; He and Garcia's review of imbalanced learning examines broader algorithmic and assessment consequences.
The second distinction concerns prediction and action. A score can rank cases successfully without being an estimate of their actual probability. Conversely, a reasonably calibrated probability may be operationally unhelpful when estimated value, claim severity, investigator capacity or a legally permissible use differs between cases. Elkan's foundations of cost-sensitive learning makes expected consequences central to the decision, while Gneiting and Raftery's proper scoring rule analysis supplies a different criterion for assessing the quality of probabilities themselves.
The research object of this dossier is supervised prediction of an explicitly defined infrequent insurance outcome. Its scope includes statistical classification, event-probability estimation, decision consequences and validation under actual deployment prevalence. Extreme-value estimation of the severity distribution beyond a high threshold, the physical generation of catastrophe hazards, and the full stochastic development of reserves are neighboring research objects. They enter here when a decision maker defines a rare class such as loss exceeding a contractually specified threshold, not as substitutes for specialized tail or reserve modeling.
The method combines original derivations from conditional probability and expected loss, critical comparison of original methodological papers, and an insurance-specific interpretation of the assumptions needed to carry methods into practice. Numerical cases below are hypothetical and exactly reproducible from disclosed values. They are not presented as measured insurance performance.
Insurance Event Imbalance
Prediction objects, risk sets and denominators
The elementary classification record is a triple , with observable predictors , binary target , and a prediction origin . The target is meaningful only after defining the eligible population, exposure unit, observation horizon, covered contract, information cutoff, and verification rule. For a commercial property policy, one record might represent a building-year; for motor claims investigation, one record may represent a claim file at first notice of loss; for cyber insurance, one record may represent an insured firm-month. A warning model for a large claim can use when ultimate insured loss exceeds a specified monetary amount within a defined coverage period. That threshold and the valuation date belong to the estimand.
Let the deployment prevalence be , and let denote the deployment conditional event probability. The recorded positive fraction is an unbiased estimate of only when the observed units represent the intended risk set and the label has been ascertained under an appropriate selection and follow-up design. Counting positive claim files among all claim files gives the probability conditional on a claim having been reported; counting positive policies among exposed policies yields a different object. Neither ratio should be substituted for the other.
Insurance exposure changes the underlying question. A policy-year with a ten-day short coverage segment is not automatically exchangeable with a full-year policy. Even if a rare-count model is reduced to a binary indicator, exposure , reporting lag and portfolio membership affect the probability of at least one event. Under a conditional Poisson count with mean , ; this is a derivation under the Poisson assumption, not a universal formula for insured claims. It makes explicit why a classifier that discards exposure length can misinterpret differences in event prevalence.
The insurance question also determines the relevant economic denominator. A rare fraud investigation flag is measured per claim eligible for review, whereas an aggregate portfolio stress event may be measured per simulated annual insurance outcome. A model's precision and operational workload depend on this denominator, making the choice a substantive scientific step rather than a data-formatting convention.
Why observed rarity can misrepresent underlying rarity
Rarity has several possible origins. Genuine infrequency results from the risk process and observation horizon. Administrative rarity occurs because only filed claims, adjudicated cases or insured events enter the database. Label rarity can arise when costly investigation leaves most cases unverified. Delayed rarity arises when an insured event has happened but reported or ultimate loss information is incomplete at the training cutoff. Sampling rarity can be induced by a case-control or stratified dataset deliberately built for modeling. These mechanisms imply different remedies.
A fraud label illustrates the problem. An investigator's suspicion flag, a concluded internal fraud assessment and a final legal finding answer different questions. Investigations are frequently selected by existing controls, so an observed negative means something different when it was thoroughly checked than when it was never investigated. Training solely on investigated claims can learn investigator selection as well as underlying fraud characteristics. It is therefore necessary to record who was eligible for verification, when verification occurred and which cases remained unresolved. The published automobile claims benchmark of Viaene and colleagues explicitly uses expert suspicion labels and early-known variables in its insurance fraud comparison, highlighting how target definition and decision-time features shape the experiment.
Large-claim labeling adds reserve development. A claim reported for $20,000 may later exceed a $100,000 threshold; the recorded class depends on the valuation date. A retrospective rule that uses the ultimate loss to label development records is legitimate when the prediction origin predates that ultimate amount, but the training design must control maturity and available follow-up. Otherwise recent records have fewer opportunities to become observed positives. This is censoring or delayed outcome ascertainment, not simply random imbalance.
The basic effective evidence count is the number and diversity of observed positive units, not the full row count. The sample size analysis of Riley and colleagues emphasizes joint requirements on total cases, outcome events and predictor parameters for binary prediction model development. In insurance, a million policy months containing a handful of distinct catastrophes may yield only a handful of independent event-generating circumstances. Repeated rows from the same event or insured entity can inflate apparent information unless their dependence is acknowledged.
Different insurance rare-event mechanisms
A routine claim occurrence indicator is a rare event when frequency is low over the chosen exposure interval. A very-large-loss indicator is a rare class nested within claim occurrence, and its prediction may require information about both claim incidence and potential severity. Suspected fraudulent claims form a selectively verified class, often affected by investigation policy. Cyber event classification may combine evolving definitions, changing defenses and incident reporting. Catastrophe loss indicators involve correlated occurrences and common hazard sources. These are not interchangeable datasets merely because their positive fractions are similar.
The disciplinary consequence is that algorithm choice follows an explicit estimand and observation design. An underwriter estimating expected claim cost needs calibrated frequency and severity components; a fraud investigator allocating a fixed number of inspections may first require ranking within a constrained queue; a catastrophe analyst interested in solvency stress may require a joint dependence and tail model beyond the binary screen. Treating their training tables as the same mathematical imbalance problem erases the operational distinction that gives the predicted class meaning.
Sampling and Reweighting
Event-conditioned sampling and logistic models
When collecting non-events is expensive, an analyst may sample positive and negative classes at different inclusion probabilities. Suppose indicates retention, with and . Assuming retention depends on class but not additionally on the features within class, Bayes' theorem yields
Thus the sampled log odds differ from target log odds by the constant . A correctly specified unpenalized logistic regression under pure case-control sampling can preserve slope information while its intercept requires correction. Model misspecification, regularization, altered feature distributions and interactions with selection can invalidate a simple intercept-only explanation. King and Zeng's case-control rare-event corrections address both rare-event estimation and data-collection efficiency; Firth's bias reduction method addresses first-order likelihood bias and is especially relevant when rare positive outcomes create unstable or separated logistic fits. These solve related but distinct estimation problems.
The empirical event fraction in an enriched dataset is not a deployment prior. If the original target prevalence is known, the selection odds can be corrected analytically. If inclusion rates depend on covariates as well as outcome, the correction must reflect those rates or the data-generating assumptions must be revised. When the target population itself is unknown, an apparently precise probability reconstruction conceals an unmeasured prior.
Reweighting, undersampling and oversampling
Three commonly used adjustments target training behavior. Undersampling removes selected majority-class observations to reduce computational cost or class dominance. Replicative oversampling repeats minority observations or changes their training frequency. Class weighting changes the contribution of an observation to the fitted objective. For a weighted binary log-loss, one can write
Weights may be chosen to correct known unequal observation probabilities, or to emphasize one kind of prediction error. These motivations have different inferential consequences. Inverse-inclusion weighting can target a representative-population risk when selection probabilities are correct; an arbitrary equal-class weighting objective generally targets a modified loss function. It should not silently be interpreted as producing a calibrated probability for the actual portfolio.
For an unrestricted probability estimator at a fixed , minimizing weighted log-loss produces
Consequently a model's probability output under class weighting can be systematically different from the unweighted conditional event probability even if its ranking remains useful. A monotone transformation can recover only under the assumptions used to derive it. This formal point helps explain the performance differences surveyed by Japkowicz and Stephen in their systematic class imbalance study and by He and Garcia in imbalanced learning methods.
An apparent improvement in recall following oversampling is therefore not sufficient evidence of an improvement in deployment utility. Model regularization, tree splitting, finite-sample variance and changed class weights may alter the ranking itself; threshold changes may simply trade false positives for true positives. Fair comparisons must distinguish ranking improvement, probability repair and the movement of a decision threshold.
Synthetic minority generation and model complexity
Chawla and colleagues' SMOTE method generates artificial minority examples by interpolating between neighboring minority observations. The innovation addresses sparsity in the training representation and has supported an extensive family of resampling methods. Its central limitation in insurance is that many records are structured by contract and law rather than by unconstrained Euclidean geometry. Interpolating deductibles, coverage limits, categorical peril codes, claim dates or judicial statuses without a valid representation can manufacture observations that are mathematically plausible in feature space but impossible as insurance contracts.
Interpolation can also blend data from fundamentally different causes of loss. Two severe claims may have the same binary outcome while arising from unrelated perils, geographic clusters or claim-development paths. A synthetic point between them is not evidence of a physically or institutionally valid exposure. Hence SMOTE should be evaluated as a predictive training intervention with controls on admissible features, grouping, timing and calibration, not as the creation of additional independent real-world events.
Every resampling operation used for model fitting belongs inside the training fold. Generating synthetic observations before the split can place descendants of a case into both training and testing partitions. The resulting metrics measure access to leaked structure rather than genuine generalization. The broader problem is detailed by Kaufman and colleagues in leakage detection and avoidance. A correct experiment preserves original, naturally imbalanced validation and test sets and fits all sampling, feature selection and calibration choices without examining held-out outcomes.
A practical comparison should include a well-regularized baseline without resampling. Method complexity has a cost: synthetic minority methods and deep representation learning can require more tuning and introduce larger variance when the independent positive-event count is small. The evidence supporting one method within a data set cannot establish universal superiority across product lines or event definitions.
Class Probability Calibration
Restoring probabilities after sampled training
Consider a classifier trained on an event-enriched sample with event prior . Suppose its output is calibrated for that sample and the class-conditional feature distributions are unchanged between the sampled training population and deployment. Define the target prior . Bayes' theorem gives a prior-odds correction,
This is the conceptual basis of the classifier-prior adjustment examined by Saerens, Latinne and Decaestecker in their changing-priors procedure. The equation does not apply automatically when features conditional on the class change, when the model output is not calibrated for the sample, or when selection depends on features in ways the correction omits.
A hypothetical enriched training set has , while the intended insurance deployment prevalence is . The prior-odds multiplier is , approximately . If a case receives a calibrated sample probability , its sample odds are 4; after correction, its target odds are approximately , and the resulting target probability is approximately , or 3.11%. This is an exact consequence of the assumptions, not evidence that any insured has that risk. An uncorrected reading of 80% would overstate the hypothetical deployment probability by more than twenty-five times.
Prior correction is particularly consequential when a risk ranking feeds premium indication, expected claims cost or fraud investigation. A score interpreted as an absolute risk must answer what would happen in the actual eligible portfolio, not what happens in a deliberately balanced modeling sample.
Proper scores and calibration diagnostics
Calibration concerns the relationship between predicted probabilities and observed event frequencies. If a collection of cases is assigned probabilities near 0.01, its appropriately observed positive fraction should be approximately 0.01, subject to sampling uncertainty and the population definition. Discrimination concerns relative ordering. They can vary independently. Niculescu-Mizil and Caruana document distortions in supervised probability estimation, and Guo and colleagues demonstrate important neural-network calibration effects in the benchmark settings examined in that study. Such results justify evaluating calibration, but do not prescribe a universally best calibration method for insurance data.
For binary outcomes the Brier score is
with smaller values preferable under the stated convention. Its historical origin is Brier's probability forecast verification. Log loss, , places a particularly strong penalty on confident errors. Both are strictly proper in their usual admissible settings, in accordance with the general result presented by Gneiting and Raftery in strictly proper scoring rules.
Under rare events, a small overall Brier score can coexist with inadequate positive-event performance because almost all terms come from non-events. A zero-probability predictor in a portfolio with prevalence 0.002 has population Brier score 0.002, yet it entirely fails to recognize positive cases and assigns zero probability to events that occur. Therefore proper scores require comparison with explicit baselines, calibration plots and task-relevant decision performance. A constant forecast equal to the true prevalence has expected Brier score , which in this illustration is : a small improvement over always forecasting zero, but not a solution to identifying which policies are at greater risk. The analysis by Van Calster and colleagues on prediction calibration underscores the need to assess absolute probability accuracy separately from rank discrimination.
Calibration should be checked by relevant portfolio subgroups, time periods, event ages and exposure bands. Flexible smoothers can be unstable in low-risk regions when positive observations are extremely sparse; a highly partitioned reliability diagram may display noisy rates or empty cells. Investigators should report denominators, events, uncertainty intervals and smoothing assumptions. A fitted intercept and slope for recalibration can be informative, but their estimation must be performed using training or validation information and judged on an independent test period.
Changing portfolio prevalence and local transport
Rarity is not fixed across all business. New underwriting rules change selection; coverage terms alter eligibility; catastrophe seasons and cyber vulnerabilities change hazard; claims procedures change ascertainment; and economic conditions can shift reporting. The operational base rate can consequently move even when the trained model is unchanged. Two phenomena should be separated. Under pure prior shift, class proportions change while the conditional distribution of features given the class remains invariant, and prior-odds correction can be justified. Under broader concept or covariate shift, class-conditional distributions or change, so a prevalence-only adjustment may fail.
Probability transport must be anchored to a specified business population and observation window. If a model fitted on one insurer's motor physical-damage claims is moved to another insurer's bodily-injury claims, the event label, exposure structure, claims process and selection may all differ. A correction based only on the global positive fraction offers no guarantee of subgroup validity. External validation and local recalibration are empirical obligations, not properties obtained automatically from the algebra.
Cost-Sensitive Objectives
Expected loss and threshold selection
Insurance decision makers act on a prediction by taking an action. Let action indicate intervention, such as review, and indicate routine treatment. In a simplified binary setting, suppose the incremental cost of intervening on a true negative is , the incremental cost of failing to intervene on a true positive is , and correct decisions have zero incremental loss. The conditional expected losses are and . Intervention is preferred when
This result is a direct derivation under its stated cost convention. Elkan's cost-sensitive decision analysis supplies the wider framework and emphasizes coherent interpretation of decision costs. The threshold is an economic quantity derived from predicted real-world probability and legitimate action consequences. It is not normally the artificial training prevalence, nor is 0.50 intrinsically correct.
If an illustrative false investigation costs $50 and a missed actionable event imposes $5,000 of avoidable incremental cost, the mathematical threshold is , approximately 0.00990, or 0.99%. The adjective "avoidable" matters. The full loss on an insured claim is not automatically the value gained by flagging or refusing that claim. Investigation may prevent only part of a loss, and customer harm, supervisory obligations and administrative expenses also belong in a coherent decision evaluation. Treating all claim severity as preventable benefit could create an inflated and ethically distorted threshold.
The simple threshold becomes more complex when intervention quality varies. If only a fraction of true positives can be acted upon, if recovery depends on case-specific severity, or if one review raises the likelihood of later recovery rather than producing a deterministic result, expected net benefit should use case-specific conditional consequences. Where regulation constrains the action, permissibility is determined before optimizing utility. A statistical score does not grant authority to deny claims or alter coverage.
Investigation capacity, claim cost and policy constraints
A claims team can review a finite number of cases per day. Under equal review costs and equal recoverable value conditional on a positive, selecting the highest calibrated probabilities maximizes the expected number of positives found. If benefit differs by case, rank by expected incremental net benefit rather than probability alone. A high-probability low-value alert may have lower priority than a moderately probable, verifiably high-value case. The optimization objective must be stated before interpreting precision at or recall at capacity.
Capacity makes an apparent improvement in overall area under a curve potentially irrelevant if it occurs at thresholds never used operationally. Evaluation should report performance near the actual workload, net value under plausible cost ranges, and the stability of selected cases when scores are re-estimated. Precision at the operational cut can be useful when it directly reflects investigation yield, but it remains sensitive to actual event prevalence and label verification.
For underwriting, apparent misclassification costs also depend on premium, customer retention, expense, adverse selection and aggregate portfolio constraints. A binary "large loss" flag can inform risk review but does not by itself specify the correct premium: pricing normally requires expected loss severity, frequency, exposures, expense and allowable rating factors. In claims operations, the decision question could be whether to inspect, request documents, triage expertise or reserve resources, each with different consequences. The architecture of the insurer's action must therefore be coupled to the target probability definition.
Ranking, probabilities and insurance actions
Class imbalance is commonly approached as though the objective were to maximize a single classification score. This conflates three separate stages. Estimation learns a conditional risk or a monotone ranking. Calibration interprets scores as probabilities at the target prevalence. Decision analysis selects an action according to those probabilities, costs and constraints. The stages may use different losses and datasets. Lipton, Elkan and Narayanaswamy's analysis of F1 thresholds shows that an F1-optimal cutoff depends on properties of the prediction set and metric; it cannot be universally identified with the cost-optimal threshold.
The insurance-specific conclusion is that improving rare-class recall is meaningful only together with its impact on false alerts and actual decisions. It can be rational to accept a low positive predictive value when the event is extremely costly and alerts are inexpensive, or rational to accept lower recall when intervention itself creates substantial hardship. The governing standard is decision-specific expected consequence supported by valid probabilities and lawful process.
Rare Loss Validation
Discrimination and prevalence-sensitive precision
Let sensitivity and false-positive rate . By Bayes' theorem, positive predictive value—precision in a binary classification task—is
This identity is central to the operational meaning of insurance class imbalance. If event prevalence becomes very small while and remain fixed, false alarms dominate the flagged population unless is correspondingly small. ROC coordinates condition separately on each actual class, which makes a ROC curve less directly informative about the absolute workload generated by a rare event. The relationship between the curve families is examined by Davis and Goadrich in precision–recall and ROC analysis; Saito and Rehmsmeier present empirical evidence of the interpretive advantage of precision–recall plots for imbalance.
A ROC area and a PR area should not be treated as interchangeable. ROC AUC measures a form of ranking discrimination, generally interpretable as the chance that a random positive receives a higher score than a random negative, with tie handling. Average precision and PR-area estimators are not identical in every software implementation. Precision depends on and the cutoff, and a random-ranking precision baseline is the event prevalence. Fawcett's introduction to ROC analysis discusses interpretation pitfalls; Jeni and colleagues' imbalanced-metric comparison documents the impact of changing skew on several familiar measures.
Accuracy can be particularly misleading. A predictor that always chooses the non-event class has accuracy , which equals 99.8% at . It has zero sensitivity. Such a classifier is not an effective warning model simply because its accuracy is high. Balanced accuracy may aid class-conditional comparison, but neither balanced accuracy nor F1 independently describes calibrated financial risk.
Selection-safe splits and uncertainty
The minimum evaluation design has a frozen prediction origin, a test population representative of operational use, and outcomes observed after sufficient follow-up. All transformations—including imputation, scaling, minority resampling, feature selection, threshold optimization and calibration—must be fit using their authorized training partition. A temporally later test interval is generally informative when insurance processes evolve; grouping by insured entity, claim family, broker, event or geography may be essential when rows share latent causes. Which split is most demanding depends on the scientific target.
A random split of claim-development snapshots can place the same claim's earlier and later records on both sides, contaminating predictive evidence. A geographic catastrophe model may obtain unrealistically narrow uncertainty when the same underlying storm appears in training and testing through different insured buildings. Claims fraud data may reveal investigation outcomes through fields added only after the prediction time. The logical defense is a decision-time feature inventory and a clear learn–predict separation, consistent with Kaufman and colleagues' leakage methodology. Varma and Simon show how parameter selection on the same validation procedure biases the reported error in cross-validation model selection; independent or correctly nested evaluation is necessary when many imbalance techniques are compared.
Sampling uncertainty is driven by the number of observed positive events and by clustering. If 20 out of 100 verified positives are detected, the measured sensitivity is 20%; its uncertainty is substantial, and clustered or selectively verified events can increase it further. For precision at a fixed workload, the number of reviewed flags matters as well. Confidence intervals should use a method appropriate to the study design and estimand; naive row-level bootstrap intervals are unreliable when correlated catastrophe events or repeat observations dominate.
Steyerberg and Vergouwe's prediction validation components organize calibration, discrimination and decision usefulness as separate dimensions, and Riley and colleagues' sample-size criteria connect development stability to event counts and model complexity. Rare-event insurance research should follow that principle rather than invoke a single unconditional minimum number of events for every model.
Reproducible mathematical illustrations
The following illustrations are worked arithmetic exercises. None involves estimated insurer data.
| Illustration | Fixed assumptions | Derived result | Interpretation |
|---|---|---|---|
| Rare-event accuracy | ; predict every case negative | accuracy, sensitivity | Aggregate accuracy conceals complete failure to identify events |
| Investigation precision | About 12.30% of flags would be positive under these fixed rates | ||
| One million observations | , same prevalence and rates | 2,000 positives; 1,400 true positive flags; 9,980 false positive flags; 11,380 total flags | Actual review workload is dominated by negatives |
| Prevalence transport | Precision increases solely because the target base rate increases | ||
| Enriched posterior correction | The sampled probability is not the deployed probability | ||
| Two-cost decision | Threshold follows assumed incremental losses | ||
| Constant probabilistic forecast | for all cases | Expected Brier score | A good aggregate probabilistic baseline need not distinguish individuals |
The one-million-record scenario can be reconstructed from events, true positive alerts, and false alarms. Precision is . Increasing event prevalence to 1% with sensitivity and false-positive rate unchanged yields precision . The difference is attributable entirely to the changed base rate. Reports that compare precision across portfolios without adjusting interpretation for prevalence confound ranking performance with the population's event composition.
These calculations also reveal why insurance costs cannot be inferred from a classification confusion matrix alone. In the hypothetical one-million-record exercise, 11,380 alerts might require review. Their financial desirability cannot be derived without knowing individual review expenditure, benefits that actions can actually realize, nonfinancial harms, capacity and contractual constraints. The table verifies arithmetic and the logical implications of conditional probability; it is not an actuarial financial projection.
External Generalization and Limitations
Insurance lines and cross-jurisdictional transfer
Methodological results developed in a general classification problem or in credit card fraud can guide insurance analysis but must be transported through insurance-specific institutions. Dal Pozzolo and colleagues' fraud detection practitioner study illustrates the combined challenges of imbalance, drifting observations and realistic assessment in a transaction setting. Insurance claims add policy terms, reporting, adjustment, disputed causation, subrogation, reserving, risk pooling and contractual coverage decisions. They also operate under different legal regimes and claims-handling practices. The scientific transfer is methodological, not a claim that the external study measured insurance performance.
In motor insurance, the decision may concern an early suspicious injury claim; in health, an unusual service pattern; in property, potential large-loss occurrence; in life, a rare mortality or lapse-associated event over a specific horizon; in cyber, a material incident or accumulation of related claims. Each case requires a new label audit, exposure basis, ascertainment process, calibration target, operational constraint and temporal holdout. Some tasks should be formulated through count, survival, severity, hierarchical or multistate processes rather than forced into a binary classification. This dossier's binary mathematics applies only after a scientifically defensible reduction to a two-class target.
Transfers across countries demand attention to insurance-law and data definitions. A fraud allegation is not a final adjudicated fact. A coverage exclusion cannot be inferred from model output. Public policy, market-conduct rules and lawful use of personal information vary by jurisdiction and time. The ISO 31000 risk management guidelines offer general organizational context for risk identification and governance, but they are not an insurance-specific legal authorization or a model-validation certificate. Any account of national regulation must be tied to its actual instrument and effective date; this monograph does not claim an exhaustive comparative law survey.
Monitoring, feedback and governance
Rare-event models produce new data as they operate. When flagged cases are more frequently investigated, future labels become selectively verified. If prevention succeeds, event rates may fall among high-score cases; conventional retrospective evaluation can then misread successful interventions as model errors. Underwriting actions change the insured portfolio, shifting the very risk distribution that the model was built to predict. Monitoring therefore needs the history of selection, intervention and verification—not only the latest confusion matrix.
Practical post-deployment evidence should preserve model version, prediction time, record eligibility, feature provenance, score, probability calibration version, action chosen, investigation capacity, realized and censored outcomes, and follow-up maturity. Reviews should separate model drift from changes in the claims process, the event definition and the business mix. Where labels are verified selectively, investigators need explicit verification sampling, adjudication audits or suitable statistical correction to estimate performance for the full eligible population. Otherwise the monitored precision estimates success among examined flags rather than predictive validity for all policies or claims.
The governance implication is comparable across insurance operations: probability estimation, contract decisions and customer treatment have distinct accountable owners. A predictive model can recommend a review priority; human, contractual and institutional mechanisms determine permissible action and recourse. Proper documentation of training prevalence, corrected target prevalence, observational selection and model uncertainty creates a reproducible basis for internal review.
Synthesis and research agenda
The principal finding is that rarity has to be located simultaneously in the insured event process, the measured label, the sampling plan and the intended action. A high-performing model cannot be recognized by a single favorable summary metric. Its effectiveness is conditional on an interpretable event, a target population, observed positive cases with adequate independence, valid calibration at the operational prior and a consequential decision. These are analytical distinctions derived from the cited literature and the mathematical identities above, not a newly named scientific framework.
Several research questions remain important across the international insurance market. First, evaluation under selectively verified fraud labels requires designs that identify what is observable and estimate uncertainty when uninvestigated cases lack reliable truth. Second, catastrophe and cyber portfolios need learning approaches that recognize dependence among rare events, common causes and shifts in accumulation. Third, severe-claim flags with immature claims development need joint treatment of label delay and out-of-sample forecasting. Fourth, calibration should be studied across exposure scales and high-risk subgroups in ways that reveal weak identification instead of masking it with smooth plots. Fifth, evidence about the net value of interventions should be separated from evidence about discrimination and classification, particularly when reviews impose financial and procedural costs.
These questions do not require inventing a new doctrine. They require a more careful empirical connection among research design, probabilistic inference, actuarial consequences and external validation.
Conclusion
Rare Event Learning and Class Imbalance in Insurance concerns the scientific conditions under which a small number of positive observations can support prediction for a much larger insured population. The decisive first step is a rigorously defined event and denominator: policy-year claim incidence, early-stage suspicious claim, large-loss threshold and catastrophe-linked event are different estimands. The second is explicit identification of the observation mechanism, including selective investigation, delayed settlement, case-control sampling and exposure heterogeneity. Without these, imbalance correction can transform an undocumented selection process into a misleading model probability.
The fundamental equations establish several durable conclusions. Changing the retained positive-to-negative ratio changes posterior odds unless restored to the target distribution. Class weighting and synthetic oversampling alter training objectives and may alter ranking, calibration and variance. Positive predictive value is a function of event prevalence as well as sensitivity and false-positive rate, so rare deployment events can generate extensive false-alert workload at apparently favorable ROC operating points. Probability calibration and cost-sensitive decisions are distinct: a high-quality probability estimate supports an action only after incremental benefits, costs, capacity and lawful constraints are specified.
Validation consequently requires deployment-representative denominators, original imbalanced test data, temporally valid features and splits, grouping for dependent insured events, uncertainty tied to the positive-event count and independent evaluation of calibration, discrimination and decision value. Transfers across insurers, products and jurisdictions demand renewed scrutiny of definitions, covariate distributions, label ascertainment, contractual context and regulatory constraints. The original contribution of this dossier is the systematic insurance-focused interpretation of these relationships and their reproducible mathematical demonstration. It is an analytical synthesis of established evidence, not a claim of an original general learning theory or proprietary empirical discovery.
References
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. https://journals.ametsoc.org/view/journals/mwre/78/1/1520-0493_1950_078_0001_vofeit_2_0_co_2.xml
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357. https://doi.org/10.1613/jair.953
Dal Pozzolo, A., Caelen, O., Le Borgne, Y.-A., Waterschoot, S., & Bontempi, G. (2014). Learned lessons in credit card fraud detection from a practitioner perspective. Expert Systems with Applications, 41(10), 4915–4928. https://www.sciencedirect.com/science/article/pii/S095741741400089X
Davis, J., & Goadrich, M. (2006). The relationship between precision–recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning, 233–240. https://doi.org/10.1145/1143844.1143874
Elkan, C. (2001). The foundations of cost-sensitive learning. Proceedings of the 17th International Joint Conference on Artificial Intelligence, 973–977. https://ijcai.org/proceedings/2001/133
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874. https://doi.org/10.1016/j.patrec.2005.10.010
Firth, D. (1993). Bias reduction of maximum likelihood estimates. Biometrika, 80(1), 27–38. https://doi.org/10.1093/biomet/80.1.27
Gneiting, T., & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477), 359–378. https://doi.org/10.1198/016214506000001437
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning, Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a
He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284. https://doi.org/10.1109/TKDE.2008.239
International Organization for Standardization. (2018). ISO 31000:2018, Risk management—Guidelines (2nd ed.). https://www.iso.org/standard/31000
Japkowicz, N., & Stephen, S. (2002). The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5), 429–449. https://doi.org/10.3233/IDA-2002-6504
Jeni, L. A., Cohn, J. F., & De La Torre, F. (2013). Facing imbalanced data—Recommendations for the use of performance metrics. Proceedings of the International Conference on Affective Computing and Intelligent Interaction Workshops, 245–251. https://doi.org/10.1109/ACII.2013.47
Kaufman, S., Rosset, S., Perlich, C., & Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data, 6(4), Article 15. https://doi.org/10.1145/2382577.2382579
King, G., & Zeng, L. (2001). Logistic regression in rare events data. Political Analysis, 9(2), 137–163. https://doi.org/10.1093/oxfordjournals.pan.a004868
Lipton, Z. C., Elkan, C., & Naryanaswamy, B. (2014). Optimal thresholding of classifiers to maximize F1 measure. Machine Learning and Knowledge Discovery in Databases, Lecture Notes in Computer Science, 8725, 225–239. https://doi.org/10.1007/978-3-662-44851-9_15
Niculescu-Mizil, A., & Caruana, R. (2005). Predicting good probabilities with supervised learning. Proceedings of the 22nd International Conference on Machine Learning, 625–632. https://doi.org/10.1145/1102351.1102430
Riley, R. D., Snell, K. I. E., Ensor, J., Burke, D. L., Harrell, F. E., Jr., Moons, K. G. M., & Collins, G. S. (2019). Minimum sample size for developing a multivariable prediction model: Part II—Binary and time-to-event outcomes. Statistics in Medicine, 38(7), 1276–1296. https://doi.org/10.1002/sim.7992
Saerens, M., Latinne, P., & Decaestecker, C. (2002). Adjusting the outputs of a classifier to new a priori probabilities: A simple procedure. Neural Computation, 14(1), 21–41. https://doi.org/10.1162/089976602753284446
Saito, T., & Rehmsmeier, M. (2015). The precision–recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3), e0118432. https://doi.org/10.1371/journal.pone.0118432
Steyerberg, E. W., & Vergouwe, Y. (2014). Towards better clinical prediction models: Seven steps for development and an ABCD for validation. European Heart Journal, 35(29), 1925–1931. https://doi.org/10.1093/eurheartj/ehu207
Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019). Calibration: The Achilles heel of predictive analytics. BMC Medicine, 17, Article 230. https://doi.org/10.1186/s12916-019-1466-7
Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics, 7, Article 91. https://doi.org/10.1186/1471-2105-7-91
Viaene, S., Derrig, R. A., Baesens, B., & Dedene, G. (2002). A comparison of state-of-the-art classification techniques for expert automobile insurance claim fraud detection. Journal of Risk and Insurance, 69(3), 373–421. https://doi.org/10.1111/1539-6975.00023