Insurance by BogdanovaThe Encyclopedia of Insurance and Risk Science

RESEARCH DOSSIER

Zero-Inflated and Hurdle Models for Insurance Claims

Abstract. The exceptional frequency of zero reported claims in insurance data is an observation requiring explanation at the level of exposure, contractual reporting, latent risk heterogeneity, and the statistical distribution of event counts. This monograph develops the probabilistic foundations and actuarial uses of zero-inflated and hurdle models, with special attention to the distinction between a mixture that admits sampling zeros and a model that separates nonoccurrence from the positive count distribution. It derives probability masses, moments, likelihoods, exposure relationships, identification conditions, and predictive quantities for Poisson, negative-binomial, zero-inflated, and zero-truncated constructions. The study connects observed claims to underlying accidents, reporting incentives, deductibles, periods at risk, incomplete development, panel dependence, and positive insurance payments. Its independent numerical analysis demonstrates exact observational equivalence between a zero-inflated Poisson law and a corresponding Poisson hurdle law at a fixed covariate setting; it quantifies the difference between zero frequency and expected frequency; and it shows why structural interpretations cannot be read directly from an estimated mixture weight. The methods are evaluated through likelihood-based inference, calibration, decision-specific scoring, model complexity, stability, and out-of-time validation. An analytical synthesis of actuarial and statistical literature identifies where each model answers a meaningful insurance question and where better exposure measurement or a simpler overdispersed distribution should take priority. Numerical values are explicitly synthetic and mathematically reproducible. No new proprietary model, named theory, empirical insurer dataset, external review, or universal superiority claim is asserted.

Keywords: Insurance claims; excess zeros; zero-inflated Poisson; zero-inflated negative binomial; hurdle regression; zero-truncated counts; claim frequency; exposure offsets; structural zeros; unreported accidents; reporting propensity; insurance ratemaking; conditional severity; two-part loss modeling; panel data; identifiability; dispersion; maximum likelihood; predictive calibration; model risk.

Introduction

A portfolio of insurance policies can contain many records with zero reported claims in a year. The statistic is immediately consequential: claim frequency influences expected loss cost, risk classification, renewal pricing, capital planning, and evaluation of preventive measures. Yet a zero in a policy-level database is the product of a measurement rule. It can mean that no covered loss occurred, that an event occurred but was never reported, that the loss lay below the deductible, that the policy was exposed for only a fraction of the period, that a report has not yet arrived, or that a recording system represented a missing record as a zero. These possibilities generate distinct actuarial propositions. A model that fits the observed zero fraction can be useful for prediction while offering weak evidence about which proposition explains the data.

The foundational mixture construction in Lambert's zero-inflated Poisson regression (1992) assigns a special zero-producing component alongside an ordinary Poisson process. Mullahy's modified count-data analysis (1986) supplies a central precursor to hurdle and other flexible counts. Heilbron's zero-altered regression account (1994) distinguishes formulations in which the zero outcome is separately modeled from the positive count behavior. Their statistical origins do not presume a single insurance mechanism. Such mechanisms must be supplied, measured, and tested by application.

The insurance-specific evidence is substantial and heterogeneous. Yip and Yau (2005) compared several added-zero distributions for insurance claim counts in general insurance with extra zeros. Boucher, Denuit, and Guillén (2007) analyzed generalized heterogeneous, hurdle, zero-inflated, and compound models in risk classification for claim counts. Subsequent work examined the difference between accident occurrence and claim reporting under a bonus-malus system. A recent open-access statistical investigation by Faroughi, Li, and Ren (2023), generalized Poisson models for insurance claims, studies exposure in both model components and the use of alternative count distributions. These studies establish an applied research program; none renders two-component regression automatically necessary for every portfolio.

The objective of this monograph is to state precisely what these families estimate, which assumptions identify their parameters, how they support prediction and insurance decision-making, and which results can survive changes in observation design. It is a fundamental methodological treatment with comparative international scope. Insurance contracts, supervisory rules, protected characteristics, data availability, and reporting obligations vary by jurisdiction. No particular national tariff rule or claims-handling law is treated as universal. The mathematical formulas below refer to defined random variables rather than undocumented portfolio-specific practices.

Zero Claims and Observation Processes

The unit of observation precedes the count distribution

Let i index an insured risk and t a defined policy interval. The observed nonnegative integer Y_it counts claims according to a documented event definition and the insurer's observation cutoff. Let e_it be the actual exposure, measured in compatible insured time or another justified unit; X_it denotes observed risk characteristics and W_it denotes administrative, reporting, and observation characteristics. A claim count per policy-year must distinguish full-year exposure from a policy that entered one month before the extract date. Otherwise the probability of recording zero grows mechanically with shorter observation, and a flexible mixture can mistake short duration for a latent zero class.

A basic conditional count model is Y_it | X_it,e_it ~ Poisson(mu_it), with mu_it=e_it lambda_it and lambda_it>0 the expected claim rate per exposure unit. It gives P(Y_it=0)=exp(-mu_it), E(Y_it)=mu_it and Var(Y_it)=mu_it. Under a log link, log(mu_it)=log(e_it)+X_it' beta. The known log exposure is an offset with coefficient one, a scientifically meaningful proportionality restriction. It is not generally an innocuous intercept correction: if insurance participation, reporting, or event probability changes nonlinearly with time insured, the offset alone does not represent the observation mechanism.

Poisson probability already includes zero. For a policy with expected count 0.1 in the observed period, the zero probability is exp(-0.1), approximately 0.90484. A portfolio can therefore contain over 90% zeros under a perfectly ordinary Poisson law. The warning developed by Warton (2005), many zeros do not imply inflation, arose in ecological abundance data, not insurance; its mathematical lesson applies whenever small heterogeneous means naturally generate numerous zeros. A sample zero percentage, by itself, is never a formal comparison of observed and correctly conditional expected zero frequency.

Zero-generating pathways in insurance

Occurrence, notification, acceptance, and payment form a sequence of linked but different variables. Let A_it count covered-risk events, R_ij indicate reporting of event j, C_ij indicate that a reported event becomes a recognized claim under contract and administration, and P_ij measure any eventual positive amount. A simplified observed count is Y_it=sum_{j=1}^{A_it} R_ij C_ij. A zero Y can coexist with A>0; a zero net payment can coexist with a recognized claim. If claim identifiers aggregate multiple transactions, the count of payment lines also differs from the count of claims. Coverage deductibles, no-claim discounts, claim withdrawal, fraud rejection, reporting delays, and data defects act at distinct stages.

A particular substantive mechanism appears in Boucher, Denuit, and Guillén's accidents-versus-claims panel study (2009): some insureds with accidents refrain from reporting to preserve a bonus. Their model uses a censorship mechanism between accidents and claims and examines a Spanish insurer's reported claims. It supports studying reporting behavior as an additional observation process. A fitted zero-inflation probability should not simply be renamed the probability of an accident-free policy. Its substantive meaning depends on which of these stages the response Y represents.

For losses reported after the observation cutoff, lag-to-report dynamics can create zeros that disappear when the development window is extended. A valuation extract as of December 31 does not automatically provide the ultimate number of claims arising in that calendar year. A model intended for ultimate frequency must either model delay or use a sufficiently mature cohort, with residual censoring assessed explicitly. Otherwise forecast accuracy can deteriorate with longer development even when the statistical fitting routine is correct.

Dispersion and heterogeneity

Poisson equidispersion is conditional: it asserts Var(Y|X,e)=E(Y|X,e). Mixing over policy-specific intensities produces extra marginal variability without a special point mass at zero. In a gamma-mixed Poisson model the marginal count law is negative binomial. With conditional mean mu and dispersion parameter k>0, one useful parameterization has Var(Y)=mu+mu^2/k and P(Y=0)=(k/(k+mu))^k. It permits a zero probability above exp(-mu) along with greater probability of large counts. In insurance this is a compelling baseline alternative when unobserved risk characteristics vary across policyholders.

The distinction between variance due to heterogeneity, zero modification, and dependence requires an observation-level specification. Random policy effects, distributional tails, exposure histories, and repeated measurements may account for features that a two-part marginal mixture would otherwise absorb. A predictive model can fit well even if the internal components are not separately causally interpretable. Model evaluation must distinguish these two standards of success.

Scope of the probabilistic claim

Throughout the study, 'structural zero' names an event generated by a component assigning probability one to zero within the specified model. It does not imply a literally risk-free person, object, or insurer. If a policy is excluded from claims by contract design or not at risk during a period, a model-based structural state may have an operational interpretation. If this state is inferred only from one zero outcome, the assignment is uncertain and usually not identifiable as a physical condition. A zero-generating component is a statistical hypothesis requiring an observation story.

Zero-Inflated Count Models

Mixture definition and likelihood

Let f(y;theta) be an ordinary count probability mass function on nonnegative integers and f0(theta)=f(0;theta). For 0<=pi<1, a zero-inflated mixture has

P(Y=0)=pi+(1-pi) f0(theta),

P(Y=y)=(1-pi) f(y;theta), for integers y>=1.

One conceptual construction is Z~Bernoulli(pi), where Z=1 selects a degenerate zero component; if Z=0, draw Y from f. Therefore even among observations arising from the ordinary count component, sampling zeros remain possible. The data analyst sees a zero but not Z. Conditional on a zero, the posterior membership probability is pi/[pi+(1-pi)f0(theta)], conditional on the fitted parameters and model assumptions. This formula makes explicit why classifying every observed zero as structural is an error.

For observed independent counts y_i and covariates, set pi_i=logit^{-1}(W_i'gamma) and mu_i=exp(log e_i+X_i'beta) in the Poisson case. The log likelihood is

ell_ZI=sum_{i:y_i=0} log[pi_i+(1-pi_i) exp(-mu_i)] + sum_{i:y_i>0} {log(1-pi_i)-mu_i+y_i log(mu_i)-log(y_i!)}.

A structural and a count equation may share covariates, use different ones, or impose justified constraints. Shared covariates do not make the fitted coefficients directly comparable, because the logit pi_i describes mixture membership while log mu_i describes mean count within the ordinary component. Lambert (1992) developed such covariate-dependent Poisson mixture estimation. Its clean algebra is central to insurance use, although substantive identification still needs more than a maximized likelihood.

Mean, variance, and predicted zero frequency

In the zero-inflated Poisson (ZIP) law,

E(Y)=(1-pi)mu,

Var(Y)=(1-pi)mu+pi(1-pi)mu^2,

P(Y=0)=pi+(1-pi)exp(-mu).

The mean is a product of the probability of entering the ordinary count component and its mean. A change in a rating characteristic may affect expected frequency through either component. A statement such as 'an additional unit of X raises claim frequency by beta percent' is generally incomplete for a covariate that also enters pi. At fixed e, the differential is d E(Y)/dx=(1-pi) dmu/dx - mu dpi/dx. A pricing implementation must recompute the combined expected count or its appropriate gradient, rather than using the count-equation coefficient alone.

For a negative-binomial base with mean mu and dispersion k, let f0=(k/(k+mu))^k, giving a zero-inflated negative binomial (ZINB) law with the same mixture construction. Its unconditional mean remains (1-pi)mu. By the law of total variance, Var(Y)=(1-pi)(mu+mu^2/k)+pi(1-pi)mu^2. This extra flexibility can be useful, but pi and k may compete to explain the same features of an empirical zero histogram. Yip and Yau's comparison of insurance zero-inflated laws considers several possible base count distributions. Their application demonstrates a research option, not a guarantee that adding both a dispersion parameter and a mixture equation improves future predictions.

Independent worked calculation: ZIP

Take the purely synthetic parameters mu=0.8 and pi=0.30 for one full observation period. The ordinary Poisson component gives exp(-0.8)=0.4493289641 for its own zero. The ZIP model consequently gives P(Y=0)=0.30+0.70(0.4493289641)=0.6145302749. Its mean is 0.70(0.8)=0.56 and variance is 0.70(0.8)[1+0.30(0.8)]=0.6944. A population of otherwise identical policies would on average have 56 claims per hundred periods, with approximately 61.45% zero records. The last two figures must not be equated: one is a mean count, the other a probability of recording no count. Neither is an empirical estimate from a company.

For an observed zero, the fitted probability of membership in the degenerate component is 0.30/0.6145302749, approximately 0.48818. More than half of zero observations under this ZIP example arise from the ordinary Poisson component. This is a concrete demonstration that 'zero-inflated' does not mean 'all zeros came from the special component.' The numerical values also show why a simplistic binary classification of policyholders from one period of experience is scientifically weak.

Mixture identifiability and boundary behavior

The likelihood may have shallow ridges if pi and mu both shift the observed zero share, especially in sparse exposure windows or small samples with few positive counts. Near pi=0, parameters encounter a boundary and familiar interior asymptotic tests can cease to apply directly. Small groups with all zero observations may induce separation in a logistic mixture component, while a very large mu with pi near one can create unstable attempts to fit rare positive observations. Penalization, explicit priors, covariate simplification, and diagnostics of the fitted likelihood surface may be necessary. They are modeling choices to document, not automatic cures for an incorrect event definition.

An essential check is whether the apparent excess-zero pattern persists conditional on modeled risk characteristics, exposure, and appropriate heterogeneity. If it does not, the mixture parameter may chiefly capture a defective baseline. A lower in-sample information criterion cannot establish that the fitted pi corresponds to a recoverable latent group. This distinction governs the interpretation of claims decisions built from the model.

Hurdle Probability Decompositions

A binary gate and a zero-truncated count

A hurdle model assigns the observation's zero status to one process and its strictly positive count to another. Define q=P(Y>0|X,W,e), with 0<q<1, and let f(y;theta) be a base count law on zero and positive integers, with f0<1. The resulting probability distribution is

P(Y=0)=1-q,

P(Y=y)=q f(y;theta)/[1-f0(theta)], for y>=1.

The positive component is conditional on exceeding zero; its normalizing factor 1-f0 is essential. Without this division the probabilities need not sum to one. A Bernoulli equation models whether any claim is observed, while a zero-truncated Poisson, negative-binomial, or other count equation models the positive number once the hurdle is crossed. Mullahy's modified count-data specification (1986) and Heilbron's zero-altered formulations (1994) provide historical mathematical foundations; Zeileis, Kleiber, and Jackman's count regression implementations (2008) explain estimation and comparison using both hurdle and zero-inflated software.

The hurdle log likelihood has the form

ell_H=sum_{i:y_i=0} log(1-q_i)+sum_{i:y_i>0} {log q_i+log f(y_i;theta_i)-log[1-f0(theta_i)]}.

Under independent observations and disjoint parameter vectors for q and theta, this expression separates into a binary likelihood and a zero-truncated positive-count likelihood. This computational separation is useful: model fit can be inspected for occurrence and positive frequency without latent zero-state allocation. However, nonindependent observations, shared random effects, or parameter constraints reintroduce coupling across the two parts. Interpreting the gate as a physical event remains a separate substantive step.

Poisson hurdle moments and tail probabilities

For a Poisson base with mean mu, the positive component has probability f_+(y)=exp(-mu) mu^y/[y!(1-exp(-mu))]. Its conditional positive mean is E(Y|Y>0)=mu/[1-exp(-mu)], and the unconditional hurdle mean is

E(Y)=q mu/[1-exp(-mu)].

The corresponding second moment is E(Y^2)=q(mu+mu^2)/[1-exp(-mu)], so

Var(Y)=q(mu+mu^2)/[1-exp(-mu)] - {q mu/[1-exp(-mu)]}^2.

The chance of any reported claim is exactly q. The predicted count at zero is 1-q, and the positive-count shape is governed by mu. For a retained threshold K>=1, P(Y>=K)=q P_f(Y>=K)/[1-f0]. These identities permit portfolio pricing and tail calculations but require a positive-count model suited to the actual task. In particular, excess zeros and heavy positive-count tails are separate phenomena. Using a Poisson positive part can still understate high claims counts even if its zero probability is calibrated exactly.

Worked calculation and observational equivalence

For a synthetic hurdle with q=0.40 and Poisson-base mu=0.8, P(Y=0)=0.60. Its conditional mean among positive observations is 0.8/[1-exp(-0.8)]=1.4527729767; hence its overall expected frequency is approximately 0.5811091907. The variance computed from the second moment is approximately 0.7083086517. Comparing this with the earlier ZIP example illustrates that similar zero proportions do not force identical means or upper-tail shapes.

A more consequential result follows directly from the formulas. Set a ZIP model's parameters to pi and mu, and choose a Poisson hurdle with the same positive-base mu and

q=(1-pi)[1-exp(-mu)].

Then the hurdle zero probability is 1-q=pi+(1-pi)exp(-mu), exactly the ZIP zero probability. For every y>=1, the hurdle probability is q f(y)/[1-f(0)]=(1-pi)f(y), exactly the ZIP positive probability. Therefore the two models generate identical distributions for the observable count Y at that covariate setting. With pi=0.30 and mu=0.8, q=0.3854697251 and both models imply the same 0.6145302749 zero probability, 0.56 mean, and 0.6944 variance. This is a complete probabilistic equality, not merely an approximation of the first moments.

The equality has an important converse qualification: every ZIP model generates such a hurdle representation, but a general hurdle model need not correspond to a nonnegative-inflation mixture with the same Poisson base. For fixed mu, a Poisson hurdle with q>1-exp(-mu) has fewer zeros than the ordinary Poisson, a zero-deflated pattern that would require pi<0 in the ZIP transformation and is excluded by the standard ZIP parameter space. Regression restrictions across covariate values can also make a parametric ZIP and a parametric hurdle specification differ in fit, even though their one-setting probability families overlap. A positive model-selection result does not by itself identify the physical origin of the extra zeros.

Two-stage prediction and the meaning of a gate

There are at least three operational interpretations of q. For policyholders with complete exposure and fully observed reporting, q can represent the probability of at least one reported claim over the interval. For a commercial coverage with a defined attachment threshold, it can represent the probability that an eligible claim crosses the policy's claims-recognition threshold. For administrative payment datasets, it can represent the chance of a positive payment line. These are distinct response definitions. Their positive counts also have different units, severities, and temporal dependencies.

The coefficients of a logit gate quantify changes in odds of any positive observed count, not necessarily changes in pure premium. The expected count combines both equations: for a parameter x affecting q and mu, d log E(Y)/dx = d log q/dx + d log mu/dx - d log[1-exp(-mu)]/dx. When both parts respond to a characteristic, conclusions based on one coefficient ignore the other. For an insurance pricing audience, a rating comparison should report predicted total frequency at standardized exposure and, where relevant, the occurrence probability and mean conditional positive count as separate audited quantities.

Hurdles for monetary claims data

A two-part model can also be applied to a semi-continuous response S representing total nonnegative claim cost per policy period: P(S=0)=1-q, while conditional positive S follows a positive density such as lognormal or gamma. This is conceptually related to a count hurdle, but a payment amount is continuous and its positive component is not a zero-truncated count. The expectation is E(S)=q E(S|S>0). If claim count and individual claim severities are observed, frequency–severity aggregation may preserve more event-level information; if only total cost is retained, a two-part model can be operationally practical. Smyth and Jørgensen's compound Poisson insurance cost model (2002) represents another coherent way to obtain a positive-cost distribution with a point mass at zero. The Tweedie/compound Poisson–gamma approach belongs to a related but distinct dossier and should not be treated as algebraically identical to a Bernoulli hurdle with arbitrary positive severity.

A positive claim amount conditioned on an above-deductible payment may follow a shifted or truncated severity distribution, and policy limits may induce censoring or a point mass at the cap. Recording a zero payment for rejected claims can create a separate administrative zero mechanism even when an actual loss occurred. Thus the response's contractual construction must be documented before a convenient two-part statistical fit is given economic meaning.

Exposure and Selection Effects

Exposure enters more than one equation

For a homogeneous arrival process with intensity lambda per insured year, a policy observed for exposure e has chance of at least one event 1-exp(-e lambda). This equation offers a natural benchmark for the hurdle gate when the gate represents claim occurrence, while log(e) is the canonical offset in the Poisson mean mu=e lambda. The gate therefore has a nonlinear relationship with exposure even if lambda is constant. A simple logistic gate with an arbitrarily fixed exposure coefficient may fit some ranges of e yet fail as e approaches zero. In particular, a model implying a positive occurrence probability at e=0 contradicts the ordinary event-clock interpretation unless there are preexisting or delayed claims included in the outcome definition.

For an actual exposure-governed Poisson process, a complementary log-log occurrence link yields log[-log(1-q)]=log(e)+log(lambda), which directly reproduces q=1-exp(-e lambda). A logistic link can be adopted when the response is an administrative reporting probability or when the exposure mechanism is more complex, but it should be assessed in the correct interpretation. Faroughi, Li, and Ren (2023) specifically study exposure in both model components rather than assuming that exposure is relevant only in the positive or count equation. The empirical performance of their candidate specifications is portfolio- and sample-dependent.

If a fixed structural mixture fraction pi is applied across exposure durations, a ZIP model has P(Y=0|e)=pi+(1-pi)exp(-e lambda). As e approaches zero, P(Y=0|e) approaches one; as e becomes large, it tends to pi. Under a genuine structurally unable-to-claim subgroup, such an asymptotic plateau can be meaningful. Under an interpretation in which pi represents underreporting, however, repeated opportunities for reporting may affect that probability. A fitted exposure-invariant pi is then a convenience, not a demonstrated behavioral constant.

Zero frequency under different exposure amounts

Consider only a model illustration: lambda=0.6 claims per full exposure year and pi=0.30. For exposure e in {0.25,0.50,1.00,2.00}, ordinary Poisson zero probabilities are approximately {0.86071,0.74082,0.54881,0.30119}; ZIP zero probabilities are approximately {0.90250,0.81857,0.68417,0.51084}. The differences are generated without changing underlying lambda or pi. These values are analytically obtained from the displayed formulas and are not observations about any country, product, or insurer. A binary model trained on unequal policy durations without exposure variables can confuse these patterns with segmentation by age, geography, coverage choice, or other characteristics correlated with duration.

Further distortion occurs if periods of zero insured exposure are retained in a dataset alongside actual in-force policies. Such rows may be valid for a portfolio transition model, but they should not be interpreted as ordinary at-risk policy-period observations. Selection into an insurance portfolio, lapses, midterm endorsements, vehicle mileage, payroll, number of insured locations, and claims event clocks can all alter the effective risk measure. The most appropriate exposure unit follows the mechanism being modeled, rather than a mechanical convention of one row equals one year.

Accidents, claims, and reporting incentives

Insurance observed claims are often a filtered sample of underlying losses. Boucher, Denuit, and Guillén (2009) examine strategic accident underreporting in experience rating. Their analysis suggests a substantive interpretation of extra zeros linked to the reporting decision, but it does not imply that all extra-zero models recover true accident frequencies. If each accident is reported independently with probability r, and accident count APoisson(lambda), elementary thinning gives YPoisson(r lambda). In this simple case underreporting reduces the observed Poisson mean and creates no separately identifiable zero-inflation component. Additional heterogeneity in r, dependence across accidents, or a latent group systematically refusing to report can yield extra variation or an added zero mechanism. The mechanism depends on the reporting process, not merely the word 'underreporting.'

This thinning calculation is vital. Given only Y, a homogeneous occurrence rate lambda and homogeneous reporting probability r appear through their product r lambda. They cannot be separately estimated from the marginal Poisson count distribution without external information, such as independent accident records, reporting audits, claim incentives, or credible constraints on one factor. Estimating a two-equation model does not manufacture the missing information. Interpretability is strengthened by additional measurements, not just an additional parameter vector.

Panel dependence and policyholder persistence

Repeated annual observations from the same policyholder need not be independent. A stable unobserved driving style, occupation, property protection standard, or regional exposure affects several periods. Random-effects and hierarchical count models can represent this persistence, and serially changing claims experience may influence reporting behavior and future coverage. Min and Agresti's random-effects two-part analysis (2005) discusses these problems for correlated zero-heavy measurements; Boucher and Guillén's insurance panel count survey (2009) situates them in actuarial risk classification.

A policy-level train/test split is not equivalent to prediction of future claims for an existing book. If the same policyholder appears in both partitions, stable unobserved effects and prior outcomes can leak information. Conversely, a purely time-ordered split may still share policyholder histories by design, which is appropriate for renewal forecasts if the historical features would have been available at the forecasting date. Validation should match the decision: new-business acquisition, renewal underwriting, next-period claims forecast, or total company loss forecast. The relevant test population, information cutoff, and loss function differ across those tasks.

Product-specific differences

Short-tail motor policies provide a natural count-data setting, but accident counts, filed claims, own-damage claims, third-party claims, and repair invoices are different response variables. Health insurance can feature many transactions per beneficiary, plan-specific copays and deductibles, and utilization decisions whose counts are not direct equivalents of accidental losses. Property catastrophe claims can cluster across exposures under common weather events, so independent policy counts can understate portfolio dependence even when the zero rate fits. Commercial liability claims may have lengthy reporting and development periods, so observed zeros can reflect immaturity. Parametric and index-based contracts can yield zero payments because the trigger has not been met; their threshold mechanism may make a two-part outcome description intuitive, but it does not guarantee independent positive severities.

The statistical families are internationally portable as mathematical constructions; their estimated parameters are not transportable between markets without studying coverage definitions, currency, inflation, judicial and administrative practice, data collection, and risk mix. The governing legal scope of a claim and its reporting time are characteristics of the contract and local institutional setting, not properties of a Poisson probability law.

Estimation and Model Comparison

Baselines before mixture complexity

A defensible analysis begins with a count definition, exposure distribution, data availability cutoff, and simple conditional benchmark. For counts, fit Poisson and at least one appropriate dispersion-aware model such as negative binomial where justified. Inspect observed versus expected zero fractions conditional on exposure and major risk covariates; inspect positive counts, residual variability, subgroup predictions, and periods with anomalous data quality. An added-zero model becomes a substantive candidate when a conditional benchmark fails in a way that corresponds to a plausible second process. Insurance applications by Yip and Yau (2005), Boucher, Denuit, and Guillén (2007), and Faroughi, Li, and Ren (2023) all give reason to compare multiple count families rather than treating Poisson versus ZIP as the entire decision set.

Warton (2005) found in an ecological study that negative-binomial heterogeneity often accommodated abundant zeros without an added mixture. This cross-domain result is an argument for model comparison, not a direct empirical finding about insurance. Insurance covariate distributions, policy conditions, reporting stages, and incentives differ. The methodological consequence is to ask whether zero inflation remains after exposure and conditional heterogeneity have been modeled adequately.

Maximum likelihood and numerical optimization

The hurdle likelihood separates into occurrence and truncated count contributions when its parameter sets are independent. Maximum likelihood can then estimate a binary generalized linear model and a positive-truncated count model, with an appropriately combined predictive distribution. ZIP and ZINB likelihoods contain the sum pi+(1-pi)f0 for zero observations; direct optimization or expectation-maximization procedures may be used. Implementation must control numerical underflow for exp(-mu), stability of log[1-exp(-mu)] when mu is small, and gradients near boundaries.

For a Poisson hurdle with very small mu, use a numerically stable computation of -expm1(-mu) rather than subtracting two nearly equal floating-point numbers. Positive-truncated models are especially susceptible to weak information when almost all positive observations equal one. Such data may identify occurrence probabilities well while providing little information about the tail of the positive count law. Confidence intervals for predicted mean count should propagate uncertainty from both components, including their covariance where shared parameters or sampling dependencies exist.

Zeileis, Kleiber, and Jackman (2008) describe hurdle and zero-inflated regression implementations, supplying a reproducible computational reference. Numerical convergence messages are necessary but insufficient: multiple starting values, Hessian inspection, sensitivity to sparse categories, and predicted probability mass summing to one are substantive checks. A likelihood maximum for a wrongly selected event definition remains a wrong scientific interpretation.

Model selection, nested tests, and information criteria

Likelihood-based selection compares different approximations to an observed conditional distribution. The Akaike information criterion, AIC=-2 ell_hat+2k, trades fit against parameter count; BIC=-2 ell_hat+k log n uses a different complexity penalty under its own asymptotic assumptions. Counts with clustering require attention to the number of independent statistical units, because treating thousands of correlated policy-year rows as independent inflates the effective information base. Information criteria are useful diagnostics, not certificates of structural zeros, future accuracy, fairness, or causal mechanisms.

The commonly used Vuong comparison deserves special care. Desmarais and Harden (2013) demonstrate that the uncorrected Vuong test for zero inflation can favor a more complicated zero-inflated count model in settings without genuine extra zeros; their article evaluates information-criterion corrections. Comparisons with a parameter on the boundary or models that overlap require further attention to regularity assumptions. In practice, a test statistic should be reported with the candidate models, nesting/overlap status, correction used, exposure and covariate design, and a forecast comparison. A single p-value cannot settle the measurement question.

For two candidate distributions that represent the same observable probability law under different parameterizations, no amount of sample size separates the latent stories at that setting. The earlier exact ZIP–hurdle equivalence shows this constructively. Distinguish the ability to estimate a response distribution from the ability to identify its decomposition into unobserved events. The former can be strong while the latter is unsupported.

Validation tailored to forecasts and insurance decisions

Prediction should be evaluated beyond a single zero-fraction statistic. A model can predict P(Y=0) well and give dangerously inadequate probabilities for Y>=2. Assess full probability mass functions by held-out negative log likelihood or another proper score, binary calibration and Brier score for Y=0 or Y>0, calibration of predicted mean count, tail probabilities, and aggregate portfolio counts. Gneiting and Raftery's proper scoring rule framework (2007) motivates evaluating complete probabilistic forecasts rather than rewarding a model for a favorable point prediction alone.

For the event I(Y>0), the Brier loss per observation is (q_hat-I)^2. For the full count prediction p_hat(y), the negative log score is -log p_hat(Y). Proper scores assess distributional truthfulness in expectation under defined evaluation distributions. They do not automatically express the insurer's asymmetric business cost of overpricing, underpricing, reserving error, or capital shortage. That decision layer should be evaluated separately through calibrated expected costs and transparent sensitivities. These two layers answer different questions and should not be confused.

Use forward-chaining or genuinely held-out calendar periods for temporal forecasts. Stratify or report results by exposure bands, insurance product, claim maturity, prior claims history, policy inception and renewal status, and material contractual features when ethically and legally appropriate. A model's pooled score can hide systematic underprediction for a small but financially important segment. Confidence bounds on segment metrics may be wide; explicitly state when the data are insufficient for a stable subgroup conclusion.

Robustness, alternative explanations, and selection bias

Perform sensitivity analyses for the censoring cutoff, open versus closed claims, changes in deductible, claim definition, policy exposure, missing-data treatment, and migration between administrative systems. Positive-count observations may be a selected subsample: insureds with at least one reported claim can differ in unmeasured risk and reporting behavior. In a hurdle model this is built into the conditional response definition, and it does not justify treating the positive component as an unconditional model of all potential losses.

If an insurer's business is shrinking, the observed zero share may rise because remaining policies are shorter, because the company is selectively retaining lower risks, or because reporting systems have changed. A flexible zero-inflation equation can absorb any of these in-sample patterns. Historical correlations are not evidence that the estimated mixture probability measures an immutable behavioral class. When an intervention such as a new deductible or reporting app changes observed counts, causal conclusions require an appropriate identification design and explicit hypotheses about selection and reporting, not only a before-and-after zero count.

Reproducibility and uncertainty record

A replicable research report should record: the claim count definition; the policy/exposure denominator; the observation and reporting cutoff; feature timestamps; exact count-family parameterizations; link functions; random-effect structure; starting values; optimizer and software versions; fitting diagnostics; evaluation partitions; all-scored observations; fitted zero probabilities; aggregate expected counts; uncertainty intervals; and material excluded records. Preserve a distinction between mathematical demonstration and actual insurer calibration. The former verifies identities; the latter evaluates performance under a real distribution with documented provenance.

Interpretation and Risk Prediction

What a zero component estimates

In ZIP, the pi parameter is the conditional probability of membership in a degenerate-zero mixture component under an assumed latent construction. The observed probability of no claim is larger: pi+(1-pi)f0. In a hurdle model, 1-q is directly the total modeled probability of no observed claim, and the positive component is a conditional distribution. These quantities cannot be interchanged. In particular, a ZIP pi estimated from claims data need not measure a biological, behavioral, or contractual impossibility of loss. A gate fitted to payment data need not describe event occurrence. Interpretation must name the count and observation mechanism.

One lesson from actuarial classification studies is that data-generating mechanisms and prediction goals may justify different constructions. Boucher, Denuit, and Guillén (2007) compare multiple claim-frequency classifications; their work supports rigorous comparative specification rather than a single preferred family. Faroughi, Li, and Ren (2023) report flexible generalized Poisson and hurdle forms across three datasets; those results motivate additional candidate models while leaving decisions to future portfolio-specific assessment. Cross-study comparisons must respect differing data definitions and validation protocols.

Pure premium and conditional expected counts

Let N be the number of observed covered claims and X_j their individual positive costs. If costs are identically distributed with finite mean m and are independent of N under the specific modeling assumptions, then E[sum_{j=1}^N X_j]=E(N)m. With ZIP frequency, the expected cost is (1-pi)mu m; with Poisson-hurdle frequency, it is q mu m/[1-exp(-mu)]. If severity is associated with count, exposure, or the hurdle state, the product formula must be replaced by a conditional aggregation, such as E[E(S|N,X)]; a blanket frequency-times-severity calculation may be biased.

A change in the contractual definition of a claim can change both N and the severity distribution. For example, removing small below-deductible claims from the count usually decreases observed N and alters conditional X_j. A model calibrated under one deductible cannot simply be transplanted to another deductible without re-expressing the observation process. Similarly, group differences in reporting propensity can affect observed claim frequency independently of actual loss-producing events. These distinctions matter for affordability, consumer treatment, and the scientific interpretation of risk classification.

Forecast horizons and probability limits

A model fitted to annual counts does not automatically imply independent monthly counts. Under an underlying Poisson process, time aggregation has a particular probabilistic structure, but a general zero-inflated annual mixture can represent a stable state across the entire year, a period-by-period selection, or a reporting artifact. Those interpretations imply different predictions for shorter intervals and policy renewals. If a long-run mixture membership is persistent, repeated zeros may increase the posterior probability of structural membership; if each interval's mixture assignment is independently redrawn, the same history conveys a different amount of information. A fitted cross-sectional ZIP curve alone does not tell us which temporal interpretation holds.

The sample's observed zero percentage is also sensitive to evaluation horizon. Extending a year to two years often reduces the fraction with zero observed claims, even if underlying per-year risks do not change. Exposure-standardization, reporting maturation, and a precisely defined horizon are therefore central to portability. Report 'probability of no reported claims in the next twelve months under stated coverage and exposure' rather than an unqualified 'zero-risk probability.'

Model risk, equity, and governance

Insurance classification can have distributive consequences. A model assigning a policy to a high-probability structural-zero class can underprice or overprice if that classification reflects uneven claim reporting, benefit access, or recording quality rather than underlying covered risk. Regulated rating factors, privacy, antidiscrimination constraints, and consumer explanations depend on jurisdiction. Scientific QA should retain the difference between predictive association and permitted operational use, and any legal conclusion requires the relevant current rules. A model family itself grants no exemption from those obligations.

Governance should separately monitor source-data quality, exposure calculation, true event maturity, zero calibration, positive-count tails, model drift, and changes in reporting practices. Some weaknesses will be fixed better by data engineering or claims-process redesign than by adding mixture components. A high AIC advantage with unstable structural probabilities is not an operational success if core predictions are sensitive to arbitrary coding decisions. Conversely, a simple hurdle decomposition can be valuable as a transparent operational reporting model even without a claim that the gate identifies a deep latent population.

Integrated synthesis

The mathematical and actuarial results support five connected conclusions. First, a zero outcome is defined by an observation process: event occurrence, claim recognition, reporting, and payment must be specified before a statistical law can be interpreted. Second, excess zeros are relative to a conditional benchmark that includes exposure, heterogeneity, and data maturity. Third, ZIP and hurdle parameterizations can represent the same observable distribution; therefore their choice alone cannot demonstrate a distinct latent cause. Fourth, the two components describe different predictive quantities, and insurance decisions should use their combined expected losses and complete probability forecasts. Fifth, model adequacy depends on the desired decision and future data, not merely on fit in a historical insurance database.

These propositions are an original analytical synthesis and set of exact mathematical deductions from established probability families and verified literature. They do not introduce a new named actuarial theory, proprietary statistical model, or universal selection algorithm. They supply a way to read model output and evidence with greater conceptual precision.

Future research directions

Three empirical directions are especially consequential. Jointly observing accidents and reported claims could separate occurrence intensity from reporting propensity under transparent identification assumptions. Linking policy exposure, claims maturity, and administrative status histories could distinguish genuinely zero event intervals from zeros caused by reporting delay. Finally, prospective comparisons of Poisson, negative binomial, hurdle, and zero-inflated regressions under shared forecasting cutoffs and insurance decision metrics could establish the conditions under which added-zero complexity produces stable value. These are research questions, not claims of completed empirical tests in this monograph.

Conclusion

Zero-inflated and hurdle models provide rigorous and flexible descriptions of insurance claims data with substantial mass at zero. The zero-inflated family combines a degenerate-zero component with a count distribution that can itself generate zeros. The hurdle family models the probability of a positive observation separately from the distribution of counts conditional on positivity. Their probability laws, likelihoods, means, variances, and count-tail predictions can be derived exactly and tested on appropriately specified observation units. Both can be extended to overdispersed count distributions; analogous two-part ideas can represent semi-continuous insurance costs when their different measurement scale is respected.

The most consequential scientific finding is the exact one-setting equivalence between ZIP and a corresponding Poisson hurdle distribution. Even perfect observation of their identical count probabilities cannot reveal which latent narrative produced the zeros. Physical explanations require additional information about exposure, claims administration, underlying accidents, timing, policyholder behavior, and contractual selection. The monograph's synthetic numerical demonstrations verify that a high zero probability may accompany meaningful positive expected frequency and that a fitted structural mixture component is distinct from the proportion of actual zero records.

For actuarial modeling, reliable practice begins with the unit of observation and exposure; compares simple and dispersion-aware count baselines; examines reporting and selection; evaluates the zero and positive components alongside their combined expected costs; and validates full predictive distributions out of sample. Internal scholarly and editorial conclusions are tied to cited primary literature, with source-access limitations recorded separately. The result is a complete analytical account of established two-part count-model methodology for insurance, suitable for a staged research package, without a public release claim or externally reviewed empirical findings.

References