RESEARCH DOSSIER
Hierarchical and Mixed-Effects Models for Insurance Losses
Abstract. Insurance losses arise from structures that are simultaneously individual, contractual, organizational, geographic, temporal, and institutional. A claim count associated with one policy depends on exposure and observable rating characteristics while sharing unobserved influences with claims from the same vehicle fleet, broker network, insurer, territory, calendar period, or insured enterprise. Claim severity introduces further conditional sampling, tail, censoring, inflation, and settlement processes. This monograph analyzes hierarchical and mixed-effects methods for representing these structures and for making defensible insurance predictions. It develops the mathematical distinction between conditional and population-average quantities, derives pooling weights and count-mixture consequences, identifies circumstances under which random-effects models reproduce actuarial credibility, and derives the effect of shared frequency–severity heterogeneity on pure-premium expectations. It reviews likelihood, approximate and Bayesian inference; compares nested and crossed designs; addresses claims development, sparse classes, zero inflation, measurement design, and subgroup fairness; and specifies validation protocols that respect the intended prediction target. The analysis connects foundational statistical results with motor, workers’ compensation, health, and intercompany insurance applications. A fully hypothetical numerical example illustrates how uncertainty and group sample size determine shrinkage. Its core finding is that useful pooling requires a correctly defined unit of shared risk and a prediction estimand consistent with underwriting, reserving, or portfolio aggregation. The manuscript offers original analytical synthesis of established methods without claiming a new statistical theory or empirical insurer performance result.
Keywords: Hierarchical models; mixed-effects models; insurance losses; generalized linear mixed models; claim frequency; claim severity; random intercepts; random slopes; credibility; partial pooling; experience rating; actuarial prediction; aggregate loss; longitudinal data; model risk; Bayesian inference.
Introduction
Insurance contracts generate observations through an institutional process: risks are selected and classified, periods of coverage are defined, exposures accrue, incidents occur, claims are reported, liabilities are assessed, and payments may continue for years. Observations from the same contract, fleet, employer, medical provider network, event, or carrier are often connected. Their proximity can be scientifically important even when every recorded row looks like a separate policy. An analysis that treats all rows as independent may understate prediction uncertainty, confuse group selection with group experience, and assign apparent precision to fragile small-cell estimates.
The foundational insight of random-effects models for longitudinal data is that repeated measurements can be represented by common latent quantities and an explicit covariance structure (Laird and Ware, 1982). The generalized linear mixed-model framework extends that logic to claim counts and other non-Gaussian measurements (Breslow and Clayton, 1993). Insurance has its own intellectual lineage: Bühlmann and Straub’s credibility analysis, Jewell’s exponential-family result, and Nelder and Verrall’s actuarial synthesis establish the relationship between experience, pooling, and hierarchical estimation.
The object of this Research Dossier is the full family of loss-related mixed models: Gaussian linear mixed models (LMMs), generalized linear mixed models (GLMMs), multilevel Bayesian models, hierarchical generalized linear models, and related longitudinal constructions. Its editorial boundary is clear. Classical credibility coefficients and their independent theory have a separate detailed treatment in actuarial credibility research; here their formulas serve to explain how group-level statistical estimation connects with that tradition. The analysis concerns mathematical methodology and comparative international use, not the determination of an individual’s premium or a regulatory filing in one jurisdiction.
The method is analytical. First, the observable data-generating and recording structures are distinguished. Second, likelihoods, marginal moments, and posterior expectations are derived under stated assumptions. Third, publisher-verified actuarial applications identify what has already been demonstrated empirically. Finally, model validation is connected to the intended decision. Mathematical examples use declared hypothetical parameters and constitute neither empirical evidence nor an estimate of market-wide loss behavior. Source access is primarily publisher metadata, original abstracts, an official standard, and available archival bibliographic records; specific claims requiring unavailable article text are deliberately excluded.
Grouped Insurance Data
The observational unit and its economic meaning
A loss record must be indexed by the quantity that has actually been measured. Let (i) denote a coverage unit, (g) a risk group, (t) an accounting or exposure period, and (k) a claim within the unit-period. Write (E_{igt}>0) for time at risk, vehicle-years, payroll, insured lives, or another appropriate exposure base; (N_{igt}) for reported or occurrence-based claim count under a declared convention; (Y_{igtk}) for the severity of a qualifying claim; and (S_{igt}=\sum_{k=1}^{N_{igt}}Y_{igtk}) for aggregate covered loss on the chosen basis. A pure-premium rate is ordinarily the expected aggregate covered loss divided by its defined exposure, subject to the product and accounting conventions. It is not necessarily a premium charged to the insured, which can include expenses, risk margins, taxes, subsidies, and commercial decisions.
A policy identifier is not always a statistical cluster. One fleet policy may contain several vehicles, each observed across renewal years; an employer may own several contracts; an insurer may serve many fleets; an event may generate losses across carriers. Statistical clustering is therefore determined by the mechanism of shared dependence. In the Singapore intercompany claim-count analysis, Antonio, Frees, and Valdez (2010) investigated vehicle, fleet, and insurer structure using nine years of pooled motor data. That study demonstrates a materially different data hierarchy from a single company’s cross-sectional price file. The same work compared several count families, including Poisson, negative binomial, zero-inflated, and hurdle formulations. Its published abstract supports those methodological facts; this monograph makes no unverified claims about exact coefficients or numerical superiority in that dataset.
Let (x_{igt}) contain measured tariff variables. A typical frequency model is
[ N_{igt}\mid b_g,u_i \sim \operatorname{Poisson}(\lambda_{igt}),\qquad \log \lambda_{igt}=\log E_{igt}+x_{igt}^{\mathsf T}\beta+b_g+u_i. ]
Here (\log E_{igt}) is an offset with its coefficient fixed at one, (b_g) is shared group heterogeneity, and (u_i) is persistent unit heterogeneity. This specification assumes exposure-proportional expected counts conditional on covariates and random effects; it is unsuitable when increasing exposure changes the underlying per-unit hazard or the denominator is defined inconsistently. If a data source records only claimants, its conditional distribution is a selected population. If multiple records contain the same event, event-level effects may be essential for accumulation, while repeated rows must be reconciled to the economic risk transfer.
Hierarchy, cross-classification, and calendar shocks
Nested groups arise when every unit belongs to exactly one fleet and every fleet to one carrier during the period. Cross-classification arises when the same insured is served by multiple hospitals or multiple claims adjusters, or when catastrophe events affect many territories and carriers simultaneously. A model might include an insurer effect (b_{c(i,t)}), a territory effect (v_{r(i,t)}), and a calendar shock (w_t). These are not interchangeable latent variables: each implies a distinct covariance network. A calendar-wide shock shared by all insurers can remain in the data even after insurer intercepts are modeled, and erroneously omitting it can make diversification appear greater than it is.
The broader longitudinal and panel-data framework treats repeated observations and dependence across individuals and periods (Frees, 2004). The methodological treatment of discrete longitudinal outcomes distinguishes joint, marginal, and conditional formulations (Molenberghs and Verbeke, 2005). The corresponding insurance design should document whether membership can change at renewal, whether past claims affect future selection, and whether the time index denotes underwriting year, accident year, report year, or payment period. The resulting estimands differ because development and exposure do not coincide automatically.
Sampling, truncation, and claims administration
Observed loss is an administrative construct. Deductibles produce left truncation of positive insured payments; policy limits can cause right censoring of underlying ground-up severity; open claims may be incompletely developed; claim reporting can change after system migration. A severity model fitted only to paid claims above a deductible estimates a conditional observed distribution unless the truncation and limit mechanism enters the likelihood. Replacing all sub-deductible claims with zero changes the object from ground-up severity to insured payment. Both objects can be legitimate, but the insurer must name which one it models.
For policy terms (d) (deductible) and (u) (limit of insurer payment), the contractual payment for a ground-up loss (X) may be represented, under a simple occurrence limit, as (Y=\min{(X-d)_+,u}). Its expected value depends on the full distribution around both thresholds. A random intercept on (\log E[X]) alone cannot recover the insured expectation if the group-specific shape and tail behavior vary. More detailed severity modeling or payment-layer likelihoods are then required.
A priori information and the available comparison set
An observed group-average loss rate combines predictable risk characteristics and historical experience. Treating the resulting difference as a pure group effect confuses underwriting composition with residual experience. The work on a priori and a posteriori classification is directly relevant because it distinguishes information available before the observation period from updates based on claims history (Antonio and Valdez, 2012). Design documentation should record when every feature becomes observable, whether it was generated by a future claim, whether the feature’s group distribution changed, and whether reporting practices changed the measurement scale.
Random Intercepts and Slopes
Gaussian prototype and its exact covariance
The linear mixed model provides a transparent mathematical starting point:
[ y=X\beta+Zb+\varepsilon,\qquad b\sim N(0,G),\quad \varepsilon\sim N(0,R),\quad b\perp\varepsilon. ]
It follows that (E[y]=X\beta) and (\operatorname{Var}(y)=ZGZ^{\mathsf T}+R). The diagonal of this matrix contains observation-level and shared-group variation, while its off-diagonal entries encode dependence. If observations (j) and (\ell) share a random intercept (b_g\sim N(0,\tau^2)), their covariance contains (\tau^2) under independent residual errors. The intraclass correlation in the balanced homoscedastic special case is (\rho=\tau^2/(\tau^2+\sigma^2)). This formula describes a Gaussian variance partition and cannot be copied unmodified to a Poisson or logistic GLMM with a mean-dependent response variance.
For a Gaussian response and known variance components, the conditional expected random effect is (E[b\mid y]=GZ^{\mathsf T}(ZGZ^{\mathsf T}+R)^{-1}(y-X\beta)). This expression displays partial pooling as a matrix operation: the data provide group-specific residual signals, and the estimated covariance structure determines how much each signal updates the group effect. The classical Bayesian linear-model treatment develops the role of exchangeability and hierarchical modeling (Lindley and Smith, 1972). Estimating (G), (R), and (\beta) introduces additional uncertainty; plugging in point estimates does not turn the resulting predictor into an exact posterior calculation.
Random slopes and group-level responses to exposure
A random-intercept model assumes that all groups share the same fitted effect of a predictor after covariate adjustment. Some insurance contexts motivate a random slope: inflation pass-through may differ by claim-handling network, experience response by fleet, and an intervention by employer. Define
[ \eta_{igt}=x_{igt}^{\mathsf T}\beta+b_{0g}+z_{igt}b_{1g},\qquad \begin{pmatrix}b_{0g}\b_{1g}\end{pmatrix}\sim N!\left(0, \begin{pmatrix}\tau_{00}&\tau_{01}\\tau_{01}&\tau_{11}\end{pmatrix}\right). ]
The contribution to latent variance at predictor value (z) is (\tau_{00}+2z\tau_{01}+z^2\tau_{11}). A negative covariance between intercept and slope can reduce heterogeneity around a particular predictor value while increasing it elsewhere. Centering (z) at a meaningful actuarial baseline changes interpretation of the intercept and the covariance, while leaving an exactly reparameterized model’s likelihood unchanged. Random slopes require within-group predictor variation; groups observed at a single (z) value provide weak separation of slope and intercept.
Poisson-lognormal effects: the marginal mean is higher
For one cluster-specific multiplicative effect, let (N_i\mid b\sim\operatorname{Poisson}(E_i\exp(\eta_i+b))), (b\sim N(0,\tau^2)). Integrating the conditional rate gives
[ E[N_i]=E_i\exp(\eta_i+\tau^2/2). ]
Thus (E_i\exp(\eta_i)) is the count expectation for a group with its latent intercept fixed at zero, not the unconditional population-average count. This distinction is essential when translating a fitted intercept to portfolio totals or predicting a completely new group. The moment-generating function of a normal variable establishes the formula; it is an analytical implication of the specified model, not an empirical claim about any insurer.
With the same shared (b) and conditional independence, write (m_i=E[N_i]). The total-variance and total-covariance identities yield
[ \operatorname{Var}(N_i)=m_i+m_i^2\left(e^{\tau^2}-1\right),\qquad \operatorname{Cov}(N_i,N_j)=m_im_j\left(e^{\tau^2}-1\right)\quad(i\ne j). ]
The excess term is dependence arising from a shared mixing variable. It affects the variance of portfolio aggregates and the estimated benefit of diversification. By Jensen’s inequality, (P(N_i=0)=E[\exp{-E_i e^{\eta_i+b}}]\geq\exp{-m_i}): heterogeneity can generate more zeros than a simple Poisson model having the same mean. More observed zeros therefore do not, by themselves, establish a separate structural zero-generating mechanism. The interpretation also depends on the dependence unit and the distribution of latent effects.
Fixed effects and correlated group composition
Calling a coefficient “random” means that its levels are modeled through a distribution, not that the underlying characteristic was assigned at random. A distributional assumption for (b_g) and the usual exogeneity requirements are substantial scientific choices. A company effect may correlate with portfolio composition, territory, claim settlement culture, or calendar period. A simple group random intercept independent of (x) can then confound covariate response with selection. The Belgian motor-insurance comparison explicitly investigates the interpretations of fixed and random effects and the residual heterogeneity they represent (Boucher and Denuit, 2006).
Practical sensitivity analysis compares correlated random-effects specifications (such as group means of time-varying predictors), fixed group effects where feasible, and a justified random-effects specification. Group fixed effects can control arbitrary time-invariant group confounders in an appropriate panel design, but do not necessarily identify time-invariant covariate effects and are not a complete solution to dynamic selection. Random effects are particularly valuable for predicting new or sparsely observed groups, provided their exchangeability class is plausible. The desired external prediction task must determine which distinction matters.
Credibility and Partial Pooling
The common mathematical structure
Consider a group’s approximately unbiased estimate (\bar y_g) of a latent group risk level (\theta_g), with (\bar y_g\mid\theta_g\sim N(\theta_g,v_g)) and (\theta_g\sim N(\mu,\tau^2)), where the variances and (\mu) are known for this derivation. Completing the square yields
[ E[\theta_g\mid\bar y_g]=Z_g\bar y_g+(1-Z_g)\mu,\qquad Z_g=\frac{\tau^2}{\tau^2+v_g}, ]
and (\operatorname{Var}(\theta_g\mid\bar y_g)=\tau^2v_g/(\tau^2+v_g)). With independent observations of known within-group variance (\sigma^2), (v_g=\sigma^2/n_g) and (Z_g=n_g\tau^2/(n_g\tau^2+\sigma^2)). Small groups borrow more information; precisely measured groups receive greater weight. The estimator is a continuous weighted average rather than a binary choice between segment-wide and individual experience.
The historical Bühlmann–Straub paper generalized this experience-weighting problem to unequal risk exposures. Jewell’s exact Bayesian result shows why some credibility calculations coincide with posterior means for exponential-family/conjugate formulations (Jewell, 1974). Credibility within hierarchical GLMs connects these principles to a broader range of actuarial response distributions (Nelder and Verrall, 1997). Equality of formulas in a particular model is not general identity of objectives: classical credibility may optimize a specified mean-squared error criterion, whereas a contemporary mixed model also represents covariances, interactions, the full predictive distribution, and structured multilevel grouping.
A fully reproducible hypothetical example
Take an overall standardized residual log-loss relativity (\mu=0.12) and between-group prior variance (\tau^2=0.09). Group A has an estimated residual level (\bar y_A=0.50) with measurement variance (v_A=0.04); Group B has (\bar y_B=-0.20) with (v_B=0.36). These are stipulated illustrative normal-summary values, not observed insurance data. The formulas give
[ Z_A=\frac{0.09}{0.09+0.04}=0.6923077,\quad E[\theta_A\mid y]=0.3830769,\quad \operatorname{Var}(\theta_A\mid y)=0.0276923; ] [ Z_B=\frac{0.09}{0.09+0.36}=0.2,\quad E[\theta_B\mid y]=0.0560000,\quad \operatorname{Var}(\theta_B\mid y)=0.0720000. ]
The corresponding posterior standard deviations are approximately 0.1664 and 0.2683. On the multiplicative scale, exponentiating the posterior means gives 1.4668 and 1.0576. These are exponentiated posterior means of the log parameter; they are not posterior means of the multiplicative risk. For a Gaussian posterior, (E[\exp(\theta_g)\mid y]=\exp(E[\theta_g\mid y]+\tfrac12\operatorname{Var}(\theta_g\mid y))), which is approximately 1.4872 for A and 1.0964 for B. The calculation exposes a recurrent actuarial error: applying an inverse link to a point estimate and describing it as a posterior predictive mean discards Jensen curvature and uncertainty.
The comparison isolates the role of measurement precision. Group B has a negative raw deviation from the overall level, but a much noisier estimate and therefore a posterior estimate close to the pool. Group A retains more of its raw signal. This supports an interpretive conclusion: credibility is a relationship between information and heterogeneity, not merely a count of observations. If the between-group variance were estimated poorly from few groups, the apparent precision of both posterior summaries would be overstated unless hyperparameter uncertainty were propagated.
Multiway and nested pooling
With company (c), fleet (f), and vehicle (i), a nested latent effect might be (b_c+u_{f(c)}+v_{i(f,c)}). Conditional on hyperparameters, fleet effects borrow across fleets in a company and vehicle effects borrow within fleets, depending on the modeled covariance structure. Whether the company itself belongs to a common population is a substantive question. In some insurance settings, companies form a convenience sample rather than an exchangeable random draw. Partial pooling then remains a working predictive device whose assumptions need diagnostic testing.
The insurance literature supplies useful practical comparisons. Ohlsson’s GLM–credibility integration addresses a high-cardinality car-model classification problem using Swedish insurer data (Ohlsson, 2008). Hierarchically structured workers’ compensation pricing applies a related idea to contract-specific and external risk factors (Campo and Antonio, 2023). The examples come from different insurance markets and establish feasibility of the approach; neither is proof that one random-effects specification dominates every tariff or jurisdiction.
Pooling and the insurance decision
The statistical benefit of pooling should be evaluated for the downstream quantity. A premium relativities table may require conditional group predictions; a corporate portfolio forecast needs marginal integration over future groups and exposure mix; an experience-rating schedule may require a transparent update of prior classification. A model can fit past group means well yet be poorly calibrated for new customers or after the acquisition of a portfolio. The decision-dependent definition of “good” prediction is a substantive requirement, not a minor choice of error metric.
Hierarchical Likelihoods
Full and approximate likelihood
Let (y_g) denote the observations in group (g), and (b_g\sim p(b\mid\psi)) random effects with hyperparameters (\psi). Under conditional independence across observations and groups, the observed-data likelihood is
[ L(\beta,\psi)=\prod_g\int\left{\prod_{i\in g}p(y_i\mid b_g,\beta)\right}p(b_g\mid\psi),db_g. ]
This integral is straightforward for some Gaussian models and generally requires numerical treatment in non-Gaussian GLMMs. Integration is necessary because fitting only the most probable latent effect and ignoring uncertainty is not equivalent to maximizing the marginal observed-data likelihood. The Breslow–Clayton analysis develops approximate likelihood inference for GLMMs, while Schall’s estimation algorithm concerns fixed effects, random effects, and dispersion parameters (Schall, 1991). Approximation accuracy matters most when event counts per group are sparse, latent variance is large, and responses are strongly non-Gaussian.
Laplace approximation replaces an integral locally using curvature near a latent mode; adaptive Gaussian quadrature can provide more accurate integration in lower-dimensional random effects; Monte Carlo methods and fully Bayesian Markov chain Monte Carlo integrate by sampling rather than a single local quadratic approximation. Penalized quasi-likelihood and other working-response approximations can be computationally efficient yet biased in some sparse binary or count settings. The practical GLMM inference review documents recurring challenges of complex random-effects estimation (Bolker et al., 2009). An estimator should be selected based on model family, event scarcity, dimension, data size, and verifiable numerical stability, rather than only the software label attached to it.
For a Gaussian linear mixed model, restricted maximum likelihood can reduce downward bias in variance-component estimation by accounting for fixed-effect estimation degrees of freedom, whereas ordinary maximum likelihood is usually preferable for comparing models with different fixed-effect specifications under the conventional likelihood-ratio framework. REML is not a general substitute for likelihood fitting in non-Gaussian models. Boundary variance estimates, flat likelihood profiles, and a small number of groups require cautious inference. A parameter estimated at zero may indicate weak data information, actual negligible variation, or an overly elaborate random-effects structure.
Bayesian multilevel inference
In a Bayesian formulation, (p(\beta,\psi,b\mid y)\propto p(y\mid\beta,b)p(b\mid\psi)p(\beta,\psi)). Proper, transparent priors for variance scales are important because weakly identified group variation can be dominated by prior choices. Posterior inference should monitor sampler convergence, effective sample size, exploration of variance tails, and sensitivity to alternative plausible priors. Model checking must compare simulated replicated claims with observed frequencies, concentration, and subgroup patterns. Visualization in the Bayesian workflow supplies the methodological rationale for iterative checks rather than reliance on a single convergence statistic (Gabry et al., 2019).
Full Bayesian inference can propagate covariance and hyperparameter uncertainty, but no computational method repairs an omitted reporting mechanism or an unjustified exchangeability class. Complexity of computation and plausibility of the scientific model are separate judgments. A model can yield numerically stable posterior draws for the wrong insurance outcome.
Frequency models and the source of overdispersion
A Poisson random-intercept model introduces extra-Poisson variation through a normal mixture of intensities. A negative-binomial model can produce overdispersion through a gamma mixture, yet the two families generally differ in higher moments, conditional dependence, and interpretation. A hurdle process separates zero from positive counts; a zero-inflated process mixes a structural-zero state with an ordinary count-generating process. Each mechanism answers a different question. It is possible to combine a random effect with a negative-binomial or hurdle component, but identifiability deteriorates if two components explain the same sparse pattern.
The time-dependent count-model study examines generalizations of Poisson and negative-binomial insurance count models (Boucher, Denuit, and Guillén, 2008). The Singapore multilevel comparison includes multiple count families within a real intercompany structure. These studies support comparative modeling; they do not establish zero inflation in a new portfolio without portfolio-specific diagnostics. Appropriate tests inspect group-specific zero fractions conditional on exposure and covariates, event clustering, repeated policy histories, and the sensitivity of high quantiles to distributional choice.
Positive severity and informative observation
Positive claim severity often requires a distribution on positive support, possibly a gamma or lognormal model; heavy-tail alternatives may be essential for large bodily injury, liability, or catastrophe losses. Under a gamma GLMM with shape (k) and log-mean (\log m_{ig}=x_{ig}^{\mathsf T}\gamma+c_g), the conditional variance is (m_{ig}^2/k) for the stated parameterization. A random effect can share information across claim groups, but does not automatically impose a suitable tail law or resolve censoring. The number of observed positive severities depends on the claim frequency process. A conditional severity likelihood is a selected sample when the same latent factor influences both claim incidence and size.
To make that dependence explicit, assume (N\mid b_N\sim\operatorname{Poisson}(E e^{\eta_N+b_N})), positive severities conditionally independent given latent factors with mean (e^{\eta_Y+b_Y}), and ((b_N,b_Y)) jointly Gaussian with variances (\sigma_N^2,\sigma_Y^2) and covariance (\sigma_{NY}). Also assume the conditional severity mean does not depend on the realized count once the latent factors and observed covariates are fixed. Conditional compound-sum expectation gives (E[S\mid b_N,b_Y]=E e^{\eta_N+\eta_Y+b_N+b_Y}). Therefore,
[ E[S]=E\exp!\left(\eta_N+\eta_Y+\tfrac12\sigma_N^2+\tfrac12\sigma_Y^2+\sigma_{NY}\right). ]
The product of separate marginal frequency and severity means instead equals the same expression without (\sigma_{NY}). Thus their ratio is (e^{\sigma_{NY}}). This is a derived correction under the explicit joint-lognormal latent structure, not a universal multiplicative factor for real claim portfolios. Positive covariance raises expected aggregate cost relative to the product of the separate marginals; negative covariance lowers it. The contractual deductible and severity selection mechanism can change the algebra. A joint frequency–severity model has value precisely when this shared latent dependence is sufficiently identified and relevant to the decision.
Claims development and longitudinal reserves
Development triangles, policy-level longitudinal claims, and open claim trajectories provide different hierarchy definitions. A development model might place random effects on accident years, development lags, lines of business, or claim-specific reporting trajectories. Calendar inflation can align several accident-year cells diagonally; failing to acknowledge that common shock induces a covariance error. Mixed models for claims reserving connect semiparametric smoothing, reserving and credibility in the work of Antonio and Beirlant (2008). Their publisher abstract specifically describes model-based access to predictive distributions through a Bayesian implementation. Yet a model for the mean of observed incremental paid amounts is not itself an adequate estimate of claims ultimate: tail development, reopening, case reserve practices, and the reporting process must also be treated.
Loss reserve forecasts require distinction between conditional expectation given observed development and the full predictive distribution including process and parameter uncertainty. Reinsurance recovery, policy limits, inflation, claims adjustment expenses, and discounting may be introduced at different stages. An actuarial result must state the order of these transformations; a pooled distribution of gross ground-up claims will not generally map to net retained losses by scaling one average coefficient.
Between-Group Heterogeneity
Variance components as questions about the portfolio
Variance-component estimates are descriptions of a specified stochastic population, conditional on covariates, group definitions, and the assumed random-effects distribution. They are not direct measures of fairness, underwriting competence, or intrinsic riskiness of people. A carrier random effect can absorb omitted product mix, reporting changes, or claims culture; a territorial effect can absorb exposure, building stock, weather, or market composition. Scientific interpretation therefore begins by enumerating what has been adjusted and what remains in the residual cluster effect.
In the intercompany motor claims study, multiple levels of observed vehicle and organizational structure made carrier-specific experience a legitimate research object (Antonio, Frees, and Valdez, 2010). In workers’ compensation hierarchical pricing, the hierarchy concerns granular contract and other explanatory factors (Campo and Antonio, 2023). These are distinct statistical units: empirical patterns at one level cannot be transferred unchanged to another. A carefully modeled high-cardinality rating factor can support predictive pooling without demonstrating that the factor itself is a causal determinant of claim frequency.
Confounding, variance allocation, and informative group size
A large group-variance estimate can mean true clustering or a missing explanatory variable. Group sizes can be informative: a high-risk employer may purchase more exposure or cancel sooner; a carrier may enter a market precisely after claims experience changed. Missing groups can be systematically absent because they failed, were merged, or left the book. If the number of observations is related to a latent group effect, treating cluster size as noninformative changes the sample-weighted target. Equal-weight group-average loss and exposure-weighted policy-average loss are different estimands even when the numerical dataset is identical.
Classical random-intercept estimation also assumes a distribution for group effects. Sensitivity analysis can vary a normal prior, heavier-tailed random effect, mixture distribution, or alternative grouping structure. A few exceptionally large insureds may dominate a variance estimate; a robust model should identify whether these are legitimate risk signals or data-quality anomalies. Shrinkage toward a contaminated portfolio mean can systematically misprice every sparse group. The appropriate response is to investigate the pool and its comparability, rather than increase the complexity of the fitted penalty alone.
Sparse strata and changing group membership
Small cells are precisely where partial pooling is useful and most vulnerable to model assumptions. A rare commercial specialty with one claim is a weak direct source of its long-run severity, while other specialty classes may have genuinely different tail dynamics. Borrowing strength is legitimate to the extent the pooling population is defensible; it should be documented through common risk definitions, coverage terms, measurement, and known hazard differences.
Group identities can evolve. A vehicle may switch fleet; an employer may change carrier; a health-care beneficiary may switch provider. If an observation contains multiple affiliations at different times, a static nested hierarchy silently assigns all experience to the wrong contemporary group. Multiple-membership or cross-classified random effects can allocate historical shared signals, with weights that sum to one when representing time or service allocation. These weights require a factual interpretation. A statistical solution cannot recover historical affiliations absent from the administrative ledger.
International transfer and regulatory context
Mathematical likelihoods are portable; their data objects and governance requirements are jurisdiction-dependent. Motor third-party liability coverage, workers’ compensation benefit design, and health reimbursement differ across markets. Foreign claims data may use different claim thresholds, payroll definitions, settlement systems, currencies, and inflation accounting. A model trained on one country’s injury claims should therefore be treated as a candidate scientific method, not as a transferable calibration parameter without local evidence.
The United States ASOP No. 56 on modeling (Actuarial Standards Board, 2019; effective October 1, 2020) supplies an official example of professional requirements concerning intended purpose, data, assumptions, model risk, validation, governance, documentation, and communication for actuaries within its scope. It is not represented here as global law or a requirement binding every international insurer. The broader analytical lesson is that model specification and decision controls should be jointly reviewable, with material limitations visible to the user of the prediction.
Model risk, fairness, and classification
A latent group coefficient can describe a historical conditional association while perpetuating a past administrative selection pattern. Whether a variable may be used in pricing, how protected characteristics must be handled, and what disclosures are required depend on the relevant law and product. Fairness metrics must be defined at a level consistent with exposure, contractual coverage, and permissible classification. Statistical regularization can reduce small-sample volatility, but it does not independently resolve the normative legitimacy of a rating variable or the regulatory status of indirect proxies.
A robust governance file should record the statistical variable, origin and update date, business purpose, data-quality risks, legal assessment, and effect of removing or adjusting the variable. This dossier does not prescribe a jurisdiction-specific tariff rule. It establishes a methodological obligation to separate predictive accuracy from eligibility to use an effect in an actual insurance decision.
Prediction and Diagnostics
Define the prediction estimand before validating
The objective can be a conditional count for an existing fleet next year, a marginal expected loss for a new fleet, a portfolio-level distribution with a common catastrophe shock, or a claim-reserve distribution conditional on observed development. Each target requires a different integration step and holdout structure. For an existing observed group, prediction integrates over its posterior or conditional random-effect uncertainty given available history. For a truly new group, prediction integrates over the population distribution of a new random effect, including hyperparameter uncertainty where material. Using a previously fitted group intercept for a new group is a form of information leakage; setting the effect to zero while calling the prediction marginal is a different error.
A hierarchy-aware validation design may contain three nested questions: (a) future time windows for already observed insureds, (b) entirely unseen insureds within known groups, and (c) unseen groups or companies. A temporal split guards against future-information leakage, a group split tests transfer to new clusters, and a combined temporal-group split tests the hardest relevant operating situation. The same fitted model can rank differently under these evaluations. When claims settle with long delays, the outcome label must be observed at a comparable maturity for every training and validation cohort.
Posterior and predictive calibration
Evaluate both point and distribution forecasts. Frequency models should reproduce exposure-standardized means, group count dispersion, zeros, and rare high-frequency cells. Severity models should reproduce payment-layer exceedance probabilities and large-claim behavior; a good average fit can coexist with severely understated tails. Aggregate-cost forecasts should be checked by calendar, product, company, and significant group, preserving known correlations. For longitudinal data, use block or group resampling rather than naive row-level resampling when the intended uncertainty includes shared effects.
Bayesian model comparison can use leave-one-out predictive assessment and related WAIC methods (Vehtari, Gelman, and Gabry, 2017), but row-wise leave-one-out has a narrower estimand than leaving out an entire group. Conditional independence at the data-row level does not by itself justify using row-wise cross-validation to predict a new insurer. Diagnostics for influential observations and importance-weight instability must determine whether a more demanding validation design or direct refitting is appropriate. Bayesian workflow visualization also supports checking latent residuals and replicated datasets before interpreting a favorable score (Gabry et al., 2019).
For frequentist mixed models, compare held-out log scores where predictive densities are available, exposure-weighted deviance or count diagnostics, proper scores for aggregate losses, and calibration plots with uncertainty. Penalized model selection criteria depend on whether the prediction target is marginal or conditional, and the definition of effective parameters in hierarchical models can differ from a fixed-effects regression. A single AIC value, convergence message, or pseudo-(R^2) does not establish economic usefulness.
A reproducible diagnostic program
A scientifically auditable implementation can follow this sequence:
- Freeze the policy, claim, exposure, coverage, development, and group definitions, with the first and last observable dates for each variable.
- Construct a dependency diagram linking unit, fleet or employer, insurer, territory, calendar period, and loss event. Identify crossed memberships and possible common shocks.
- Fit a transparent exposure-adjusted baseline with no latent effects, then nested and crossed alternatives justified by the dependency diagram.
- Compare Poisson, negative-binomial, hurdle or zero-inflated counts according to separately articulated mechanisms; inspect expected zeros and cluster dispersion.
- Fit severity conditional on the chosen claim and payment basis; audit deductible, limit, truncation, and right-censoring treatment; test frequency–severity dependence where justified.
- Inspect variance-component uncertainty, distributional assumptions, boundary fits, singular covariance matrices, optimization tolerances, posterior convergence, and prior sensitivity.
- Validate future known-group, new-policy and new-group predictions separately; report scoring and calibration on the economic scale and with credible or confidence intervals.
- Quantify the effect of a model change on annual aggregate cost, claim tail, reserves, and subgroup outcomes; record legal and operational signoff separately from the scientific fit.
Each action has a reason. The initial baseline measures what the hierarchy contributes. Multiple candidate count mechanisms prevent a random effect from being used as an explanation for every discrepancy. Group-aware holdouts test whether shrinkage genuinely transfers information, while calibrated uncertainty prevents false certainty from a large integrated database. Source and input versioning make later deviations investigable.
Research agenda and interpretable synthesis
Several unresolved directions deserve direct empirical study. Hierarchical frequency–severity dependence requires reliable observation of both count and payment processes across common group definitions. Model migration under changing insurance coverage demands transparent mapping of old to new exposure and loss bases. Cross-market transfer requires independent validation that the grouping distribution is transportable, especially when catastrophe, health, or labor institutions change. Claims reserving needs joint development-time, reporting, and calendar-shock models whose prediction targets are matched to reserve accounting. These are proposed research questions, not assertions that a novel named theoretical system has been established.
The analytical synthesis is specific. A hierarchy is the model’s representation of the insurance system’s dependence network. Partial pooling is the statistical redistribution of estimation precision within that network. Credibility is one important decision-oriented expression of that redistribution. A random-effects coefficient has meaning only relative to its group definition, covariate adjustment, and distributional assumptions. The economic output comes after the appropriate predictive integration and contractual transformation. These propositions follow from the established likelihood mathematics and the verified actuarial applications rather than from an invented principle.
Conclusion
Hierarchical and mixed-effects methods provide an integrated scientific framework for estimating insurance losses when contracts, insured units, institutions, locations, and periods share unobserved influences. Their central mathematical achievement is the explicit representation of dependence and the corresponding partial pooling of uncertain group experience. The Gaussian model gives exact covariance and shrinkage formulas under its assumptions; generalized linear mixed models extend the framework to counts and positive severities; joint latent components can correct aggregate loss expectations for shared frequency–severity heterogeneity.
The research establishes five substantive conclusions. First, the statistical unit of grouping must follow the risk-generating and administrative process rather than convenient database fields. Second, conditional and marginal predictions differ on nonlinear links, and the difference has practical consequences for new groups, premiums, and portfolio aggregates. Third, apparent excess zeros, overdispersion, and group variance admit multiple explanations that require targeted comparison rather than automatic model expansion. Fourth, credible partial pooling depends on justified exchangeability and verified variance components, particularly in small or changing groups. Fifth, validation must reproduce the future decision setting: time, group membership, claims maturity, contract basis, and the treatment of shared shocks.
The available original literature demonstrates the methodological maturity and insurance relevance of this family of models across motor insurance, intercompany portfolios, workers’ compensation, and claims reserving. The manuscript’s original contribution is the coherent derivation and cross-context synthesis of prediction targets, dependence structures, and modeling controls; it does not claim empirical superiority of one algorithm or a new theoretical doctrine. Its formulas and hypothetical calculations are reproducible, while applications to any specific insurer require independent portfolio data, professional judgment, and the law applicable to the insurance product. The release state of this research package remains separate from scientific completion and will be determined by the recorded editorial and technical gate.
References
Laird, N. M., and J. H. Ware (1982). “Random-Effects Models for Longitudinal Data.” Biometrics 38(4): 963–974. Publisher or archival record.
Breslow, N. E., and D. G. Clayton (1993). “Approximate Inference in Generalized Linear Mixed Models.” Journal of the American Statistical Association 88(421): 9–25. Publisher or archival record.
Schall, R. (1991). “Estimation in Generalized Linear Models with Random Effects.” Biometrika 78(4): 719–727. Publisher or archival record.
Nelder, J. A., and R. J. Verrall (1997). “Credibility Theory and Generalized Linear Models.” ASTIN Bulletin 27(1): 71–82. Publisher or archival record.
Bühlmann, H., and E. Straub (1970). “Glaubwürdigkeit für Schadensätze.” Mitteilungen der Vereinigung Schweizerischer Versicherungsmathematiker 70: 111–133. Publisher or archival record.
Jewell, W. S. (1974). “Credible Means Are Exact Bayesian for Exponential Families.” ASTIN Bulletin 8(1): 77–90. Publisher or archival record.
Ohlsson, E. (2008). “Combining Generalized Linear Models and Credibility Models in Practice.” Scandinavian Actuarial Journal 2008(4): 301–314. Publisher or archival record.
Antonio, K., and J. Beirlant (2008). “Issues in Claims Reserving and Credibility: A Semiparametric Approach with Mixed Models.” Journal of Risk and Insurance 75(3): 643–676. Publisher or archival record.
Boucher, J.-P., and M. Denuit (2006). “Fixed versus Random Effects in Poisson Regression Models for Claim Counts: A Case Study with Motor Insurance.” ASTIN Bulletin 36(1): 285–301. Publisher or archival record.
Antonio, K., E. W. Frees, and E. A. Valdez (2010). “A Multilevel Analysis of Intercompany Claim Counts.” ASTIN Bulletin 40(1): 151–177. Publisher or archival record.
Antonio, K., and E. A. Valdez (2012). “Statistical Concepts of a Priori and a Posteriori Risk Classification in Insurance.” AStA Advances in Statistical Analysis 96: 187–224. Publisher or archival record.
Campo, B. D. C., and K. Antonio (2023). “Insurance Pricing with Hierarchically Structured Data: An Illustration with a Workers’ Compensation Insurance Portfolio.” Scandinavian Actuarial Journal 2023(9): 853–884. Publisher or archival record.
Bolker, B. M., M. E. Brooks, C. J. Clark, S. W. Geange, J. R. Poulsen, M. H. H. Stevens, and J.-S. S. White (2009). “Generalized Linear Mixed Models: A Practical Guide for Ecology and Evolution.” Trends in Ecology & Evolution 24(3): 127–135. Publisher or archival record.
Vehtari, A., A. Gelman, and J. Gabry (2017). “Practical Bayesian Model Evaluation Using Leave-One-Out Cross-Validation and WAIC.” Statistics and Computing 27: 1413–1432. Publisher or archival record.
Gabry, J., D. Simpson, A. Vehtari, M. Betancourt, and A. Gelman (2019). “Visualization in Bayesian Workflow.” Journal of the Royal Statistical Society: Series A 182(2): 389–402. Publisher or archival record.
Actuarial Standards Board (2019). “Actuarial Standard of Practice No. 56: Modeling.” Adopted December 2019; effective October 1, 2020 (United States). Publisher or archival record.
Boucher, J.-P., M. Denuit, and M. Guillén (2008). “Models of Insurance Claim Counts with Time Dependence Based on Generalization of Poisson and Negative Binomial Distributions.” Variance 2(1): 135–162. Publisher or archival record.
Frees, E. W. (2004). “Longitudinal and Panel Data: Analysis and Applications in the Social Sciences.” Cambridge University Press; original print edition 2004, later digital edition. Publisher or archival record.
Molenberghs, G., and G. Verbeke (2005). “Models for Discrete Longitudinal Data.” Springer Series in Statistics; Springer, New York. Publisher or archival record.
Lindley, D. V., and A. F. M. Smith (1972). “Bayes Estimates for the Linear Model.” Journal of the Royal Statistical Society: Series B 34(1): 1–18. Publisher or archival record.