Insurance by BogdanovaThe Encyclopedia of Insurance and Risk Science

RESEARCH DOSSIER

Insurance Data Linkage and Entity Resolution

Abstract. Insurance data linkage is the inferential and administrative process of establishing which records from policies, underwriting, insured assets, claims, payments, loss events, external registers, and historical systems refer to the same entity or describe a specified relationship between distinct entities. Entity resolution transforms fragmented observations into an analytically usable representation of persons, organizations, objects, events, and their changes through time. The scientific significance of this transformation lies in its effect on the statistical denominator, the allocation of claim outcomes, severity distributions, exposure histories, fraud signals, pricing variables, reserving estimates, and regulatory fairness assessments. This monograph develops the relationship between classical probabilistic record linkage, contemporary Bayesian and graph-based entity resolution, the many-to-many structure of insurance relationships, and downstream actuarial inference. It establishes explicit distinctions among identical identity, related entities, shared incidents, and duplicate transactions; derives the likelihood-ratio decision rule and the consequences of constrained matching; analyzes time-dependent identity and event attribution; and calculates transparent synthetic examples of false-positive and false-negative linkage, subgroup distortions, posterior match probabilities, and expected exposure-weighted claims. It evaluates what validation evidence permits claims about accuracy, calibration, causal interpretation, and fairness, and examines applicable privacy and professional governance under international, European, U.S., and UK sources. Its original contribution is an integrated critical interpretation and a reproducible set of analytic demonstrations, rather than a newly named theory, statistical estimator, or empirical study. The central conclusion is that insurance entity resolution is part of the inferential specification of the insured population and loss process. Linkage decisions must therefore be evaluated by their effect on the particular scientific and contractual question they are intended to answer.

Keywords: Insurance data linkage; entity resolution; record linkage; probabilistic matching; Fellegi–Sunter; Bayesian record linkage; identity uncertainty; claims deduplication; insured assets; temporal entity resolution; exposure measurement; actuarial bias; model fairness; privacy-preserving record linkage; data quality; auditability; international insurance statistics.

Introduction

An insurance record is an observation produced by a contractual, organizational, technical, or legal process. A person may be a policyholder in one system, an insured life in another, a claimant in a third, a driver or beneficiary in a fourth, and the representative of an organization in a fifth. Each system can correctly record its own local transaction while failing to preserve a consistent account of the relationships between them. An insurer can consequently possess many accurate individual records and still produce an inaccurate estimate of insured experience when records are joined across systems.

The central question is deceptively simple: which observations belong together? Its answer requires a definition of what “together” means. Two claim rows may refer to one loss event while representing distinct covered claimants. Two policies may have a common mailing address without having the same policyholder. A legal entity may change its name without changing its legal identity; an acquisition may preserve some contractual rights while transferring other obligations. A person may appear under different name spellings and addresses across years. A single identifier can be highly reliable within one issuing system and meaningless outside that system. In each case the scientific unit must be specified before the matching algorithm can be evaluated.

The modern field developed from administrative and vital-record linkage. Newcombe and colleagues (1959) supplied an early historical illustration of automatic linkage of vital records. Fellegi and Sunter (1969) established a formal theory for record linkage that explicitly allows links, nonlinks, and unresolved cases. Christen (2012) subsequently brought together data matching and duplicate detection across statistics, databases, and computing. Winkler (2006) surveyed record-linkage methods and their limits, including the importance of identifiers and the construction of comparison procedures. These works provide a conceptual and methodological foundation; their conclusions do not, by themselves, establish that a particular insurer has achieved adequate linkage accuracy.

Insurance introduces distinctive consequences because the joining operation determines who is exposed, when coverage exists, whose loss is attributable to a covered risk, whether a payment is new, and whether the inferred experience belongs in a particular actuarial analysis. A false join can redirect a claim to a different insured person or property; an omitted join can make a genuine claim disappear from an experience dataset. Misassigned vehicle or building losses can contaminate geographic and hazard predictors. Duplicate payment records can inflate paid loss while a mistaken deduplication can suppress legitimate multi-coverage recoveries. Linkage is therefore an element of measurement, not merely a computational preliminary.

The present research has three aims. It defines insurance entities and permissible relationship types; explains the statistical assumptions, uncertainty, and practical constraints of linkage methods; and derives downstream consequences for insurance estimation, modeling, operations, and governance. The scope is international, with jurisdiction-specific instruments expressly identified. The paper synthesizes primary academic publications and institutional material confirmed at publisher or official-source level. Mathematical examples use declared hypothetical values, checked by independent calculation; they do not describe insurer experience or estimate real-world prevalence of matching errors. The analysis distinguishes verified source claims, deductive results from stated assumptions, and operational recommendations.

This research treats entity resolution as an inferential problem with an operational implementation. Its defining question is not how many database rows have been joined, but whether each claimed identity or relationship is supported at the level required by the decision that uses it. That distinction guides every chapter.

Insurance Entities and Keys

Statistical units, legal identity, and role-specific identifiers

An entity is the object about which an identity claim is made: a natural person, legal organization, insured vehicle, building, account, policy, contract, claim, accident, peril occurrence, or payment. An identity relation asserts that two records refer to the same member of a given entity class. A relationship asserts an association between two distinct entities, such as owns, resides at, insures, operates, benefits from, belongs to a household, or arises from an occurrence. Confusing the two propositions creates structural errors before probability is calculated.

At minimum, a defensible insurance analytical system distinguishes a person identifier, organizational entity identifier, asset identifier, policy identifier, coverage or endorsement identifier, claim identifier, occurrence identifier, transaction identifier, and source-record identifier. The precise set varies by product and jurisdiction. A policy number can identify an insurer's contract or account namespace, but it does not globally identify every insured person named on the contract. A claim number can identify a local claim file even where one incident produces multiple claims. A payment transaction has its own accounting identity and can be reversed, corrected, subdivided, or recovered without constituting another underlying loss.

Legal entity references can sometimes be linked through official cross-organization identifiers. The Global Legal Entity Identifier Foundation explains that the Legal Entity Identifier identifies a legal entity through a unique 20-character code and connects it to reference data. This reduces a class of organizational ambiguity but does not infer that two subsidiaries are the same entity, that a corporate successor inherited every insurance obligation, or that an LEI identifies an individual policyholder. The underlying legal relation must be interpreted separately from the identifier. Insurance identity is correspondingly situated in a typed, date-sensitive system of facts.

A record carries at least three kinds of information: identifiers supplied or generated by a source, attributes such as name, address, date of birth or property description, and provenance indicating who asserted those attributes, when, for what purpose, and under which transformation. A normalized name is an observable feature and a possible matching signal. A source-specific key is an identifier with an issuer and scope. A confirmed link is a relationship claim supported by evidence and decision rules. These layers should be preserved, because silently replacing contradictory observations with a single “golden” value destroys useful evidence about how an identity was established.

In health and life insurance, a person may be simultaneously applicant, payer, insured life, beneficiary, and claimant, with distinct legal roles. In property insurance, title ownership, mortgage interest, risk location, scheduled item and physical building can diverge. In marine and aviation insurance, insured interests, operators, voyages, cargo consignments and occurrence locations may have different governing identities. In commercial programs, a multinational parent, named insured, additional insured, local subsidiary, broker, fronting carrier and reinsurer cannot safely be collapsed into a corporate-group node. The model must preserve both the entity and its role in the specific contract.

Insurance relationships and their cardinalities

A one-to-one assumption is useful when matching a unique person row in file A to a unique person row in file B, but insurance databases often involve one-to-many and many-to-many relations. One insured may hold several policies; a policy may cover several persons, sites, vehicles or legal entities; a single catastrophe may cause many claims; a claim may receive many payments; and a single payment can settle several claim components. The problem is not solved by applying a one-to-one person-matching algorithm to every pair of tables. Its matching constraint must follow the entity type and relationship definition.

Consider a group motor policy that covers five vehicles and lists three authorized drivers. There are five asset objects, three person objects, one policy object, and multiple edges specifying insured interest, driver authorization and time interval. Repeating the policy number over eight or more rows can be correct data normalization. An algorithm that labels these rows “duplicates” and removes all but one destroys contractual multiplicity. Conversely, if the same payment transaction is replicated by a system import, counting it twice without a reversal or transfer identifier creates an accounting error. Cardinality and event semantics determine whether repeated records are legitimate.

A practical relational ontology therefore records node type, source-specific record identity, stable internal resolution ID, edge type, validity interval, observation time, and evidence. This is a descriptive architecture for organizing data, not a new theory. The choice of a stable internal ID is an engineering decision; it should not assert a legal identity beyond the evidence. An unresolved candidate may be retained as two hypotheses with probabilities rather than forcibly merged.

To interpret frequencies, define exposure unit before joining. For a one-year property policy the unit might be building-years or insured-location-years; in motor insurance it might be vehicle-years; for life coverage it might be life-years under specified benefit conditions. A row count is an exposure denominator only if the row-generating process has the required one-to-one relationship to the chosen unit. When a household holds three policies and two vehicles, a household count, person count, policy count and vehicle-year count answer different questions. Linking them changes the available information but does not make the counts interchangeable.

Data-generating processes, observation and selection

Observed insurance data reflect underwriting selection, renewal, lapse, claim notification, payment development, administrative system changes, legal eligibility and the purpose of each data source. A loss may occur before notification; an insurer may record a claim only after a reporting threshold; an external registry may update addresses after a delay. Missingness is often systematic. Claims reported by phone may have different name completeness from online claims. Legacy policies may lack universal identifiers. Catastrophe data may include geocoding uncertainty. These mechanisms affect whether linkage errors correlate with risk outcomes.

Let an underlying entity be ii, the insurer's observed record be rr, and a source indicator be ss. A statistical treatment may describe observed attributes as XrsX_{rs} with an unobserved assignment Zrs=iZ_{rs}=i. The observation model determines which attributes are present conditional on the actual entity and the source; the linkage model infers candidate assignments. When high-frequency claimants have more interactions and therefore more aliases or records, treating every observed row as an independent entity oversamples the outcome under analysis. When low-income or mobile policyholders have inconsistent address fields, selective nonmatching can distort the effective sample even when overall match rates appear acceptable.

A robust study documents the population to which inferences refer, the mechanism that generates records, the coverage of each source, the observation dates, and the sample selection induced by candidate generation. All four are needed to interpret downstream comparisons. An estimate of claim frequency for all insured exposures, for claims with a verified external registry match, and for people consenting to data sharing are three different estimands. An analyst cannot bridge those populations without assumptions and supporting evidence.

This observation is especially important when using external data for predictive models. The coverage of the external register may depend on geography, reporting practice or public-sector eligibility. Linking succeeds conditional on both availability and matchability. A high predictive score trained on the linked subset may reflect selection into that subset instead of prospective claim risk. The dataset construction decision belongs in the model's scientific description, including its admissible target population and conditions for use.

Deterministic and Probabilistic Linkage

Exact keys, normalization, candidate selection and blocking

Deterministic linkage uses exact equality of one or more attributes or identifiers after an explicitly defined standardization process. It can provide strong evidence where authoritative identifiers are complete, correctly scoped, stable, and reliably transcribed. Its errors arise from recycled keys, scope collisions, clerical transcription, stale identifiers, legal identity changes, missing data, and attributes that are shared by distinct entities. A key that identifies a vehicle registration at a specific date is not automatically a permanent physical-vehicle identifier. A normalized postal address identifies a location string and can house many insured people or units.

Normalization often includes Unicode character handling, punctuation removal, standard postal formats, name tokenization, date parsing, transliteration and phonetic comparison. Each procedure embeds a decision about information loss. Transliteration can improve comparisons across scripts while conflating distinct names; nickname dictionaries can recover aliases while creating accidental equivalence; truncation can make different apartment identifiers appear equal. Original values and the exact versioned transformation should remain available for audit. Normalization that is useful for candidate retrieval may be too aggressive for a final identity decision.

At enterprise scale, evaluating all possible pairs is costly: two files of sizes nn and mm contain nmnm possible pairs. Blocking or indexing reduces the candidate set by requiring some shared key, postal prefix, locality, phonetic token, or learned retrieval score. Multiple blocking passes can recover candidates missed by any one rule. Yet a pair omitted by blocking cannot subsequently be recovered by a perfect comparison classifier. An evaluation must therefore distinguish candidate-generation recall from pair-classification recall; the end-to-end recall is bounded by the former. Christen (2012) develops the wider matching workflow from preprocessing to evaluation, and Elmagarmid, Ipeirotis, and Verykios (2007) survey approximate duplicate-detection techniques.

A high-quality candidate strategy uses different rules for different entity types. Person candidates may use name, birth date and address history; buildings may use geospatial footprints, parcel references and street addresses; vehicles may use registration date and a manufacturer identification; loss occurrences may use peril, event window, location and incident features. Household matching uses membership and temporal co-residence; legal-entity matching uses registered identity and corporate events. A single universal similarity score discards useful domain structure.

Likelihood ratios, priors and decision loss

Fellegi and Sunter's (1969) three-way linkage framework expresses the evidentiary value of a comparison vector γ\gamma through probabilities under matching and nonmatching hypotheses. Write MM for the proposition that the records represent the same target entity and UU for the alternative. The general evidence ratio is

Λ(γ)=P(γ∣M)P(γ∣U).\Lambda(\gamma)=\frac{P(\gamma\mid M)}{P(\gamma\mid U)}.

If comparisons γ1,…,γk\gamma_1,\ldots,\gamma_k are conditionally independent given match status, the log-ratio can be written

W(γ)=∑j=1klog⁡(mj(γj)uj(γj)),mj=P(γj∣M),uj=P(γj∣U).W(\gamma)=\sum_{j=1}^{k}\log\left(\frac{m_j(\gamma_j)}{u_j(\gamma_j)}\right), \qquad m_j=P(\gamma_j\mid M),\quad u_j=P(\gamma_j\mid U).

The product form is a modeling assumption, not an identity. Name, address and household relations are correlated; duplicate forms may be generated from a common upstream source. Uncritical multiplication can greatly exaggerate evidence. Estimation also depends on the composition of candidate pairs: the prevalence of genuine matches in all possible pairs differs from that after blocking. Thresholds fitted on one candidate population may be miscalibrated after changing a blocking rule or adding a new source.

With match prior probability π\pi, Bayes' rule gives posterior odds

P(M∣γ)1−P(M∣γ)=π1−πΛ(γ).\frac{P(M\mid\gamma)}{1-P(M\mid\gamma)}=\frac{\pi}{1-\pi}\Lambda(\gamma).

This equation clarifies why apparently strong field agreement can be insufficient in an enormous search space. In a strictly hypothetical example, date-of-birth agreement has likelihood ratio 0.90/0.002=4500.90/0.002=450 and postal-code agreement has ratio 0.75/0.02=37.50.75/0.02=37.5. Assuming their conditional independence, the combined ratio is 16,87516{,}875. With prior π=0.0001\pi=0.0001, the posterior match probability is approximately 0.6280.628, while a prior π=0.01\pi=0.01 gives approximately 0.9940.994. Both results arise from the same observed comparisons. They differ because the candidate population and prior prevalence differ. These are mathematical illustrations, not estimates for real individuals, ZIP codes or insurance portfolios.

Suppose a false positive link incurs loss CFPC_{FP}, a missed true link incurs CFNC_{FN}, and a decision must be made for an unconstrained pair. Expected loss from linking is (1−p)CFP(1-p)C_{FP}; expected loss from not linking is pCFNpC_{FN}, where p=P(M∣γ)p=P(M\mid\gamma). Linking has lower loss if

p>CFPCFP+CFN.p>\frac{C_{FP}}{C_{FP}+C_{FN}}.

An optional review or defer decision with its own cost creates a third region. In claims settlement, a high-cost misattribution or unauthorized disclosure may justify more restrictive linking than a low-stakes analytical exploration. However, the costs must be specified by decision context rather than manipulated to justify a desired acceptance rate. Thresholds appropriate for population estimation can differ from those appropriate for individual claim adjudication. Neither probability nor score should substitute for the applicable legal and evidentiary standards governing contractual decisions.

Bipartite assignment, multi-file clusters and calibration

Pairwise classification does not automatically produce a coherent collection of entity assignments. If one person record from file A is confidently matched to two distinct unique-person records in file B, an unconstrained classifier may accept both links although the presumed file uniqueness rules prohibit that result. Under an explicitly justified one-to-one assumption, matching becomes an assignment problem: candidate edges compete, and the optimal configuration depends on the collection of pair scores and the admissible matchings. Sadinle (2017) develops Bayesian bipartite matching with uncertainty, including partial estimates that preserve unresolved cases.

For multiple files, an entity may be represented by several observed records. A graph can connect records to latent entity identities, preserving duplicates and uncertainty across sources. Steorts, Hall, and Fienberg (2016) describe a Bayesian graphical approach to deduplication that places latent individuals at the center of the linkage structure. It is useful to distinguish the mathematical assumption of one record per entity in a particular source from the identity of a person across all sources. A claim ledger has repeated records of a single claim by design; a unique-person register might not. Constraints should be justified separately for each source and entity type.

Graph closure also needs care. If A is provisionally linked to B and B to C, transitive merging treats A and C as the same entity. That conclusion can be valid for confirmed identity links, but not for shared address, employer, beneficiary, owner, incident, or other nonidentity relationships. A broad “connected component” algorithm that ignores edge semantics can merge entire households, corporate groups or catastrophe clusters. Probabilistic identity graphs should therefore separate identity hypotheses from typed nonidentity relations and retain the evidence supporting any closure.

A calibrated match probability is one whose predicted probability corresponds, on appropriate validation cases, to the observed frequency of truth. A score of 0.9 is not automatically a 90% probability. Calibration requires representative adjudicated or otherwise authoritative truth labels, ideally evaluated across source combinations, missingness patterns, periods, products and relevant populations. Accuracy, precision and recall must be reported alongside prevalence, candidate construction and the error costs of the downstream use. The UK Office for National Statistics specifically warns that match rate is not linkage accuracy and calls for estimates of precision and recall.

Linked records may carry uncertainty even after a most-likely assignment has been selected. For analytical estimands, integration over plausible match configurations can be more defensible than treating the selected graph as known. Let LL index possible consistent linkage configurations, DD the observed data, and T(L,D)T(L,D) the downstream statistic under each configuration. Under a specified posterior linkage model,

E[T∣D]=∑LT(L,D)P(L∣D).E[T\mid D]=\sum_{L}T(L,D)P(L\mid D).

The law of total variance similarly separates uncertainty within a linkage configuration from the uncertainty induced by alternative configurations. This representation identifies an existing inferential principle, rather than proposing a novel estimator. It is appropriate only when the posterior assignments and the statistic's meaning are coherent, particularly where claims share an occurrence or exposures are correlated. Naively summing unrelated pair probabilities can violate one-to-one or contractual constraints and therefore fail to represent any feasible insurance population.

Temporal Entity Changes

Valid time, recording time and stable identities

Identity is evaluated at a time, and the relevant time depends on the analytical question. A person can change surname, address or marital status; a policy can be amended; a corporation can merge or dissolve; a property can be rebuilt or subdivided; a vehicle can be transferred between owners. Historical evidence may arrive after the event to which it relates. A database that keeps only the latest attribute values can therefore produce plausible matches that are historically false.

The practical distinction is between valid time, the period during which a proposition applies to the modeled world, and recording time, the period during which the information was available to a system. A property address corrected in 2026 may have described the actual risk location in 2024, while a claimant identity confirmed in 2026 may not have been known to the insurer at the 2024 underwriting decision. These dates answer different questions. An experience study needs the best supported reconstruction of the insured history; a prospective predictive validation must restrict inputs to what was known when the prediction would have been made.

The central temporal objects can be expressed as records R=(e,a,vfrom,vto,trecorded,s)R=(e,a,v_{from},v_{to},t_{recorded},s), where ee identifies the resolved entity hypothesis, aa is a source assertion or relationship, the interval [vfrom,vto)[v_{from},v_{to}) expresses its asserted effective period, trecordedt_{recorded} is the time the assertion entered the analytical environment, and ss identifies its source and version. This notation organizes established bitemporal considerations for insurance use; it is a descriptive representation, not a proposed universal schema. Where legal validity is disputed, the dates are themselves assertions with provenance rather than presumed truths.

Consider a household moving from one address to another during a policy year. If the analyst links claim geography to the current policyholder address instead of the covered location on the date of loss, an old water-damage claim can be assigned to the new house. An apparently accurate person match is insufficient because the wrong relationship interval changes the exposure allocation. Similarly, an insurer may discover that two claims represent one hailstorm occurrence without merging the insured buildings or claim files. The occurrence identity and its time window must be resolved independently of customer identity.

Temporal linkage also encounters changing identifier namespaces. An insurer might acquire a portfolio, migrate policy systems and reissue local policy numbers. A mapping table may preserve historical continuity, but the nature of the mapping matters: a contractual novation, renewal, administrative renumbering and new contract are distinct events. Recording an equivalence between old and new identifiers is justified only for the specific relationship supported by the migration evidence. A durable entity-resolution service must allow the equivalence itself to be versioned and challenged.

Policy chronology, asset replacement and corporate succession

Policy chronology gives the temporal boundary for many insurance estimands. Coverage inception and expiration, endorsements, limits, deductibles, cancellations, reinstatements, waiting periods and other contractual conditions determine whether a loss belongs to the insured exposure under study. A matching procedure that associates a claim with the right person but wrong policy term can produce a false coverage attribution. The claim's reported date may differ from its occurrence date; the correct choice depends on the product, contract and analytic purpose. Reserving by accident year, underwriting-year experience and paid-year analysis consequently require different time keys.

Insured assets may persist despite alterations, but their risk attributes change. A building's street address can be reformatted, its use converted and its insured value revised. A repaired vehicle can retain physical identity while its condition changes. Conversely, a redevelopment may create new insured structures within a previous parcel. These examples call for separate asset identity, asset version, insured-interest and exposure-period information. A single “most recent asset” record cannot substitute for an event history where frequency, severity and coverage must be assigned to periods.

Corporate identifiers illustrate why entity resolution cannot equate ownership with identity. A parent and subsidiary may share executives and addresses while maintaining separate legal personalities. Mergers, asset transfers, dissolution and internal restructuring can create different successor relationships. The GLEIF legal-entity reference framework assists with identification and ownership information, but legal succession and policy coverage still require competent evidence under the relevant law and contract. A data engineer may record an acquisition date; a legal or coverage analyst must determine which rights and obligations transferred.

A practical insurance system consequently needs a policy for conflicting sources. An official register, policy endorsement, claimant statement, broker feed and historical snapshot can disagree for legitimate reasons: different record times, different entity roles or uncorrected errors. Rather than selecting the last update as truth, the system should preserve competing assertions, decision dates, source reliability and the justification for the resolved value. This permits re-evaluation when evidence changes and prevents an audit trail from being overwritten by an attractive canonical profile.

Temporal leakage and prospective model evaluation

Temporal leakage occurs when information that was unavailable at the prediction time enters model training or evaluation. Identity resolution can introduce leakage without ever inserting an explicit future outcome variable. A final claim identifier created after investigation, a corrected address learned after loss, or a fraud-cluster designation derived from later claims can indirectly reveal subsequent events. Entity resolution performed once on the full dataset can carry future knowledge back into historical training rows.

A defensible prospective evaluation reconstructs the information state at a declared decision time. It versions not only predictors and labels but also candidate retrieval, match rules, identifier dictionaries, external source availability, threshold settings and graph membership. If the prospective test concerns a 2023 underwriting decision, using an identity mapping confirmed in 2025 changes the simulated information set. It may be appropriate for a retrospective corrected-experience estimate; it is inappropriate when the objective is to estimate how the 2023 decision model would have performed using contemporaneously available evidence.

The chronology also affects train-test separation. Randomly splitting transaction rows can place the same insured person, occurrence or duplicated claim on both sides of the partition. The resulting leakage can inflate model discrimination and produce misleading confidence about transportability. Depending on the question, a valid split can group by entity, policy, occurrence, origin system, calendar period or geography. There is no universally correct grouping; the rule must protect the unit whose independence or prospective novelty the evaluation claims to test.

Evaluation over time should quantify changing linkage performance. A new insurer acquisition may alter key formats; a new external registry may improve name resolution for some regions; a catastrophe may suddenly create highly similar claims. Report precision, recall, ambiguous-case rates and error impact by vintage, source, product and material subgroup. A single annual match rate can conceal structural breaks. A historical linkage quality statement applies to the population and methods actually tested, not indefinitely to every new system and dataset.

Temporal integrity is thus a joint property of identity, relationship and evidence availability. It extends the ordinary demand for accurate records into a demand that the correct historical question be asked of those records.

Duplicate and Household Relationships

Claims, incidents, payments and reserve movements

The word duplicate is dangerously underspecified in claims operations. A duplicated electronic message, a repeated invoice, a reopened claim, a supplemental claim, a second claimant, a second layer of coverage, an independent recovery and a fraudulently repeated submission may all share substantial textual or numerical similarity. They have different legal and accounting meanings. Entity resolution should assign distinct object types before attempting suppression, aggregation or fraud inference.

A defensible claims representation includes an occurrence, an insured interest, a coverage segment, a claim case, a component or line item, financial transactions and recoveries, each with its own identity and relations. One hurricane may produce hundreds of property claims; a single property loss may engage building and contents covers; a claim can be reopened for supplemental expense; one indemnity payment can be reversed and replaced by a corrected transfer. Deduplicating all common occurrence identifiers would undercount loss. Counting every ledger movement as an independent claim would overcount loss.

The appropriate unit depends on the target. For frequency, a carrier might count occurrences per policy-year or claims per exposure-year. For pure premium, aggregate ultimate claim cost divided by exposure may be suitable. For paid-loss reconciliation, the relevant identities are financial transactions and their accounting reversals. For fraudulent duplicate submissions, the relevant comparison includes claimed loss circumstances and documentary provenance; similarity creates a lead for review, not a finding of fraud. Multiple true claims from the same catastrophe should not be converted into fraud signals merely because they share peril, time and geolocation.

Consider a synthetic financial ledger containing an initial payment of $2,000, its documented reversal of −$2,000, and a corrected payment of $1,850. The net paid amount is $1,850. An algorithm that removes the reversal because it “duplicates” the first payment produces $3,850; one that removes the corrected payment because its claimant and incident match the initial item produces zero. Both are entity-resolution errors at the transaction-semantics level. The correct treatment depends on event types and ledger links, not text similarity alone. A reliable claims model preserves gross and net measures, corrections, currencies and effective dates.

Exposure and reinsurance aggregation introduce further layers. A ceded loss can be allocated to several treaties or layers while remaining one original occurrence. A catastrophe event identifier may group claims for accumulation analysis without equating the insured objects or payment rights. A legal recovery from another party affects net economics but does not erase the underlying gross loss. Reporting rules should make the counting level explicit at every stage so that a join can be audited against its intended measure.

Households, organizations and shared risks

Household relationships matter in motor, life, health, homeowners and other lines, but a household is neither an invariant person nor an unchanging address. Its boundaries can depend on common residence, family ties, financial dependency, policy definition, or jurisdiction-specific eligibility. A single resident may belong to several operationally meaningful groups over time. Policies with the same street address may refer to distinct apartments, co-located businesses or separate households. A household identifier built from last name and ZIP code is a hypothesis about association, not a statement of identical persons.

The distinction becomes actuarially material when household-level attributes are joined to individual experience. If drivers share a vehicle, the same exposure should not automatically be counted once per driver. If several people in a household hold individual life cover, individual benefit exposures remain separate despite shared address or family relationships. If a claims event affects multiple residents, the common event can induce correlated outcomes without making their claims duplicates. Statistical models must distinguish within-household dependence, repeated measures, common shocks and identity ambiguity.

For commercial insurance, related companies and beneficial ownership networks can support risk accumulation and sanctions or fraud screening, subject to the relevant legal purposes. Yet a corporate family is not a single contracting insured. Ownership percentage, control and registered identity are relationships with dates and jurisdictional legal consequences. The LEI provides an authoritative organizational identification mechanism where coverage exists; ownership links and succession require their own interpretations. A multinational group should not be reduced to a single exposure unit unless the intended measure explicitly covers consolidated group risk.

An entity graph that preserves these distinctions can support several analytical perspectives without changing the underlying records. A claims investigator may traverse shared device, address and payment-account relationships; an actuary may aggregate by insured asset; a solvency analyst may measure common-catastrophe concentrations; a consumer-conduct review may analyze outcomes by natural person. The graph remains an evidence structure whose edges are typed, dated and source-attributed. Each application defines a permitted view over that structure.

Product-specific cases and interoperable data

Property insurance particularly requires careful parcel, building and insured-location resolution. Mailing addresses and geocodes can be displaced from the physical insured site; parcel boundaries may shift, and a campus can contain multiple structures. Hazard studies should retain geospatial uncertainty and physical-building identity. A false join between claims and locations can create artificial weather or flood correlations, while an omitted join can make exposed properties appear loss-free. Any location-based validation should use a temporal and physical unit appropriate for the peril.

Motor insurance links policyholder, vehicle, authorized driver, registration history, accident, injury claimant and repair invoice. Vehicles can move between policies and owners, while claims involving several vehicles describe a common event without identical property. Health insurance adds healthcare-provider, patient, episode, billing and benefit-coverage identities; records may contain legally sensitive diagnoses or identifiers. Life insurance distinguishes insured lives, policy owners, annuitants, beneficiaries and claimants. Group benefits require membership and eligibility intervals. Each line has an identifiable linkage structure and its own potentially costly category mistakes.

Technical interoperability supports reliable data exchange but cannot itself determine whether records denote the same real-world entity. A published insurance message specification can standardize the names and expected meanings of policy and claim elements, while reference matching must still account for source quality, identifier jurisdiction, lifecycle and missing values. Such separation is essential for cross-border programs, inherited business, delegated authority and third-party administrators. Technical conformity describes a message; empirical linkage validation describes whether a factual identity assertion is correct.

An insurance enterprise can use common resolution primitives—source provenance, normalized comparison attributes, candidates, link likelihoods, cardinality constraints, decision states and audit revisions—while retaining distinct domain rules for each relationship. This combination yields interoperability without assuming that every record-sharing task is one identical statistical problem. It also gives an internal editor or regulator a more intelligible path from a reported outcome back to the original sources.

Linkage Error and Predictive Bias

False matches, missed matches and estimand changes

A false positive link declares two observations to concern the same target entity when they do not. A false negative or missed link leaves observations apart when they concern the same entity. Their consequences are not symmetric. A false match can attach a third party's loss history, credit-like feature, diagnostic code or fraud suspicion to a policyholder; a missed match can omit an authentic loss or split historical exposures. Both can affect pricing and claims decisions. The direction and magnitude of bias depend on the joint distribution of linkage error, outcome, predictors and population selection.

Let the true insurance experience be N=∑iYiN=\sum_iY_i events over exposure E=∑ieiE=\sum_i e_i, with frequency λ=N/E\lambda=N/E. Let NTPN_{TP} be true claims correctly linked to the target exposure population, NFPN_{FP} external or other-population claims falsely attached, and NFNN_{FN} target claims missed. When denominators are otherwise correctly constructed and all target claims fall into one of these categories, the observed linked-claim count is

Nobs=N−NFN+NFP,λ^obs=N−NFN+NFPE.N_{obs}=N-N_{FN}+N_{FP},\qquad \hat\lambda_{obs}=\frac{N-N_{FN}+N_{FP}}{E}.

The induced difference is (NFP−NFN)/E(N_{FP}-N_{FN})/E. This simple identity does not estimate false links from observed data; it establishes a conditional arithmetic consequence. In actual insurance data, entity splitting can also inflate EE, entity merging can reduce it, and systematic selection can change the target population. There the total bias combines numerator, denominator and conditional-outcome distortions. It may be positive, negative or vary by subgroup.

The UK ONS quality requirements for linked data distinguish precision from recall and emphasize that match counts do not establish validity. Health and population research offers direct evidence that linkage errors can influence downstream estimates; Harron and colleagues' (2016) mother–baby record-linkage study illustrates an applied setting in which identity and relationship structure are essential. Applying those empirical lessons to insurance requires explicit treatment of different exposure processes, contracts and reporting systems. It is unjustified to transfer an error rate measured in one population directly to a commercial insurance book.

A false link can cause measurement error in the predictor, outcome, or entity cluster simultaneously. Under ordinary independent classical measurement-error assumptions, some regression slopes attenuate, but insurance linkage errors often violate those assumptions. Record quality may correlate with age, migration, claims frequency, distribution channel, provider, or socio-economic proxies. The resulting bias can reverse direction, generate spurious interactions or alter which policies are selected for training. A researcher should identify the actual error process before importing a familiar textbook correction.

Numerical demonstrations and sensitivity analysis

Consider a fully synthetic portfolio with 1,000 valid exposure-years and 80 true claims. Suppose a matching system correctly links 68 claims, misses 12 genuine claims and falsely links six external claims. The resulting observed count is 74, giving an observed frequency of 74/1,000=0.07474/1{,}000=0.074, compared with the true 80/1,000=0.08080/1{,}000=0.080. The absolute error is −0.006 per exposure-year, and the relative error is −7.5%. Precision is 68/74≈91.89%68/74\approx91.89\%, recall is 68/80=85%68/80=85\%, and pairwise F1=2(68)/(2(68)+6+12)≈88.31%F_1=2(68)/(2(68)+6+12)\approx88.31\%. The familiar composite metric does not communicate the sign of the actuarial bias or the consequences of errors for different claims.

Under the explicit assumptions that the validated precision and recall apply to the target claims, the relevant linked-claim population is otherwise representative, and false-positive and false-negative counts are estimated for the same period and unit, a count-level correction can be written N^≈Nobs precision^/recall^\widehat N\approx N_{obs}\,\widehat{precision}/\widehat{recall}. Substituting the synthetic oracle values yields 74×(68/74)/(68/80)=8074\times(68/74)/(68/80)=80. This equality is constructed from the generating truth; in operational data the accuracy rates themselves must be estimated, their uncertainty propagated and their transportability justified. When false matches have systematically different severity or exposures are also mismeasured, correcting a total count alone does not recover an unbiased pure premium or claim-cost distribution.

Now split the same hypothetical portfolio into two equally exposed groups with 40 genuine claims in each. Group A correctly links 36 and falsely includes two, producing 38 observed claims; group B correctly links 32 and falsely includes four, producing 36. The underlying true frequency is 0.08 in both groups if each has 500 exposure-years, while the observed frequencies are 0.076 and 0.072. A model or conduct assessment may infer an artificial difference of 0.004 in absolute frequency. Group A has 90% recall and approximately 94.74% precision; group B has 80% recall and approximately 88.89% precision. Equal source-world risk and unequal linkage performance produce a measurable but spurious outcome disparity. The example illustrates a mechanism for differential bias, not an empirical statement about any protected group or insurer.

Loss severity creates additional asymmetry. Suppose a dataset contains ten true claims of $2,000 each, totaling $20,000. If two unrelated $10,000 claims are falsely appended as independent losses, the observed count becomes 12 and observed total becomes $40,000; the observed mean is $3,333.33 rather than the true $2,000. If those two payments were duplicate ledger movements for already counted claims instead, the same apparent total inflation requires a different remedy, since the object being duplicated is a transaction, not a new occurrence. A frequency model, a severity model and a pure-premium model are affected differently by the same incorrect database join. This is why aggregate accuracy must be paired with outcome-weighted diagnostics.

The posterior match example from the preceding chapter can be checked independently. Its inputs give likelihood ratios 450 and 37.5; their product is 16,875. For prior π=10−4\pi=10^{-4}, posterior odds are 16,875(10−4/0.9999)16{,}875(10^{-4}/0.9999), giving probability about 0.628. When the false-positive decision cost is nine times the false-negative cost, the pairwise decision threshold is 9/(9+1)=0.99/(9+1)=0.9; the same candidate should remain unresolved or unlinked under that stylized loss rule. With a prior of 0.01, the posterior is roughly 0.994, crossing the threshold. No individual claim should be paid or denied by direct application of this toy model.

A reproducible sensitivity study should vary missed-match and false-link rates, compare alternate exposure denominators, impose plausible subgroup and time trends, and re-evaluate metrics under several coherent linkage configurations. Where linkage probabilities are available, repeated draws from the supported configuration distribution can reveal the range of rate and severity estimates induced by identity ambiguity. Intervals should reflect both finite sample variability and linkage estimation error. Neither plugging the same noisy labels into several classifiers nor repeating one arbitrary split establishes uncertainty about the underlying identity truth.

Validation, fairness and statistical uncertainty

A validation sample must have a declared sampling design and a defensible truth-adjudication process. If reviewers examine only high-score pairs, they cannot estimate the missed-match rate among pairs never proposed by blocking. If labels come from the same identifiers used by the algorithm, the validation may reproduce the algorithm's blind spots. A more credible audit stratifies across score bands, sources, missingness patterns and materially relevant groups; includes sampled apparent nonlinks; measures candidate-retrieval loss; and documents uncertain adjudications. Where a full gold standard is impossible, partial verification and interval estimates are more informative than an unsupported “98% accurate” headline.

Precision TP/(TP+FP)TP/(TP+FP) measures the fraction of declared links that are true; recall TP/(TP+FN)TP/(TP+FN) measures the fraction of all true links recovered. A true-negative denominator may be enormous in cross-file matching, making accuracy (TP+TN)/all pairs(TP+TN)/all\ pairs deceptively high. Calibration should evaluate predicted probabilities against independent truth labels. Additional metrics can measure end-to-end entity clustering, constraint violations, duplicate-payment consequences, exposure-weighted errors and severe-loss-weighted misattributions. A high pairwise F1F_1 score may coexist with an unacceptable rate of wrongful high-value claim attribution.

Validation must also separate different kinds of fairness. A linkage model can have similar overall precision in two groups yet produce higher claim-cost misallocation in one group because severity differs. A model can have equal recall among observed candidates while one group's records fail blocking at a higher rate. A commercial premium system can thus create differential outcomes from identity processing before the pricing model is fitted. Fairness assessment should examine data-source coverage, subgroup retrieval, pairwise false positives and false negatives, downstream outcomes, and appeal or correction mechanisms. The assessment should be suited to relevant local law, line of business and insurance decision, not a single universal parity measure.

The analytical consequence extends to claims reserving and solvency. Accident-year triangles and experience studies require consistent claim and transaction identities across periods. Migrated claims may receive fresh identifiers, reopening may trigger new financial rows, and recoveries may post under different local accounts. Incorrect joins can shift development patterns or double count unsettled obligations. The appropriate QA combines accounting reconciliation, event-level verification and actuarial reasonableness checks. The U.S. Actuarial Standards Board's ASOP No. 23 on Data Quality calls for consideration of data appropriateness, sufficiency, reasonableness, significant limitations and relevant external information when performing covered actuarial services. It does not itself supply a numeric linkage acceptance threshold for every insurance operation.

Scientific validity depends on traceable relationships between source record, matching evidence, linkage decision, model input, statistical unit and final inference. An analyst should be able to calculate how the estimate changes under a plausible alternative matching resolution. A successful identity-engineering deployment is useful evidence of implementation; it is not, without those further checks, evidence of a correct actuarial estimand.

Privacy, Traceability and Quality Assurance

Lawful purpose, minimization and sensitive information

Record linkage can substantially increase the information available about a person by combining sources that were originally collected for different purposes. This creates privacy risks independently of whether a match is correct. A false positive can disclose someone else's medical, financial or claims information; a true but unauthorized cross-context join can still violate legal or contractual restrictions. Data minimization, lawful basis, confidentiality, access control and transparency therefore belong in the design of the linkage task rather than appearing as a final administrative review.

For processing within its territorial and material scope, the European Union's General Data Protection Regulation establishes principles in Article 5, bases of lawful processing in Article 6, additional conditions for special categories of personal data in Article 9, provisions for automated individual decisions in Article 22, and requirements concerning data protection impact assessments in Article 35. Their applicability depends on the particular controller, jurisdiction, data, purpose and operation. Article 22 addresses certain solely automated decisions with legal or similarly significant effects and includes defined exceptions and safeguards; it does not impose an identical rule on every statistical join. Insurance data processing may also be subject to sector, employment, consumer, health and national rules beyond that Regulation.

In the United Kingdom, the Information Commissioner's Office's data-sharing code of practice provides statutory guidance on fair and lawful sharing. Its current web presentation explicitly notes that guidance is under review following the Data (Use and Access) Act, so legal application requires checking current jurisdiction-specific materials at the time of use. A partnership between an insurer and an external record provider should distinguish their controller or processor roles and define permitted data, purposes, retention, correction, onward sharing and accountability. An insurer may possess a legitimate business objective yet still need a distinct legal justification for a proposed linkage.

The NIST Privacy Framework offers a voluntary, risk-based vocabulary for governance, identification and control of privacy risks. NIST's Digital Identity Guidelines, Revision 4, published in 2025, address identity proofing, authentication and federation in their specified government-system context. Identity proofing under those guidelines and statistical record linkage are related but distinct activities: an authenticated user session does not establish that every legacy claim record should be linked to that user. Standards should be applied according to their actual scope rather than treated as universal insurance law.

Privacy-preserving linkage and its residual risks

Privacy-preserving record linkage attempts to compare records across organizations while reducing exposure of directly identifying attributes. Schnell, Bachteler, and Reiher (2009) studied Bloom-filter approaches to private linkage. Durham and colleagues (2014) developed composite Bloom filters for secure linkage. Such methods can improve matching utility under defined threat assumptions, but encoding identifiers is not automatically anonymization. Repeated tokens, low-entropy fields, auxiliary information, protocol weaknesses and inferential outputs can expose identity information or reveal membership.

A privacy-preserving design needs an explicit adversary model: which parties know raw identifiers, what encoded comparison values can be observed, who obtains candidate scores and final links, whether keys are shared, and what external information may be available. The threat model should consider an honest-but-curious data processor as well as malicious participants where relevant. Strong transport security and encryption at rest protect data in particular phases but do not solve unauthorized inference from permitted outputs. Accuracy and security must be evaluated together; an encoding choice that prevents useful linkage can introduce differential missingness and affect statistical results.

The minimum necessary identity fields for a stated task may differ from those needed for actuarial modeling. Separating a trusted linkage environment from an analytical environment can limit direct exposure, with pseudonymous internal keys and access-controlled mappings. Such separation is a safeguard, not a guarantee of de-identification. Linkage outputs, especially unique rare combinations and event networks, can remain personal data under applicable law. Researchers should limit disclosures, maintain correction routes and justify retention periods based on documented purposes.

A governance design also assigns authority for contested links. Human adjudicators can be necessary for high-stakes cases, but review itself requires training, independence, appropriate evidence and documented conflicts. An insurer must prevent circular verification, in which the reviewer accepts the same faulty key that generated the candidate as independent corroboration. Review decisions should record evidence, outcome, rationale, reviewer or process authority, effective date, and whether the decision is provisional or final for a defined operational purpose.

Documentation, operational controls and scholarly reproducibility

A reproducible insurance linkage process retains source provenance and extraction versions; schema and field semantics; normalized and original attributes where lawfully retained; candidate generation and blocking rules; match model and training data lineage; priors, thresholds and constraint definitions; review states; cluster and edge revision history; quality sampling design; and all downstream dataset versions. Audit records should allow a qualified reviewer to reconstruct the relationship between raw input, identity decision, exposure denominator and published estimate without unrestricted disclosure of identifiable records.

The system should explicitly prohibit silent transformation of an uncertain candidate into an unquestioned master identity. A defensible workflow distinguishes proposed, automatically accepted, human-confirmed, rejected and unresolved relationships, with rules for reversal and historical versioning. Updating an identity graph changes potentially many features and reports; that effect should be tested and documented. Model performance should be re-evaluated after material changes in data sources, migration, blocking, algorithm version or contractual rules. A fixed numerical performance threshold may be useful operationally only when justified by business consequences and legal constraints.

Insurance-specific supervisory expectations reinforce that accountability extends beyond model code. The International Association of Insurance Supervisors' 2025 AI supervision application paper addresses risk-based proportionality, governance, robustness, transparency, fairness and redress in the supervision of AI systems. Its relevance is direct when entity resolution contributes to an AI-supported insurance decision; the paper does not convert all matching procedures into AI systems or itself create uniform binding law. Professional data-quality duties under ASOP No. 23 likewise must be interpreted in their U.S. actuarial scope.

A complete scientific audit compares three objects: the identity truth or supported reference evidence, the decision rule and the downstream estimand. The first requires adjudication or credible uncertainty bounds; the second requires statistical testing and implementation verification; the third requires re-running insurance calculations with documented alternatives. A high-quality engineering benchmark tests speed, candidate recall, precision and consistency; a high-quality actuarial benchmark additionally tests exposures, counts, amounts, time trends, subgroup differences and sensitivity to mislinkage. Privacy and documentation tests examine who may inspect and correct the result. This layered analysis forms an integrated audit procedure using established concepts, not a newly declared doctrine.

A final quality review should ask whether all linked scientific sources support their precise adjacent statements, whether standards are correctly dated and scoped, whether synthetic numerical examples are independently reproducible, and whether unresolved questions would materially affect the conclusion. This monograph records the sources underlying its claims and explicitly limits inference from paywalled abstracts, illustrative calculations and secondary application contexts. The analytical synthesis can guide future insurance research without implying that any insurer, data broker or regulator has adopted these results.

Research agenda and analytical synthesis

The unresolved scientific problems are substantial. High-quality evidence about identity error in different insurance products remains uneven, because proprietary systems rarely expose independently adjudicated ground truth. Valuable studies would quantify how false links and missed links vary by claim type, transaction channel, missingness, legacy migration, insured mobility and temporal distance between records. Other studies should compare the inferential value of calibrated probabilistic links with hard-threshold graph construction under reserve, pricing and fairness objectives. Cross-institution collaboration may require privacy-preserving methods, but such work should assess re-identification exposure and lawfulness as carefully as numerical accuracy.

Another research question concerns standards of validation under product change. A matching algorithm trained on mature automobile claims may behave differently on newly acquired property business, catastrophe-heavy claims or life policy beneficiaries. A credible transfer evaluation would report both expected link performance and induced error in exposure-weighted insurance outcomes. Changes in law, identifier infrastructure and administrative practice may alter both. Such research is feasible through auditable synthetic benchmarks, appropriately governed confidential studies, or independently reviewed administrative-data collaborations; none of those empirical datasets has been introduced in this work.

The integrated interpretation is specific. The quality of an insurance linkage does not reside in a universal match score. It resides in supported identity and relationship assertions, their temporal meaning, their consequences for the insurer's chosen population and estimand, and their lawful and accountable use. This interpretation follows from the combined statistical, contractual and institutional evidence. It yields a practical direction for future measurement and quality assurance while retaining the established literature's methodological vocabulary.

Conclusion

Insurance data linkage establishes the identity and relationship claims on which insurance statistics depend. Because policies, persons, organizations, insured assets, events, claims and payments are distinct entities, a correctly functioning system must preserve their different cardinalities, time intervals, source provenance and contractual meanings. Exact identifiers can reduce ambiguity where their scope and accuracy are established. Probabilistic linkage provides an evidentiary representation of ambiguous comparisons; Bayesian and graph-based approaches permit constrained assignment, clustered identity and explicit uncertainty. All approaches remain dependent on appropriate source data, realistic statistical assumptions and an independently validated relationship to the intended use.

The quantitative demonstrations prove several conditional results. False links and missed links change the reported claims numerator even when the exposure denominator is held constant; a corrected count can be recovered from known precision and recall in a deliberately restricted hypothetical setting, but this is not a general guarantee of unbiased insurance estimation. Differential linkage accuracy can generate apparent subgroup differences when true loss frequencies are equal. Changing the candidate prior alters posterior match probabilities despite identical comparison evidence. Financial-ledger repetitions, common catastrophes and multi-policy relationships show why apparent duplicates cannot be removed without an entity and transaction ontology.

Temporal evaluation adds a further requirement: historical inference and prospective modeling use different information sets. A linkage confirmed after the outcome may improve retrospective reconstruction while contaminating a simulation of what an insurer knew before that outcome. Privacy and professional obligations require purpose-specific linkage, traceable decisions, source quality reviews, meaningful correction procedures and evaluation of risks created by combining records. The GDPR, relevant UK guidance, NIST publications, U.S. actuarial data-quality practice and IAIS supervisory material supply different, expressly bounded institutional perspectives.

The original synthesis is that entity resolution is part of the scientific definition and measurement of insured experience. In underwriting, pricing, reserving, claims, fraud analysis and regulatory evaluation, a matched dataset is credible only to the extent that its identity assertions can be justified and their errors incorporated into the interpretation of results. This conclusion identifies a stable research requirement for insurance and risk science; it does not claim a new theory or an empirical discovery. Future progress depends on better independent truth evidence, uncertainty-aware downstream analysis, temporally faithful validation and insurance-specific reporting of consequences.

References