RESEARCH DOSSIER
Underwriting Data Vendor Due Diligence and Reliability
Abstract. This Research Dossier examines the reliability of external information used in insurance underwriting, with particular attention to vendor market structure, conditional coverage, data provenance, identity resolution, classification error, drift, confidentiality, fairness, procurement, audit rights and operational continuity. The research combines selected primary supervisory and professional documents with published research and constructed quantitative illustrations. It distinguishes the reliability of individual observations from their validity for a decision, the characteristics of issued portfolios from those of eligible applicants, and supplier assurance from insurer accountability. Findings show how measurement errors, uneven coverage and common dependencies can propagate through acceptance, pricing and portfolio outcomes, and why effective recourse and reproducible decision evidence are material. Comparative analysis covers selected international supervisory principles and United States, European Union and Australian instruments. No proprietary insurer data or new theoretical doctrine is claimed.
Keywords: Insurance underwriting; third-party data; vendor due diligence; data quality; risk classification; selective labels; concept drift; operational resilience; consumer recourse; insurance governance
Introduction
Underwriting data vendor due diligence is the evaluation of an external information supplier and the information it delivers before and throughout its use in insurance risk decisions. Its scientific object is the relation between external observations, underwriting actions, subsequent loss experience, and the institutional ability to identify and correct error. Reliability is the demonstrated fitness and continuity of information for a specified underwriting use, population, time horizon and decision rule.
An insurer purchases access to observations whose production generally takes place beyond its immediate control. The observations may concern an applicant, insured object, business activity, physical hazard or previous experience. The commercial agreement therefore connects two distinct processes: producing a description of risk and committing insurance capacity. A reliable description of one process can still be unsuitable for the other. A record that accurately reports a building's last recorded roof material may fail to describe its current condition; a score that predicts default may be inappropriate for a particular insured peril.
The principal questions are how to establish decision-specific fitness, how to preserve confidentiality and recourse, and how to estimate the consequences of error, unequal coverage and disruption. The analysis uses comparative supervisory documents, applicable U.S. professional standards, accessible scholarly records and industry research. It develops transparent analytical examples to explain mechanisms. No vendor is rated and no insurer-level raw data are available. Numerical illustrations are explicitly constructed. The literature search is purposive, not a systematic review or meta-analysis; its cutoff is November 19, 2026. Where full texts were unavailable, only claims established by accessible records are used.
The parent Science is Underwriting. The present inquiry concentrates on supplier selection and assurance, while risk classification, pricing, contract interpretation and general operational resilience enter where they determine the consequences of information dependency. The original contribution is an analytical connection between the quality of a delivered observation, the admissibility of its use, the sensitivity of a decision and the practical capacity to reverse errors.
External Data Market Structure
External suppliers occupy different positions in an information supply chain. Primary collectors record events or properties. Aggregators reconcile information from multiple collectors. Identity-resolution services match observations to the person, property or enterprise being assessed. Analytics providers transform information into variables or scores. Platform operators make those outputs available through interfaces or embed them in automated workflows. A single supplier can perform several of these roles, while a branded product can depend on numerous upstream companies.
The relevant distinction for the insurer is the function performed in its decision, rather than the supplier's marketing category. A raw record establishes an observed fact subject to verification. A derived variable summarizes or transforms observations. A score estimates or ranks a quantity under specified assumptions. A recommendation proposes an action. Confidence in the underlying observation does not automatically validate the transformation or the proposed decision. A vendor can correctly transmit an inaccurate source record, and a mathematically valid transformation can be applied to an inappropriate population.
The market's economic attraction comes from pooling collection costs across users, accessing specialized information and shortening assessment times. A smaller insurer may obtain capabilities that would be impractical to reproduce internally. The opposing economic force is dependency: the supplier controls update schedules, data definitions, coverage changes and sometimes the evidence needed to challenge a decision. Switching becomes expensive if an insurer's eligibility rules, rate classifications and historical decision files were developed around a proprietary score.
Information asymmetry exists in the procurement relationship itself. The vendor has greater knowledge of upstream selection and transformations; the insurer understands its own product, target outcomes and regulatory obligations. A purchase specification that asks for a high accuracy percentage without defining the target, denominator or reference truth delegates the essential measurement decision to the contracting party that has an incentive to emphasize favorable results. Procurement must establish a common testable specification before comparing offers.
In individual life insurance, the NAIC's accelerated underwriting overview (2026) describes the development of regulatory guidance on external data and predictive analytics; the relevant committee adopted guidance in August 2024. The setting illustrates why speed is a legitimate commercial objective: traditional evidence collection can delay decisions or cause abandonment. However, reducing the amount of conventional evidence changes the information set available when a risk class is assigned.
Property underwriting may use parcel boundaries, construction characteristics, aerial interpretation, hazard intensity or protective features. Automobile underwriting may use driving records and, where permitted, consumer reports or telematics. Commercial underwriting may rely on corporate registries, financial measures, industry codes and inspections. A business interruption exposure may be inferred from supplier relationships that are incomplete or rapidly changing. These examples describe possible arrangements, not a census of vendor adoption.
A useful market assessment establishes whether the insurer receives a reusable record, a time-limited response, a licensed derivative or a decision service. The distinctions have consequences for record retention, independent validation and exit. A purchased data entitlement does not establish a lawful processing purpose or permission to redistribute the underlying information. Rights to original records, derived information and audit evidence require separate identification.
Vendor Coverage and Sampling
Coverage is the probability that a usable vendor observation can be obtained for an eligible exposure in a specified population and period. It differs from API availability, the proportion of requests returning a syntactically successful response, and the proportion of responses that contain every field needed for a decision. Denominators must distinguish eligible proposals, submitted requests, matched entities, valid field values and eventual policies. Each quantity answers a separate question.
Let A denote the set of proposals eligible for a vendor-enabled rule, and M the event that an observation is usable. Coverage is Pr(M = 1 | A). A carrier should report this probability for relevant cohorts and repeat it through time. If a vendor reports a 95 percent match rate among returned records, the insurer must know how many eligible proposals were never submitted, how many requests failed and how many matches were ambiguous. Conditioning on successful records conceals the population with the weakest evidence.
Consider a hypothetical portfolio with 8,000 applications in cohort A and 2,000 in cohort B. Usable data exist for 7,600 A applications and 1,200 B applications. Coverage is 95 percent and 60 percent, respectively; the pooled rate is 88 percent. The remaining 1,200 applications include 800 in B. Consequently, two thirds of unobserved applications come from a group constituting one fifth of the initial population. A pooled 88 percent figure hides the concentration of missing data. Cohorts could represent distribution channels, geographic regions or product eligibility bands; they are not labels for protected classes in this illustration.
Missingness can be associated with the process of collection. An applicant might have recently moved, have limited documented history, or belong to a business category absent from a commercial registry. A remote property may lack high-resolution imagery. The reason that an observation is unavailable thus carries information about the collection process and potentially about the underlying risk. Treating a missing field as a neutral value assumes that its absence does not matter, an assumption that must be tested. Treating every absence as adverse creates a separate decision rule that demands its own actuarial and legal justification.
For property data, coverage must be exposure-weighted as well as count-weighted. If the missing 5 percent of buildings contains a large share of insured value, the loss consequence of missing observations can be substantial. For life insurance, face amount, age mix and duration may determine the financial materiality of missing evidence. Financial weighting does not replace policyholder counts; both reveal different consequences.
The data supplied for model development may also have different eligibility conditions from production data. A vendor's demonstration file may exclude records with uncertain identity. Its development labels may originate from established customers in another geography. A carrier should reconcile the distribution of inputs, eligibility filters and observation dates before interpreting a validation result.
The outcome problem is deeper than missing inputs. Claims are ordinarily observed for bound policies, subject to reporting, coverage and development. An application that is declined or never binds does not yield the same insured outcome. Lakkaraju and colleagues (2017) examine this general problem in selectively observed outcomes, using settings outside insurance. Applied to underwriting, historical decisions create a selected set of labels. Estimating loss on issued policies alone cannot establish the risk that would have been realized under a different acceptance policy.
Let D indicate binding and Y the potential covered loss conditional on a specified policy. Data often reveal Y only when D = 1. Comparing a proposed rule with the incumbent therefore requires accounting for differential selection, offer terms and take-up. Weighting can address observed differences under appropriate overlap and conditional-independence assumptions. It does not recover outcomes for categories that were always declined, nor eliminate bias from unmeasured causes of binding. A report must specify the estimand and whether the strategy identifies it.
Random pre-issue reviews can obtain an independent reference classification on otherwise accelerated applications; post-issue reviews can evaluate evidence received after issuance. Neither automatically identifies the counterfactual insured loss of a declined applicant. A separate follow-up process may be required, and its lawfulness, costs and impact on consumers must be considered. Sampling should be designed before inspecting vendor performance, with separate random and targeted audit streams.
For an independent random sample, a proportion estimate p has approximate standard error sqrt(p(1-p)/n), where n is the sample size. If 95 out of 100 verified observations agree with a reference, the estimated agreement is 95 percent but the approximate standard error is 2.18 percentage points. Exact or Wilson intervals are preferable for small samples and extreme proportions. Correlated observations from the same building, household or broker reduce effective independent sample size. Sample plans should therefore use appropriate clustered methods and report uncertainty rather than interpreting a threshold as an absolute fact.
Provenance, Identity Resolution and Data Quality
Provenance specifies the origin and transformation history of the observation used for a decision. At minimum it identifies the collector or source category, observation time, supplier acquisition time, delivery time, geographic and entity identifiers, transformation version and permitted-use restrictions. The insurer needs to preserve the relevant evidence as it existed when the decision was made. Re-querying a database after a correction or update can produce a substantially different record.
Wang and Strong (1996), in Beyond Accuracy, present exploratory evidence for multiple dimensions of information quality, including fitness for its context. In underwriting these dimensions have distinct consequences. Accuracy concerns agreement with the reference fact; completeness concerns the necessary information present; timeliness concerns the age of facts in relation to the relevant decision; consistency concerns coherent representations; accessibility concerns the ability to retrieve and interpret the evidence when needed. These characteristics can conflict: broad and timely data may have weak validation, while authoritative records can arrive too late.
The Actuarial Standards Board's ASOP No. 23 (2016) provides a professional framework for selecting, reviewing and using data, including reliance on information supplied by others. Its scope is actuarial services and it does not itself require a formal data audit. An insurer's separate procurement assurance must therefore establish the appropriate level of inspection for its use. ASOP No. 12 (2005, updated 2011) provides the corresponding U.S. professional context for risk classification. The standards do not determine the terms of every commercial data contract.
Identity resolution is especially material because a correct observation about the wrong entity constitutes an incorrect underwriting input. Deterministic matching on a reliable unique identifier may be appropriate where available and permitted. When matching uses names, addresses, dates or coordinates, a system must accommodate ambiguity, historical addresses, missing apartment numbers, parcel subdivisions and duplicate records. It should distinguish a confirmed match, an uncertain match and no match. The reference reviewed in a linkage audit should be independent of the matcher's own output.
A simple constructed example illustrates the impact of base rates. Among 10,000 candidate record pairs, assume 500 are true matches. A linkage rule detects 98 percent of the true pairs and falsely accepts 1 percent of the 9,500 nonmatches. It produces 490 correct and 95 incorrect matches. Its apparent combined accuracy across pairs is 9,895 / 10,000 = 98.95 percent; its accepted-match precision is 490 / 585 = 83.76 percent. More than one in six accepted matches is wrong under these assumptions. The relevant quality statistic is determined by how the match enters underwriting, rather than by the highest available percentage.
A reference truth can itself be uncertain. A conventional underwriting review may disagree with another reviewer, a property inspection may be outdated and a diagnosis code may be an imperfect description of a clinical state. An adjudicated validation sample should document its reference standard, timing, conflicting records, resolution procedure and rate of unresolved cases. An agreement measurement against an imperfect reference is an agreement measurement, not necessarily the absolute error rate.
The supplier must distinguish observed, estimated, imputed and inferred values. A roof age inferred from imagery has a different evidential meaning from a verified completion date. A data schema should preserve this distinction and record uncertainty where available. Downstream transformations must propagate quality indicators instead of discarding them. A rule that interprets an unknown date as zero is an input-processing error even if the original vendor response was correct.
Gebru and colleagues (2021) propose datasheets for datasets documenting the motivation, composition, collection and intended applications of datasets. The idea is applicable to procurement because it provides a common vocabulary for questions an insurer must ask of external sources. A datasheet is evidence of documentation, not proof that a data field is accurate or that an insurance use is permitted. Its claims still require validation against representative records and actual contractual rights.
Version control should cover the upstream source, field definitions, data mapping and model logic. An indicator may retain its name while its source changes. Unit conversions, category mappings and absent-value conventions must be tested against interface specifications. The same raw data should be processed through both historical and proposed versions during a migration to identify differences. Known failure cases should become maintained regression fixtures.
An error can propagate across teams that assume another group has resolved it. Sambasivan and colleagues (2021) discuss data practices in high-stakes AI systems based on interviews with 53 practitioners across several regions. Their research is not an insurance-specific estimate of defect incidence. It identifies a relevant organizational concern: the work required to maintain data quality can be distributed across many people and institutional boundaries. Applied to underwriting, the responsibility to correct a source defect must be allocated before it occurs.
Quality and Drift Testing
Vendor validation must be conducted at several levels. Interface validation checks schema, response status, identifiers and timestamp formats. Field validation checks observation agreement, missingness and uncertainty. Decision validation identifies how the fields change eligibility, price, terms or referral. Outcome validation determines whether the new decision process performs appropriately over the relevant future loss horizon. The first two levels are necessary regardless of whether a vendor supplies a score or raw information.
An initial validation should compare a baseline process using existing inputs with a challenger using the new supplier. The target should be declared in advance: frequency, severity, mortality, expected claim cost, a reference risk class, or another justified decision quantity. If the target is a reference risk class, the study evaluates agreement and misclassification. It cannot be described as direct validation of future mortality or losses without realized outcomes.
A prediction's discrimination and calibration are distinct. Discrimination measures the ability to order cases by the target risk. Calibration compares estimated magnitudes with observed outcomes. A model may preserve relative ordering while systematically underestimating loss levels. For a binary event, the Brier score is the average of (p_i - y_i)^2. In a constructed four-case example with p = (0.1, 0.2, 0.4, 0.8) and y = (0, 0, 1, 1), the score is (0.01 + 0.04 + 0.36 + 0.04)/4 = 0.1125. This example is an arithmetic demonstration, not evidence of a vendor's performance.
For expected losses, a portfolio calibration ratio may compare total observed incurred losses with total predicted losses for matched exposure and valuation periods. The choice of incurred or ultimate amounts, treatment of deductible and limits, reporting lag, and actuarial development are essential. A raw paid-loss comparison shortly after policy issuance can falsely suggest that predicted losses are excessive. In life insurance, observed deaths require sufficient exposure and duration; proxy classes are leading indicators rather than substitutes for long-run experience.
Seeman and Herzog (2024) describe a mortality slippage monitoring study drawing on more than 33,000 lives from 30 accelerated underwriting programs through year-end 2023. They distinguish random holdouts conducted before final decisions from post-issue audits and calculate implied mortality effects from risk-class differences using internal mortality assumptions. The evidence supports the usefulness of independent monitoring and the distinction between proxy and realized experience. It does not establish the effect of all data vendors or supply an unbiased estimate for every carrier. The insurer samples, age mix, audit methods and model assumptions limit transferability.
Incremental predictive utility should be measured on temporally separated data and preferably on an external population representative of production. A random record-level split can create leakage if repeat policies, related properties or duplicate business entities occur in both sets. Training and tuning must be separated from the final evaluation. Sources should be restricted to information actually available at the underwriting decision time. Later records can be used as verification targets, not inserted into historical inputs.
Drift describes a change over time in the relevant distribution or relation. Changes in the input distribution, the loss distribution and the conditional relation between inputs and outcomes have different implications. The literature on concept drift adaptation reviewed by Gama and colleagues (2014) is a general machine-learning research context. Here, the underwriting-specific distinction is between a change in the supplier's collection mechanism and a change in the underlying insured risk itself.
For example, a new imagery supplier may classify roof types differently without any physical change in the building stock. Alternatively, the annual loss distribution may change because of increasing repair costs despite an unchanged description of roof types. A third situation arises when the insurer changes eligibility criteria and thereby changes the insured population. The same movement in a dashboard metric can have any of these causes. Monitoring should include version events, cohort composition and outcome timing rather than attributing every shift to deterioration of the vendor.
Population stability indices or distribution-distance measures can provide early warning, but none has a universally justified underwriting threshold. For bins with reference shares q_j and current shares p_j, a common population stability index sums (p_j - q_j) ln(p_j/q_j). Zero shares require an explicit rule; binning, sample size and weighting affect the statistic. A high value flags distributional change, not necessarily incorrect predictions or unlawful discrimination. A low value cannot establish stable conditional loss relationships.
Operational validation must measure stale responses, response latency, incomplete fields and errors by cause. Outcome monitoring requires slower feedback. A two-speed system can separate readily available indicators from mature claims evidence. Fast indicators can include valid-response rate, schema errors, external-source age, completeness, referral rate and override frequency. Slow indicators include calibration, ultimate loss ratios adjusted for mix and exposure, classification errors from adjudicated audits, and confirmed outcome differences.
The owner should predefine thresholds, actions and escalation. A threshold is a trigger for an investigation or defined restriction, not a universal scientific constant. A statistically significant change in a large sample may be financially negligible; a very rare error may be catastrophic if it affects a high-limit risk. Testing therefore combines uncertainty, exposure materiality and consumer consequences.
Decision Sensitivity and Portfolio Consequences
A supplier defect matters through the action it changes. Suppose an underwriting score S is calculated from internal observations X and external data Z, and the action a is determined by a rule d(X,Z). The relevant counterfactual for a detected source error is d(X,Z_corrected). This comparison identifies which decisions would have changed, subject to any additional manual review and the policy in force. The insurer should not assume that every data defect altered the decision, nor that a defect below an acceptance threshold was harmless: it may have changed price, exclusions or the quality of an explanation.
A constructed portfolio illustrates the financial mechanism. Assume the insurer receives 1,000 otherwise comparable applicants, of whom 100 have an expected annual insured loss of $1,000 each and 900 have an expected annual loss of $100 each. Under a simplified classification, correct expected losses total $190,000. Suppose the vendor causes 20 high-risk applicants to be put into the lower class and 30 low-risk applicants to be put into the higher class; class premiums are set equal to class expected losses, with no expenses or margin. Total premium is then 110 x $1,000 + 890 x $100 = $199,000. The misclassified low-risk applicants are overcharged by $27,000 in aggregate while the misclassified high-risk applicants are undercharged by $18,000. Positive aggregate premium error of $9,000 conceals individual harms and an incorrect distribution of prices.
If the more expensive offer drives away low-risk applicants, the initially positive aggregate difference may disappear. For example, if all 30 incorrectly high-priced low risks reject coverage while the other applicants bind, premiums become $169,000 and expected losses become $187,000. The resulting expected underwriting loss is $18,000 in this deliberately simplified calculation. The result depends on the assumed behavioral response and should not be generalized as an estimate of real elasticity. It demonstrates that the decision and take-up process can amplify information errors.
Threshold concentration matters. A small continuous change in a property risk score may have little effect for most locations but change an eligibility decision for locations close to a cutoff. A discontinuous decision rule transforms modest measurement errors into material changes for a subset of applicants. The distribution of distance to thresholds should be examined alongside marginal data accuracy. Analysts should measure the frequency and magnitude of changed decisions under realistic perturbations, including the correlated errors that may accompany a source revision.
Correlation also matters at the portfolio level. If errors are independent across risks, some may offset one another in aggregate. Treating a shared geographic layer, common classification table or centralized vendor scoring service as an independent per-policy source of noise is inappropriate. A single mapping error can affect many policies at once. A review of exposure concentration should estimate the total insured value, policy limits and contractual obligations linked to a common input source.
A vendor's commercially attractive increase in quote conversion may interact with risk selection. Conversion can rise because legitimate friction was removed, because the product became easier to understand, or because risk requirements were weakened. Outcome evaluation must distinguish conversion at the same risk profile from a change in the composition of applicants or policies. A vendor value assessment should measure incremental information under a controlled comparison and relate operational savings to risk and consumer outcomes.
The economic case for a source includes fees, integration and verification costs, additional underwriting work, false referral expense, missed adverse information, wrongly adverse decisions, remediation and exit costs. Some effects can be valued through expected costs; lawfulness, confidentiality and the ability to provide required explanations constitute constraints on the use of the data rather than costs that can simply be traded against additional profit. The business case remains conditional on these constraints.
Fairness, Confidentiality and Consumer Recourse
Data reliability includes evaluating whether an observation is suitable for the insured event and whether its patterns of collection, transformation and use create unequal errors or prohibited classifications. A variable can be predictive within a historical dataset and still be inadmissible under applicable law. The legal inquiry must consider the jurisdiction, insurance product, proposed action and nature of the information.
The Federal Trade Commission's 2007 credit-based insurance scores report provides an important historical example. Its official summary reports that credit-based scores predicted automobile insurance risk in the study while score distributions differed across racial and ethnic groups. The report summary does not establish the lawfulness of a specific present-day scoring product, the size of its effect in another market or causal mechanisms. The two findings illustrate why statistical utility and consumer distribution require separate investigation.
A sound review measures coverage, match precision, false-positive and false-negative rates where a credible reference is available, calibration conditional on relevant exposure and the distribution of underwriting actions. Group comparisons must be legally permissible and privacy-protective. The appropriate groups and metrics depend on the jurisdiction and the decision. A difference in acceptance rates is an observable disparity, whereas unlawful discrimination is a legal conclusion requiring the applicable standards and factual analysis. A lack of a detected difference may reflect low statistical power.
For a binary adverse flag, let T denote a reference condition. The false-positive rate is Pr(flag = 1 | T = 0); the false-negative rate is Pr(flag = 0 | T = 1). In a hypothetical validation, 10 false flags among 1,000 reference-negative cases yield a 1 percent false-positive rate. Five false flags among 100 reference-negative cases yield 5 percent. Equal absolute error counts or high pooled accuracy would obscure the relative burden. If cohort sizes are limited, uncertainty intervals and review of the reference standard are essential.
Differential rates can emerge before modeling begins. A data aggregator might have more reliable address histories for one channel than another. A model may learn the differential availability of the external evidence rather than the intended underlying risk factor. A vendor audit should investigate whether more direct or less burdensome information provides adequate decision utility. The trade-off is evaluated for the specific use, using independent holdout evidence and lawful assessment methods.
The insurer requires a legal basis and defined purpose for processing. The European Union's GDPR (2016) establishes an accuracy principle in Article 5, including reasonable steps to rectify or erase personal data that are inaccurate in light of the processing purpose. This discussion uses that narrow provision; the appropriate lawful basis, information rights, automated decision restrictions and other obligations must be separately determined under the applicable provisions when implementing a particular system. An insurance data contract cannot itself grant authority that the insurer lacks.
In the United States, the FTC's consumer report guidance (2016, updated 2025) explains that applicable Fair Credit Reporting Act obligations depend on the product and use of consumer reports. The guidance addresses the purpose for obtaining a report, permission in relevant medical-information cases, and adverse-action notice where the report contributes to the adverse decision. Other legal obligations may apply even when a supplier or data product falls outside this statutory category.
Consumer recourse is part of evidence quality. An applicant must have a meaningful path to identify incorrect information where disclosure is required or provided. A complaint should be linked to the actual source, decision and relevant version. The carrier should track the dispute independently of its vendor, obtain corrected evidence, reevaluate the original action when warranted, and document the outcome. A correction at the vendor that never reaches the underwriting decision system leaves the original harm unaddressed.
Confidentiality should be preserved by controlling the collection, transfer, access and storage of information. Development samples, dispute records, audit extracts and production records may require different controls. A vendor's request to reuse insurer data for secondary purposes must be assessed separately from the original service. Retention should support applicable recordkeeping duties without retaining unrelated personal data indefinitely. Access to identifying records and sensitive information should be restricted and logged. An audit process must itself maintain confidentiality.
Procurement and Outsourcing Controls
Due diligence begins before the purchase with a defined use case and responsible underwriting owner. The intended product, jurisdiction, applicant population, eligibility rules, material fields and decisions must be specified. Vendors should be evaluated against a baseline using common sample definitions and measurement periods. A procurement comparison that allows each candidate to choose a different validation sample measures supplier presentation rather than relative fitness.
The insurer should examine legal ownership and organizational continuity, source rights, collection practices, service architecture, change-management processes, financial capacity, subcontracting and available independent controls. The extent of investigation depends on the materiality of the service. A data source used only for a low-stakes manual prompt requires a different assessment from a single supplier controlling an automated eligibility decision for an entire product. Materiality should reflect the ability to substitute a source, potential affected policyholders and portfolio exposure, as well as contract value.
The international supervisory context is explicit. IAIS (2024) ICP 8.8 on outsourcing requires supervisors to ensure appropriate insurer oversight and accountability for outsourced material activities. Its guidance addresses review and approval, risk and continuity assessment, written agreements, expertise, and periodic review. ICP 9.3 addresses supervisory access and review of outsourced material activities. These are international supervisory principles implemented through applicable jurisdictional frameworks; the classification of an individual data supply arrangement depends on its role and local rules.
New York's final Circular Letter No. 7 (2024) states supervisory expectations for external consumer data and AI systems in underwriting and pricing. Its vendor provisions emphasize insurer responsibility, written acquisition and oversight processes, correction procedures, and appropriate contractual audit arrangements and regulatory cooperation where available. The guidance's scope and phrasing must be preserved: it does not impose an identical audit contract on every vendor worldwide.
The NAIC's AI regulatory overview (2026) describes the December 2023 model bulletin and examples from insurer surveys. Adoption of the model bulletin by the association is distinct from the way an individual state incorporates or applies it. This distinction matters for research because conceptual convergence in oversight cannot be treated as identical enforceable obligations.
In Australia, APRA's CPS 230 version commencing July 1, 2026 governs operational risk for entities in scope. Underwriting-service providers are among the categories presumptively treated as material unless justified otherwise. The standard provides for assessing material arrangements, legally binding terms and supervisory access, as well as specified exceptions or exemptions. An ordinary data feed is assessed according to its actual function and materiality rather than automatically equated with a fully outsourced underwriting function.
The contract establishes the operational expression of the due diligence. The service description should specify required fields, accepted entities, collection dates, definition changes, source categories, coverage boundaries, data quality measures and exception codes. The insurer should obtain relevant rights to verify claim-level information, retain decision evidence, receive correction notifications and investigate significant failures. A full proprietary source code transfer is not necessary for all uses; sufficient effective validation, traceability and regulatory access are central. A refusal to permit any material independent challenge can make a product unsuitable even when its demonstration results are encouraging.
Change provisions should distinguish routine updates from material modifications to collection, transformation, coverage, matching, infrastructure or subcontractors. Every material change should carry a traceable version and effective date. An insurer may require a test dataset and a parallel-run window, subject to privacy limits and negotiated rights. A service-level agreement should define measurements from the recipient's perspective and describe exclusions, escalation and remedies. The source of a data-quality failure is not always the same as the source of an API outage; both should be addressed.
The contract should address confidentiality and permitted uses, incident reporting, secure disposal, and access to relevant records after termination. Appropriate audit rights may involve independent qualified auditors, a controlled audit environment, standardized control reports or direct inspection where necessary. General control certifications or attestations can provide useful evidence about a particular scope and period; they cannot establish relevance of every underwriting variable or fitness of the supplier for the insurer's decision population.
Technical and scientific requirements must remain connected. A service may meet uptime targets while supplying stale values. A data validator may pass because it only checks field types. A model may pass backtesting but become unreviewable when the insurer loses historical access to source versions. The procurement approval must therefore specify the exact authorized uses, version or material-change rules, control owner and conditions for restricted or suspended operation.
Incident Handling, Audit Rights and Operational Reliability
Operational reliability is the sustained availability of usable, timely and authentic information. An uptime measure computed from HTTP success responses is insufficient if a vendor returns records that lack required fields, exceed an approved age or correspond to the wrong entity. Metrics should separately identify transport availability, usable response rate, response time, stale values and data integrity failures.
Assume a process receives 2,000 eligible requests daily and 60 percent depend on a vendor for immediate automated processing. A complete five-hour outage during a ten-hour uniform arrival window delays or reroutes approximately 600 requests. If the manual fallback can resolve 80 cases per hour during the five hours, 400 are processed and 200 remain backlogged, before any other workload or later arrivals are considered. If actual arrival rates are concentrated, the estimate changes. An operational test should therefore use time-of-day volumes and observed fallback capacity rather than a daily average alone.
An outage may produce ambiguous behavior. Retry storms can overload both the supplier and insurer; duplicate requests can cause inconsistent versions, excess costs or erroneous writes. Caching can preserve continuity but returns observations from an earlier time. The acceptable age differs between a permanent property characteristic and a rapidly changing sanction or eligibility field. Pending decisions must preserve their state so a later successful request cannot overwrite a manual determination without appropriate controls.
A failure strategy should be selected according to the decision's consequences: suspend the automated path, request alternative evidence, route for qualified human assessment, or use validated data within its approved freshness window. Automatically accepting or declining all unavailable cases creates an unvalidated classification rule. Temporary procedures should be documented with the start and end times, eligible products, decision authority and affected cases. Recovery requires reconciliation of any decisions or offers made during the disruption.
Two vendor brands do not necessarily provide independent continuity. They may share a primary registry, identity-resolution service, cloud infrastructure or downstream analytic component. In a stylized model, suppose a common upstream dependency fails with probability 1 percent during a defined period and, conditional on its functioning, two alternative services fail independently with probabilities 2 percent each. The probability that both services are unavailable is 0.01 + 0.99 x 0.02 x 0.02 = 0.010396, or 1.0396 percent. Multiplying independent marginal outage probabilities would wrongly treat the common component as independent. The calculation illustrates dependence, not measured service availability.
The framework in APRA's 2026 CPS 230 expressly addresses ongoing service provision, risks from dependencies and orderly exit for material arrangements. Applied to underwriting data suppliers, a continuity exercise should therefore verify actual alternate data rights, operational capacity, staff training, reconciliation and the availability of historical evidence. A nominal exit clause is of little value if it cannot be executed with the retained information.
Security incidents can compromise confidentiality, availability or integrity. A credential exposure may leave the service technically available while making the source untrustworthy. A malicious change to a dataset can affect a concentrated class of risks without obvious interface failures. Incident triage must consider whether consumer data were exposed, a source was altered, or an incorrect decision was made. Notification duties are determined separately for relevant jurisdictions and agreements.
An investigation should establish event timing, affected versions, impacted proposals and policies, the type and magnitude of the data defect, decisions that relied on it, and whether action is required. Preserve the original record and a corrected version in accordance with retention obligations. Identify all downstream uses, not simply the service making the initial request. Recovery requires verifying data correction, testing affected decision logic, reconsidering cases where appropriate, and satisfying any applicable reporting, notice or remedy duties.
An audit right is a means of access, not the completed audit. Effective inspection defines the scope, sampled period, evidence access, qualifications of reviewers, independence, findings and corrective actions. The supplier may provide an annual report on general controls while the insurer conducts targeted data validation for a specific field. Material defects require a traceable issue owner, agreed remediation deadline, retest and a decision on continued use. APRA's service-provider arrangements and New York's vendor guidance exemplify the relevance of such access and monitoring under their own institutional conditions.
A tested exit plan includes a complete inventory of dependencies and downstream consumers, permissible export format and retention terms, replacement or manual decision procedures, and a comparison of new and historical classifications. Portability of a file does not guarantee portability of a decision: a replacement score may measure a different target and have a different scale. The replacement must be independently validated. The original decision records remain important for complaints, regulator inquiries, claims disputes and historical experience studies.
Comparative Synthesis and Reproducibility
The regulatory comparison is functional rather than a claim of legal equivalence. The IAIS framework deals with supervisory oversight and insurer accountability for outsourcing. U.S. professional standards address actuarial work and data reliance. New York expressly identifies underwriting and pricing uses of external consumer information and AI. Federal U.S. consumer-report rules have their own product and action triggers. The GDPR establishes a cross-sector personal-data requirement where applicable. The EIOPA (2025) AI governance opinion describes a proportionate insurance-sector supervisory approach; the present comparison relies on the official description, not the detailed text of the opinion. Australia's CPS 230 articulates operational-risk requirements for APRA-regulated entities.
These authorities converge around an institutional problem: an insurer must be capable of understanding and managing material dependencies even where information is produced elsewhere. Their particular legal mechanisms, affected actors, definitions, thresholds and remedies differ. A legal crosswalk should identify the responsible authority, operative provision or guidance, product scope, effective date and local implementation. The present international comparison is a selected examination of several regimes, not an exhaustive survey of world insurance law.
The differences across lines are also substantial. Life insurance may experience long delays before outcome validation obtains statistically credible mortality experience. It can use independent reference underwriting as an interim indicator with documented assumptions. Property insurance may have geographically correlated measurement errors and severe tail exposures. Automobile insurance may use rapidly changing individual or vehicle observations and can be sensitive to lawful-use restrictions on consumer reports. Commercial insurance may have heterogeneous and sparse exposures, changing legal entities and complex relationships that make broad market-level validation less informative.
The same statistical metric can have different scientific meaning across these settings. In a life program, class agreement might be translated into implied mortality using a calibrated mortality basis. In a property program, the correspondence of vendor building records to independent inspections may be tested at the location level, while error propagation is evaluated against insured value and catastrophe modeling. In commercial underwriting, manual referral quality and the accuracy of an industry or entity classification might deserve greater emphasis. Financial exposure, uncertainty, the time to outcome evidence and possible consumer harm determine which tests are necessary.
A reproducible vendor review should preserve an evidence inventory, data-use specification, sampling frame, eligibility and exclusion rules, data dictionaries, source version, permitted reference datasets, error definitions, model and decision version, outcome period, loss valuation date and analysis code. Access to proprietary or sensitive records may be restricted; reproducibility can be achieved through controlled independent access, aggregate disclosure or a secure research environment when lawful. A public scientific report should explain its restrictions rather than imply that confidential data were independently replicated.
An evaluation can be made more rigorous through a prospective comparison. Define the baseline decision path and a candidate vendor-assisted path; evaluate both on a suitably selected sample with information frozen at the original decision date. Where ethically, legally and operationally feasible, randomize independent review rather than the adverse decision itself. Use the same reference adjudication and blind reviewers to the vendor's output where possible. Compare decision differences, additional manual work, elapsed time and appropriately mature outcomes. Document all withdrawals and missing responses; preserve the originally assigned review group in analysis when possible to reduce selection bias.
Material changes should trigger review with an explicitly defined baseline. For a vendor providing a new data version, compare old and new outputs on identical input requests, inspect changed records, classify differences and test the decision consequences. For an insurer changing underwriting policy, freeze source observations while varying the rule. This separates source effects from decision effects. A simultaneous shift in both prevents straightforward attribution unless a design or sensitivity analysis can isolate their contributions.
Competing explanations should be evaluated for observed deterioration. A rise in referrals may be caused by a vendor mismatch, a new distribution channel, legitimate risk changes, an altered underwriting threshold or a changed failure-handling rule. A loss-ratio increase may reflect exposure mix, claims inflation, reporting practices, catastrophes, deteriorating source reliability or unrelated pricing inadequacy. Across periods, comparing actual experience with risk-adjusted expectations and tracking provenance changes makes the diagnosis more reliable. None of these comparisons alone establishes causality.
A release or continued-use decision should state which population and actions were authorized and why the evidence is adequate for that scope. Validation failures can be resolved by corrected records, narrowed use, a validated fallback, revised mappings or a new independent study. Residual uncertainty must be described in relation to possible decision consequences. An aggregate vendor score is an insufficient substitute for the evidence relevant to a material failure mode.
The author's synthesis is that vendor reliability in underwriting must be evaluated at the location where the externally supplied information becomes an insurance commitment. Source provenance establishes what was observed; population analysis establishes for whom evidence is available; validation establishes how informative and stable it is; legal and contractual review establishes admissibility and accountability; incident and exit controls establish whether erroneous commitments can be contained and corrected. These are interconnected applications of established methods and supervisory requirements, not a new theory, model or standard.
Conclusion
External data can increase the speed and informational reach of underwriting, but its decision value depends on the properties of the supplied data and the institution using them. A supplier's average quality, predictive accuracy or uptime cannot independently establish reliability for a specific insurance decision.
The research establishes that the relevant validation population is the eligible proposal population, not only successfully matched or issued cases. Missingness and selective outcome observation require explicit treatment. Identity errors and source changes can affect many policies through a shared dependency. Errors near underwriting thresholds can alter eligibility and price even when overall field error rates are small. Portfolio averages can conceal misclassification and consumer harm, as the constructed risk-class example demonstrates.
The comparative analysis identifies an institutional continuity between scientific validation, procurement, legal accountability, complaint resolution and operational resilience. Each is necessary to establish the scope within which external observations may support insurance commitments. The degree of investigation should correspond to materiality, uncertainty and the potential consequences of a wrong or unavailable observation. Approval should be specific to the assessed decision and population, and renewed when material source, model or use conditions change.
The evidence supports bounded qualitative and analytical conclusions. It does not estimate a universal rate of vendor error, prevalence of unlawful discrimination, or aggregate effect on insurer profitability. The illustrative calculations establish mechanisms under their stated assumptions. Regulatory references concern selected jurisdictions and provisions; the work is a research monograph rather than individualized actuarial or legal advice. Further empirical work could use permissioned linked proposal, decision, vendor-version and mature loss records to estimate the effects under a prospective sampling design. No claim of priority, new theory or external peer review is made.
References
Actuarial Standards Board. (2016). Actuarial Standard of Practice No. 23: Data Quality. Effective April 30, 2017. Source
Actuarial Standards Board. (2005, updated 2011). Actuarial Standard of Practice No. 12: Risk Classification (for All Practice Areas). Source
Australian Prudential Regulation Authority. (2026). Prudential Standard CPS 230: Operational Risk Management. Version commencing July 1, 2026. Source
European Insurance and Occupational Pensions Authority. (2025, August 6). Opinion on Artificial Intelligence Governance and Risk Management. Source
European Parliament and Council of the European Union. (2016). Regulation (EU) 2016/679 (General Data Protection Regulation). Official Journal L 119, 1–88. Source
Federal Trade Commission. (2007, July). Credit-Based Insurance Scores: Impacts on Consumers of Automobile Insurance: A Report to Congress. Source
Federal Trade Commission. (2016, November; edited January 2025). Consumer Reports: What Insurers Need to Know. Source
Gama, J., Zliobaite, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4), Article 44, 1–37. Source
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daume III, H., & Crawford, K. (2021). Datasheets for Datasets. arXiv:1803.09010v8. Version of work published in Communications of the ACM, 64(12), 86–92. Source
International Association of Insurance Supervisors. (2024). Insurance Core Principles and ComFrame: ICP 8, Risk Management and Internal Controls, and ICP 9, Supervisory Review and Reporting. December 2024 framework, online tool. Source
Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017). The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Source
National Association of Insurance Commissioners. (2026). Accelerated Underwriting. Insurance Topics. Source
National Association of Insurance Commissioners. (2026, April 3). Artificial Intelligence. Insurance Topics. Source
New York State Department of Financial Services. (2024, July 11). Insurance Circular Letter No. 7: Use of Artificial Intelligence Systems and External Consumer Data and Information Sources in Insurance Underwriting and Pricing. Source
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P. K., & Aroyo, L. M. (2021). 'Everyone wants to do the model work, not the data work': Data Cascades in High-Stakes AI. ACM CHI Conference on Human Factors in Computing Systems. Source
Seeman, L., & Herzog, K. (2024, August). Accelerated Underwriting: Mortality Slippage Study and Monitoring Best Practices. Product Matters!, Society of Actuaries. Source
Wang, R. Y., & Strong, D. M. (1996). Beyond Accuracy: What Data Quality Means to Data Consumers. Journal of Management Information Systems, 12(4), 5–33. Source