Insurance by BogdanovaThe Encyclopedia of Insurance and Risk Science

RESEARCH DOSSIER

Underwriting Human Judgment and Structured Expert Elicitation

Abstract. This monograph investigates the contribution of underwriting practitioners to insurance decisions under sparse, changing, or contested exposure evidence. It compares direct insurance research with structured expert elicitation, calibration, aggregation, actuarial model integration, and supervisory governance. The analysis distinguishes factual assessments from risk appetite, probabilistic forecasts from contract decisions, predictive performance from causal policy evaluation, and shared information from independent evidence. Reproducible hypothetical calculations demonstrate probability conditioning, opinion pooling, scoring, and Bayesian updating. The synthesis situates these methods across insurance lines and international institutional settings while preserving limitations arising from selection, reporting delays, restricted source access, and absent proprietary insurer observations. The contribution is a comparative analytical synthesis of established methods, not a new theory or a claim of measured operational effectiveness.

Keywords: Underwriting; human judgment; structured expert elicitation; expert calibration; opinion pooling; proper scoring rules; actuarial models; model overrides; insurance governance; risk uncertainty.

Introduction

Underwriting human judgment is the evidential and interpretive contribution of practitioners to decisions concerning the acceptance, modification, referral, or rejection of proposed insurance risks. Structured expert elicitation is a family of procedures for obtaining, documenting, and combining judgments about specified uncertain quantities. Its scientific purpose is to transform information dispersed among specialists into explicit, inspectable assessments, especially where exposure records are sparse, unstable, or contested.

The underwriting decision is an institutional act. It converts a provisional description of an exposure into a contract with a specified insured interest, attachment, limit, premium, exclusions, period, and responsibility for residual loss. Judgment enters at several distinct points: determining what the evidence establishes, interpreting a model's domain of application, evaluating unknown parameters, and choosing among authorized contract alternatives. The last of these involves organizational objectives as well as beliefs about loss. Eliciting a probability requires the prior separation of factual belief from the appetite for bearing that probability.

This distinction gives the dossier its independent subject. The mathematics of pricing and the general design of classification systems belong to related disciplines. Here the object is the acquisition and validation of information that an underwriter contributes when ordinary statistical estimates leave a consequential gap. Three questions organize the inquiry: when does practitioner judgment contribute incremental information; through which procedures can disagreement and uncertainty be recorded; and how can later observations test whether the judgments were useful?

The research is an analytical, comparative synthesis of published insurance studies, elicitation research, professional standards, and supervisory material. It includes reproducible hypothetical calculations, not newly collected insurer data. The source record distinguishes accessible article text, abstracts, institutional summaries, and official provisions. Comparisons across countries concern documented institutional mechanisms rather than estimates of national performance. The study does not establish a new theory or claim a novel discovery.

Judgment in Specialty Risks

A mature class with many comparable insured periods permits substantial reliance on observed frequencies and severities. Specialty risks frequently reduce comparability: a new industrial process, unconventional construction, interdependent digital infrastructure, unusually high attachment, or a changing legal environment may alter the distribution that supported earlier experience. The underwriting question becomes whether a specific observation changes a relevant conditional distribution and whether the organization can recognize that change before binding.

Consider a proposed policy for a chemical plant. Loss records describe past operations; engineering records describe current process conditions; maintenance logs reveal a change in equipment; a broker explains the proposed insured values. An experienced specialist can contribute by identifying a mechanism connecting these records to a fire or interruption scenario. The scientific value lies in the mechanism and its evidential support. Years in a particular job are an imperfect substitute for establishing relevance to the current plant, occupancy, process, and policy wording.

A useful taxonomy distinguishes process knowledge, exposure knowledge, and interpretive knowledge. Process knowledge explains how an insured event may arise; exposure knowledge identifies the people, assets, locations, or activities subject to that event; interpretive knowledge determines how a contract converts physical or financial loss into an insured claim. A person may possess extensive competence in one component and little competence in another. Elicitation therefore allocates questions at the level of the uncertain variable rather than asking any single specialist to supply every number.

The direct insurance literature demonstrates why the distinction matters. Wright, Bolger, and Rowe (2002), in their study of risk judgments, compared underwriters with students and reported relatively small expert advantages on several measures alongside shared distortions in estimated magnitudes. Ranking the risks was easier than estimating their absolute probabilities. This evidence concerns the particular life-threatening events and respondents studied; it does not establish an overall accuracy rate for insurance professionals.

Kunreuther and colleagues (1993), in Insurer ambiguity and market failure, reported risk and ambiguity aversion in survey-based premium-setting judgments of actuaries, underwriters, and reinsurers. An ambiguity-related premium difference may embody a capital cost, an aversion to uncertain estimates, strategic negotiation, or several mechanisms together. A price alone cannot identify the underlying subjective distribution because price also contains expenses, capacity, margins, competition, and contract terms.

The resulting scientific implication is that a file should record an estimate of the relevant event and the subsequent willingness to insure it as different quantities. A specialist who reasonably increases the uncertainty surrounding a fire frequency may leave its mean unchanged; a decision authority can nevertheless choose additional protections or decline because the range crosses an exposure constraint. Only an explicit separation permits a reviewer to tell which process generated the decision.

Expertise, Cognitive Bias, and Information

Kahneman and Klein (2009) examine the conditions for intuitive expertise: the environment must contain learnable regularities and practitioners must have adequate opportunities to learn them. Applied to insurance, these conditions are unevenly distributed. Frequent small claims can give relatively rapid feedback. A rare catastrophic mechanism may be recognized through engineering knowledge even when no similar insured loss has occurred in the practitioner’s career. Conversely, apparent success in a long-tailed class may reflect temporary absence of reported losses rather than accurate anticipation of ultimate liabilities.

The informational contribution of judgment is best evaluated relative to a baseline. Let X be the information used by an existing predictive system, Z be additional predecision evidence interpreted by an expert, and Y be the defined outcome. A specialist adds predictive information when the conditional distribution of Y given X and Z differs usefully from that given X and when the observed improvement persists under a suitable validation design. A change in forecast that merely restates the same input in more authoritative language is not evidence of an incremental contribution.

Anchoring can occur when the existing model score or prior renewal rate is displayed before independent investigation. Availability can arise when a recent large claim dominates a judgment about frequency. Confirmation can arise when an underwriter seeks support for an attractive placement while failing to investigate mechanisms that would make acceptance inappropriate. Motivated reasoning may emerge when production targets affect the interpretation of incomplete information. These are plausible mechanisms for errors in the decision process. Their presence and magnitude in a particular insurer require evidence.

Lo, Patel, and Calder (2015) examined judgment in P&C companies through a prediction survey with 120 participants in a single organization. Across four forecast questions they found no evidence for the hypothesized differences between underwriters and analysts. Their study provides a useful corrective to occupational stereotypes. It does not prove equivalence of professional roles, and the sample and tasks cannot establish universal conclusions about real binding decisions.

There are at least three competing explanations for a disagreement between an actuary and an underwriter: they may have unequal information; they may interpret shared evidence differently; or they may optimize different objectives. The first can be addressed by controlled information sharing, the second by independent forecasts and assumption analysis, and the third by separating forecast assessment from decision authority. An organizational procedure that simply demands consensus cannot distinguish these explanations.

The temporal order of evidence matters. An initial judgment should state whether it was made before the specialist saw the model output, after seeing the output, or after seeing further information. This allows an analyst to identify changes attributable to the presentation of a baseline rather than newly acquired evidence. The resulting record supports a more meaningful evaluation than a final unversioned score.

Defining the Elicitation Target

An elicitation question has an observation unit, a horizon, a measurable outcome, a conditioning information set, and resolution rules. For example, the probability of at least one insured business interruption claim exceeding $100,000 over a particular twelve-month policy period is a different target from the probability of an interruption of any size at the premises. The contract terms determine the threshold and the claims record determines the measurement. The date at which the case becomes sufficiently developed to score must also be specified.

Several distinct uncertainty objects recur. A prospective event probability refers to a future realization, a frequency parameter refers to an expected count per exposure unit, a severity parameter refers to the conditional loss distribution, and a model parameter specifies a feature of a representation of these outcomes. Experts may be more comfortable reasoning about frequencies per thousand comparable periods than abstract probabilities. The facilitator should translate between those formats only when the underlying probability model and exposure unit justify it.

O’Hagan (2019) describes expert knowledge elicitation as a way of representing knowledge through probability distributions, with protocols intended to reduce cognitive bias. This provides an established methodological basis for separating the quantity under inquiry from the confidence attached to its possible values. If experts instead supply an opinion about a deterministic contract interpretation or missing recorded fact, the record should distinguish disputed evidence from future stochastic variation.

A simple synthetic example illustrates the importance of the target. Suppose a specialist estimates a 10% probability of a process shutdown and a 30% conditional probability that a shutdown generates an insured claim above a threshold. Under the stated conditioning, the joint probability of the two events is 0.10 times 0.30, or 3%. This calculation does not establish the total chance of exceeding the threshold if other mechanisms can also generate such claims. An event tree should enumerate those additional mechanisms and model overlaps before summing probabilities.

A probability distribution for loss conditional on a specified policy requires a coherent mapping from physical consequences to insured payments. Deductibles, waiting periods, sublimits, exclusions, event aggregation terms, and claims development can all change the observed target. In a renewal, revised wording may make historical feedback unsuitable for direct scoring. The elicitation form must preserve the exact version of the question, wording, data cutoff, and assumptions, so that later reviewers compare like with like.

Questions should separate quantities that can be resolved from assumptions used for exploratory stress tests. A scenario describing a simultaneous grid outage and communications failure may be valuable for examining portfolio exposure even when no credible event probability can be elicited. Labeling it a conditional scenario protects against converting an illustrative mechanism into a falsely precise probability.

Structured Elicitation Protocols

Formal protocols organize the interaction between subject specialists, facilitators, and analysts. A facilitator specifies the quantity and controls ambiguity. Specialists identify and interpret relevant evidence. An analyst translates the judgments into coherent values and distributions, checking that the mathematical representation preserves the information originally supplied. An accountable decision authority determines their use within policy limits.

The IDEA method organizes an inquiry through investigate, discuss, estimate, and aggregate. Hemming and colleagues (2018) implemented it in a Great Barrier Reef case study, involving 76 participants and forecasts for 14 events. The reported benefits of pooling and discussion arise from that particular natural resource management design. The study supports testing the procedure in an insurance setting; it does not measure an underwriting profit effect.

An insurance adaptation starts with a decision brief, common evidence pack, and elicitation target. The facilitator first invites experts to inspect evidence individually. Before open discussion, each supplies an initial judgment with bounds and a rationale. Participants then discuss conflicting assumptions, additional observations, missing records, and alternative explanations. They revise privately after hearing the evidence and retain their first and second judgments. The analyst checks consistency, aggregates if appropriate, and supplies both the combined distribution and the disagreements to the decision authority.

The Sheffield Elicitation Framework developed by O’Hagan and Oakley provides another established approach to elicitation. A facilitated group can reason toward a shared distribution after individual judgments are expressed. Such a distribution is a collective representation of uncertainty under an explicitly agreed interpretation. It requires particular care where junior participants might defer to a senior signatory or commercially powerful participant. Confidential initial estimates and an independent facilitator reduce the risk that agreement merely reflects authority.

The classical model, reviewed by Colson and Cooke (2018) in Expert elicitation, adds calibration questions with known or subsequently observed answers and uses performance on those questions to combine expert distributions. Its calibration design is particularly attractive where many targets are not rapidly observable. The difficulty is constructing questions sufficiently close to the target domain without allowing participants to retrieve the answers from current files or memorize the evaluation set.

The three approaches serve related but distinct purposes. IDEA makes the progression of individual information and discussion visible; facilitated behavioral aggregation can resolve interpretive differences about a shared quantity; performance weighting supplies a formal aggregation mechanism contingent on informative calibration data. The procedure should be determined before the desired decision is known. Choosing the method retrospectively because it gives a preferred premium or acceptance result defeats its role as an evidence discipline.

For ordinary small-value proposals, a carefully specified individual assessment with an explicit reason code, controlled escalation, and periodic validation may be proportionate. A lengthy panel procedure carries real delay and expense. High-limit, novel, disputed, or portfolio-sensitive exposures justify greater deliberation because errors may consume capacity or affect many contracts. The appropriate degree of rigor depends on the materiality and reversibility of the decision as well as data scarcity.

Consensus, Disagreement, and Aggregation

Expert disagreement is information about the process of estimation. It may concern the meaning of the question, the source of a factual statement, a causal mechanism, the applicability of historical data, or the location and dispersion of a distribution. These types require different responses. A disputed equipment inspection calls for verification; a disagreement about a physical failure pathway may require an engineering review; persistent uncertainty about the predictive distribution calls for sensitivity analysis or a documented distributional combination.

A linear opinion pool combines distributions using nonnegative weights summing to one. If F_i(x) is the cumulative distribution from expert i and w_i is that expert’s weight, the pooled distribution is F(x) = sum_i w_i F_i(x). The mixture is a valid cumulative distribution and preserves variation between component distributions. It generally differs from fitting a distribution through averaged quantiles. In particular, the mean of component medians need not equal the median of the combined distribution.

A hypothetical pair of experts provides a useful counterexample. Expert A assigns equal probability to loss amounts of $0 and $100,000. Expert B assigns equal probability to $100,000 and $200,000. With equal weights, the combined distribution assigns probabilities 0.25, 0.50, and 0.25 to $0, $100,000, and $200,000, respectively. Its expected loss is $100,000. Averaging corresponding endpoint values would instead suggest a distribution with equal mass at $50,000 and $150,000. Both constructions share the same mean but represent very different tail probabilities and possible outcomes.

The choice of aggregation should therefore depend on the object. When experts describe alternatives for one uncertain parameter, a mixture expresses uncertainty over those alternatives. When their estimates concern distinct sites or times, separate distributions may be needed. When participants all derive estimates from a common catastrophe model or vendor, their disagreement understates shared model error; treating the estimates as independent observational samples would create false precision.

Performance weights require special attention to validation. Colson and Cooke (2017), in cross-validation of the classical model, compare weighting methods using 33 studies. Their abstract reports that the performance-weighted pool had lower out-of-sample statistical accuracy than equal weighting, although it performed better on a combined accuracy and information index in 26 studies. This distinction matters because narrow estimates may improve an informativeness metric while reducing empirical coverage. The preferred rule should align with the loss criterion and be evaluated outside its fitting sample.

An equal-weight pool is an essential benchmark because it is transparent and has relatively few fitted parameters. A performance-weighted pool may be justified by sufficient domain-relevant calibration questions, predeclared scoring, reasonable weight stability, and independent validation. With very few resolved cases, aggressive weights can make a single unusual result dominate the estimate. Weight floors or shrinkage toward equal weights are familiar statistical safeguards; any particular choice must be justified against validation, not promoted as an independently established new method.

Consensus should be recorded with assumptions and dissent. Suppose a specialist identifies a plausible unmodeled route to correlated loss. Even if the remainder of a panel assigns it low probability, it may affect a tail exposure boundary. The decision authority can separately analyze the scenario while retaining the predictive pool. The majority view is evidence of agreement; it is not a guarantee that a minority’s causal objection has been addressed.

Calibration, Scoring, and Retrospective Assessment

Calibration concerns the relationship between probabilities assigned before outcomes and frequencies observed later. If a set of comparable events all receives probabilities around 0.20, an empirically calibrated forecast should resolve positively roughly one fifth of the time over repeated predictions. Small groups can deviate substantially through ordinary random variation. Calibration assessment must therefore accompany sample size, uncertainty, time window, outcome definition, and any dependence between cases.

Resolution is a separate property: the ability to distinguish settings where an event is more likely from settings where it is less likely. A constant base-rate forecast may be well calibrated in aggregate and yet unhelpful for distinguishing individual proposals. An underwriting assessment needs both credible absolute probabilities and information relevant to differences in exposure. Discrimination alone, such as ranking ability, does not establish the accuracy of probabilities.

Gneiting and Raftery (2007) analyze strictly proper scoring rules. For binary observations, the Brier loss is (p − y)^2, where p is a forecast probability and y is zero or one. For a true Bernoulli probability q, expected loss is q(1 − p)^2 + (1 − q)p^2 = (p − q)^2 + q(1 − q). The expectation is uniquely minimized when p equals q. The derivation assumes a stated binary target and does not depend on the underwriting application.

For a synthetic demonstration consider four exposures with outcomes 0, 1, 0, and 1. A baseline issues forecasts 0.2, 0.3, 0.4, and 0.6. An expert-assisted procedure issues 0.1, 0.5, 0.3, and 0.7. The baseline mean Brier loss is (0.04 + 0.49 + 0.16 + 0.16)/4 = 0.2125. The assisted loss is (0.01 + 0.25 + 0.09 + 0.09)/4 = 0.11. The difference of 0.1025 favors the assisted method on these four invented observations only. The calculation demonstrates how comparison works; it establishes no estimate for operational underwriting.

The log score in loss form is −log(p) when the event occurs and −log(1 − p) otherwise. It penalizes confident error more strongly and becomes unbounded for a realized event assigned zero probability. Implementations may require a predeclared boundary treatment for numerical stability. A lower Brier loss and a lower log loss can rank forecasts differently in a finite sample. Reporting both, with a stated primary endpoint, clarifies which features of forecasts drive any apparent improvement.

For continuous losses, intervals, quantiles, and full distributions can be assessed using coverage, quantile loss, or distributional scores. A central 90% interval that covers 90% of outcomes in aggregate may still be too narrow in one line and too wide in another. A useful evaluation stratifies by class, horizon, data regime, and model validity domain; it avoids making numerous post hoc comparisons without acknowledging their uncertainty.

A calibration question must be resolved under a consistent rule. Claims reported but not settled introduce a distinction between reported and ultimate loss; a policy canceled early changes exposure time; an underwriting year may contain varying reporting lags. Scoring an immature claim as a known ultimate loss can reward overconfidence simply because development has not occurred. Freeze the valuation date or use a suitable observation model, carry unresolved items separately, and update the validation record as the outcomes mature.

Interaction with Actuarial Models

Actuarial models and expert assessments can contribute to distinct layers of an inference. A statistical estimate provides an evidence-conditioned baseline. A domain specialist supplies additional observations or identifies a failure of the assumptions underlying that baseline. Their combination is meaningful when each contribution has traceable provenance and a clearly defined statistical relationship to the other.

For illustration, assume event probability theta has a beta prior with parameters 2 and 18. Its mean is 2/20 = 0.10. Suppose a comparable sample of ten independent Bernoulli exposure periods produces two events. Under the binomial likelihood and exchangeability assumptions, the posterior is Beta(4,26), with mean 4/30 = 0.1333 repeating. The example illustrates an established Bayesian update. The prior parameters are hypothetical and the ten periods are assumed comparable solely for the calculation.

In practice a specialist’s prior may already rely on the same claims history used in the likelihood. Updating that prior with those same observations treats the evidence twice. The analyst must record what information informed the elicited distribution and use either independent new information, a joint model of the overlapping evidence, or a transparent sensitivity analysis. This discipline becomes especially important when an underwriter has access to dashboards calculated from overlapping sources.

For model overrides, retain the unmodified estimate, the proposed change, the precise reason, the evidence supporting it, and the terms on which the recommendation is accepted. Overrides may reflect corrected data, a recognized regime change, omitted risk factors, contractual mismatch, or an authorized risk-appetite decision. The latter changes the action rather than the factual probability and should be recorded as such. A single generic override code prevents a reviewer from separating model repair from commercial discretion.

The US Actuarial Standards Board’s ASOP 12 addresses professional actuarial services relating to risk classification. Its ASOP 56 addresses actuarial modeling. Their applicability is determined by the professional work and scope of each standard. These sources support treating judgment-informed assumptions and classification changes as part of accountable actuarial work where applicable; they are not global statutes imposing the same professional obligations on every underwriter.

Feedback should include both the initial and adjusted forecast. Otherwise future model retraining may absorb human modifications without revealing whether the modifications added information. A counterfactual comparison can often be made for the same bound policy's observable outcomes, provided both probabilities refer to the same policy and information date. It cannot identify how changing acceptance would affect the mix of policies. Forecast validation and policy evaluation therefore require separate designs.

Decisions must also reflect dependence. A local risk may have a favorable mean loss estimate while adding materially to a shared event accumulation. If multiple specialists rely on the same assumptions about a cloud service, earthquake zone, or supplier network, an independence-based combination conceals systemic exposure. Portfolio aggregation needs scenarios and dependence assessment alongside the individual file’s expert-derived estimate.

Comparative Applications Across Insurance Lines

In property and engineering insurance, relevant expert inputs include the verified condition of protective systems, failure mechanisms, restoration time, and uncertainty in values at risk. The event may be physical, while the insured payment depends on business interruption definitions and the contractual period of indemnity. A property engineer can estimate physical restoration time; a claims specialist can identify the portion that is likely to qualify as covered interruption. Separate elicitation improves clarity at this interface.

In life insurance, assessment concerns mortality over a specified period given evidence permitted by applicable law. The NAIC accelerated underwriting overview explains that insufficient evidence in streamlined processes can lead to traditional assessment and highlights governance and discrimination concerns associated with external data. A human referral can obtain missing relevant information, challenge a data discrepancy, or identify cases outside the model’s scope. Unstructured reversal alone, however, does not show that classification has improved.

For casualty insurance, long reporting and settlement lags constrain feedback. The target may be occurrence of an event, claim notification, ultimate cost above an attachment, or a measured liability at a fixed maturity. Expert beliefs about legal developments and inflation should be maintained as explicit scenarios or distributions over the relevant assumptions. Retrospective scoring at an early valuation date must not be described as a test of ultimate severity.

For cyber insurance, some exposures change rapidly as technical configurations, dependencies, and remediation change. Questions should specify the relevant version and date of the environment, coverage and triggering event. An expert can assess the existence or relevance of a dependency without offering a reliably estimated annual attack rate. The degree of confidence in those two quantities must be recorded separately. Shared providers create a reason to examine portfolio accumulation in addition to individual expected loss.

In marine and aviation specialty risks, operational mechanisms, maintenance regimes, routes and aggregations may differ widely within broad classes. An elicited forecast needs the operating conditions actually covered and the unit of exposure. In agricultural risks, weather, agronomy, geographic correlation, and index design require distinction between physical loss and contractual payout. In parametric insurance, a probability of payout is not necessarily a probability of loss; eliciting both with an explicit relationship can help characterize basis risk.

These comparisons identify mechanisms shared across lines: nonexchangeability, changed coverage, feedback delay, model domain limitations, and correlated exposure. They do not rank national or line-specific professional accuracy. For a specialty portfolio, it is often more meaningful to validate assessments by the type of unknown—event frequency, recovery duration, severity, exposure amount, or conditional dependence—than to pool all specialists into a single undifferentiated success measure.

Governance, Fairness, and Institutional Comparison

The authority to supply evidence differs from the authority to bind an insurance contract. A specialist may identify a changed hazard distribution while a portfolio manager determines permissible line size and an authorized underwriter decides the commercial terms. A reproducible record distinguishes assessment competence, approval responsibility, and the legal status of the resulting contract.

The IAIS Insurance Core Principles and ComFrame provide an international supervisory framework. ICP 8 addresses risk management responsibilities and deviations, and ICP 16.7 connects the underwriting policy to exposure limits, emerging risks and reinsurance. ComFrame adds group-specific provisions, including data procedures under CF 16.7.a. These are supervisory principles and standards whose implementation depends on jurisdictions and applicable entities; they do not prescribe one expert elicitation algorithm.

In the Lloyd’s market, Underwriting Profitability links managing-agent oversight with business planning, appetite, and controls. An independent review of a material expert override can connect the file assessment to those organizational commitments. A committee record should identify whether a contested decision was resolved by additional evidence or by exercise of authorized discretion.

In the European Union, Article 48 of Solvency II assigns the actuarial function a role in expressing an opinion on the overall underwriting policy. This requirement concerns the level of the policy and its actuarial governance, so it should not be interpreted as a mandate for that function to personally approve each individual assessment. In Australia, APRA’s GPG 240 Insurance Risk discusses insurance risk in relation to underwriting, claims, product design and pricing. A prudential practice guide and an applicable prudential standard have different normative roles. An international comparison must preserve those distinctions.

Fairness requires examining both the input and the decision. A seemingly neutral proxy can encode a characteristic whose use is restricted in the relevant jurisdiction. Human discretion can correct a model’s error or introduce a new unjustified disparity. An accurate prediction is not by itself sufficient to establish legal or ethical permissibility. The scientific file requires provenance and relevance; the conduct review requires separate examination of applicable classifications, legal restrictions, notice and review rights, and treatment of similarly situated applicants.

Where data concern health, finances or private business operations, retain only what is necessary for the defined decision and evaluation. Separate the record available to analysts from personally identifying records where the evaluation permits it. Record authority for handling data, access restrictions, retention periods and correction mechanisms under the applicable local rules. The comparative scope of this dossier does not substitute for jurisdiction-specific legal review.

A discrepancy across applicant groups should trigger an analysis of data quality, case mix, permissible explanatory variables, referral rates and outcomes. Observed parity or disparity in approval rates is neither proof of discriminatory causation nor proof of its absence. Selection mechanisms and contractual terms can affect the outcome measures. Governance needs evidence about how human decisions and model outputs interact at successive steps.

Research Design and Reproducibility

A prospective insurer study should distinguish incremental predictive quality from causal effects of decision processes. The unit of analysis may be a submitted proposal, an offered contract, a bound policy, a policy-year, or a portfolio segment. The analysis population and target must be stated before the observations are collected. A proposal dataset includes risks that were declined or withdrawn, while a claim outcome dataset ordinarily contains only policies that became insured.

The simplest paired forecast study records the baseline and the expert-assisted estimate for the same predefined outcome before a decision. On policies with observable mature outcomes, it can compare losses using the same scoring rule. However, this estimates performance conditional on the historical binding process. Inferences about declined risks require additional defensible outcome evidence, a justified identification design, or explicit partial-identification bounds. Imputing zero claims to rejected proposals creates an artificial advantage for any rule that declines more cases.

A separate experiment could assign qualified specialists to an ordinary or structured elicitation process for comparable, de-identified files, with random assignment or a counterbalanced crossover design. The primary endpoint might be Brier loss for later resolved probabilities, with secondary criteria including time, missing-evidence identification, completeness of rationales, and disagreement. The case sampling design should include routine and specialty exposures so that a change in task composition does not masquerade as an intervention effect.

To examine reliability, independently duplicate a random subset of cases, blind reviewers to one another, preserve case order, and predefine category or probability agreement measures. Simple percent agreement is easy to interpret but depends on the prevalence of outcome categories. Cohen’s kappa adjusts categorical agreement for expected chance agreement under its marginal-frequency calculation; for numeric ratings, intraclass correlation or absolute differences may be suitable depending on the measurement design. Reliability and accuracy answer distinct questions: two people may agree on a systematically erroneous probability.

Statistical comparison of prediction errors should account for clustering by expert, broker, event, location, underwriting year, or insured group as relevant. The uncertainty of the paired loss difference can be evaluated with a suitable cluster-aware resampling design. Temporal holdouts test whether the procedure persists across changes in business conditions; random data splits may leak information between related losses. Predefine how cancellations, incomplete development, nonresponse and ambiguous resolution are handled.

A reproducible scientific record contains the frozen target questionnaire, evidence version, protocol version, participant eligibility and conflicts, individual initial and revised estimates, fitted distributions, calibration questions and outcomes, aggregation code or formulas, scoring code, resolution dates, sample selection and exclusions, and the decision record. Software versions and hashes support computational replication; they do not remove the need to describe the scientific design.

The hypothetical calculations in this monograph are fully specified in prose and can be reproduced without proprietary data. The literature search was targeted rather than a registered systematic review. Direct insurance studies were prioritized; studies from other fields were used for methodological comparison with explicitly restricted transfer. Bibliographic records and available source text were examined for the stated claims, but access restrictions prevent an assertion of full-text examination of every article. No empirical insurer dataset, individual underwriting file, or confidential information was available.

Comparative Synthesis

The relationship between human judgment and statistical estimation is determined by the informational structure of the task. The strongest case for expert contribution arises when the specialist has verifiable information relevant to the target which the baseline lacks, or when the applicability of the baseline assumptions has changed. Expertise by title, tenure, or professional prestige is not sufficient evidence of that contribution. A well-defined target and transparent evidence record are the prerequisites for evaluation.

The direct insurance studies reviewed here reveal limits to simple cultural expectations about professional judgment. Experts can rank risks without estimating their magnitudes well; underwriters and analysts need not exhibit systematically different forecasts on controlled tasks; an ambiguity-related premium can reflect financial objectives and preferences as well as beliefs. These findings support a research design that separates knowledge, uncertainty, and authorized action. They do not establish that every underwriter shares the same biases or that formal procedures invariably improve insurance outcomes.

An elicitation protocol can make expert information inspectable by preserving initial assessments, clarifying different interpretations, recording revision, and documenting a combination. The scientific benefit of the record is independent of whether a panel reaches agreement. In certain decisions, the presence of unresolved disagreement supplies a useful reason to obtain further evidence, limit capacity, or apply scenario analysis under established authority rules.

The classical model provides an example of performance-based aggregation with empirical calibration, but the choice of scoring criteria and validation sample matters. Empirical support from other domains establishes the feasibility of formal elicitation and the importance of validation. It does not measure its effect on underwriting loss ratios, consumer welfare or solvency. Those are distinct endpoints that require insurer-level studies and appropriate causal identification.

The actuarial integration problem is fundamentally a provenance problem. A model output and a specialist judgment may be derived from common observations. Combining them without identifying shared information can create unjustified precision. Explicit baseline preservation, conditional forecasts, independent evidence checks, and controlled override processes allow the contribution of each information source to be studied.

Supervisory comparisons reinforce the importance of connecting the file-level assessment to institutional accountability. Different frameworks establish responsibilities for actuarial work, underwriting policies, risk appetite, and governance; none of the sources reviewed establishes a universal binding elicitation format. A proportionate process can satisfy a scientific aim while remaining adaptable to contractual and jurisdictional differences.

The author’s contribution is the integrated analysis of these interfaces: the separation of probability from underwriting appetite, the distinction between case-level forecast validation and acceptance-policy evaluation, the importance of common information provenance for hybrid estimates, and the retention of disagreement as a potentially decision-relevant part of the evidence. These are analytical consequences of established research and insurance mechanisms, not claims to a new doctrine or demonstrated empirical effects.

Conclusion

Underwriting human judgment is scientifically valuable to the extent that it converts relevant practitioner knowledge into explicit, testable information concerning a defined insurance exposure. The degree of value is conditional on the nature of the evidence, the baseline’s information set, the predictability of the environment, and the opportunities for feedback.

Structured expert elicitation allows insurers and researchers to formulate that contribution in a reproducible way. Target specification, independent initial assessment, documented discussion, coherent probabilistic representation, transparent aggregation, and prospective scoring are mutually reinforcing elements. They enable disagreement to be interpreted and model interaction to be audited rather than obscured in an undifferentiated underwriting decision.

Calibration and proper scoring can test judgments against mature, comparable outcomes. Research must preserve proposal selection, reporting delay, coverage changes, and dependence. A paired comparison on bound policies tests predictive accuracy within the selected population; it cannot by itself establish the consequences of changing which policies are accepted.

Governance completes the connection between knowledge and action. Expert uncertainty informs the decision, while applicable rules determine permissible evidence, classification, authority, risk appetite, and contractual commitments. An assessment contributes to institutional reliability when it is auditable, revisable, and disciplined by feedback. The available evidence justifies such processes and their empirical testing; their incremental effects across individual insurance lines and jurisdictions remain research questions.

References

  • Actuarial Standards Board. (2005). Actuarial Standard of Practice No. 12: Risk Classification (for All Practice Areas). Full bibliographic source.

  • Actuarial Standards Board. (2019). Actuarial Standard of Practice No. 56: Modeling. Effective October 1, 2020. Full bibliographic source.

  • Australian Prudential Regulation Authority. (2024). Prudential Practice Guide GPG 240: Insurance Risk. Full bibliographic source.

  • Colson, A. R., & Cooke, R. M. (2017). Cross validation for the classical model of structured expert judgment. Reliability Engineering & System Safety, 163, 109–120. DOI: 10.1016/j.ress.2017.02.003. Full bibliographic source.

  • Colson, A. R., & Cooke, R. M. (2018). Expert elicitation: Using the classical model to validate experts’ judgments. Review of Environmental Economics and Policy, 12(1), 113–132. DOI: 10.1093/reep/rex022. Full bibliographic source.

  • European Parliament & Council. (2009). Directive 2009/138/EC (Solvency II), Article 48, Actuarial function. EIOPA Single Rulebook. Full bibliographic source.

  • Gneiting, T., & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477), 359–378. DOI: 10.1198/016214506000001437. Full bibliographic source.

  • Hemming, V., Walshe, T. V., Hanea, A. M., Fidler, F., & Burgman, M. A. (2018). Eliciting improved quantitative judgements using the IDEA protocol: A case study in natural resource management. PLOS ONE, 13(6), e0198468. DOI: 10.1371/journal.pone.0198468. Full bibliographic source.

  • International Association of Insurance Supervisors. (2024). Insurance Core Principles and ComFrame, ICP 8 and ICP 16. Online tool. Full bibliographic source.

  • Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515–526. DOI: 10.1037/a0016755. Full bibliographic source.

  • Kunreuther, H., Hogarth, R. M., & Meszaros, J. (1993). Insurer ambiguity and market failure. Journal of Risk and Uncertainty, 7(1), 71–87. Full bibliographic source.

  • Lloyd’s. (n.d.). Principle 1: Underwriting Profitability. Full bibliographic source.

  • Lo, J., Patel, N., & Calder, A. (2015). Judgmental topics in P&C companies: Findings from a prediction survey. Variance, 9(1), 101–113. DOI: 10.66573/001c.120853. Full bibliographic source.

  • National Association of Insurance Commissioners. (2026). Accelerated Underwriting. Insurance Topics. Full bibliographic source.

  • O’Hagan, A. (2019). Expert knowledge elicitation: Subjective but scientific. The American Statistician, 73(sup1), 69–81. DOI: 10.1080/00031305.2018.1518265. Full bibliographic source.

  • O’Hagan, A., & Oakley, J. E. (n.d.). Sheffield Elicitation Framework (SHELF). University of Sheffield. Full bibliographic source.

  • Wright, G., Bolger, F., & Rowe, G. (2002). An empirical test of the relative validity of expert and lay judgments of risk. Risk Analysis, 22(6), 1107–1122. DOI: 10.1111/1539-6924.00276. Full bibliographic source.