RESEARCH DOSSIER
Insurance Advice Suitability Testing and Recommendation Quality
Abstract. Insurance advice translates a client's exposures, resources, preferences and constraints into a recommendation concerning uncertain future losses and contractual protection. This monograph examines the scientific evaluation of that translation across insurance lines and regulatory systems. It distinguishes legal suitability, substantive recommendation quality, procedural adequacy, comprehension, implementation and realized outcomes; develops their operational measurement through file review, standardized scenarios, mystery shopping and longitudinal observation; and analyzes identification problems created by incomplete information, selection, conflicts of interest and stochastic claims. Comparative documentary research covers the European Union, United Kingdom, international supervisory standards, Hong Kong, and bounded United States and Australian examples. A descriptive reconstruction of a published European mystery-shopping cross-tabulation and transparent synthetic insurance and audit calculations illustrate why process scores and client welfare cannot be treated as interchangeable. The synthesis establishes the importance of a contract-specific, client-specific, time-indexed evidentiary record and a feasible alternative set. It supplies methodological recommendations for supervisory testing without asserting a new theory or extrapolating selected empirical findings to entire markets.
Keywords: insurance advice; suitability; demands and needs; recommendation quality; insurance distribution; market conduct; consumer protection; mystery shopping; conflicts of interest; causal inference; insurance-based investment products
Introduction
Insurance coverage is acquired before the state of the world that establishes the actual value of the contract. The person choosing coverage must compare uncertain losses, contingent contractual rights, current premiums and future financial capacity while facing a distribution institution that typically knows more about the products than the purchaser. Advice is an institutional response to this decision problem. Its quality is a property of the relationship between the recommendation and the decision situation, including those features of the situation that can reasonably be established at the time.
A customer may follow an appropriate recommendation and never make a claim. An inappropriate contract may yield a favorable payout by chance. A compliant sales process may still produce an inferior choice among feasible alternatives, and a relatively brief consultation may adequately resolve a straightforward exposure. These possibilities make insurance advice evaluation a problem of measurement under uncertainty. The object of assessment is the justified decision, with actual outcomes used as additional evidence.
This study asks three questions. Which criteria distinguish recommendation quality from duration, volume and customer satisfaction? How can suitability and alternatives be tested when information is incomplete? How do competence, remuneration and oversight influence documented recommendation fitness? The answers require institutional, contractual, decision-theoretic and empirical analysis in conjunction.
The jurisdictional material concerns selected instruments and supervisory studies rather than an exhaustive legal survey. The primary comparative sources include legal texts and supervisory publications retrieved for this research; empirical material consists of reported observations, an independently reconstructed published cross-tabulation and synthetic examples. No confidential advice files, adviser-level microdata or original field sample were collected. Numerical illustrations are explanatory rather than calibrated models. The study examines general cross-line advice evaluation; specialist annuity, life remuneration and long-term care sales rules receive only the treatment needed to establish boundaries.
Insurance Advice and Recommendations in Institutional Context
The institutional function of advice
Insurance advice is the production and communication of a recommendation concerning the acquisition, modification, retention or termination of insurance coverage in relation to a particular decision maker. Its economic function is to reduce the cost of acquiring and interpreting relevant information and to help align contractual response with exposure. A recommendation can concern a product, insurer, limit, deductible, endorsement, term or combination of these features. Its relevant unit is the available contract as a whole rather than the marketing name.
Insurance distribution includes information provision, product arrangement, advice and other legally defined activities. A price comparison may describe quotations without recommending one. A standardized digital journey may convey or imply a personalized choice; its classification depends on the applicable law and the actual interaction. The analyst must distinguish the customer's perception of a recommendation from its legal classification. The former helps explain reliance; the latter determines which regulatory duties apply.
The Insurance Distribution Directive (European Parliament and Council, 2016, arts. 17 and 20) provides a particularly useful institutional distinction. Distributors must act in the customer's best interests, proposed contracts must be consistent with identified demands and needs, and advice involves a personalized explanation of why a particular product fits them. The level of information is proportionate to product complexity and customer type. The directive supplies a legal starting point for evaluation rather than a quantitative definition of welfare.
A distribution channel determines the range of encounters and alternatives available. A tied intermediary may have a narrower product universe than a broker, and a direct insurer ordinarily evaluates its own offerings. The relevant comparison must therefore identify the actual market search obligation, the disclosed scope of the channel, the customer's ability to obtain alternatives and the precision of the recommendation. A recommendation described as market-wide requires different evidence from one explicitly confined to a defined panel. A narrow panel is an important condition of inference: a strong recommendation within it does not establish market-wide optimality.
Contractual content and the unit of review
A policy's protective value is determined by its insuring clauses, conditions, definitions, exclusions, deductible structure, limits and claims obligations. The nominal sum insured can be misleading when sublimits or valuation conditions materially restrict recovery. An adviser reviewing property coverage must connect catastrophe exposures to the actual insured perils, liability coverage to the trigger and exclusions, and income protection to eligibility, benefit definitions and waiting periods. Investment-linked contracts require additional attention to costs, downside risk, surrender and liquidity.
The recommendation record therefore needs two distinct kinds of information: the client facts used to identify the decision and the legally operative features establishing the product response. A product brochure can support an inquiry, but critical coverage findings require the relevant policy version or authoritative product terms. Illustrations and projected returns must be labeled according to their assumptions; guaranteed benefits, discretionary benefits and scenario-dependent values require separate treatment.
The Insurance Core Principles of the IAIS (2024, ICP 19.8) articulate internationally relevant expectations concerning advice appropriate to disclosed circumstances, supporting records and controls. These expectations address supervision and intermediaries' conduct. Their operational implication is that an institution must be capable of showing the factual basis of a recommendation, not merely retaining proof that the customer signed a form.
The unit of observation consequently has at least three coordinates: the advice encounter, the recommendation and the subsequent contract. A shopper may receive several options; a client may implement only one element; an application may be refused by underwriting. Encounter-level measures of questioning cannot automatically be compared with policy-level claim outcomes. A unique, privacy-preserving identifier and product-version reference are essential to link these units without collapsing them.
Ex ante quality and ex post outcomes
The appropriate reference time is the date when the recommendation was made. Material facts include known current risks, reasonably foreseeable changes, applicable contractual terms and alternatives that were actually available. Information becoming available after that date can support learning, but retrospective results should not be inserted into an assessment as though the adviser knew them earlier.
An ex ante evaluation asks whether the recommendation was supported by the evidence and compatible with relevant constraints. An ex post evaluation asks whether the implemented contract produced the protection or financial performance expected under the circumstances that occurred. Realized claims, lapses, complaints, renewals and cancellation reasons inform outcome monitoring. They are influenced by hazards, behavior, underwriting, claims administration and unrelated changes in the client's circumstances. Their interpretive value therefore depends on how explicitly the causal chain is reconstructed.
The possibility of luck has practical consequences for disciplinary and research decisions. A denial that follows an exclusion disclosed, understood and reasonably accepted under a justified trade-off differs from a denial caused by an exclusion directly contradicting an identified essential need. A favorable claim under a badly structured contract does not erase the exposure that remained uninsured. Evidence about the realized event supplements the original decision record and allows the review to distinguish avoidable mismatch from legitimate uncertainty.
Standards of Suitability and Consumer Outcome Alignment
Demands and needs, suitability and appropriateness
Terminological precision is essential because the same word may denote distinct requirements across instruments. Under Article 20 of the EU distribution directive, the demands-and-needs assessment concerns proposed insurance contracts and the client's information. When advice is provided, the distributor must also give a personalized recommendation with reasons. Article 30 establishes a more elaborate suitability assessment for advised insurance-based investment products (IBIPs): knowledge and experience, financial situation including capacity to bear losses, and investment objectives including risk tolerance. Non-advised IBIP distribution has an appropriateness regime centered on knowledge and experience, subject to its applicable conditions and warnings. Execution-only exceptions are conditional and do not displace the demands-and-needs baseline. EIOPA's studies of IBIPs therefore cannot be treated as direct measurements of the requirements applying to every non-investment insurance sale.
The European Commission's consolidated delegated conduct regulation (2017, arts. 9–12 and 14) elaborates suitability, information reliability, sustainability preferences, cost-benefit analysis where relevant to switching, and the suitability statement. Insufficient information can preclude a personalized recommendation under its applicable regime. Where the product set contains no suitable product, the distributor cannot resolve the problem by calling the least unsuitable product suitable. Article 12 maintains the responsible institution's duties when automated or semi-automated systems participate in the process. These provisions are specific to the regulation's scope.
In the UK, the FCA's ICOBS client-needs rules (2026a, sections 5.2 and 5.3) distinguish needs identification and advised sales. Its advice provisions and guidance provide for reasonable care and attention to relevant limitations, excesses and unmet needs within the applicable scope. The FCA's fit assessment should be evaluated using the actual applicable rules and guidance for the product, rather than using the EU IBIP checklist as a generic substitute.
A recommendation-quality study must thus define the legal test separately from its research benchmark. A legal finding asks whether an operative rule was satisfied. A research finding may also examine missed alternatives, quality of explanations, sensitivity to uncertainty and expected client welfare. These extra criteria may reveal room for improvement without establishing a statutory violation.
Feasibility and meaningful alternatives
Consider a client with initial liquid resources W, premium P, potential state-dependent loss X(s) and contractual payment I(s). A simplified end-of-period resource expression is W − P − X(s) + I(s). It omits tax, financing, timing and other balance-sheet items; those enter the analysis when material. For an eligible set of contracts, expected utility can express the ranking of financial positions under stated probabilities and risk preferences. In actual advice the necessary variables are often incompletely known, and the evaluator may only be able to compare plausible rankings under alternative assumptions.
Constraints precede ranking. An infeasible premium, underwriting ineligibility, a legally unavailable contract or a material uncovered requirement may exclude an option. The feasible set must also reflect the ability to implement protection in time, including evidence of actual availability and conditions. A quote made after closing a placement window is a poor counterfactual for the original decision. The comparison set may include retention of risk, a different deductible, higher limits, an endorsement or a decision to seek further advice. A comparator must be attainable for the client on the terms considered.
Recommendation quality involves both the selection and explanation of trade-offs among feasible alternatives. An analyst may identify dominance under a limited set of assumptions: if one option has weakly lower costs and at least as favorable protection in every relevant state, with a strict improvement in at least one, it is financially superior within those assumptions. Actual product comparisons often resist dominance because costs, protection, service and timing differ. An evaluator should articulate the dimensions in which a recommendation sacrifices one advantage for another.
Synthetic coverage comparison
Take two disjoint annual loss states: fire with probability 0.02 and loss of USD 20,000; flood with probability 0.005 and loss of USD 50,000; otherwise zero modeled loss. Policy A costs USD 400, covers fire with a USD 1,000 deductible and excludes flood. Policy B costs USD 650, covers both modeled perils with a USD 2,000 deductible. Assume full payment above the applicable deductible, sufficient limits, no other exclusions, no claims costs or tax, and a client with USD 5,000 liquid reserves and premium budget of USD 700.
Expected retained loss under A is 0.02 × 1,000 + 0.005 × 50,000 = USD 270. Total expected premium and retained loss is USD 670. Under B the respective amounts are 0.02 × 2,000 + 0.005 × 2,000 = USD 50 and USD 700. Expected indemnity is USD 380 for A and USD 600 for B. B costs USD 30 more on this expected-cost measure while sharply reducing exposure to a USD 50,000 retained flood loss beyond the assumed reserve. Both premiums satisfy the initial affordability ceiling.
The expected-cost difference B minus A, holding fire probability constant and writing flood probability as q, is 270 − 48,000q. It becomes zero at q = 0.005625, or 0.5625%. The example therefore separates expected-cost ranking from protection against an unaffordable state. If the premium ceiling falls to USD 500, B becomes unavailable on the stated terms. The appropriate response is to examine feasible endorsements, different terms or other risk-management options, rather than report a recommendation outside the budget as implementable.
These inputs are deliberately synthetic. They establish neither flood frequencies nor insurer pricing, and the expected-cost calculation does not incorporate risk aversion, claims delay, insolvency, policy interpretation or competing needs. Its function is to demonstrate why a suitability review must retain both residual-risk and affordability information and clarify which assumptions justify a recommendation.
Incomplete information and client-directed decisions
Material uncertainty is a decision condition that should be explicitly recorded. Unknown items can include dependents' resources, property replacement values, existing insurance, collateral obligations, investment horizon and foreseeable affordability changes. Missing information is not a false answer, and inferred attributes require provenance. A reviewer should distinguish a fact supplied by the client, a verified record, an estimate based on an explicit method, and an unresolved variable.
If an essential variable remains unknown, the range of reasonable decisions may be wider than a single recommendation. Additional questions, conditional recommendations, coverage comparisons or escalation may resolve the issue. In legal regimes requiring complete enough information before personalized advice, the institution must observe the corresponding restriction. The IBIP suitability provisions give a clear example within their defined scope.
Clients may choose differently from an adviser's documented suggestion. Evaluation must record the relevant facts, rationale, alternatives, explanation, client's decision and any legally applicable restriction. A client override is an event in the process and does not by itself establish that the original recommendation was inappropriate. Equally, consent or waiver cannot establish that an otherwise unsuitable recommendation was suitable.
Metrics and Evidentiary Benchmarks
The dimensions of measurement
An operational evaluation begins by specifying the claim its result is intended to support. A file-completeness statistic describes records; a case-quality score describes coded recommendations; a claims outcome describes a later event. The evidence required for each is different. These measures can be integrated for supervision while retaining their denominators and interpretations.
For substantive fit, assess critical coverage requirements, feasibility, exclusions, affordability, duplicate insurance and relevant alternatives. For procedural quality, assess the accuracy and completeness of elicited information, the relevance of further questions and the traceability of the rationale. For communication, test whether the client received and understood the material differences likely to affect the decision. For implementation, establish whether the issued contract matches the recommendation and whether unresolved conditions were addressed. For downstream outcomes, assess actual loss response, lapse, complaints, financial effects and changed needs, accounting for selection and timing.
Some defects are material irrespective of a high aggregate score. A clearly established requirement remaining wholly uninsured, unaffordable premiums, or a prohibited recommendation under the governing regime requires separate identification. An additive score may help summarize many observations, but compensating a critical defect with excellent administrative presentation is a measurement error. Critical criteria should be defined in advance, with an adjudication route for genuinely uncertain cases.
A checklist also needs a category for inapplicability. A sustainability preference criterion specified for an advised IBIP cannot be transferred indiscriminately to a basic property policy. An assessment designed for a complex personal recommendation may overstate failings in a simple non-advised journey. Applicability is a feature of the legal and empirical assessment, not missing data to be scored zero.
Denominators, missing data and agreement
Let a study select 200 advice encounters, of which 150 can be classified after applying its evidence rules. Suppose 120 of these meet the predefined suitability benchmark, 30 do not, and 50 remain unclassified. The observed classifiable proportion is 120/150 = 80%. Across all selected encounters, the identification interval without assumptions about the unclassified cases is 120/200 to 170/200, or 60% to 85%. Neither extreme need be plausible; the interval reports what the observed data alone establish. Reasons for nonclassification matter. A closed office, incomplete product record, no recommendation and refusal to supply critical information are distinct observational events.
If the legal benchmark requires documentation that is missing, the missing record can itself establish a documentation defect. The underlying substantive suitability assessment may remain indeterminate. Conflating the two misstates the basis of the conclusion. Report documentation failure, unresolved fit and affirmative mismatch separately.
Independent reviewers also require calibration. In a synthetic set of 100 classifiable cases, reviewers agree on pass for 40, agree on fail for 45, and disagree in 15 cases (10 pass by reviewer A only and 5 by reviewer B only). Observed agreement is 85%. Their pass proportions are 0.50 and 0.45; expected chance agreement based on marginal proportions is 0.50, producing Cohen's kappa of (0.85 − 0.50)/(1 − 0.50) = 0.70. This is an illustration, not a universal acceptance threshold. Report category-specific agreement and the content of disagreements; high agreement on routine cases can conceal disagreements on rare critical defects.
The coding manual should specify observable evidence, permissible inference, priority rules where sources conflict, case-level reasons, reviewer blinding and adjudication. The intended inference determines the severity of a classification error. A false pass in a supervisory screening system can conceal harm; a false fail in a disciplinary decision can create unjustified liability. A screening indicator has a different evidentiary threshold from a legal determination.
Validity, comparator design and documentation
A high-performing metric must capture the object it claims to measure. Content validity requires the criteria to cover significant risks and contractual features. Construct validity requires separating irrelevant process proxies from meaningful fit. Criterion validation examines agreement with appropriate independently assessed cases or other defensible benchmarks. Predictive evidence can test associations with subsequent outcomes while recognizing their causal limitations.
A fair comparator requires matching on product availability, customer eligibility, decision timing, exposure and disclosed preferences. Retrospective comparison with products admitted to the market later is invalid. Comparisons should account for changes to terms and exclusions, including individual underwriting conditions. When the reviewer cannot observe the entire market, the result should specify the searched universe.
Evidence is strongest when it supports a reconstruction of the recommendation at the time. A complete record contains material inputs, relevant verification, product documentation, alternative assessment, reasons for the preferred choice, unresolved limitations, communication and the final client decision. Digital systems should retain the relevant versions of decision rules and product data as well as the visible result. Data retention and access must be reconciled with privacy, purpose limitation and the applicable rules of the jurisdiction.
Outcome-monitoring expectations in the FCA's Consumer Duty monitoring provisions (2026b) illustrate a supervisory reason for retaining multiple indicators and investigating persistent poor outcomes. They do not imply that every poor outcome establishes poor advice. Monitoring should provide a route back to the underlying cases, identify repeated patterns and distinguish the product's inherent risk from avoidable distribution failures.
Mystery Shopping and Field Evaluation
Design of a standardized encounter
Mystery shopping observes a distribution process through a shopper acting under a specified scenario. A well-defined scenario states the product need, budget, existing coverage, relevant knowledge, vulnerability conditions when appropriate and the facts the shopper should disclose. The script should distinguish information volunteered from information supplied only in response to a question. Excessive scripting can create an artificial interaction; insufficient standardization undermines comparability.
Suppose the research question concerns whether advisers recognize an essential coverage exclusion. Identical baseline profiles may be assigned to comparable distribution channels, with one critical exposure varied according to a predeclared design. The resulting outcomes can include whether the exposure was elicited, whether its contractual relevance was recognized, whether products with differing terms were offered and whether the recommendation rationale explained the trade-off. If the exposure is varied, random assignment supports an interpretation about the effect of the scenario manipulation under the design, not a causal estimate of actual long-run financial harm.
The shopper needs training in accurate reporting and the preservation of documents lawfully obtained during the encounter. Product quotations are time-sensitive. Record timestamps, relevant market and product conditions, refusals and unsuccessful visits. Avoid exposing individual staff identities in public reporting unless the research has an adequate legal and ethical basis. Predefine whether recording is lawful, which forms of consent apply, how data are secured, and how simulated engagements are terminated without creating unintended transactions.
Mystery shopping estimates observed conduct under a specific set of presentations. It does not directly observe an institution's entire book, long-term customer relationships, or actual underwriting after an unsubmitted application. Repeated visits, case variation and complementarity with actual file review strengthen interpretation.
The European coordinated exercise
EIOPA's coordinated mystery-shopping report (2025) documents 568 visits across eight participating Member States, of which 454 were valid. The investigation concerned sales discussions of insurance-based investment products. It reported a mean meeting duration of 42 minutes and found weak or limited association between features of the process and recommendation consistency. In its reported assessment, 84% of recommendations in the relevant tabulation met at least one criterion. Meeting at least one criterion is a low threshold relative to meeting every applicable critical criterion and cannot be interpreted as an all-criteria suitability pass.
The report also states that 36% received the key information document during the discussion. That measure concerns a particular stage; it does not establish by itself whether a document was later supplied when legally required before contract conclusion. The study is an exercise in comparative observation of distribution processes, not a case-by-case legal compliance determination.
The distinction between sample denominators is especially important. Visits made, valid visits, detailed product discussions and cases included in a particular cross-tabulation are different populations. An observed percentage from one should not automatically be applied to another. The report's combined ratings compress multiple observations into ordered categories. Their interpretation therefore depends on the constituent criteria and scoring approach.
Independent descriptive reconstruction
Figure 5 in the published European sales study displays a cross-tabulation of process and consistency scores. Reading its displayed cells with process score y and recommendation-consistency score x, both coded zero through five, yields the following reconstruction. The table represents the printed cell counts rather than raw customer-level data.
| Process score | x=0 | x=1 | x=2 | x=3 | x=4 | x=5 | Total |
|---|---|---|---|---|---|---|---|
| 0 | 3 | 4 | 0 | 7 | 2 | 1 | 17 |
| 1 | 9 | 8 | 0 | 26 | 3 | 37 | 83 |
| 2 | 7 | 5 | 2 | 15 | 1 | 25 | 55 |
| 3 | 22 | 19 | 3 | 47 | 1 | 55 | 147 |
| 4 | 9 | 5 | 0 | 14 | 0 | 13 | 41 |
| 5 | 11 | 6 | 0 | 11 | 0 | 6 | 34 |
| Total | 61 | 47 | 5 | 120 | 7 | 137 | 377 |
The displayed figure contains 377 observations. Coding both ordered scales as equally spaced solely for a descriptive calculation gives weighted means of approximately 2.997 for x and 2.568 for y. The weighted Pearson correlation is approximately −0.136. Its interpretation is limited by the ordinal character of the scores, the missing cases relative to the 454 valid visits, and possible clustering and contextual differences. It does not estimate a causal effect or a population parameter. The sign should not be interpreted as evidence that better processes reduce recommendation quality.
A threshold illustration clarifies the mismatch. Among the 75 observations with process scores four or five, 31 have consistency scores zero or one (41.3%). Among 100 observations with process scores zero or one, 43 have consistency scores four or five (43.0%). The categories are illustrative and have no independently validated legal or scientific pass threshold. Their value is to show the possibility of substantial discordance in this observed table. Neither threshold result is derived from the full sample of 454.
The reconstruction can be reproduced by summing all cells n(x,y), weighting x, y, x², y² and xy by the counts, subtracting the product of the means from the mean cross-product, and dividing by the product of the two resulting standard deviations. This is original descriptive analysis of published aggregate data, not a claim of original data collection, statistical discovery or a new model. No significance test or country ranking is justified from the aggregate counts alone.
Complementarity with register and file research
The EIOPA third IDD application report (2026) reports supervisory developments and continuing issues in the implementation of distribution rules, based on its described collection of evidence. It also draws on the preceding mystery-shopping exercise. Counting these publications as independent replications of the same observed association would inflate evidentiary confidence. Their complementary value lies in the different scope of the institutional review and the earlier field observations.
A different type of evidence comes from ASIC's retail life advice review (2014). The targeted historical review examined 202 advice files, reporting 127 passes and 75 failures. The latter count corresponds to about 37.1% of the reviewed files. A targeted surveillance cohort is not a probability sample of all Australian advisers or all clients. The study also belongs to its historical legal and market context. Its relevance here is the feasibility of substantive file review and the presence of material advice problems within the selected cases, rather than a current national prevalence estimate.
File review and mystery shopping have different sources of bias. Completed files provide access to the implemented contract and professional rationale, but exclude encounters that never became an application. Mystery shopping can observe initial recommendations without relying on completed transactions, while documentation and subsequent implementation may be absent. Linking results at the institutional or product level can guide examination, provided differences in cohort, timing and selection are respected.
Incentives, Competence and Product Complexity
Incentive compatibility and product choice
An adviser allocates effort among acquiring product knowledge, eliciting customer information, comparing alternatives, explaining restrictions and completing transactions. Remuneration, targets, time constraints and product complexity influence the costs and returns of these activities. When adviser compensation varies among feasible alternatives, recommendation patterns may shift for reasons independent of customer welfare.
This mechanism can be studied through product-level remuneration comparisons, contract changes, randomized disclosure or fee conditions where lawful, and quasi-experimental policy changes. A descriptive association between high commissions and sales of certain products does not identify the effect of commissions by itself. Product demand, perceived quality, underwriting acceptance and distribution economics may also affect compensation and sales. A comparison should document these potential confounders and specify the variation that identifies the intended effect.
Anagol, Cole and Sarkar's Indian life insurance experiment (2017) supplies direct empirical evidence from a specific life insurance market. The working-paper account (2015) adds detail on recommendations favoring forms of insurance that the study classified as dominated under its assumptions, limitations of agent knowledge and product substitution around commission disclosure. These observations establish the possibility of advice distortions under the investigated conditions. They do not establish that every commission-bearing contract is inferior, that all market participants act alike, or that the same estimates apply to other insurance lines. The working paper and published record concern the same research and are treated as a single empirical contribution.
The incentives question must be separated from competence. A poorly informed adviser may recommend an inferior option without deliberately preferring remuneration. A knowledgeable adviser with a commercial conflict may deliberately restrict a comparison. The observable recommendation can look similar in both cases, while the targeted supervisory response differs. Compensation-related screening should therefore be paired with product-knowledge tests, case review and independent assessment of alternative availability.
Article 17 of the EU insurance distribution rules addresses conduct in the customer's best interest and restrictions on remuneration arrangements and sales targets that conflict with that duty. A scientific comparison can evaluate incentive patterns more broadly, but its findings should be tied to the legal scope before describing an observed practice as prohibited.
Expertise, complexity and communication
Expertise is multidimensional: product knowledge, legal understanding, exposure identification, ability to compare competing provisions, quantitative competence and communication skill. Credentials and training hours are useful administrative information. Independent case exercises and periodically reviewed real files provide additional evidence of applied capability.
Complexity can arise from contract duration, exclusions, multiperil interactions, surrender costs, embedded investments, claims triggers and contractual obligations. Longer documentation does not necessarily mean greater complexity for a particular client; a single poorly understood restriction may dominate a decision. Complexity evaluation should focus on characteristics material to the specific choice, including the number and consequences of contingent terms that must be understood.
A good recommendation explanation should identify the relevant client needs, the product features that address them, important residual exposure and the significant reasons for choosing among alternatives. Comprehension can be examined with neutral scenario-based prompts, such as asking which loss would remain uninsured under a particular exclusion. Self-reported confidence alone cannot establish understanding. Such prompts should be accessible and should distinguish misunderstanding caused by the explanation from lack of prior domain knowledge.
Financial advice can itself be expensive to acquire. Reuter and Schoar's financial advice literature review (2024) identifies demand- and supply-side constraints and heterogeneity in the benefits and risks of advice. Its general financial setting provides a reason to analyze the access counterfactual rather than presume that additional deliberation always improves welfare. The relevant insurance comparison may be between professional advice, a simple transparent product, informed self-direction, delayed purchase and remaining uninsured. The retrieved abstract supports this general interpretation; it does not supply an insurance-specific effect estimate.
Digital advice and supervisory responsibility
Automated or assisted systems may elicit customer details, classify needs, compare contracts and generate recommendations. Their evaluation requires many of the same substantive tests used for human advice, plus traceability of decision rules, versions, data transformation and override conditions. A system may reproduce product biases consistently, yielding low variance and systematically poor recommendations. Reproducibility is useful evidence but is distinct from correctness.
Testing should include ordinary cases, boundary cases, missing information, inconsistent responses, changes in eligibility, high-consequence exclusions and clients whose needs are outside the supported product range. For each, the evaluator should establish expected behavior based on applicable rules and product facts, then compare the actual system response. A system's ability to halt, ask for clarification or escalate is part of its suitability evaluation.
Under the EU automation responsibility provision (European Commission, 2017, art. 12), the use of automated or semi-automated systems does not diminish the responsible distributor's obligations under the applicable IBIP regime. In empirical research, version-specific logging and change controls enable repeatable comparisons and distinguish improvements from shifts in the case mix. Digital channels should also be examined for how choices are framed, which alternatives are displayed, and whether relevant restrictions are communicated before commitment.
Cross-Product and Cross-Jurisdiction Comparison
The comparative legal method
A comparative study should map functionally similar requirements without treating unlike legal instruments as identical. The source hierarchy matters: enacted or delegated law, binding rulebook provisions, supervisory guidance, professional codes, model instruments and international supervisory standards have different forms of authority. Dates, scope, exclusions and national implementation should be explicit. The present comparison is documentary and selective, with no claim to give exhaustive jurisdiction-specific legal opinions.
| Instrument or jurisdiction | Relevant evaluation object | Boundary of interpretation |
|---|---|---|
| European Union, IDD | Demands and needs; personalized reasons when advised; additional IBIP tests | Directive and national implementation; delegated IBIP rules have defined scope |
| United Kingdom, ICOBS and PRIN | Client needs, appropriate advised sales and outcome monitoring | Product and customer scope of operative FCA rules and guidance |
| IAIS, ICP 19 | Supervision of appropriate conduct and advice controls | International supervisory benchmark, implemented through jurisdictions |
| Hong Kong broker code | Client circumstances, product comparison and rational grounds | Broker-specific conduct code and applicable local law |
| United States, NAIC model 275 | Best-interest and suitability architecture for annuity recommendations | State model and local adoption, product-specific |
The Hong Kong Insurance Authority's licensed broker conduct code (2019, General Principle 6) provides a useful comparison because it addresses the collection of adequate information about the client, comparison across an adequate range of products and insurers, the reasonable basis for advice and handling of incomplete information and client decisions. Evaluation under the code requires attention to the broker's institutional role and applicable requirements. Its published provisions cannot be extended automatically to agents, insurers or other jurisdictions.
In the United States, the NAIC's annuity best-interest overview describes the 2020 revisions to Model Regulation 275. Its stated approach includes acting in the consumer's best interest and avoiding placement of the producer's financial interest ahead of the consumer's interest, supported by supervision. The model is not a single federal rule applicable to every insurance sale. State laws and adoption details require explicit examination for a legal determination. The model is included for comparative structure, not for a complete analysis of annuity suitability.
The IAIS conduct supervision standard enables comparison at the level of supervisory objectives. National implementation can differ even where the high-level objective is similar. A cross-country score should consequently publish both the common research benchmark and each local legal coding rule. Legal compliance rates created from non-equivalent duties should not be combined into a single rank without a defensible harmonization method.
Product-specific tests within the common problem
For property insurance, relevant indicators include peril coverage, valuation, limits, replacement costs, deductibles and the insured's resources to manage retained loss. Liability insurance introduces claims triggers, limits, aggregation, exclusions, retroactive dates where applicable and the character of the client's legal exposure. Health and disability insurance may involve provider access, benefit definitions, waiting periods, eligibility and continuity. Changes in employment or health may affect feasible alternatives, subject to the governing legal and contractual rules.
For long-duration and investment-related contracts, affordability over time, protection needs, liquidity and surrender consequences receive additional weight. Investment risk and risk-bearing capacity enter the analysis for relevant products. Existing protection and replacement decisions require careful comparison because termination may destroy valuable contractual rights. The field of available new contracts is not equivalent to the rights under an existing contract.
Mechanisms such as low-probability catastrophic loss, duplicate cover and exclusions are shared across many lines. Their measurement depends on the contract and customer. A general quality test should preserve common functions such as need identification, feasible options and rationale while specifying line-specific critical conditions. This allows benchmarking without manufacturing false equivalence.
Cross-market comparison and measurement invariance
The same nominal score can acquire different meanings when markets have different default coverage, distribution obligations, insurance literacy, compulsory insurance, product availability and social safety nets. Translation may alter a question's interpretation. A shopper script that represents an ordinary client in one jurisdiction may be uncommon elsewhere.
A rigorous cross-market protocol begins with a common decision problem and context-specific facts. It maps comparable outcome criteria, documents differences, pilots the scripts and coding instructions, and examines whether similar evidence yields similar assessments. If a criterion depends on local law, publish the local legal result and the common research result independently. Avoid attributing differences to the national regulatory regime without a design that addresses confounding.
The same caution applies across time. Changes in product supply or economic conditions can transform an enquiry even when the script remains unchanged. For repeated monitoring, preserve a stable subset of scenarios, track difficult or unavailable products, and update additional scenarios when material risks change. The objective is continuity of inference rather than mechanical repetition.
Research Synthesis, Identification and Supervisory Design
Identifying advice effects
A causal question should first specify intervention, population, outcome and comparison. For example, what is the effect of a structured alternative-comparison prompt on the probability that a recommendation addresses an explicitly disclosed critical exposure among a defined set of advice encounters? This differs from asking whether advisers who voluntarily use prompts produce better contract outcomes. The second question is vulnerable to selection by competence, employer, products and clientele.
Randomized assignment of acceptable testing scenarios or workflow interventions may identify the effect of the assigned condition within the study. Analysis should distinguish assignment from actual adherence and account for interference where advisers share information. In observational studies, confounding can arise from case complexity, income, vulnerability, adviser experience and product range. Matching and adjustment compare cases on observed characteristics; their interpretation still requires assumptions about unmeasured confounding, common support and comparable outcome definitions.
For a before-and-after regulatory change, the relevant comparative design must examine trends, concurrent changes, anticipation, case-mix shifts and the timing of implementation. A discontinuity-based design needs evidence that assignment near the threshold is plausibly comparable and that actors did not manipulate the score or running variable in a way that invalidates the design. These are established empirical design principles applied to insurance advice, rather than a new identification method.
Downstream outcomes introduce additional selection. Only clients who purchase a policy enter an insurer's policy records; only those experiencing particular events become eligible to claim. Comparing claim denial proportions among claimants can therefore yield a very different conclusion from comparing expected protection over all eligible advice encounters. In a study of claim outcomes, specify the initial population, the selection process into contracting and claims, and the event that defines success or harm.
Sampling, precision and audit interpretation
A probability sample from a defined archive can estimate the prevalence of specified recorded defects, subject to nonresponse, missing records and classification validity. A risk-based supervisory sample can efficiently detect problems but cannot directly estimate prevalence without an appropriate selection design and weighting. Reporting should identify the sampled population and the selection mechanism.
As a synthetic planning example, for a simple random sample of independent observations, an anticipated proportion of 0.5, a normal approximation with multiplier 1.96 and absolute half-width 0.05 yield 1.96² × 0.25/0.05² = 384.16, rounded up to 385. If encounters are clustered with average cluster size five and assumed intraclass correlation 0.1, the elementary equal-size design-effect approximation is 1 + (5 − 1) × 0.1 = 1.4, suggesting about 539 completed observations using the rounded base. This ignores finite population correction, unequal cluster sizes, stratification, incomplete cases and small numbers of clusters. Actual study planning must examine these conditions and the primary endpoint; the calculation is not a universal sample-size recommendation.
Statistical precision is only one dimension of evidence. A large sample with invalid product matching or an ambiguous benchmark can establish a misleading estimate with narrow intervals. A smaller research sample with verified contractual facts and transparent decisions may provide useful evidence about mechanisms. Supervisory institutions benefit from using both forms of investigation for their respective purposes.
Institutional control and corrective action
Monitoring can combine standardized scenario tests, independently coded files, product-specific comparisons and longitudinal customer information. A practical sequence begins by identifying material harm and exposure, defining evidence requirements, choosing sampling methods and measuring the baseline. Review findings should be translated into corrective action appropriate to the demonstrated problem and authority of the institution.
A pattern of missed exclusions suggests a review of product information, the questioning process and training. A pattern concentrated in high-remuneration products can justify investigation of compensation and product-selection rules. Failures in digital edge cases can motivate changes to the rules or escalation pathway. Repeated evidence of unaffordable recommendations may require investigation of the underlying affordability assessment and incentives. The response should follow from evidence about the defect and its mechanism rather than the generic label of poor quality.
Corrective programs should include implementation verification. Outcome measures after the intervention need stable definitions or a transparent account of changes. Improved documentation may reflect more accurate records without immediate improvement in substantive recommendations. Improved case-quality scores may result from a more favorable sample rather than the intervention. Independent follow-up and appropriate comparison groups support stronger interpretation.
Integration of evidence and limits
The primary legal sources establish distinct obligations concerning needs, recommendation rationale, suitability and supervision. The institutional studies show that process measurements and substantive assessments can diverge, and selected field and file research demonstrates the relevance of remuneration, knowledge and oversight. The synthetic calculations show why explicit constraints, alternative sets, denominators and assessor agreement matter even without a new empirical dataset.
The strongest conclusion concerns the structure of inference. A recommendation is a time-indexed relationship among a particular client's circumstances, a defined contract, an attainable set of alternatives and the rationale for selection. Evidence of that relationship cannot be replaced by its administrative representations. Process data are essential for understanding how a recommendation arose; contractual data determine the substantive feasibility of protection; downstream outcomes illuminate the consequences. Their links must be tested and their limitations retained.
The evidence does not establish a universal prevalence of unsuitable advice, a single international legal standard, or a causal estimate of advice's welfare impact. The European reconstruction uses ordered scores and published aggregates rather than underlying observations. The Australian historical surveillance is selected, and the Indian experiment concerns a specific market and period. The general financial advice review is used at abstract level. These limitations constrain inference, rather than removing the substantive comparative and methodological findings.
Conclusion
Insurance advice suitability testing evaluates the justified fit between contractual protection and a client's decision circumstances at the time of recommendation. Its scientific foundations lie in the combination of insurance contract analysis, decision analysis under uncertainty, comparative conduct regulation, measurement theory and empirical identification.
Demands and needs, legal suitability, substantive recommendation quality, explanation, implementation and realized outcomes require distinguishable indicators. Their relationship is observable only through an adequate evidentiary record. The pertinent benchmark must identify the governing rule or research criterion and preserve material conditions including feasibility and loss-bearing capacity. Comparisons must use alternatives available at the relevant time.
The evidence examined in this study demonstrates the limitations of process-only measures. In the EIOPA cross-tabulation, substantial discordance exists between ordered process and recommendation-consistency categories. The descriptive reconstruction cannot establish causal mechanisms, but it establishes why the dimensions should be separately observed and why the denominator of each analysis matters. Historical file review and experimental research illuminate distinct mechanisms and support complementary methods.
A defensible supervisory test begins with a defined decision problem, identifies critical criteria, records applicable legal scope and uses appropriately designed observations. It addresses incomplete data, distinguishes uncertainty from affirmative defect, calibrates human coding, validates contract-specific facts and evaluates downstream consequences within a stated causal framework. Such a test improves the intelligibility of supervisory evidence and gives corrective action a demonstrable connection to the failures it seeks to prevent.
The original contribution is a comparative analytical synthesis, a reproducible reconstruction of reported aggregate observations, and transparent synthetic applications of insurance and audit calculations. No new named theory, doctrine or claim of priority is advanced. Future empirical research can apply the designs described here to verified representative samples, versioned policy terms and ethically linked outcome records. The value of that research will depend on the precision of the decision problem, measurement and identification rather than the mere volume of recorded sales activity.
References
European Parliament and Council (2016). Directive (EU) 2016/97 of 20 January 2016 on insurance distribution (recast). Official Journal L 26, 19–59. Consolidated text, October 9, 2024. Source
European Commission (2017). Commission Delegated Regulation (EU) 2017/2359 of 21 September 2017 supplementing Directive (EU) 2016/97 with regard to information requirements and conduct of business rules applicable to the distribution of insurance-based investment products. Official Journal L 341, 8–18. Consolidated August 2, 2022. Source
Financial Conduct Authority (2026a). FCA Handbook, Insurance Conduct of Business Sourcebook (ICOBS), Chapter 5: Identifying Client Needs and Advising. Live Handbook, retrieved November 19, 2026. Source
Financial Conduct Authority (2026b). FCA Handbook, PRIN 2A.9: Monitoring of Consumer Outcomes. Live Handbook, retrieved November 19, 2026. Source
International Association of Insurance Supervisors (2024). Insurance Core Principles and Common Framework for the Supervision of Internationally Active Insurance Groups. December 2024. Source
Insurance Authority, Hong Kong (2019). Code of Conduct for Licensed Insurance Brokers. September 2019. Source
European Insurance and Occupational Pensions Authority (2025). Uncovering the IBIP Sales Process: Results from EIOPA’s First Coordinated Mystery Shopping Exercise on Insurance-Based Investment Products. EIOPA-BoS-25-164, June 17, 2025. Source
European Insurance and Occupational Pensions Authority (2026). Third Report on the Application of the Insurance Distribution Directive. EIOPA-BoS-26-077, March 30, 2026. Source
Australian Securities and Investments Commission (2014). Review of Retail Life Insurance Advice. Report 413, October 9, 2014. Source
Anagol, S., Cole, S., & Sarkar, S. (2017). Understanding the Advice of Commissions-Motivated Agents: Evidence from the Indian Life Insurance Market. Review of Economics and Statistics, 99(1), 1–15. DOI: 10.1162/rest_a_00625. Bibliographic record and abstract. Source
Anagol, S., Cole, S., & Sarkar, S. (2015). Understanding the Advice of Commissions-Motivated Agents: Evidence from the Indian Life Insurance Market. Harvard Business School Working Paper 12-055, working-paper version. Source
Reuter, J., & Schoar, A. (2024). Demand-Side and Supply-Side Constraints in the Market for Financial Advice. Annual Review of Financial Economics, 16, 391–411. DOI: 10.1146/annurev-financial-110921-012809. Source
National Association of Insurance Commissioners (n.d.). Annuity Suitability and Best Interest Standard. Topic overview, retrieved November 19, 2026. Source