There is a version of the pitch for AI in compliance that goes something like this: upload your regulatory documents, the system reads them, tells you what changed, tells you what to do, and you spend your compliance budget on higher-value work. That pitch is not dishonest. It is also not the full picture, and compliance teams that buy into it without understanding the boundaries will either be disappointed when the system fails to deliver the complete solution, or will over-rely on automated outputs in contexts where human judgment is the only defensible mechanism.
I want to be specific about where the technology we have built at Kalipso genuinely helps, and where it does not. The honest answer, from someone who has been building and testing this system for two years, is that the boundary matters more than either the optimistic pitch or the sceptical dismissal.
What Natural Language Processing Does Well in Regulatory Documents
Regulatory documents are, from a natural language processing standpoint, good candidates for structured text extraction. They follow predictable patterns: FCA policy statements have an executive summary, chapters addressing specific issues, and annexes with instrument text. EBA technical standards have recitals, articles, and annexes. The structure is consistent enough that extraction of the operative provisions, the actual rule changes that create obligations, can be systematised.
Named entity recognition within regulatory texts performs well on several classes of entity that matter for compliance purposes: regulated firm types, financial product categories, regulatory framework references, effective dates, and cross-references to other instruments. A system trained on regulatory document structure can identify that a particular paragraph of a policy statement amends a specific COBS rule, identify the effective date of that amendment, and classify the amendment by the regulatory framework it modifies. This is useful. It replaces a reading task that would otherwise fall on a compliance officer or external counsel.
The FCA published over 400 regulatory items in 2025. Running a classification pass over that volume automatically, and presenting the compliance team with a prioritised view of items classified as material to their firm type, is a real reduction in the reading burden. We measured this internally: for a firm with a mid-complexity regulatory footprint (investment management, SMCR, AML, Consumer Duty obligations), the pre-filtered alert set from Kalipso represents roughly 15 to 20 percent of the total FCA publication volume in a given month. That 80 percent reduction in triage volume is a meaningful productivity change.
Materiality Scoring: The Three Signals
Beyond classification, materiality scoring is the layer that determines how urgent a given alert is for a compliance team. We use three signals to score materiality. The first is change type: a new obligation imposed by a final instrument scores higher than a consultation paper proposing a future change. A final rule scores higher than guidance. This is not a controversial claim, but implementing it requires reliably distinguishing between publication types, which requires reading the document rather than relying on the label the regulator applies to it. Regulators do not always label publication types consistently.
The second signal is rule proximity: how directly does the extracted change reference a rule or obligation applicable to the firm's specific permissions? A change to COBS 2 is more material to an investment firm than a change to MCOB, which governs mortgage and home finance activities. Mapping permissions to rule references requires maintaining a current model of which rule chapters apply to which firm types, and that model needs to be updated as the FCA's regulatory architecture evolves.
The third signal is firm-type applicability: does the publication's scope explicitly include or exclude the firm's regulatory permission set? This signal is the noisiest of the three. Many publications are ambiguous on applicability. A supervisory statement addressed to "investment firms" covers a statutory category that includes firms with very different permission profiles. Getting the applicability signal right requires not just reading the publication but understanding the regulatory context in which firm-type categories are defined.
Where Automation Stops: The Judgment Layer
The three signals together produce a ranked list of alerts with a plain-English summary of the change and an initial assessment of whether the firm needs to review its existing compliance documentation. That output is useful. It is not a compliance decision.
The compliance decision requires something that automation cannot currently provide: the judgment of whether a specific rule, in its specific context, applies to the firm's specific product structure, client population, and operational model. A change to how Consumer Duty price and value assessments must be documented is materially relevant to a firm with a complex legacy product book and very different in practical significance to a firm with a simple, recently launched product range. That distinction requires a compliance officer who knows the firm, not a classification system that knows the regulation.
We have sometimes been asked whether we plan to extend Kalipso to generate remediation steps, not just alerts. The answer is: partially, for well-defined cases. For a change that modifies a specific record-keeping requirement, we can generate a template checklist of what the new requirement implies for a standard firm. That checklist is a starting point. Whether it applies to the specific firm, and whether the firm's existing processes already satisfy the requirement, requires a compliance officer to assess. Generating a checklist and generating a compliance decision are different things, and the distinction is not a technical limitation we expect to engineer away; it is a fundamental property of judgment.
Volume Is the Argument for Automation, Not Full Replacement
The clearest argument for automation in regulatory change management is volume, not completeness. 400+ FCA items per year, plus PRA, ESMA, EBA, and ICO, means that the alternative to a classification system is either a very large compliance team that reads everything, or a small team that reads selectively and accepts that it will miss things. Neither is satisfactory at the compliance standards that UK financial regulators apply. Automation handles the reading volume; compliance officers handle the judgment. The division of labour is not a compromise; it is the correct allocation of what each does well.
We are not saying that the classification and scoring system we have built is perfect. We track false positives (alerts that turn out to be irrelevant to the firm's permissions) and false negatives (relevant publications that scored below the alert threshold) in production, and we use that feedback to improve the classification models. The error rate is not zero. The comparison is not against zero errors; it is against the alternative, which is a compliance team manually triaging 400 publications a year without classification support. That baseline has its own error rate, and it is typically higher than what the automated system produces once it has been calibrated to a specific firm's profile.
What This Means for Compliance Teams Evaluating These Tools
When evaluating any automated regulatory monitoring system, including Kalipso, the right questions are: what does it reliably do, what does it reliably not do, and who is responsible for the gap? A system that reads 400 publications a month and presents 60 to 80 pre-classified alerts is valuable. A system that claims to replace the compliance judgment on whether a specific alert requires a policy update is making a claim that the technology does not support. The difference matters for how the tool is used and for who is accountable for the compliance decisions that flow from it.
In our view, the accountability never moves from the compliance team to the tool. The tool changes the workflow; it does not change the regulatory obligation. That is the right frame for using any of these systems, including ours.
