AI Hallucination Has a Price Tag: Product Liability, Operational Risk, and the Governance Gap in the Era of Generative AI
Generative AI is no longer experimental. It is embedded in products and workflows across every sector of the Romanian and wider EU economy — from compliance platforms and contract-review engines to medical triage software, customer-service chatbots, industrial maintenance systems, and credit-scoring tools. The outputs these systems produce are fluent, authoritative in tone, and structurally plausible. They are also, with a frequency that is now empirically documented, wrong.
The liability consequences of that wrongness have never been more concrete. Directive (EU) 2024/2853, the revised EU Product Liability Directive, explicitly brings software and AI-driven outputs within the scope of strict liability for the first time. Its evidence-shifting provisions make it considerably easier for claimants to establish causation. And the withdrawal of the proposed AI Liability Directive (AILD) means that the PLD is now the sole EU-level strict liability instrument applicable to artificial intelligence — a fact whose implications most businesses have not yet absorbed.
We are already advising Romanian businesses, technology integrators, and professional-service firms on how to reconcile these new obligations with the operational resilience requirements of DORA (Regulation (EU) 2022/2554), with national professional-duty standards, and with the insurance frameworks that are supposed to cover them when things go wrong.
What this article covers:
What Are AI Hallucinations? Why Every Business Should Care
An AI hallucination occurs when a generative model produces output that is fluent, authoritative in tone, and structurally plausible, yet factually wrong. The term originates in the technical literature on large language models (LLMs), but its consequences are commercial, regulatory, and legal.
In a legal workflow, a hallucination might mean a fabricated case citation, an invented statutory provision, or a non-existent regulatory deadline. In a medical context, it might mean a fictitious clinical study, a misquoted dosage guideline, or a fabricated drug interaction. In financial services, it might mean an invented prudential threshold, a deformed reporting timeline, or a non-existent regulatory exemption. In industrial applications, it might mean a fabricated safety parameter, a misquoted tolerance standard, or an incorrect maintenance interval. In customer service, it might mean a non-existent refund policy, a fabricated warranty term, or an invented discount.
The mechanism is the same in every case: the model generates a statistically plausible sequence of tokens, not by retrieving verified information, but by predicting what text should come next based on patterns learned during training. The output reads convincingly enough to pass a cursory review, which is precisely what makes it dangerous.
The problem is not theoretical — and it is not confined to any single industry
Courts in the United States have already sanctioned lawyers for submitting AI-generated briefs containing entirely fictitious case law (Mata v. Avianca, S.D.N.Y. 2023; Park v. Kim, 2d Cir. 2024). In British Columbia, a tribunal held Air Canada liable for a policy invented by its customer-service chatbot (Moffatt v. Air Canada, 2024 BCCRT 149), rejecting the airline’s remarkable defence that the chatbot was “a separate legal entity, responsible for its own actions.” In the EU, regulatory attention is intensifying as generative AI tools move from experimental use into production workflows across regulated industries.
Two features of AI hallucination make the risk particularly acute:
For any entity deploying these tools — whether a law firm, a bank, a hospital, a manufacturer, or a retailer — the question is no longer whether hallucinations occur, but how often, and what happens when one reaches a court, a regulator, a patient, or a client.
The Revised EU Product Liability Directive (2024/2853): What Changed for AI-Enabled Products
Directive (EU) 2024/2853 represents the most significant overhaul of EU product liability law in nearly four decades. Its relevance extends far beyond legal technology: it applies to every product with an AI component, across every sector, in every Member State. Three structural changes matter most.
Software and AI as a “Product”
Under the revised Directive, “product” now expressly includes software — including AI systems — regardless of whether they are supplied as standalone applications, embedded in physical products, or delivered as a service (SaaS, cloud, API). Recital (13) lists operating systems, firmware, applications, and artificial intelligence systems explicitly, and provides that the developer of software, including the provider of an AI system within the meaning of the AI Act (Regulation (EU) 2024/1689), “should be treated as a manufacturer.”
This means that an AI tool used for legal research falls within scope — but so does a credit-scoring model, a medical triage chatbot, a predictive maintenance system, an automated customer-service agent, and an AI-assisted contract generator. The manufacturer, importer, or — in certain circumstances — the integrator or deployer may be treated as the “economic operator” liable for a defective product.
The distinction some vendors have attempted — “we don’t sell a product, we provide a service” — collides directly with the Directive’s text, which covers software irrespective of the mode of supply.
Strict Liability and the Defectiveness Standard
Strict liability under the Directive means that the injured party does not need to prove fault or negligence on the part of the producer. They need to demonstrate three things: (i) the product was defective, (ii) damage occurred, and (iii) a causal link exists between the defect and the damage.
A product is defective when it does not provide the safety that a person is entitled to expect, taking into account all circumstances, including:
A hallucinated output — a fabricated citation, an invented regulatory deadline, a fictitious clinical guideline, a non-existent safety parameter — that leads to a lost case, a regulatory penalty, a patient injury, a failed transaction, or a financial loss could satisfy the defectiveness criterion if the output falls below the level of safety a person is entitled to expect.
Evidence Shifting Under the PLD
The evidence-shifting provisions are perhaps the most consequential innovation. Where a claimant faces “excessive difficulties” in proving defectiveness or the causal link — due to the technical or scientific complexity of the product — a court may presume the defect or the causal connection, or both (Art. 10(4)).
Given the opacity of large language models — systems whose internal mechanisms cannot be fully explained even by their own developers — this provision is almost tailor-made for AI disputes. The burden then shifts to the economic operator to rebut the presumption, effectively inverting the traditional litigation dynamic.
Additionally, Art. 9 creates a disclosure mechanism: a court may order the defendant to disclose technical evidence under its control — internal reliability evaluations, test results, model documentation, training data logs, performance metrics. Refusal to disclose is not neutral: the court may presume defectiveness on the basis of the refusal (Art. 10(2)(a)). A vendor who refuses disclosure while simultaneously claiming that the product is not defective contradicts itself procedurally — and the Directive was designed to make that contradiction actionable.
What the PLD Does NOT Cover — and Why It Still Matters
Intellectual honesty requires marking the boundaries. The PLD’s scope for damages is limited: death or personal injury (including medically recognised psychological harm); destruction or corruption of property (excluding property used exclusively for professional purposes); and destruction or corruption of data not used for professional purposes (Art. 6(1)). Pure economic loss — regulatory fines, contract renegotiation costs, lost business opportunities, reputational harm — falls outside the Directive.
This means that a financial institution fined by BNR for a DORA non-compliance caused by an AI hallucination cannot recover the fine through the PLD. A law firm that loses a client because of a fabricated citation in a memo cannot claim reputational damages under the Directive. These losses must be pursued through national civil law — tort (Art. 1357 et seq. of the Romanian Civil Code) or contract (Art. 1350 et seq.) — where the burden of proof is heavier but the scope of recoverable damages is broader.
However, the two frameworks are complementary, not alternative: a finding of defectiveness under the PLD produces objective evidence of the standard the product failed to meet, and that evidence migrates into the national-law claim. The instrument that would not pass the PLD’s safety test does not become safe merely because a bank is using it instead of a consumer.
Key Timeline
|
Instrument |
Key Date |
Effect |
|
Directive (EU) 2024/2853, published in Official Journal |
November 2024 |
Entered into force; Member States must transpose |
|
Transposition deadline (all EU Member States, including Romania) |
December 9, 2026 |
National implementing legislation must be enacted |
|
Application of national measures |
From transposition date onward |
Claims under the new regime become actionable in national courts |
Romania’s transposition window is closing. Businesses that deploy AI-enabled products — whether as vendors, integrators, or professional users — should treat the December 2026 deadline as the hard cut-off for having governance, documentation, and contractual protections in place.
Beyond Legal Tech: AI Hallucination as a Cross-Industry Product Defect
The discussion of hallucination risk often centres on legal AI, because that is where the most visible incidents have occurred and where the empirical data is strongest. But framing the problem as a “legal tech issue” dangerously understates its scope. Hallucination is a structural property of the entire class of generative AI models. It is cross-model and cross-domain. Any product that integrates a generative AI component as a building block inherits this property — and, under the PLD, inherits the liability exposure that comes with it.
The supply-chain cascade
The PLD treats component liability expressly. Art. 8(1)(b) provides that the manufacturer of a defective component is liable when the defectiveness of the final product derives from that component. Art. 12 provides that, where multiple economic operators are liable for the same damage, their liability is joint and several — the victim chooses whom to pursue, and the co-debtors sort out contribution afterwards.
The practical consequence is this: when a composite product (a platform with integrated AI) causes harm because of hallucination in the AI component, both the model developer (component manufacturer) and the integrator who built the final product (product manufacturer) are jointly and severally liable. For Romanian integrators building on foundation models accessed via API, this is the provision that matters most — and the one that makes post-integration reliability testing not a best practice, but a measure of patrimony preservation.
The supply chain, schematised:
[Foundation model provider] → [Integrator / product manufacturer] → [Importer / distributor]
OpenAI, Anthropic, Google SoftX SRL — “AI-powered platform” local commercial channel
component manufacturer final product manufacturer Art. 8(1)(c)/(3)
Art. 8(1)(b) Art. 8(1)(a); Art. 8(2)
──────────────────────── joint and several liability, Art. 12 ────────────────────────
The argument “it’s not our AI, it’s OpenAI’s” is not a defence — it is a confirmation that the integrator chose a component it knew, or should have known, produces fabricated outputs with a documented frequency, and delivered it to the public under its own brand without domain-specific testing.
Five sectors, one defect
The same structural defect — generative hallucination — lands differently across sectors. What differs is the nature and severity of the harm, and the density of overlapping regulatory frameworks. What does not differ is the mechanism.
|
Sector |
Hallucination example |
Harm type |
Overlapping frameworks |
|
Healthcare |
AI cites a fictitious clinical study; recommends a wrong dosage based on fabricated guidelines |
Personal injury, death — PLD maximum |
MDR (Reg. 2017/745), AI Act (high-risk), PLD |
|
Financial services |
AI fabricates a DORA article; invents a prudential threshold; hallucinates a regulatory exemption |
Regulatory fines, credit losses, misreporting |
DORA (Reg. 2022/2554), AI Act (high-risk for credit scoring), OUG nr. 14/2026, PLD |
|
Legal services |
AI fabricates case citations; invents statutory provisions; confirms false legal premises |
Professional negligence, client loss, sanctions |
Professional-duty law (Legea nr. 51/1995), PLD, Civil Code |
|
Industry / manufacturing |
AI misreports a safety parameter; fabricates a maintenance standard; gives a false “all clear” |
Equipment failure, workplace injury, environmental damage |
Machinery Regulation (2023/1230), AI Act (safety components), PLD |
|
Consumer services |
Chatbot invents a return policy, fabricates a warranty term, promises a non-existent discount |
Contractual obligation, consumer claims, aggregate litigation |
Unfair Commercial Practices Directive (2005/29/CE), Representative Actions Directive (2020/1828), PLD |
The scale multiplier. A single defective physical product affects a finite batch. A single defective AI model is the same functional copy running simultaneously in thousands of products and millions of user sessions. One model update that worsens hallucination rates produces simultaneous defectiveness across every product that integrates it — overnight, without the integrators being notified, without any possibility of physical recall. The defect does not sit in a “lot”; it propagates across an ecosystem. This is risk at a scale that product-liability law has not seen since the major pharmaceutical and automotive series — with the critical difference that those defects were at least static. This one pulses with every update.
Operational Risk: Measuring Hallucinations and the “Unsafe Rate”
In my view, the most significant gap in current AI governance is the absence of a standardised methodology for measuring hallucination frequency in domain-specific outputs. Without a consistent benchmark, businesses cannot demonstrate due diligence, insurers cannot price risk accurately, and regulators cannot set thresholds for enforcement action. Each published benchmark also has a direct legal consequence: it fixes the state of knowledge — the “state of the art” against which the PLD’s development-risk defence (Art. 11(1)(e)) is measured. Every measurement published makes that defence harder to invoke.
What Is the Unsafe Rate?
The unsafe rate is not an accuracy metric — it is a risk metric. It measures the percentage of AI-generated outputs that simultaneously satisfy three conditions: (i) they are fluent — written in convincing, professional language; (ii) they are wrong — containing fabricated, distorted, or materially inaccurate information; and (iii) they are credible — a non-specialist (a director, a compliance officer, a non-specialist lawyer, a clinician outside the subspecialty) would have accepted them as correct without further verification.
The unsafe rate measures not how often the model errs, but how often it errs dangerously: producing output that passes a reasonable person’s natural scepticism filters. An obvious error — a confused, incoherent, or expressly hedged response — is not “unsafe”: it self-signals. A fluent, authoritative error is unsafe: it defeats the user’s defences.
Compliance and Governance: DORA, Contracts, and Professional Diligence
Effective AI governance requires a layered approach: operational controls, contractual protections, and professional supervision. The compliance obligations differ depending on whether you are the AI vendor, a professional-services firm using the tool, or a corporate adopter integrating it into business processes.
Compliance Obligations by Entity Type
|
Entity Type |
Key Controls Required |
Practical Implication |
|
Software vendor / AI provider |
Product safety documentation, incident records, corrective action procedures, defect monitoring, post-market surveillance |
Primary target for strict liability claims; ensure PLD-compliant technical file; preserve all internal reliability evaluations (they will be discoverable under Art. 9) |
|
Professional-services firm (law firm, consultancy, audit firm) |
Diligence records, supervision logs, client disclaimers, per-output verification protocols |
Professional negligence risk; the obligation to “study the case thoroughly” (Art. 38, Legea nr. 51/1995 for lawyers) cannot be delegated to a tool with a known, published error rate |
|
Corporate adopter (in-house legal, compliance, risk, operations) |
Procurement SLA with benchmark requirements, acceptance testing, risk register entry, board reporting |
Manage supply-chain exposure and insurer expectations; document the chain of responsibility; for financial institutions, Art. 5 DORA makes the management body personally responsible |
|
Integrator (builds products incorporating AI components) |
Domain-specific reliability testing post-integration, component-manufacturer due diligence, PLD technical file |
Art. 8(1)(a) PLD: the integrator is the product manufacturer, jointly and severally liable with the component manufacturer (Art. 12); untested integration = unmanaged patrimony risk |
Contractual and Procurement Clauses for AI Outputs
Every contract governing the supply or use of an AI tool should address hallucination risk explicitly. From what I am seeing in practice, the following clauses are becoming standard in well-advised deployments:
Litigation and Liability Scenarios: Who Gets Sued and How to Defend
To illustrate the practical operation of AI product liability under the new framework, consider five scenarios — drawn from multiple sectors — that are, in my assessment, increasingly likely to arise in Romanian and EU courts.
Scenario 1 — Consumer harm via AI chatbot. A consumer-facing chatbot generates an answer that invents a return policy or misquotes a consumer-protection statute, leading the user to miss a warranty deadline or rely on a non-existent right. Under the PLD, the operator may face strict liability. The user invokes the evidence-shifting provisions, arguing that the AI’s opacity makes it excessively difficult to prove how the defect arose. The court may then presume defectiveness. (Cf. Moffatt v. Air Canada, where the tribunal held the airline liable for its chatbot’s fabricated policy.)
Scenario 2 — Court filing with fabricated citation. A lawyer uses an AI research tool that generates a fictitious case citation. The citation is included in a court submission. The opposing party identifies the fabrication, the court sanctions the lawyer, and the client incurs additional costs. The law firm faces a professional negligence claim; the AI vendor faces a potential product-liability claim via the contractual indemnity chain. (Cf. Mata v. Avianca; Park v. Kim.)
Scenario 3 — Regulatory filing error in financial services. An in-house compliance team relies on AI-generated regulatory analysis to prepare a DORA incident report. The AI hallucinates the reporting deadline — stating 72 hours for the final report when the actual deadline is one month from the intermediate report. The filing is incomplete, the regulator identifies the error, and a sanction follows. Under OUG nr. 14/2026, the BNR can impose fines of up to 10% of annual turnover or RON 23 million, plus personal sanctions on management-body members.
Scenario 4 — Medical AI generates fabricated clinical guidance. A triage application with an integrated LLM component cites a non-existent clinical study to support a dosage recommendation. A clinician follows the recommendation. The patient suffers harm. The integrator (product manufacturer) and the model developer (component manufacturer) face joint and several PLD liability for the maximum category of harm: personal injury. No proof of fault is required.
Scenario 5 — Industrial AI misreports equipment status. A predictive-maintenance system with a generative AI component reports a critical equipment parameter as “within normal range” when it is not. The equipment fails; a workplace injury occurs. The same PLD supply-chain liability applies. The integrator cannot argue that the hallucination was OpenAI’s fault — the integrator chose the component, integrated it, and delivered it under its own brand.
In each case, the key defences available to the economic operator include: the development-risk defence (that the defect was not discoverable given the available state of scientific and technical knowledge — but increasingly difficult to invoke as published benchmarks and empirical studies accumulate); contributory fault by the claimant (for example, failing to verify an obviously suspect output); and evidence that the product was not defective when placed on the market (but Art. 11(2) closes this defence for software under the manufacturer’s control).
Directors and Officers: Personal Exposure and the Business Judgment Rule
Directors’ personal liability for AI failures is an emerging area that too few boards are taking seriously. Under Romanian corporate law — specifically Art. 144¹ of Legea nr. 31/1990, introduced by O.U.G. nr. 82/2007 — directors owe a duty to exercise their mandate “with the prudence and diligence of a prudent administrator.” Critically, Art. 144¹(2) codifies the business judgment rule in Romanian positive law: a director does not breach the duty at paragraph (1) if, at the time of the business decision, the director “was reasonably entitled to consider that he or she was acting in the interest of the company and on the basis of adequate information.”
Three consequences flow from this provision, each reinforcing the case for documented AI governance:
D&O Insurance: When the Insurer Refuses Coverage
The practical consequences of this analysis materialise in the context of Directors & Officers liability insurance. D&O policies exclude gross negligence, intentional conduct, and fraud. If the board authorised AI deployment in critical functions without documented governance, testing, or oversight — and the error rate was publicly known at the time of the decision — the insurer has a strong case for refusing coverage on grounds of gross negligence. The director is then exposed personally, with their own assets, for the harm caused.
Board-level risk management checklist:
Insurance, Indemnities, and the Coverage Gap
Insurance coverage for AI hallucination losses sits at the intersection of several traditional policy types — none of which was designed for this risk:
No existing policy type fully covers PLD-era AI risk. Businesses should take three immediate steps:
On the indemnity side, sample language should require the AI vendor to “defend, indemnify, and hold harmless the Customer against all claims, losses, and costs arising from any material inaccuracy, fabricated citation, or factual error in the AI-generated output, except to the extent caused by the Customer’s failure to follow the Vendor’s documented usage guidelines.” This language should be reviewed and tailored by local counsel, including with reference to Romania’s Civil Code provisions on contractual indemnities (Art. 1350 et seq.).
The benchmark becomes, in this context, an underwriting instrument — similar to a cybersecurity rating, an ISO audit, or an ESG due-diligence report. An entity with a documented, measured unsafe rate obtains coverage at reasonable terms; an entity without one pays the risk premium of the unknown — or does not obtain coverage at all.
Romania-Specific Enforcement and Regulatory Landscape
Romania does not yet have a standalone AI liability statute, but the transposition of Directive (EU) 2024/2853 — due by December 9, 2026 — will bring AI-enabled products within the existing product-liability enforcement framework. As of the date of this article (July 2026), no Romanian transposition bill has been published for public consultation. The gap between the significance of the act and the absence of public discussion is itself information about the market’s level of preparedness.
Romanian businesses should monitor four enforcement channels:
Romanian firms should also track the transposition process closely. The implementing legislation may include Romania-specific procedural rules — including on the Art. 9 disclosure mechanism, on the protection of trade secrets in AI-related disputes, and on the admissibility of benchmark evidence — that go beyond the Directive’s minimum requirements. Art. 18 of the Directive permits Member States to derogate from the development-risk defence for specific product categories — an option that the Romanian legislator should at least consider for AI systems used in high-stakes professional decisions.
Conclusion: 10-Step Immediate Action Plan
The convergence of Directive (EU) 2024/2853, DORA, AI Act, and professional-duty obligations creates an urgent compliance window. The risk is not sector-specific — it applies to every entity that develops, integrates, deploys, or relies on AI-enabled products.
For specialist advice on AI product liability, DORA compliance, PLD transposition, and AI governance frameworks, contact Razvan Alexandru Olaru.
posted 2 hours ago
posted 3 hours ago
posted 4 hours ago
posted 4 hours ago
posted 4 hours ago
posted 10 hours ago
posted 10 hours ago
posted 11 hours ago
posted 11 hours ago
posted 12 hours ago
posted 12 hours ago
posted 12 hours ago
No results available
Find the right Legal Expert for your business
Sign up for the latest legal briefings and news within Global Law Experts’ community, as well as a whole host of features, editorial and conference updates direct to your email inbox.
Naturally you can unsubscribe at any time.
Global Law Experts is dedicated to providing exceptional legal services to clients around the world. With a vast network of highly skilled and experienced lawyers, we are committed to delivering innovative and tailored solutions to meet the diverse needs of our clients in various jurisdictions.
Global Law Experts is dedicated to providing exceptional legal services to clients around the world. With a vast network of highly skilled and experienced lawyers, we are committed to delivering innovative and tailored solutions to meet the diverse needs of our clients in various jurisdictions.
Send welcome message