Testing the Limits: Experiment Shows AI Falls Short in Drafting Patent Applications

This article addresses considerations with respect to using AI assistants when drafting a patent application based on an experiment by the author.

The author took a case from her portfolio where the application had already been submitted with the patent office, thus there are no privacy issues.

The application is in the mechanical field and refers to a piece of equipment for producing a particular type of tissue together with the method of producing same. The matter is straightforward for an experienced patent attorney because:

• The invention represents an improvement of an earlier piece of equipment and method of the same inventor in which new features of the equipment are added and some additional steps, thus the closest state of the art is known; and
• The inventor provided the attorney with detailed differences between the old and the new technical solutions, although not in a structured way, with the disadvantages of the prior art and with a self-explanatory drawing from which it is relatively easy to infer the distinguishing features.

In the first phase, the following items were provided to four different AI assistants, one of which was a local model:

• The old technical solution making the object of a published patent application;
• The detailed differences as stated by the inventor;
• The new technical solution (without the differences clearly structured);
• The self-explanatory drawing; and
• Legal provisions as PDF documents (the law and the implementing regulation).

The following input order was given to the AI assistants: based on the documents and the applicable law, draft a patent application following the structure of the published patent application.

Separately from the above exercise, the author identified in the second phase of the experiment, without the aid of an AI assistant, the distinguishing features of the new technical solution over the earlier technical solution. The author addressed separate targeted questions to the AI assistants as follows: what are the technical effects of the distinguishing feature X of the new solution? The same question was repeated for the product features and for the new steps of the method.

The answers given by the AI assistants were then filtered, firstly by the author’s team and then by the client. The correctly identified technical effects and advantages were inserted in the patent application.

None of the AI assistants drafted a sufficiently good patent application.

Concretely:

• None of the AI assistants managed to grasp the distinguishing features sufficiently, although their identification was quite easy in this case. The percentage of correctly identified features ranged from 0% to 33%. Without the distinguishing features correctly identified, there is no basis for a serious patent application.
• Even for the few distinguishing features correctly identified, they were not sufficiently disclosed and explained to enable the examiner to understand their technical effects that are necessary to demonstrate inventive step. Either the technical effects were missing or they were wrongly understood. Remember that the second phase of the experiment was directed to the technical effects that were verified with the inventor.
• The language was inconsistent throughout the entire text, although it is a basic requirement in patent drafting to use the same term for the same feature.

With regard to the second phase of the experiment, the degree of correctness of the identification of the technical effects ranged from one-third to two-thirds depending on the AI assistant.

The use of AI assistants for drafting patents does not yet produce satisfactory results. AI assistants can be of help for responding to targeted questions, but not yet for drafting.