Model theft meaning, in plain words

Model theft copies an AI's inner workings through repeated queries. What NIST calls model extraction, what OWASP lists as defenses, and why limits can be evaded.

Written by: Mio Kanzaki(Security Analyst (AI))Reviewed by: 千葉圭介Updated

Model theft is an attack that copies an AI's inner workings through repeated queries. The report of NIST (the US National Institute of Standards and Technology) calls the same attack model extraction. It explains that the aim is to draw out, by sending queries, information about the AI's structure and the numbers that decide its contents (parameters).

How does model theft work?

It is like ordering a popular shop's dish many times to check the taste and guess the recipe. The provider of an AI wants to keep secret the structure and parameters of a model built with time and money. According to NIST's report, an attacker repeats queries to the AI service and tries to draw that secret out of the answers.

NIST introduces research showing that extracting a completely identical model is impossible. Instead, the picture is that another model with close performance can be rebuilt.

Measures include limits on queries, detection of suspicious queries, and a design that resists side-channel attacks, which use secondary information leaking from a device. NIST says that a determined attacker with resources can evade them, so they should be used with care. If you publish an AI on your own site, the free URL check shows from outside the state of the side you publish.

Why does model theft matter?

If you offer your own AI to outsiders through an API or similar, copying could hand the result of your development cost to another company. OWASP's LLM10 explains that uncontrolled inference at scale can lead to theft of intellectual property by replicating the AI's behavior. Examples it names include extraction through an API and replication of functions.

The measures OWASP gives include rate limits on queries, access control, records and monitoring for anomalies, and adding watermarks. Even if you only use an outside AI service, a design in which queries can be used without limit is a burden in terms of cost.

To talk through what to put in place before you publish an AI, contact us.

Related terms

Next step

If you offer your own AI to outsiders, check that user authentication, limits on query counts, and records of use are in place. The free URL check shows the state of published entry points, and for preparation that goes with publishing an AI, contact us.

Frequently asked questions

Does model theft make an identical copy of the AI?

NIST's report introduces research showing that exact extraction of a model is impossible. The picture is not an identical copy but another model rebuilt with similar performance.

Does limiting the number of queries prevent model theft?

NIST's report says limits on queries and detection of suspicious queries serve as measures, but that a determined attacker with resources can evade them, so use them with care. OWASP also lists many measures, including rate limits.

Frequently asked questions

Does model theft make an identical copy of the AI?

NIST's report introduces research showing that exact extraction of a model is impossible. The picture is not an identical copy but another model rebuilt with similar performance.

Does limiting the number of queries prevent model theft?

NIST's report says limits on queries and detection of suspicious queries serve as measures, but that a determined attacker with resources can evade them, so use them with care. OWASP also lists many measures, including rate limits.

See what an attacker can see, today

The free URL check returns a grade in under a minute. For costs and next steps, book a free consultation.