AI guardrails meaning, in plain words

AI guardrails are safety mechanisms that put limits on what an AI outputs and does. The methods OWASP lists, and why they are not a complete defense.

Written by: Mio Kanzaki(Security Analyst (AI))Reviewed by: 千葉圭介Updated

A guardrail is a safety mechanism that puts a limit on the output and actions of an AI. The AI Guidelines for Business (Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry) tell businesses that develop and provide AI to consider guardrail technology and similar as a method to minimize risks such as inappropriate output.

How do AI guardrails work?

It is like the protective fence beside a mountain road, which stops a car from falling into the valley when it leaves the road. In AI, it catches an answer the AI must not give, or an operation it must not perform, before it happens.

The methods OWASP lists show the roles. They are setting the AI's role and scope by instruction, fixing the format of the output and checking it with a program, and narrowing input and output. Keeping permissions to the minimum and putting human approval in front of high-risk operations are listed alongside them.

However, it is not a complete mechanism. OWASP says no sure way to prevent prompt injection is known, and adds that the AI should be treated as an untrusted user. LLM02 says that even when the types of data are restricted by instruction, the restriction may be bypassed.

The relationship with jailbreaking is on its glossary page. Checks of features that build in AI are covered by the AI and LLM assessment.

Why do AI guardrails matter?

An AI produces a great deal of output, and people cannot watch all of it. If you decide in advance, as a mechanism, the range of what may come out, you can keep the damage small when an error or a takeover happens.

As an operator, first decide the information the AI must not touch and the operations it must not execute. Then add mechanisms that check output and input afterward. If the order is reversed, then once the added mechanism is bypassed, nothing is left to protect.

To confirm that they work, a check that tries from the attacker's side is useful. It is explained under AI red teaming. The free URL check shows what is visible from outside.

For a talk about how to design guardrails, contact us.

Related terms

  • Jailbreak: a trick that gets around the limits placed on an AI by careful wording.
  • AI red teaming: a check that tries to find an AI's weaknesses from the attacker's side.

Next step

First decide the information the AI must not touch and the operations it must not execute. Then choose the mechanisms that check input and output. The points to examine are in the AI and LLM assessment. What is visible from outside can be seen with the free URL check, and we take questions on design through a free consultation.

Frequently asked questions

If we have guardrails, will AI problems not happen?

It cannot be said that they will not. OWASP says no sure way to prevent prompt injection is known, and that the measures are meant to reduce the damage. Combine guardrails with measures such as narrowing permissions.

What methods do guardrails use?

OWASP lists setting the AI's role and scope by instruction, checking the format of the output with a program, and narrowing input and output. The AI Guidelines for Business give guardrail technology and similar as an example of a method to minimize risks such as inappropriate output.

Frequently asked questions

If we have guardrails, will AI problems not happen?

It cannot be said that they will not. OWASP says no sure way to prevent prompt injection is known, and that the measures are meant to reduce the damage. Combine guardrails with measures such as narrowing permissions.

What methods do guardrails use?

OWASP lists setting the AI's role and scope by instruction, checking the format of the output with a program, and narrowing input and output. The AI Guidelines for Business give guardrail technology and similar as an example of a method to minimize risks such as inappropriate output.

See what an attacker can see, today

The free URL check returns a grade in under a minute. For costs and next steps, book a free consultation.