Written by: Mio Kanzaki(Security Analyst (AI))Reviewed by: 千葉圭介Updated
An LLM security assessment checks the generative AI apps built into your business for holes such as information leaks and AI that has been given too much authority. It is meant for companies that run a chatbot, or a mechanism that finds internal documents and answers from them, on an LLM (large language model, an AI that reads and writes text). AI agents, which use tools to do work on their own, are included. As a first step, write down what information the AI can read and what operations it can run.
Cyber Shield uses the ten viewpoints published by OWASP, reviews the setup and has an analyst operate the app to confirm each point. The assessment is from JPY 298,000 (excl. tax). Below we cover why it is needed now, what we check, what you keep and what it costs.
Why does a generative AI app need its own assessment?
Generative AI does not always give the same answer to the same input, unlike a traditional program. The AI is also not good at telling a command from a text it was given to read. As a result, it may follow an instruction hidden in text that arrived from outside.
Public documents reflect this. In IPA's Top 10 Information Security Threats 2026, organization edition, "cyber risks related to the use of AI" ranks third, and 2026 is the first year it was chosen. IPA also published a guide to using generative AI and AI agents safely in July 2026.
An assessment of a traditional web app examines screens and communication. A generative AI app has a further entrance: what the AI reads and what it runs. Checking the screens alone cannot confirm that entrance.
What does an LLM assessment check?
Our viewpoints rest on OWASP's Top 10 for LLM Applications 2025. It organizes the risks most likely to be serious in generative AI apps into ten items, and it was published in November 2024.
The table turns the ten items into plain words and adds what we confirm. We do not publish the text used in attacks.
| OWASP item | In plain words | What we confirm |
|---|---|---|
| Prompt injection | An instruction hidden in text the AI reads makes it act beyond its role | How far the AI trusts and acts on text that arrives from outside |
| Sensitive information disclosure | Information that must not be shown comes out through the AI's answers | Whether another user's data or internal documents mix into answers |
| Supply chain | The model or parts in use come from sources that cannot be trusted | The origin and updates of outside models, libraries and add-ons |
| Data and model poisoning | Malicious content gets into the data used for learning or lookup | Who can add to the documents the AI refers to, and how |
| Improper output handling | The AI's answer is passed to a screen or a system without checks | Whether answers are used directly for display or for commands |
| Excessive agency | The AI is trusted with more operations than the work needs | The tools the AI can use and how wide their permissions are |
| System prompt leakage | The AI's internal instructions contain secret information | Whether the instructions hold keys or internal details |
| Vector and embedding weaknesses | A setup error in document search lets the AI read documents it should not | Whether permission boundaries hold in search |
| Misinformation | The AI gives a plausible error and people act on it | Whether an error can feed straight into a business decision or action |
| Unbounded consumption | Heavy use drives costs up or stops the service | Whether limits exist on the number of uses and the size of input |
For the mechanics and fixes of individual flaws, see prompt injection, information leaks through an LLM and excessive permissions for AI agents.
What this assessment does not cover
The assessment is limited to apps you manage and their settings. It does not include testing the platform the AI provider runs. To assess the API door the AI uses on its own, see the API assessment. Holes on the screen side belong to the web application assessment.
How do we check?
The work runs in three steps: review of the setup, review of the settings and an analyst's hands-on check. Not many holes can be judged mechanically by a tool. Because the AI's answer changes each time, the main work is for a person to create a situation and read the result.
First we ask for a list of the information the AI can read and the tools it can use. Next, once the consent form fixes the target and the period in writing, we operate the app within that scope. For example, we check whether a user without rights can touch a document they should not be able to see.
An analyst confirms that each finding can be reproduced and judges whether it is a problem that really occurs. We then show how to fix the setup or the design. Examples are narrowing permissions, checking output, and separating the data the AI refers to.
We sign an NDA before the assessment. Testing a system without permission may break Japan's Unauthorized Computer Access Law, so we limit the targets to those you have consented to.
What do you receive when the assessment ends?
You receive a report. For each hole it states which function is affected, why it is a problem and how to fix it. It also ranks what to fix first, such as narrowing the permissions the AI holds and reviewing what it connects to.
One re-assessment after the fixes is free. It confirms that each repaired function now behaves as intended, and we add the outcome to the report.
What does it cost, and what makes it higher?
The generative AI/LLM app assessment is from JPY 298,000 (excl. tax).
| Item | Price | Notes |
|---|---|---|
| Generative AI/LLM app assessment | from JPY 298,000 (excl. tax) | Includes a report |
| Individual vulnerability fix | from JPY 10,000 per item (excl. tax) | Quoted by content |
| Re-assessment after fixes | Free | Once |
Three things raise the amount. The first is the number of functions assessed: a chat-only app and one that also searches and runs tasks need different amounts of checking. The second is the number of systems the AI connects to, because every internal system or outside service widens the range of permissions to examine. The third is the kind and volume of data the AI refers to.
The formal quotation is given after we see the setup, and it follows the free assessment. All prices are on the pricing page. If you cannot yet say which systems the AI touches, contact us.
How does a request go from first contact to delivery?
The general flow applies to this assessment too, with these points specific to an AI app.
- In the free consultation we ask how you use AI, which systems it connects to and what information it refers to.
- Both sides sign the NDA first, at no charge.
- Once the consent form is signed, a quick scan and a person's review make up the free assessment, and you receive an issue report.
- We estimate what it costs to fix the findings. A full redesign of the AI app, if one is needed, is estimated separately.
- On order, we cover all items and close the gaps.
- A PDF describes each change before the app is handed back.
- You confirm how the app behaves on the real screens.
- Delivery comes next, and the month after it carries free support.
Common objections
Our AI is still at the pilot stage. Is an assessment too early?
No. The questions an assessment asks are which data the AI may read and which tools it may use. They are easiest to settle before real data is connected or before people outside the company start to use the app.
Will the assessment stop our live AI or disturb operations?
The consent form fixes the target and the period. For operations on a live system that could have an effect, we agree how to handle them beforehand. For worrying functions such as payment or deleting data, working in a test environment is also possible.
Frequently asked questions
What is an LLM security assessment?
It is a check on apps that use generative AI, such as a chatbot built into your work, a search over internal documents or an AI agent, for holes such as information leaks and excessive permissions. We use OWASP's ten items as viewpoints, review the setup, have an analyst operate the app, and put the results in a report.
How much does it cost?
It is from JPY 298,000 (excl. tax). The amount depends on the number of functions assessed and the number of internal systems the AI connects to, and the formal quotation follows the free assessment. An individual vulnerability fix is from JPY 10,000 per item (excl. tax), quoted by content, and one re-assessment after the fixes is free.
Can prompt injection be stopped by AI settings alone?
Settings alone are not enough. Wording the instructions carefully helps, but you should assume it can be broken. So you narrow the permissions you give the AI and what it connects to, and add a way to check its output. The assessment confirms that these layers have no gaps.
We only use an outside AI service. Do we still need an assessment?
Sometimes. The AI provider protects the underlying platform. What your app sends to the AI, and what it does with the AI's answer, is the owner's responsibility. If you connect internal documents or customer data, it is worth checking.
Does an assessment prove the app is safe?
No. An assessment tells you which holes the setup has at that moment. Replacing the model, adding documents it can read or adding tools it can use each change the risk. When you change the setup, check again.
Next step
If you want to see how the web side of your service looks today, use the free URL check. From one URL it reads only public information, runs no attacks and cannot see how your AI app handles permissions.
If the systems the AI connects to or the scope of the assessment are unclear, contact us by form, phone or online meeting. The NDA is free, and we sign it before we touch the app.
Written by Mio Kanzaki (AI security analyst). Supervised by 千葉圭介. Updated 19 November 2026.
Frequently asked questions
What is an LLM security assessment?
It is a check on apps that use generative AI, such as a chatbot built into your work, a search over internal documents or an AI agent, for holes such as information leaks and excessive permissions. We use OWASP's ten items as viewpoints, review the setup, have an analyst operate the app, and put the results in a report.
How much does it cost?
It is from JPY 298,000 (excl. tax). The amount depends on the number of functions assessed and the number of internal systems the AI connects to, and the formal quotation follows the free assessment. An individual vulnerability fix is from JPY 10,000 per item (excl. tax), quoted by content, and one re-assessment after the fixes is free.
Can prompt injection be stopped by AI settings alone?
Settings alone are not enough. Wording the instructions carefully helps, but you should assume it can be broken. So you narrow the permissions you give the AI and what it connects to, and add a way to check its output. The assessment confirms that these layers have no gaps.
We only use an outside AI service. Do we still need an assessment?
Sometimes. The AI provider protects the underlying platform. What your app sends to the AI, and what it does with the AI's answer, is the owner's responsibility. If you connect internal documents or customer data, it is worth checking.
Does an assessment prove the app is safe?
No. An assessment tells you which holes the setup has at that moment. Replacing the model, adding documents it can read or adding tools it can use each change the risk. When you change the setup, check again.