AI and LLM Pentesting: What Should Actually Be Tested

Prompt testing matters, but many serious AI failures happen because the surrounding system trusts model output too much or gives the model unsafe reach into data and tools. The assessment needs to cover the workflow, not only the model response.

• Prompt injection from user input, retrieved content, documents, or web data
• Unsafe tool execution and over-permissive action handling
• Cross-user data exposure through memory, retrieval, or output assembly
• Authorization failures around AI-triggered actions
• Weak boundaries between model output and system decision-making

Once a model can search, fetch, write, send, or trigger workflows, it becomes part of the control plane. At that point the security question is no longer “can the model say something wrong?” but “can untrusted input influence actions the system should not allow?”

• The exact trust boundary that failed
• What untrusted input influenced
• What action or data exposure became possible
• Which control should have limited the model or workflow
• How to fix the issue at the system design level

Before launch, identify every place model output can change state, trigger tools, expose data, or influence privileged logic. Those are the paths that deserve direct security review before customers and buyers rely on the feature.