Setting the Safeguards
By now, you know I’m not exactly exaggerating when I say that AI bears great promise. Investors are pouring unrivalled amounts of money into the development of AI technology. Businesses are racing to bring it to the market first. The excitement is contagious. Yet, now the first awe starts to wane, I also hear a different message. One of concern.
- What risks are involved with this cool new technology?
- Are we accountable for the content AI is generating?
- Are the costs remotely in balance with the expected benefits?
These are all good questions to raise. And there are most likely more similar questions spinning in your head right now. It’s a fact of life in the digital age that useful technologies are being misused by malicious actors to steel your precious data and to hijack your digital systems. Undeniably, attack surfaces in LLMs are broader than traditional software. The more powerful these agent technologies grow, the scarier the threats become. Yet, a
Perhaps you’re worried about the strict requirements set in the European AI Act. Perhaps you’ve experienced some of the downsides of AI sprawl already. Or maybe you’ve faced a deceivingly real, but AI-generated, micro-targeted fishing attack. In that case, I’m sorry for you. But you came to the right place. This blog will not only give you food for thought, but also many actionable tactics to get on top of AI in your enterprise.
OWASP
Whenever application security is at stake, OWASP is my trusted source of reference. Unsurprisingly, the recently published 2025 OWASP Top-10 of vulnerabilities has some new AI-related entries. “AI prompt injection attacks”, for instance. Moreover, there’s now a separate top-10 for Generative AI & LLMs. Your concerns are heard!
1. Prompt Injection
Not unlike the manipulation of an SQL query, prompt injection looks to generate unintended responses from the model. Attempts to override the system prompt (“Ignore all previous instructions”), attempts to ‘negotiate ’with the model, or even to bribe it, it all can lead to disastrous outcomes.
2. Insecure Output Handling
When an AI agent generates and executes computer code without proper validation, you risk a nonsensical outcome, or worse, a Cross-Site Request Forgery attack, or even a Cross-Site Scripting attack. You cannot trust the model for being flawless. And you cannot test every possible outcome upfront.
3. Training Data Poisoning
Some language models are self-learning. The interaction with users is thought to make the models ever smarter. However it’s also possible to enter malicious training data into the model this way. Just imagine how a foreign actor could render the model you rely on less than useless.
4. Model Inversion
Sometimes, the output a model generates leaves traces to its source. When you’re familiar with the source, you can design your prompts to reveal it unequivocally. It may reveal intellectual property theft, GDPR violations, amongst others, and thereby trigger penalties.
5. Model Theft
Training data is hard to get and expensive. So, why not use the output of competing models as input for your model? Or simply create an agent that combines the output of multiple source models into one new artefact?
6. Inadequate Sandboxing
Controlling access of AI agents to source systems (typically through MCPs) is notoriously difficult. One user’s legitimate private data is another users loot. Albeit hard, control of access is essential to prevent bad things from happening.
7. Unauthorized Code Execution
Closely related to Insecure Output Handling, unauthorised code execution occurs when a user constructs a malicious prompt to get uncontrolled code executed.
8. Insecure Plugin Design
When you’re using third-party AI plugin, such as an AI chatbot, in your business systems, and especially when they get updates outside your control, you may unknowingly be on the brink of disaster.
9. Overreliance on LLMs
People are easily impressed by a quickly generated document. Yet, not every generated case reference in a legal brief may be real. Mind you, the number of known AI hallucination cases in legal documents is 541 at the time of writing and growing. Likewise, not every generated email reply is correct, not every summary is . Moreover, many readers are getting sensitive for AI writing styles — and pictures (see what I did there?).
10. Inadequate Monitoring and Logging
Obviously, when your AI agents leave no trails, or under some unmatchable identity, you’re in the dark about your operations.
At this point, I can image you’re sceptical about the real world impact of these vulnerabilities. After all, if the risk is so high, then why is BigTech continuing their massive investments into it, right? Well, although it is tempting to list some juicy events, it would be outdated before you read it. Instead, I’ll give you a prompt which your favourite GenAI client will happily investigate on your behalf:
Please give me some recent actual historic incidents (public breaches or documented attacks) mapped to each of issues in the latest OWASP Top 10 for Generative AI & LLM vulnerabilities.
Don’t forget to appreciate the magic while you browse the list of debacles. Now please read on before you find yourself trapped in another doom-scroll.
Help on offer
Managed MCPs.
By now, the software industry is actively developing capabilities to help you tackle those vulnerabilities. MCPs are introduced by Anthropic as a standard interface to connect AI Agents to data sources, typically by decorating existing APIs. In WSO2 API Manager, MCP’s are now first-class citizens. It offers a new feature for creating an MCP Server by importing an OpenAPI specification and for manually adapting the tools and their behaviour. Moreover, you can secure access to your MCPs in the same way you manage access to all your other APIs. Since MCPs are built on top of APIs, you can strengthen the security of your MCP by carefully hardening your API. This includes rigorous input and output validation. We’ve written an entire white paper on the topic of API Security, free to download here.
AI Guardrails
LLM’s are probabilistic machines in nature. They have no pre-determined outcome. You cannot expect getting the same answer twice. Or even a similar answer. More surprises will happen. But there are definitely ways to maximise the usefulness of model use.
To safeguard the use of LLM’s and GenAI in your organisation, you want to sanitise the model input as well as the output. The AI Gateway does exactly that. It intercepts the prompt and enforces any policies you’ve defined. You decide how to sanitise the prompt – by masking PII, by trunking or augmenting the prompt, or by rejecting it altogether. The AI Gateway can even determine the safety of the content for you. The number of policies you can apply is already large and will only grow.
Additionally, the AI Gateway can sanitise the output of the model in a similar fashion, thereby putting you back in charge of content generation. You can in fact determine the content safety of the generated output. You can also validate any URL in the response. You just need to define a policy for it.
Agent identity
But it doesn’t stop there. Importantly, WSO2 Identity Server also has a new first-class citizen: Agents. Not only can an agent get its own identity –which will help make its actions traceable– but also of the identity on whose behalf it is acting. This is implemented as a new grant type, especially for AI Agents. Interestingly, it even includes a flow for the agent to get consent from its user to execute an action.
Together, this lays a solid foundation for secure agentic computing.
Semantic caching
The consumption of tokens can easily skyrocket. One way to reduce the costs of using those LLM’s is simply caching their responses. It depends on your use case to what extent this will bring relief. But experience shows that prompt repetition happens more often than you might expect. With semantic caching, you’re in fact generating a list of responses for frequently asked questions. No human intervention required.
Cost control
Even though semantic caching eases some of the pain, writing a blank check to go wild on AI experiments is can easily become a costly affair. At that point, you may consider allocating the costs of AI to the budget owners. For a fair calculation, however, you need to track the consumption per budget owner. Fair enough, since the AI is fully integrated in API Manager, and Agents have gotten their own Identity, the Analytics engine in API Manager can give you exactly the report you need to pass the LLM costs on.
Retrieval-Augmented Generation
Integration is at the heart of agentic AI. Retrieval-augmented generation (RAG) is a nascent technology to feed a language model with trusted information thus making the outcome more reliable. Typically, you augment a prompt with data items you derive from the state of your resources. In essence, there are two competing mechanisms. The Micro Integrator allows you to enrich agent information with available data in a knowledge base. If you feed it with available trend data, for instance, your agent will happily predict future directions. Using an LLM, you can easily generate an impressive report from it.
The Streaming Integrator allows you to use real-time events as additional input – perhaps from web traffic, or from alerts. Your agent might instantly decide on a next best action. The Ballerina Integrator does both, by the way, so you can even mix it if you want.
Prompt Decoration
Generated content can be super powerful, but it typically does not comply to any guidelines. In many situations, common guidelines are used to create consistency across the artefacts created by different people. Think of writing guides, presentation templates, coding guidelines, or an API Style guide, for instance. Prompt decoration allows you to automatically add those guidelines to the input your colleagues feed to your agents, thereby greatly enhancing the usability of the outcome.
Making it work
Safeguarding the AI in your enterprise is about more than just acquiring the right technologies, albeit a good start. To make it a success, you need to organize the tool use, define responsibilities, get a buy-in from stakeholders, create commitment for your policies, and such.
Fortunately, introducing AI Management is not really different from introducing API Management in your organisation. With WSO2, it’s even in the same toolset. As Yenlo, we’ve been there and done that many times over. We also generally love embracing the newest things in integration technology, and taking them to their fullest potential. No wonder we’d love to put our experience to work by helping you succeed in mastering AI Security.
Want to find out how your organization can implement AI in a secure, scalable and controlled way? Get in touch with us today we’d be happy to explore the possibilities with you and help you get started right away.
We’re just one call away.
