Cybersecure AI: AI as both target and attacker

Through GPT-NL, TNO is developing a sovereign Dutch language model to strengthen digital sovereignty. To ensure the model can be deployed safely and responsibly, we investigate both safety and security risks.

By analysing model behaviour, conducting experiments and evaluating real-world use cases, we identify vulnerabilities, potential misuse scenarios and opportunities for improvement. These insights are used to strengthen the model's robustness, reliability and security, helping to create generative AI systems that are not only powerful but also trustworthy and aligned with European values.

How do you build an AI application securely when the risks are not yet fully understood? TNO's workshop methodology helps organisations identify the key risks associated with LLMs and AI agents within a specific use case. The approach is also used in the safety assessment of practical GPT-NL applications.

The outcome is a clear action plan outlining which risks require attention, which mitigations are needed and which aspects should be continuously monitored to ensure safe and responsible deployment.

TNO provides an independent and trusted framework for AI security assessment, enabling organisations to make informed, risk-based decisions that strengthen the protection of their AI solutions. The framework can also serve as a foundation for future standards, methodologies and regulatory guidance.

TAISHA collects information about an AI application, including the model used, the type of data involved and the deployment context. Based on these characteristics, the tool identifies the security measures most relevant to your organisation and highlights key priorities and risks. Its output provides a practical starting point for risk management, compliance assessment and board-level decision-making.

LLMs and RAG systems introduce new security risks, including prompt injection, jailbreaking and the unintended disclosure of sensitive information. These vulnerabilities depend heavily on the deployment context and are therefore often missed by generic red-teaming tools and benchmarks.

To address this challenge, TNO and its PCSI partners developed ProViLE (Prompt for Vulnerabilities in LLM-based Applications with Extensions). This open-source methodology and tool enables organisations to conduct AI red-teaming exercises on their own LLM applications. In four steps, ProViLE helps users identify relevant attack scenarios, create attack prompts and evaluate system responses.

What sets ProViLE apart is its context-driven approach. Rather than relying on generic tests, it uses prompts tailored to a specific application, dataset and risk profile. This produces more realistic attack scenarios and deeper insight into vulnerabilities that matter in practice.

AI systems are complex and often unpredictable. Integrating AI into existing software introduces new vulnerabilities, meaning traditional cybersecurity measures are not always sufficient.

AI red teaming stress-tests an AI system to identify security weaknesses. Techniques include adversarial prompting for LLMs, adversarial samples for computer vision systems and data extraction testing for models trained on sensitive information.

AI safety and security alignment refers to the methods, technologies and processes used to ensure AI systems operate safely, reliably and in accordance with human goals, societal values and legal requirements. It focuses on preventing harmful behaviour, protecting sensitive information and improving resilience against manipulation, misuse and cyberattacks.

• Manipulated inputs (adversarial attacks) that cause incorrect outputs or unintended behaviour.
• Unintentional disclosure of sensitive information.
• Vulnerabilities in open-source components that propagate through supply chains.
• Manipulated training data that teaches a model incorrect or harmful patterns.