Research, references, and field notes on AI systems, prompt injection, agent security, responsible AI, governance, and compliance for LLM and agent systems.
Curated research, system cards, and technical write-ups that are useful for understanding how AI systems are being evaluated, attacked, governed, and deployed in practice.
Watch video
Attacking AI is a one of a kind session releasing case studies, tactics, and methodology from Arcanum’s AI assessments in 2024 and 2025. While most AI assessment material focuses on academic AI red team content, “Attacking AI” is focused on the task of assessing AI enabled systems.
Microsoft Incident Response walks through how to detect prompt abuse operationally, tying prompt injection risk back to logging, telemetry, and incident response workflows.
OpenAI frames prompt injection as an evolving agent-security problem that increasingly resembles social engineering rather than a simple string-matching issue.
OpenAI announced plans to acquire Promptfoo, highlighting automated AI security testing, red teaming, and evaluation as core enterprise requirements.
Recent notes and references across prompt injection, agent security, evaluations, responsible AI, and adjacent AI work.
Krebs on Security covers April 2026 patching activity, including a record-sized Microsoft release and active exploitation notes.
NIST’s AI RMF hub now highlights the upcoming Trustworthy AI in Critical Infrastructure profile alongside the playbook and related implementation resources.
MITRE maps incident patterns in an open-source agentic ecosystem to ATLAS techniques, showing how AI-first systems create distinct execution paths for attackers.
The European Commission’s AI Act hub centralizes the EU’s risk-based AI compliance framework, implementation material, and links to governance, enforcement, and standardisation resources.
These topic hubs connect current research with the parts of AI security, governance, evaluation, and system behavior that are most useful in practice.
Methods, case studies, and tooling for red teaming AI systems end to end.
Open topicPrompt design patterns, instruction hierarchy, and defensive prompt construction.
Open topicPrompt injection attacks, mitigations, detection, and design patterns for safer AI applications.
Open topicControls and attack paths for browsing, tool use, memory, identity, and action-taking agents.
Open topicSafety evaluations, system cards, preparedness, and security measurement for frontier models.
Open topicResponsible AI, governance, standards, and regulatory reference material for teams mapping AI systems to policy and operational controls.
Open topicAdversarial machine learning attacks, taxonomies, and mitigations across the ML lifecycle.
Open topicFocused on AI research across model behavior, AI security, responsible AI controls, governance, and compliance for LLM and agent systems, while building AI operational software for compliance and financial tracking.