Why it matters
NVIDIA's AI Red Team reports recurring failures across six months of enterprise-agent assessments: weak user-level access control, command and file tools that enable code execution, unrestricted network egress, and secrets exposed through environment variables or CLI caches. Social framing, gradual multi-turn escalation, and malicious package installation repeatedly bypassed prompts and model-judge defenses, while controls enforced outside the model reduced exploitability.
My takeaway: Bind every agent session to an authenticated user and cap the agent at that user's permissions. Avoid general shell tools; otherwise isolate them in a hardened sandbox, block executable file writes, and allowlist commands and package sources. Enforce default-deny egress outside the runtime, broker short-lived task-scoped credentials without exposing them to the model, and treat prompts or LLM judges as supplemental detection rather than security boundaries.