AI Engineer · August 24, 2025

2025 is the Year of Evals! Just like 2024, and 2023, and … — John Dickerson, CEO Mozilla AI

video AI Engineering

2025 is the Year of Evals! Just like 2024, and 2023, and … — John Dickerson, CEO Mozilla AI video thumbnail

Why it matters

AI Engineer session on 2025 is the Year of Evals! Just like 2024, and 2023, and …, presented by John Dickerson, CEO Mozilla AI. It adds practical context for how teams are building and operating AI systems in production.

My takeaway: 2025 is the Year of Evals! Just like 2024, and 2023, and … — John Dickerson, CEO Mozilla AI is a model-evaluation signal. The practical read is to tie capability claims to evidence, launch criteria, and regression tests rather than relying on demos or benchmark headlines.

Keep exploring

More curated notes connected through AI Engineering.

Google Cloud Security Blog · tool

Now in preview: Find and fix software vulnerabilities with CodeMender

Google opened a preview of CodeMender, an AI code-security agent delivered through Gemini Enterprise Agent Platform and AI Threat Defense. It is designed to inspect code, identify and validate potentially exploitable defects, and produce targeted fixes, with Google’s specialized Gemini 3.5 Flash Cyber model initially restricted to governments and trusted partners.

AWS Security Blog · guide

Authenticate legitimate AI agent traffic with AWS WAF Bot Control

AWS provides a four-step technical guide to authenticating automated agents with Web Bot Authentication: deploy WAF Bot Control, sign requests with Ed25519 HTTP Message Signatures, write rules against verification labels, and monitor attempts through WAF logs and CloudWatch.

Google DeepMind Blog · guide

Securing the future of AI agents

Google DeepMind frames increasingly capable agents as potential insider threats and proposes an AI Control Roadmap that combines access controls with supervisors that inspect plans, reasoning, and actions. Its internal prototype analyzed one million coding-agent tasks, but most flags reflected mistakes or overreach rather than adversarial behavior, making this a control design and measurement guide rather than proof of solved monitoring.