Day 4 · Lesson 1 — Agent Security & Evaluation
Security asks if it stayed in bounds. Evaluation asks if what it did was any good.
Days 1–3 built the system. Day 4 asks the question that decides whether you can ship it: can you trust it? Trust splits cleanly into two axes that must both hold, and for you specifically, the security half maps almost pillar-for-pillar onto the VERDICT framework.
From Day 1: what verifies the non-deterministic parts of an agent's behaviour?
Evals. Day 4 is the deep version of that idea, the full evaluation framework, paired with its twin, security. Together they are the two axes of trust.
In deterministic software, trust is binary. It compiles, tests pass, the token is valid. An agent shatters that: it can hold a valid token and still act with misaligned intent. So trust must be redefined across two distinct axes:1
SECURITY did the agent stay INSIDE the boundary? (safe, no malicious action)
EVALUATION is what happened INSIDE that boundary (quality, alignment,
actually worth shipping? "did it build what I meant?")Crucially, they're independent: a vibe-coded agent can pass every security check and still misread your intent, ignore conventions, or silently break a feature. Security proves no harm; evaluation proves value. You need both. This is the governance "readiness map" from Day 1, at the level of a single agent's trust.
The paper names the goal Effective Trust: a continuous metric earned, verified, and dynamically enforced on runtime context, rather than a gate passed once at deployment. Static identity is a poor perimeter; the model is "Context-as-a-Perimeter."1
Because the model itself might fail or be compromised, security can't live inside the AI. It's an external "safety envelope" across seven disciplines, each owned by a different role:1
| # | Pillar | Guards against |
|---|---|---|
| 1 | Infrastructure & Networking | container escape, exfiltration — ephemeral sandboxes, egress governance |
| 2 | Data | context leaks, poisoned RAG — CMEK/mTLS, least privilege, tenant partitioning |
| 3 | Model | semantic attacks — protect system instructions as attested artifacts |
| 4 | Application & Runtime | rogue tool use — LLM firewalls, hooks, Agent Gateways |
| 5 | Identity & Access | confused deputy — SPIFFE IDs, ABAC, JIT downscoping |
| 6 | Observability & SecOps | invisible failures, drift — Red/Blue/Green triad, circuit breakers |
| 7 | Governance | compliance, liability — audit trail, attestation, logic reviews |
This is the convergence I've flagged since Day 1. Google's 7-pillar model (defence-in-depth, by role) and your VERDICT model (a runtime governance operating model) describe the same trust surface from two angles:
| Google pillar | ≈ VERDICT |
|---|---|
| 1 Infra/Networking (sandbox, egress) | R Runtime Control — containment |
| 2 Data (least-priv, tenant partitioning) | C Compliance + I scoped access |
| 3 Model (protect instructions) | V Validation of inputs |
| 4 App/Runtime (firewall, hooks, gateway) | V + R gates & kill |
| 5 IAM (SPIFFE, ABAC, JIT) | I Identity |
| 6 Observability/SecOps (drift, breakers) | E Evidence + R Runtime Control |
| 7 Governance (audit, attestation, reviews) | T + D + C |
Not a clean 1:1. Google leads with where controls live (infra→governance); VERDICT leads with what is enforced at runtime. But every VERDICT pillar has a home here, and Google's "Effective Trust" is exactly VERDICT's thesis: trust is continuously earned, not granted once. Your framework and this whitepaper are describing the same L3-Controlled goal.
Hold both axes for every agent you ship: can it do harm? (security) and did it do the right thing well? (evaluation). Passing one is not passing both.
Assign the pillars, they map to real roles (infra, data, IAM, SecOps). Trust isn't one person's job; the seven owners are how "Effective Trust" gets staffed.
Adopt Context-as-a-Perimeter: retire identity-as-a-gate for continuously-earned trust. Whether you call it the 7 pillars or VERDICT, it's the same L3 governance climb.
Recall, don't re-read.
Security tells you the agent stayed in bounds. Evaluation tells you —
Two independent axes. An agent can be perfectly secure and still misread intent or break a feature. Security = no harm; evaluation = real value.
"Effective Trust" means trust is —
A continuous metric across supply chain, identity, runtime behaviour, and context, rather than a one-time gate. This is VERDICT's "prevents AI from going unchecked," in Google's words.
Google's Pillar 5 (IAM: SPIFFE, ABAC, JIT) corresponds most to which VERDICT pillar?
Identity (I): cryptographic per-agent IDs and just-in-time scoped access are exactly how VERDICT closes the confused-deputy gap from Day 2.
Take one production agent and score it on both axes honestly: which of the 7 pillars are actually in place, and do you have any evaluation of whether it does the right thing? Most agents are strong on one axis and blind on the other. That blind spot is your Day-4 backlog.
Kartakis, Eidelman, Bakkali & Subasioglu, Vibe Coding Agent Security and Evaluation (Day 4) is the source; the Mandiant AI risk report grounds the threat landscape.
Up next → Day 4 · Lesson 2: Securing the Vibe Loop. Pillars 1 & 4 up close: ephemeral sandboxes, the slopsquatting supply-chain attack, egress governance, and the app-logic flaws vibe coding produces by default.
Related: The Governance Layer (VERDICT) · Day 1 L1: the dial · Course home