Day 4 · Lesson 2 — Agent Security & Evaluation

Securing the Vibe Loop

The agent writes untrusted code and runs it. Contain the blast radius.

Vibe coding's engine is a loop: the agent writes a script, runs it, reads the error, rewrites. That generative churn means the code cannot be implicitly trusted, so Pillars 1 & 4 wrap it in containment, a hardened supply chain, and a hard truth about the code it produces: it compiles, and it's still often wide open.

Recall first (spacing)

From Lesson 1: security's job is to prove the agent —

Built something valuable
Used the cheapest model
Stayed inside the boundary

Stayed in bounds, and did no harm. This lesson is the first, most physical boundary: where the agent's code actually runs, and what it's allowed to reach.

Sandbox the loop (Pillar 1)

Any skill-generated code executes first in an ephemeral, network-isolated sandbox, a hardened container, VM, or kernel-level environment (e.g. gVisor). Two non-negotiables: it blocks raw host access, and it resets state between runs. So even a container-escape attempt or a severe vulnerability can't persist or touch the host while the agent iterates.¹ The sandbox embodies a working assumption: all generated code is untrusted until proven otherwise, not just the known-bad kind.

Defend the supply chain: slopsquatting

A supply-chain attack unique to agentic coding: LLMs frequently hallucinate packages that don't exist. Attackers monitor for these hallucinated names and pre-publish malware under them ("slopsquatting"). Because an autonomous agent can alter the dependency graph without human confirmation, one hallucination pulls malware straight into the build.¹

DEFENCES
  · source deps ONLY from vetted / internal registries
  · cryptographic version pinning
  · CI/CD verifies SBOM entries + digital signatures (Binary Authorisation gate)
  · egress: non-interactive access only — offline caches / sanitised crawlers,
    never let the agent touch a live payload directly

Note the egress point: a domain allowlist is not enough, because it can't stop an indirect prompt injection hidden inside an allowed third-party page.

The uncomfortable truth about vibe-coded apps (Pillar 4)

Vibe coding prioritises "it works" over "it's safe," so generated apps fail two predictable ways:¹

Trusts the browser too much — dumps API keys, password checks, and session flags into the client. Anyone with dev-tools open reads the credentials or flips their own access.
Leaves the backend wide open — spins up a DB or admin dashboard but skips default-deny controls; row-level security is missing, private data is on the public internet.

And the trap: users trust it because it compiles, the "It Works, Ship It" fallacy. Clean compile, gaping hole.

Where to enforce: IDE advises, CI/CD enforces

The balance that keeps velocity and safety:¹ hard-blocking insecure prompts in the IDE is easily bypassed and just creates friction. So shift left as advice, enforce hard in CI/CD: developer-advisory linters give real-time IDE guidance, while deterministic SAST + SCA gates in the pipeline are the unyielding checks generated code must pass before production.

This is Day 1's dial made literal: the IDE is the vibe-friendly advisory zone; the CI/CD gate is where agentic-engineering discipline is enforced.

The governance layer, on containment

Governance overlay · Day 4 Lesson 2

Sandbox + supply chain = the Runtime Control and Validation pillars, physically

The ephemeral sandbox is R Runtime Control made concrete. It caps the blast radius of any single run. SBOM/signature verification and version pinning are V Validation gates on what enters the build. Egress governance is C Compliance on where data can go. And note the split-enforcement pattern: advisory-in-IDE + deterministic-in-CI is exactly the Day 2 lesson: never rely on a control the agent (or developer) can talk its way past; put the hard gate where it can't.

Ladder read: "run generated code straight on the host" is L1 recklessness; ephemeral sandbox + Binary Authorisation + CI gates is the L3 Controlled baseline.

Through your three lenses

Individual (IC)

Run agent-generated code in a throwaway sandbox, never on your host. Pin dependencies and pull only from vetted registries, assume any package name the model invents could be malware.

Team

Advisory linters in the IDE, deterministic SAST/SCA gates in CI. Don't fight the velocity in the editor; enforce the floor at the pipeline where it can't be bypassed.

Organisation

Mandate Binary Authorisation (SBOM + signatures) and non-interactive egress as platform policy. Slopsquatting and client-side-secret leaks are systemic risks, not per-project ones.

Check the reflex

Recall, don't re-read.

"Slopsquatting" exploits the fact that LLMs —

Leak API keys in logs
Hallucinate package names
Run in shared sandboxes

Attackers pre-publish malware under hallucinated dependency names; an autonomous agent pulls it in without confirmation. Defend with vetted registries, pinning, and SBOM/signature gates.

The most predictable flaw in a vibe-coded app is —

Slow database queries
Overly strict access control
Secrets and logic on the client side

AI takes the path of least resistance: API keys, password checks, and session flags dumped into the browser, plus default-deny controls skipped on the backend. Compiles fine; wide open.

Where should hard security enforcement live?

Deterministic CI/CD gates
Blocking prompts in the IDE
The model's own judgement

IDE blocks are bypassable and cause friction, so keep them advisory. Enforce with deterministic SAST/SCA in the pipeline, where the agent can't route around it.

Carry this into the week

Check one agent that writes code: does its output run in a sandbox or on a real host? Does anything verify its dependencies before build? If the answers are "host" and "nothing," you have both a container-escape and a slopsquatting exposure, the two most concrete Day-4 risks.

Go deeper (primary source)

Vibe Coding Agent Security and Evaluation (Day 4), "Sandboxes and Supply Chain Defence" & "Securing Application Logic" pp.13–17; Wiz's secure vibe-coding guide is the slopsquatting source.

Notes

Kartakis et al., Vibe Coding Agent Security and Evaluation (Day 4), Pillars 1 & 4 pp.13–17 (ephemeral sandboxing; hallucinated packages / slopsquatting; egress; application vulnerabilities; IDE vs CI/CD enforcement).

Up next → Day 4 · Lesson 3: Identity & High-Stakes Actions. The confused deputy returns (from Day 2), plus zero ambient authority, JIT downscoping, and the "Vibe Diff" that stops you rubber-stamping code you don't understand.