The Methodology

The 7 Control Layers Every AI Deployment Must Rest On

The model is one piece of the agent economy. The infrastructure companies — the ones that decide where the agent runs, who it's acting for, what it can know, what it can spend, and who can stop it — those are the ones that determine whether your AI agent ships to production.

"Most teams have a model picked out before they've answered a single infrastructure question. And then they hit a ceiling they didn't know was there — not because the model failed, but because nobody decided where the agent lives, who authorized it, or what happens when someone needs to stop it."
📍

Where Is It Placed?

Is this a permanent fixture in production, a temporary campaign that runs for weeks, or a prototype that may graduate? The runtime decision shapes everything downstream.

🏛️

Does the Base Support More?

Will additional statues be added later? Is this single-tenant or multi-tenant? Can the identity layer isolate Client X from Client Y? Build the foundation for where you're going, not where you are.

🪨

Is the Pillar Durable?

If the model fails, does the system degrade gracefully? If a tool API returns 503, does the agent loop or escalate? Can a human stop it at multiple layers, or does "stop" just mean telling the model to stop?

The Seven Layers

For every AI deployment — production, demo, or campaign — answer these before a single line of code. A TBD on any row means the system has no foundation at that load point.

1

Runtime

Where does the agent live?
Execution
Docker container? Serverless edge (Cloudflare Workers)? Self-hosted VPS? The runtime determines latency, cost, and what tools are available.
State & Memory
A model is stateless. An agent needs durable memory — conversation history, guest preferences, booking state — that survives disconnects and wake-ups.
Scheduling
Can it run on a timer? Wake from idle? Recover from a crash mid-task? Scheduled work separates a production agent from a chatbot.
Tool Execution
How are tools called? Timeout handling? Retry logic? What happens when a tool fails halfway through a workflow?
Cloudflare Agents SDK · AWS Bedrock Agent Core · Vercel AI Gateway · Self-hosted Docker
My approach: Docker on VPS with Traefik routing. Hermes :8642, Paperclip :3100, Agent Zero :50080. PostgreSQL for state. FAISS for vector memory.
2

Identity

Who is the agent acting for?
Principal
Is the agent acting as a specific user? The company? Another agent? Fuzzy authority is the #1 source of agent incidents in production.
Delegated Authority
Agents should not get broad permanent credentials. Instead: scoped, time-limited, revocable tokens with explicit consent for sensitive operations.
Cross-System Auth
Does the agent span Google, Slack, GitHub, Salesforce? With what tokens? Who manages the credential lifecycle across systems?
Async Approval
Can a human approve while away? Is there a time-bound escalation? What happens if the human doesn't respond in time?
Okta/Auth0 · Microsoft Entra · WorkOS · AWS Agent Core Identity
My approach: Token-gated client dashboards with per-client isolation. Consent registry — scope-bound, withdrawal-enabled. Every email stays in Drafts until a human presses send.
3

Data

What can the agent know?
Governance Perimeter
Does the agent query INSIDE the governance boundary or AROUND it? An agent that retrieves ungoverned documents will eventually answer from stale or restricted data.
Semantic Layer
What is "revenue"? Which customer hierarchy is authoritative? Agents make the meaning-making layer more important, not less. Conflicting definitions produce conflicting outputs.
Access Control
RAG queries must only retrieve documents the user is authorized to see. An agent that can't tell public docs from confidential commitments shouldn't answer support questions.
Freshness
How current is the data? An agent answering from a knowledge base that hasn't been updated since launch is not an agent — it's a guessing machine with confidence.
Snowflake Cortex Agents · Databricks Mosaic AI · BigQuery + Gemini
My approach: PostgreSQL for structured records, FAISS for vector memory, InfraNodus for knowledge graph analysis. RAG queries are permission-scoped. No agent gets unrestricted data access.
4

Tools & Approval

What can the agent change?
Read Access
Which systems can the agent read from? CRM? Calendar? Knowledge base? Define the read perimeter explicitly — agents are creative at finding data you didn't intend them to access.
Write Access
Which systems can the agent write to? Never grant write access to systems the agent hasn't proven it can read correctly. Write access should always lag behind read access by at least one validation cycle.
Approval Gates
Which actions require human sign-off? The gate is not a bottleneck — it's a dial. Tight at launch, loosened as trust builds, but never removed entirely.
Scope Boundaries
Can the agent exceed the original request? A support agent that starts modifying billing records is a security incident. Define what "in scope" means and enforce it at the tool layer.
Framework-level enforcement · Tied to identity layer · HITL middleware
My approach: AI reads from CRM, calendar, knowledge base. AI drafts responses — emails, reviews, proposals. AI phones patients via Vapi (Module 2). AI NEVER sends, publishes, or transacts without human approval. Every write action logs to the SHA-256 audit trail.
5

Payment

What can the agent spend?
Transaction Limits
Maximum per-action spend? Per-day cap? Per-category limit? An agent with no spend limits is a financial control failure waiting to happen.
Approval Thresholds
Above what amount is human approval required? Set the threshold low at launch, raise it as the system proves itself. The threshold is a policy decision, not a technical one.
Fraud Prevention
How are unauthorized transactions detected and blocked? The payment networks (Visa, Mastercard) are building agent-specific fraud detection. Your payment layer needs to integrate with theirs.
Payment Rails
Stripe Agent Commerce Suite is the default for most startups. Enterprises may have their own stack. The key decision: build your own agentic payment layer or partner with operators who have.
Stripe Agent Commerce Suite · Visa/Mastercard agent rails · Custom payment stacks
My approach: Zero agent-initiated payments for our own operations. For client deployments: Stripe integration with hard limits. Payment freeze as a kill switch layer — if the agent needs to stop, payment stops first.
6

Observability

What gets traced?
Agent Runs as Work
Trace agent runs as work units, not API traffic. What was the goal? Which tools were called? Who authorized the action? Did a human accept the result?
Cost Tracking
Tokens consumed? API calls made? Cost per business decision? If you can't measure cost per decision, you can't justify the system to the CFO.
Failure Patterns
Agents fail differently from ordinary software. They call the right tool with the wrong question. They complete a task technically while violating the user's intent. Logs alone don't catch these.
Independent Perspective
Do not rely on agent self-reporting. Have an independent evaluation layer that tells you whether the agent actually got the job done correctly.
Datadog LLM Observability · LangSmith · Langfuse · BrainTrust · AWS Agent Core (OpenTelemetry)
My approach: SHA-256 hash-chained audit trail on every decision. Campaign tracking wired into guest memory. Uptime Kuma health monitoring on all endpoints. Independent evaluation — never self-reported.
7

Kill Switch

Who can stop it?
Runtime Kill
Can the run be cancelled or paused? At the orchestration layer — not by asking the model nicely to stop.
Identity Kill
Can the credential be revoked? Instantly, across all systems, with a single action. If it takes three different admin panels, you don't have an identity kill switch.
Payment Kill
Can the payment instrument be frozen? Can a spending limit be hit mid-transaction? Payment is the most important kill switch — stop the money, stop the damage.
Framework Kill
Can the workflow be interrupted before a sensitive node? A framework like LangGraph can pause execution before a dangerous action. The kill switch must work at MULTIPLE layers — not just one.
Critical rule: If the only way to stop the agent is to tell the model "stop," you don't have a kill switch.
My approach: HITL gates at every sensitive action. Deploy gateway with secret-key auth. Consent withdrawal instantly revokes data access. Cron watchdog auto-restarts failed services. Kill at runtime, identity, payment, and framework layers.

The Methodology Has a Name

Andrej Karpathy — the engineer behind Tesla Autopilot and OpenAI's founding team — formalized the four principles AI coding agents must follow to avoid predictable failure. The same four principles govern every AI implementation I build. Same discipline, different surface — codebases for him, business operations for me.

Karpathy's Rule
1. Think Before Coding
Don't assume. Surface tradeoffs. Stop when confused.
My Deployment
Paid Audit First
I map every workflow, system, handoff, and exception before a single module is built.
Karpathy's Rule
2. Simplicity First
Minimum code. No features beyond what was asked. No speculation.
My Deployment
Module 1. Nothing Else.
One function. One department. One metric. Proves itself before Module 2 begins.
Karpathy's Rule
3. Surgical Changes
Touch only what you must. Every changed line must trace to the request.
My Deployment
HITL Gates Everywhere
AI reads. AI drafts. Human approves. Audit trail logs. Never touches unasked.
Karpathy's Rule
4. Goal-Driven Execution
Define success criteria. Loop until verified. No "make it work."
My Deployment
Measurable, Not "Engagement"
Time saved per FTE. Error reduction. Compliance adherence. Audit pass rate.

Read the full post →  ·  View the Karpathy Guidelines repo →  ·  141k stars, 14.5k forks

How to Use This Framework

  1. 1. Pick a specific agent workflow — a support refund agent, a customer email agent, a claims agent, or an agent that reads conversations and checks usage.
  2. 2. Run it through the seven questions above. Where does it run? Who is it acting for? What can it know? What can it change? What can it spend? What gets observed? Who can stop it?
  3. 3. Mark any TBD rows. Those are your pre-production blockers. Do not deploy until every row has an answer.
  4. 4. Assign owners for each layer. Agents do not respect org charts. If nobody owns the identity layer, the identity layer never gets solved.
  5. 5. Deploy the first module. Measure. Validate. Only then start the next.

This Is How I Build

Every AI implementation I build starts with these seven questions. No exceptions. No skipping layers. No TBDs in production.

Work With Me Read the Blog