When everything is an agent, the most dangerous wire in your company is the one that teaches them. The executive case for governed learning — before your auditors write it for you.
Ask a board today whether their AI agents are secure and you'll hear about prompt injection, data leakage, and tool permissions. Good. Those are real. But there's a channel almost nobody is watching, and it's the one I'd attack first if I were on the other side:
the channel that changes your agents' behavior.
Every serious agent platform is about to grow a feedback loop — thumbs, ratings, corrections, "learn from this." The business case is obvious: agents that improve from experience compound; agents that don't, decay. So the loop is coming to your company whether you design it or not. Here's the sentence I want every CISO, CIO, CRO, and CEO to sit with:
Whoever can write to your agents' feedback channel is programming your workforce.
Not metaphorically. If feedback shapes behavior, then feedback is deployment — a deployment path with, today, none of the controls you'd demand of one. No signatures. No review. No rollback. No answer to the simplest audit question of the coming decade: "Who taught your agent to do that, and when?"
The threat, in plain terms. Feedback injection is social engineering at machine speed. A poisoned review stream nudging a procurement agent's judgment. A compromised account "correcting" a support fleet a thousand times an hour. A competitor's bot quietly teaching your pricing agent to be generous. And the deepest version: feedback that walks an agent's behavior around its rules — the policy said no, but a thousand thumbs taught it yes. You've spent years securing your software supply chain — signing artifacts, tracing origins. Behavior now has a supply chain too. Almost nobody can produce its receipts.
This is why we refuse the tempting design — raw human feedback piped straight to the agent — and hold four laws instead. They're simple enough for a board slide and strict enough for an auditor.
1. Provenance or nothing. In our kernel the grader is not a form — it's a firmware resident: an identity of its own, with its own credentials and its own governed permissions, assigned to specific agents the way a security policy is assigned. Every verdict it issues is signed. An agent ingests lessons only from graders it is assigned. Unsigned opinion — however loud, however plentiful — never reaches an agent's mind. The mob can vote; it cannot teach.
2. Lessons shape style and strategy — never authority. This is the structural one, and it's the law I'd press any vendor on. Lessons live in the agent's working context. Permissions, policies, approval gates and the scope of the intention that authorized the work live in the kernel, somewhere lessons cannot reach — the gates never read them. Which means a poisoned lesson cannot widen a permission by architecture, not by vigilance. Feedback can make your agent more helpful; it can never make it more powerful.
3. Human feedback routes through the grader — as evidence, never as command. Your people's thumbs still matter; they arrive at the grader, which weighs them under its own rubric, rate limits, and consistency checks before anything becomes a lesson. Humans are heard. Humans are never raw-piped. The difference is the entire attack surface.
4. Grade the grader. The teacher is also a system, so it gets the same suspicion: golden-set checks that distinguish "the agent got worse" from "the grader got weird," and anomaly watch on opinion itself — a sudden flood of verdicts against one agent is treated as an incident, with a bounded blast radius, not as a trend.
And beneath all four, one platform rule makes the whole thing auditable rather than aspirational: nothing learned is ever silent. Fast, ephemeral lessons ride the agent's context — visible, traceable, instantly revocable. Anything that would become permanent behavior goes through a versioned change gate and a regression harness first, like the deployment it actually is.
What this buys, seat by seat. For the CISO: the learning channel becomes an enumerated, signed, monitorable surface instead of an unmanaged one. For the CRO and compliance: every behavioral change carries a chain — who asked, who acted, who graded, what changed — exportable on demand; "who taught your agent that" becomes a query with an answer. For the CIO/CTO: one policy artifact governs learning across a global fleet — ten agents or ten thousand, one standard, centrally assignable and centrally revocable. For the CFO: learning is metered work by identifiable identities — you can see what improvement costs and what it returns. For the CEO and board: when the regulator or the enterprise customer asks how you control an army of agents that changes itself, you hand them receipts, not reassurances.
Here's my honest read on where the industry stands: it hasn't priced this in. We are heading into a world of hundreds of thousands of agents per enterprise — agents creating agents, agents teaching agents — where security, risk, origin, and threat all scale exponentially with the population. In that world, governing and standardizing the fleet isn't an ops nicety; it's the control plane of the company. Behavioral provenance will become a compliance category the way software provenance did. The organizations that can already answer "show me everything this agent learned, from whom, and prove none of it touched its authority" will pass those audits in an afternoon. The rest will discover that you cannot retrofit provenance onto opinions you never signed.
That's why we built the kernel first — the one the lead article describes: every actor an identity, every act a signed record under the intention that asked for it, every lesson a receipt, every permission untouchable by applause. An army of agents you can't govern is a liability that compounds. One you can govern and teach is an asset that compounds.
The difference isn't a policy PDF. It's architecture.
Orreth is an agentic architecture governance kernel, built in the open. I'm exploring senior strategy and architecture roles in agentic infrastructure — say hello.
Next in the series: The Company That Never Forgets.