Deep Engineering #57: Rory Preddy on Cutting Agent Costs Without Ranking Engineers
Agent profiles as shared infrastructure, spend caps inside the authorization boundary, and the metric that replaces tokens burned
Featured - Engineering Reliable Agentic AI Systems
Building an agent is easy. Building one that survives production takes architecture, verification, and stopping conditions that actually stop. Four hands-on hours with Rick Hightower on loop engineering, evaluation harnesses, context efficiency, and safe MCP tool integration, live on 29 August.
Deep Engineering readers save 40% with code - DEEPENG40.
✍️ From the editor’s desk,
Welcome to the 57th issue of Deep Engineering!
On 28 July, GitHub’s changelog recorded a change that sounds administrative and is not. Individual Copilot app activity is now attributed to users in the enterprise-user and organization-user reports, with a per-user section reporting session counts, request counts, prompt counts, and a token usage breakdown covering output tokens, prompt tokens, and average tokens per request. Per-engineer token attribution is now a REST call against an API most enterprise administrators already query.
Nothing about that is objectionable on its own, because a team that cannot see its consumption cannot govern it. The risk lives in what gets built next, since the shortest path from per-user token data to a management artifact is a ranking. Amazon and Meta both stood up internal leaderboards ranking engineers by tokens burned, then dismantled them within a quarter once engineers began padding their usage, which we covered in our special issue on token maxxing. The visibility has now improved considerably, and the temptation has improved with it.
Rory Preddy, AI Advocate in Developer Relations at Microsoft and GitHub, worked five years in cloud advocacy watching the same question mature from how much will we spend into how much can we save, and he argues the answer was never individual behaviour. Today’s issue comes out of our live session, where his position was that cost discipline belongs in the artifacts a team inherits rather than in anything resembling a performance conversation.
You can also read this practical deep dive on the mechanics of token efficiency by Preddy.
Let’s get started.
Agent-written TLA+
An agent wrote our TLA+ spec. The model checker explored 14.3M states and caught a real race.
Expert Insight
Cost discipline belongs in the agent profile, not the performance review
by Saqib Jan with Rory Preddy
The logic behind a token leaderboard feels right from a distance, because agents produce measurable volume, volume varies between engineers, and variance invites comparison. Rory Preddy, AI Advocate in Developer Relations at Microsoft and GitHub, has observed this sequence play out once before in cloud, and his position is simply that “Rather than spend more, spend less.”
Coming from somebody whose job is persuading developers to adopt AI, that is a more interesting claim than it would be from a finance team, and the reasoning underneath it is what makes it useful. Preddy is not arguing for austerity or for slowing teams down. He is arguing that cost discipline is a property of the artifacts a team inherits rather than a property of the people using them, which relocates the problem from individual behavior to engineering infrastructure. For any leader currently deciding what belongs on an AI dashboard, that distinction determines whether the dashboard helps or quietly makes things worse.
Spending less is an engineering target, not a budget instruction
Preddy formed this view during five years as a cloud advocate, in rooms with the CTOs of banks whose balance sheets run larger than some national economies. He describes asking those CTOs what worried them most, and the answer changed shape over time in a way worth attending to. Early on they wanted to know how much they were going to spend, because a four-year hardware forecast let them buy servers and depreciate the cost against a schedule they controlled. Later the question inverted entirely, and what they wanted to know was how much they could save.
That inversion is the part that transfers to tokens, and it transfers with one difference that makes it more urgent rather than less. A server that gets over-provisioned wastes capital on a predictable schedule, and the waste appears on a depreciation line somebody reviews. An agent loop that resends its context on every iteration wastes money continuously, invisibly, and at a rate nobody set, because the cost arrives as consumption rather than as a purchase. Cloud waste taught organizations to rightsize compute against a bill they could at least forecast. Token waste removes the forecast.
Preddy’s instruction to teams starting now runs from the floor upward, so his advice is to look for “what’s the minimum and then cost save, not maximum.” The engineering content of that instruction matters more than its thrift. A team that starts at the largest model with the longest context has no baseline against which to measure any later improvement, so it cannot tell whether a change helped. A team that starts at the smallest model completing the work has a floor, a known cost, and a reason to escalate when the work genuinely demands it. The cheap path becomes the measured default rather than the fallback nobody trusts.
Discipline is arriving as a function, not a phase
That parallel with cloud is not incidental to Preddy’s argument, and he expects the rest of the pattern to follow too. “There’s a whole industry out there just for token efficiency,” he says, pointing to engineers who wrote good deployment scripts, became cloud engineers, and then watched cost optimization separate into its own discipline with its own titles and its own tooling. He puts it more flatly when he calls it “This is our industry.”
The institutional scaffolding has arrived faster this time than it did for cloud. The FinOps Foundation’s State of FinOps 2026 reports that 98 percent of practitioners now manage AI spend, up from 31 percent two years ago, and it ranks FinOps for AI as the top forward-looking priority with AI cost management as the single skillset teams most need to develop. In June the Linux Foundation announced its intent to launch the Tokenomics Foundation, working with the FinOps Foundation on open industry standards, benchmarks, and best practices for the economics of AI infrastructure. Preddy reads that as the point where dashboards, standards, and protection mechanisms stop being individual good habits and become things an organization can be held to.
For engineering leaders the timing carries a specific implication. A function forming now will settle its conventions within a few quarters, and the organizations contributing to those conventions will be the ones already treating agent cost as an engineering concern rather than a procurement one. Teams still running enablement sessions about prompt length will inherit whatever standards other people write.
Cost advice without a budget produces a cheaper unbounded system
Before any of that scaffolding helps, a team has to answer a question most cost guidance skips. The sharpest moment in our conversation came when Preddy declined the premise of a question about authorization boundaries and asked something more basic in its place. “How much money do I have?” he said, and then named the assumption he keeps encountering, because “a lot of what you’re asking me assumes that you have infinite amount of tokens.”
He is identifying a real gap in how the industry discusses this. Almost all published cost guidance optimizes mechanics, so prompts get shorter, retrieval gets tuned, caches get warmed, and routing gets layered in. Very little of it starts by asking what the workload is permitted to cost. Optimization applied to an unbounded system produces a cheaper unbounded system, which still has no ceiling and still fails in the same direction. The budget has to exist before the tuning means anything, because the budget is what converts an optimization into a decision.
What makes this harder than it looks is that the system cannot supply the number for you. A study published through Microsoft Research this spring, How Do AI Agents Spend Your Money?, presents the first systematic analysis of token consumption in agentic coding tasks, and it found that runs on the same task can differ by up to 30 times in total tokens, that input rather than output tokens drive the overall cost, and that frontier models fail to predict their own token usage and systematically underestimate what a task will cost. A workload whose executor cannot estimate it will not be estimated reliably by a spreadsheet either. That finding closes off the approach most organizations reach for first, which is forecasting from measured averages, and it pushes the answer toward setting a limit and enforcing it rather than predicting a figure and hoping.
The consequences of skipping that step have landed at named companies. Forrester’s July analysis of AI cost management cites Uber burning its AI budget in four months, Microsoft ending Claude Code licenses after also burning its yearly AI budget, Tesla limiting AI spending to 200 dollars per week, and Priceline absorbing an unexpected surge in AI development renewal costs. None of those reads as an engineering failure. Each one reads as a missing number.
Foundation is an artifact leaders ship
Setting the number is the first move, and the second is deciding who builds the thing that keeps teams inside it. That is where Preddy’s position turns into a leadership argument rather than a productivity tip. He reaches for the idea of a cybernetic teammate, an agent working alongside a team rather than replacing anyone, and then reframes it as something an engineering leader produces rather than something a vendor sells. The artifact is the agent profile, along with the instructions file and the container definition that travel with it, and his claim about ownership is direct. “Your leaders should go in and create the agent profiles,” he says.
His reasoning turns on availability rather than authority, and it is the more humane version of the argument. Not every engineer works somewhere they can walk to a senior colleague and admit they are burning too many tokens, and not every senior colleague is reachable at the moment the question arrives. Preddy frames the profile as the answer to that absence, describing the message it carries as “I’m not around to help you out, but I’ve created a nice agent profile” with the conventions already encoded. A profile that declares only the tools a task needs, names the model tier, and sets the compaction expectation makes the disciplined path the default path for everybody who inherits it, including the engineer who joined last week and has nobody to ask.
The second-order effect is what leaders should weigh most carefully, because it determines which of two very different programs they end up running. Token waste treated as individual behavior produces enablement sessions, then dashboards, then comparisons, then rankings, and each step follows plausibly from the last. Token waste treated as a missing artifact produces a file one person writes once and every engineer benefits from without thinking about it. The first approach asks engineers to carry knowledge that changes every quarter. The second encodes that knowledge where it survives staff turnover and stops depending on who attended which session.
One detail in Preddy’s own workflow reinforces the point. He builds these files using the cheapest model available with reasoning switched off, which means the foundation costs almost nothing to produce. The barrier to doing this properly is not budget or tooling. It is that nobody has been made accountable for it.
A spend cap belongs with the permissions
The foundation shapes the default, and a cap is what holds when the default gets overridden. Engineering organizations already accept that an agent’s authority needs governing, so they scope which systems it may read, which it may write to, and which credentials it holds. Preddy’s contribution is to place money in that same category rather than treating it as a separate financial concern reviewed on a different cycle by different people. A per-session budget that warns at a threshold and stops at a limit behaves exactly like any other permission, because it constrains what the agent may do without a human present.
The practical consequence is a change in when the decision gets made and by whom. Treated as finance, a spend cap arrives quarterly, applies at the account level, and reaches the engineer as a rate limit they cannot explain. Treated as authorization, it arrives at the same review where the agent’s other permissions get set, applies at the session level, and reaches the engineer as a known boundary they helped define. Preddy demonstrated the enforced version during our session, where a run halted after crossing its allocated credits and offered the operator a choice to add credits, raise the ceiling, or remove it entirely.
That halt is the whole value, and it is worth being precise about why. An agent meeting a wall has produced a decision point, with a human present, a specific task in view, and the cost of continuing visible on screen. An agent with no wall has produced an invoice, arriving weeks later, aggregated across teams, with nobody able to reconstruct which run caused it. The same money moves in both cases. Only one of them leaves an organization able to govern the next run.
Token counts measure activity rather than relief
A budget and a cap tell you when to stop. Neither tells you whether the spending accomplished anything, which is the question a dashboard is supposed to answer. A contributor to our earlier issue on token maxxing offered a formulation Preddy endorsed when we put it to him, that token usage behaves like CPU utilization, so the honest unit is the cost of relieving a bottleneck rather than the volume consumed. He agreed, then complicated it usefully by pointing out that the model chosen to relieve the bottleneck changes the answer, which means the metric has to account for the routing decision rather than treating all tokens as equivalent.
The research supports him on the underlying point. The same Microsoft Research analysis found that higher token usage does not translate into higher accuracy, and that accuracy often peaks at intermediate cost before saturating as spending rises. A number that rises while the outcome does not is the definition of a metric worth distrusting.
Preddy’s own experience supplies the rest, and it is more persuasive because it is self-implicating. He assigned a merge conflict to the smallest model available, and it consumed 51 million tokens without resolving anything, a run we cover in detail in his companion deep dive. Read as a token count, that looks like heavy engagement from a productive engineer. Read as bottleneck relieved, it produced nothing whatsoever. Any metric unable to distinguish between those two readings will reward the wrong behavior from the day somebody starts reporting it upward, and it will do so most strongly for the engineers who least understand what they are doing.
This is also why the leaderboard fails on its own terms rather than only on cultural ones. A ranking by tokens consumed can be improved by consuming more tokens, which requires no additional value and no additional skill. A measure built on work closed, so a merge conflict resolved, an accessibility fix shipped, a suite returned to green, can only be improved by closing more work. The second measure is harder to define and harder to collect, which is exactly why organizations reach for the first. Preddy’s argument is that the difficulty is the job, because the easy number and the useful number have never been further apart than they are now.
The principle holding it together
What makes Preddy’s approach more than a list of preferences is the single idea running through every part of it, which is that cost control belongs in the system rather than in the person operating it. Every choice he describes follows from that.
The foundation is a file rather than a training session, because a file survives turnover and a session does not. The spend cap is a permission rather than a budget line, because a permission gets enforced at the moment of use and a budget line gets reviewed after the fact. The metric is work closed rather than tokens consumed, because an outcome cannot be inflated by burning more of the input. The smallest viable model is the default rather than the exception, because a default is what happens when nobody is paying attention, and most of the cost accrues exactly then.
That principle also explains what Preddy leaves out. He never asks engineers to be more careful, and he never suggests awareness alone fixes anything, even while arguing for visibility. His own session history showed input tokens outweighing output roughly 100 to 1, and one session climbing from 49,000 to 174,000 input tokens across 66 turns with no compaction at all. He describes himself falling into bad practices like everybody else. A discipline depending on the operator remembering will fail for the person who wrote the discipline, which is the strongest available argument for encoding it somewhere else.
What this asks of a leader
The sequence that follows is short enough to start this week. Write the agent profile and the instructions file yourself, or make one person accountable for them, and treat them as shared infrastructure reviewed like any other. Set a per-session budget in the same review where the agent’s permissions get set, rather than in a separate conversation with a different owner. Report the work closed rather than the tokens consumed, and decline to build the leaderboard even though the API now makes one trivial.
The deeper point for any leader standing up AI measurement is that visibility and ranking are not the same thing, and the tooling arriving now makes it very easy to confuse them. Per-user token data has legitimate uses, including chargeback, capacity planning, and helping a manager notice that a team is consuming heavily without much to show for it. It becomes destructive at the moment it gets used to compare engineers against each other, because that is the point at which the number starts changing the behavior it was meant to observe.
Preddy’s closing thought aimed at the people doing the work rather than the people measuring it, and it belongs here because burnout and cost discipline turn out to share a cause. His instruction was not to “end each day exhausted, not from the work itself, but from managing of the work.” An engineer who inherits a foundation somebody else built gives the day to the problem. An engineer who inherits nothing gives it to managing agents, and pays for the privilege in tokens. “Be kind to yourself,” he said, which for a leader translates into something concrete. Build the foundation, and there will be nothing worth ranking.
In case you missed
I burned 51 million tokens on one merge conflict, and the model was not the problem
By Rory Preddy - Cache the prefix, compact the context, route to the smallest model that fits, and put a ceiling on every loop
🛠️ Tool of the Week
Langfuse is an open source AI engineering platform that traces agent runs and attributes cost and token usage per trace. A spend cap needs a number behind it, and a per-trace cost is that number.
Traces multi-step agent runs so cost attaches to a workflow rather than to a raw API call
Tracks token usage and cost per trace, per model, and per user for chargeback and per-team budgets
Integrates with OpenTelemetry, LangChain, LiteLLM, and the OpenAI SDK, so routing across providers stays visible in one place
Self-hostable, which keeps prompt and completion data inside your own boundary
📎 Tech Briefs
Model Context Protocol ships its 2026-07-28 specification - Sessions and initialization are removed for stateless requests, with authorization and versioned extensions now explicit.
npm adds publish-time malware scanning and dual-use metadata - New packages now face pre-install scanning, metadata disclosure, and stricter publishing controls for security-sensitive capabilities.
GitHub Actions holds potentially malicious workflows for approval - Suspicious runs pause before execution, moving CI supply-chain defense ahead of the workflow rather than after.
Kubernetes v1.37 reaches code and test freeze - Code changes now require exceptions, shifting attention to release-blocking tests and regressions before August’s release.
Accenture launches a tokenomics practice for enterprise AI spend - Token spend gets tied to business outcomes, making AI cost governance a consulting line item.
📣 Contribute to Deep Engineering
Pitch a practical deep dive under your byline. Or if you lead a team, we would like to interview you and build an engineering leadership feature around your story.
Subscribe to Deep Engineering newsletter and message us through the chat option, or email us at saqibj @ packt.com.
That’s all for today. Thank you for reading this issue of Deep Engineering.
We’ll be back next week with more expert-led content.
Keep building,
Saqib Jan
Editor-in-Chief, Deep Engineering
If your company wants to reach senior developers, software engineers, and technical decision-makers, speak to us about partnering with Deep Engineering.