Join this interactive workshop to learn how to turn Claude Code from a session-by-session assistant into a repeatable engineering system, using structured context, reusable skills, scoped rules, hooks, and guardrails that work across real codebases and team workflows.

🗓️ Friday, June 20 · 10:30 AM EDT onwards

Use code DEEPENG50 for 50% off.

Register here

✍️ From the editor’s desk,

Welcome to the 52nd issue of Deep Engineering!

A recent study pulled apart the architecture of Claude Code and found that only 1.6 percent of the codebase is actual AI decision logic, while the remaining 98.4 percent is the deterministic infrastructure that surrounds the model, including the permission gates, the context management, the tool routing, and the recovery logic that keep the whole thing usable. The agent loop at the center of it turns out to be a simple while loop, which means the genuine engineering effort sits in the systems built around the model rather than in the model itself.

This analysis lines up almost exactly with what most engineers working with AI tools are discovering through daily practice, which is that the thing slowing them down is rarely the intelligence of the model and is far more often the fact that none of the context they carefully supply in one session survives into the next. Supplying that missing context has quietly become the engineer’s job, and it gets paid at the start of every single session without anyone ever counting it.

This week Sam Keen, an agentic engineering researcher, and former engineer at AWS and Nike, and the author of Clean Architecture with Python, shares a practical way to stop paying that cost for good. His piece walks through how to convert the context you re-explain on repeat into a system that compounds across sessions, using the mechanisms Claude Code already gives you, and how to recognize the moment that system starts quietly working against you rather than for you.

Let’s get started.

Featured Newsletter: Engineering At Scale

A weekly column that makes databases, system design, and architecture easy to follow, with clear explanations, practical insights, and career advice for engineers building at scale.

→ Subscribe to Engineering At Scale

The Hidden Cost of Starting From Scratch

Submitted by

When you open a fresh Claude Code session, the assistant knows nothing about your project. It does not know where your tests live, it does not know the patterns this codebase uses, and it does not remember the conventions you walked it through the day before. So you explain all of it again, and then you do the same thing again tomorrow.

That re-teaching is a tax, and it is the easiest one to overlook because it never shows up on any gauge, which means you pay it at the start of every session without ever once counting what it costs you.

The bottleneck isn’t the model

The real limiting factor in AI-assisted development right now is not how clever the model is, because the models are already more than capable enough for the work most teams are asking of them. The limiting factor is that none of that intelligence carries from one session into the next, and the job of supplying the missing context every single time has quietly been handed to you without anyone naming it as work.

Picture a senior engineer who forgets everything about your codebase overnight, every single night, and shows up the next morning brilliant and fast and genuinely helpful but starting again from absolute zero. You would not describe that person as a force multiplier, you would describe the arrangement as exhausting, and yet that is the default relationship most people have with their coding assistant. They end up blaming the model for the drag when the actual problem is that nothing they explained yesterday is still present today.

Write the context down once

The fix is to stop starting from scratch, which means writing the recurring context down once in a place the harness reads automatically so that it is present in every session without you having to lift a finger. Claude Code gives you several mechanisms for doing exactly this, and three of them are foundational. The most useful way to think about the three is by the specific kind of cost each one removes from your day.

Agent files, written as CLAUDE.md, are your project’s standing memory, the place where the conventions and the layout and the way things get done here all live. You write them once and they load into every session, so you stop re-explaining the project from the beginning each time. The part most people underuse is that these files load hierarchically, which means a personal file can ride along on every session on your machine, a broader file can cover all of your coding work, and a project-specific file can sit on top of both, each one layering onto the last. The thing to remember is to keep them lean, somewhere around a couple hundred lines each rather than letting them sprawl.

Skills capture the procedures you would otherwise walk through by hand every time, the multi-step moves where you first do one thing and then check another before continuing. A procedure you would normally re-explain becomes a procedure you simply invoke, and because a skill is not limited to instructions alone, it can bundle the scripts the agent runs, which means the repeatable move can carry real executable code rather than prose describing what the code should do.

Hooks handle the corrections you would otherwise find yourself repeating, the lint nit and the formatting rule and the check you keep having to ask for. They fire at fixed points in the harness lifecycle, before or after a tool runs for instance, and because they sit outside the model’s control they run every single time regardless of whether the model would have remembered to do them. You give the note once and then you never have to give it again.

The pattern underneath all three mechanisms is the same one, because each of them converts a recurring cost into a single one-time write. That is what compounding actually means in this context, and it is the direct opposite of starting from scratch, since the investment you make is small and the payback lands in every session that follows it.

Your setup can rot, here is how to catch it

There is an honest catch worth naming here, and it is the place where the /context command earns its keep, because a compounding system can quietly rot over time. You install a skill pack and then forget it is even there. A CLAUDE.md that started out tight slowly fills up with things that mattered once and no longer do. The investment turns into freight without you noticing, and freight is really just the from-scratch tax wearing a slightly nicer costume.

The /context command is how you tell the difference, and it does one genuinely useful thing, which is that it makes the invisible visible. One command gives you a colored grid and a per-category breakdown of exactly what you are carrying before the conversation has even started. You do not need to master it, you only need to glance at it often enough to notice when something is wrong.

The last time I ran it, the single biggest chunk of my standing context was not the project memory I had carefully written, it was 14.4k tokens of skill packs I had installed on a whim and then never used even once. I had assumed my context was quietly working in my favor, and a five-second look told me otherwise. Culling them took about a minute, using /skills to deactivate the ones I never reach for and /plugin to drop a whole pack that had arrived bundled with something else.

That same readout flagged a second problem with a different fix, because the CLAUDE.md in my working directory had grown to 3,500 tokens without my noticing. I sat down with Claude and compacted it, cutting the irrelevant and the quietly duplicated, and it came back at 1,200 tokens, which is the same project memory carried at roughly a third of the weight.

Clarity helps the agent for exactly the same reason it helps a human reader, because a bloated and half-contradictory CLAUDE.md does not only cost you tokens, it actively muddies the very instructions you are leaning on to get good work out of the model. A lean file is easier for the model to follow in the same way that a tight brief is easier for a colleague to follow.

The specific number you land on does not really matter, but the habit does, so skim /context the way you would skim a credit-card statement, not obsessively but often enough to catch the recurring charge you forgot you ever signed up for.

Do the upfront work once

Starting from scratch feels free because the cost is smeared so thin across every session you will ever run, but it is not actually free, and it is in fact one of the largest and quietest line items in the way you work.

So put in the upfront work of writing a lean memory file, building a few skills you genuinely use, and setting a couple of hooks that hold the line for you. Then keep curating it, because models change and your projects change, and a glance at /context now and then is what tells you which parts of your setup are still earning their place. The goal was never a clever prompt. The goal is a setup that already knows your project before you say a single word to it.

🛠️ Tool of the Week

Repomix — an open source tool that packs an entire repository into a single, AI-friendly file ready to feed to a coding agent, with token counting built in.

Highlights

• Packs a whole repository into a single structured file optimized for LLM consumption, removing the manual copy-paste that eats the start of every session.

• Counts tokens per file and for the whole pack, so you see what context costs before you spend it rather than after.

• Ships an official skill for Claude Code, Cursor, Codex, and Copilot, letting agents run Repomix directly inside the workflow.

Learn more about Repomix

📎 Tech Briefs

• Anthropic suspends Fable 5 and Mythos 5 worldwide - A US export control directive citing national security forced Anthropic to disable both frontier models for all customers, days after launch, with all other models unaffected.

• OpenAI Codex 0.140.0 ships - Codex adds Claude Code imports, unified mentions, and encrypted Amazon Bedrock API-key authentication.

• GitHub Copilot CLI 1.0.63 released - A new deferTools option reduces MCP context bloat when tool search is enabled.

• Xiaomi open-sources MiMo Code - Xiaomi’s terminal coding agent targets long tasks and offers free limited-time MiMo Auto access.

• ChatGPT adds memory summary controls - Users can delete memories, turn memory off, and directly correct their memory summary.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Keep building,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company wants to reach senior developers, software engineers, and technical decision-makers, speak to us about partnering with Deep Engineering.

Learn how to use Claude and Codex as development partners for building reliable Rust applications faster. This hands-on workshop with Francesco Ciulla shows how to scaffold, refactor, debug, test, and productionize AI-assisted Rust code with confidence.

Use code DEEPENG50 for 50% off.

Register here

✍️ From the editor’s desk,

Welcome to the 51st issue of Deep Engineering!

The Rust team released version 1.96.0 on May 28, shipping new Copy-compatible range types, stabilized assert macros, and Cargo security fixes. It is the kind of release that tells you something important about where the language is. Now on a steady six-week release cadence, Rust’s toolchain keeps getting more integrated, and each release moves the language further from its reputation as a difficult, specialist tool and closer to something engineering teams can simply rely on.

That maturity is part of what is changing the production calculus for engineering teams this year. Francesco Ciulla, author of The Rust Programming Handbook and head of developer relations at Zerops, has used both Rust and Go in production. His view on the Rust versus Go debate is neither dismissive of Go nor evangelistic about Rust.

Ciulla discussed how Rust and Go solve different backend problems, where Go still wins, where Rust’s flat latency and binary size arguments become genuinely decisive, and why his thinking on committing to Rust for a production backend has changed.

Today’s expert insights are based on the broader conversation about Rust adoption we had with Ciulla. You can read our previous issue or watch the full Q&A here.

Let’s get started.

Featured Newsletter: Java Tips and Tricks

Subscribe to Java Tips and Tricks on Substack for practical Java insights, modern Java features, software design discussions, and updates from the Java ecosystem.

→ Subscribe to Java Tips and Tricks

Expert Insights

Rust and Go Are Better Compared Service by Service

by Saqib Jan with Francesco Ciulla

Most engineering teams debating Rust versus Go for their next backend service are usually trying to answer the question too early, assuming the decision starts with the language rather than the service.

Francesco Ciulla, author of The Rust Programming Handbook and head of developer relations at Zerops, thinks that framing misses the point. The useful question is not whether Rust is better than Go, or whether Go is more practical than Rust. It is whether the specific service being built has constraints that make one language’s trade-offs more valuable than the other’s.

The distinction Ciulla draws is practical. Go still earns its place in cloud infrastructure, CLI tooling, hiring, and teams that need a service working quickly without introducing a new adoption burden. Rust, in his view, becomes much harder to ignore when a service is constrained by memory use, latency predictability, high-concurrency performance, or the need to remove whole classes of runtime failure from the system. That is why the Rust versus Go debate becomes less useful the longer it stays at the language level. The decision only starts to make sense when it moves down to the service level.

Most teams are asking the wrong question

Most engineers who have followed the Rust versus Go conversation online will have encountered it as a tribal argument, with advocates on both sides treating the choice as a matter of identity rather than engineering judgment. Ciulla rejects that framing, and part of what makes his view useful is that he is not arguing from a Rust-only position. He currently works with Go at Zerops, where the company’s CLI is written in Go, and he has run Rust services in production on his own projects.

“I would never say that Go is a bad programming language,” Ciulla says. “You can feel how powerful Rust is because we are talking about completely different scenarios and still Rust has something to say. But that does not make Go bad.”

The more useful framing, he argues, is to stop treating Rust versus Go as a general-purpose language comparison and start treating it as a service-level engineering decision. The question is not whether an organization should adopt Rust instead of Go. The question is whether a specific service in the system has properties where Rust’s trade-offs are worth paying for, or whether Go’s simplicity, ecosystem, and hiring advantages matter more.

That reframe changes the conversation because it moves the decision away from preference and toward evidence. Once the team is looking at the service rather than the language, the relevant questions become much more concrete. Is memory use a real constraint? Is tail latency a business problem? Does the service sit on a critical path? Does the team have anyone who can review Rust code well enough to put it into production safely? Without those questions, the debate quickly becomes ideology dressed up as architecture.

Go wins on hiring, tooling, and the Docker ecosystem

Ciulla is careful not to turn the comparison into an anti-Go argument. Go is a natural fit for CLI tools, cloud infrastructure tooling, and the Docker and Kubernetes ecosystem. Docker is written in Go and Kubernetes is built on Go. For teams building tools that live inside that ecosystem, Go is the default for reasons that have little to do with language preference and everything to do with integration, community, and the availability of patterns the team can learn from.

The hiring argument also runs in Go’s favor, and Ciulla thinks engineering leaders should be honest about it. “In terms of finding Go engineers, probably at the moment it is easier,” he notes, “because there are probably more of them.” For a company that needs to move quickly and cannot afford a long search for specialized Rust talent, that matters. Staffing is not separate from engineering judgment. It is one of the constraints that determines whether a technical choice can survive contact with production.

Go also has lower adoption friction in general-purpose backend contexts where the performance ceiling is not the binding constraint. If a team is building a service that needs to be working by the end of the week, deployed reliably, and understood by the next engineer who touches it, Go’s simplicity and the breadth of its ecosystem are real advantages. Ciulla makes the same point more generally when talking about technology choices under deadline pressure. “When you need something simple, and you’re familiar already with Java or JavaScript, why don’t you use it?” The same principle applies to Go for teams already operating in that ecosystem.

That matters because Rust adoption is not free. It requires a different mental model, a compiler that forces decisions earlier, and at least one person on the team who knows the language well enough to validate what is being shipped. For a service that does not need Rust’s performance profile, those costs may not be worth paying.

Rust wins on performance and it is not a close contest

Where Ciulla becomes more direct is performance. On raw performance, he does not see much ambiguity. “Check the benchmarks yourself and send me the link where Go beats Rust,” he says. “Sometimes they are at the same level. In terms of pure performance, there is no story.”

That bluntness is useful, but the more important argument in his interview is not about benchmark wins. It is about latency predictability. Languages that rely on garbage collection, including Go, Java, and Node.js, can introduce pauses when the collector runs. An HTTP request that arrives during one of those pauses may experience higher latency than one that does not, even though the service logic is identical.

“By not having a garbage collector on the backend side, you basically have flat latency,” Ciulla explains. “You don’t rely on luck, or on the user not being the unlucky one. It’s a problem that is removed.”

For most web applications running at moderate scale, that distinction may not matter enough to justify a language change. Ciulla’s point is not that every API should be rewritten in Rust. His argument is that services with strict latency requirements, high concurrency, or service-level objectives tied to consistent tail latency should be evaluated differently from ordinary backend services. In those cases, the lack of a garbage collector is not an aesthetic language feature. It changes the runtime behavior the team has to reason about.

The resource efficiency argument is equally concrete. Ciulla describes running a Rust web server in production and observing roughly four megabytes of RAM in development and five megabytes in production. On a one-gigabyte droplet, that means many small Rust services can sit idle at once without creating the same memory pressure a heavier runtime might introduce. That is not a benchmark designed to win an online argument. It is an operational fact that changes what an infrastructure team has to provision, monitor, and pay for.

That is where Rust’s case becomes strongest. Not when the team wants a language that is fashionable, and not when someone wants to win the Rust versus Go debate, but when a specific service is expensive, latency-sensitive, memory-constrained, or sitting on a path where runtime predictability matters.

Deployment changes the operational argument

One of the more practical points Ciulla makes is that Rust changes the shape of the deployment artifact. When you run cargo build on a Rust project, you get an architecture-specific executable binary. A Windows machine produces a Windows executable. A Mac produces a Mac executable. A Linux machine produces a Linux executable. You can also cross-compile by changing the target architecture through the compiler, which lets a team produce a Linux binary from another environment when the workflow requires it.

Ciulla’s preferred production workflow is to build the Rust binary directly inside the Docker image build process. “My flow is that I prefer to build the Rust binary directly when I build the Docker image,” he explains, “so I have something which is just deployable everywhere, a Linux executable running in a Docker container.”

The appeal is straightforward. The team gets a lightweight binary compiled for the right architecture, packaged inside a portable container. “The dream for operations is having an executable inside a Docker container,” he says. “You get something which is lightweight and can run everywhere Docker is installed.”

That matters for teams already containerizing services, which is most teams building at meaningful production scale. A Rust binary inside a container keeps the deployment artifact small while preserving the operational consistency teams expect from Docker. The container still matters because real systems rarely deploy one service by hand. They orchestrate many services, restart them, replace them, and scale them across environments. As Ciulla puts it, “We are not in the 90s anymore.”

The argument is not that Rust removes the need for containers. It is that Rust and containers fit together cleanly. Rust gives the team a compact executable artifact. Docker gives the team a repeatable deployment boundary. Together, they reduce some of the runtime and packaging overhead that teams accept as normal in other ecosystems.

Rust is ready for more production backends

The most interesting part of Ciulla’s position is that it has changed. Two years ago, he would not have committed to building a paid SaaS product with a Rust backend. A year ago, he was still hedging. In 2026, he removes the qualification entirely. “If I had a paid product, I would use Rust,” he says. “Let’s remove the probably.”

That shift is not based on general enthusiasm for the language. Ciulla describes himself as skeptical by default, and his advice throughout the interview is to try technologies directly rather than rely on what advocates or critics say online. His confidence comes from the maturity he now sees in the Rust backend ecosystem, especially around Axum, which he says he would now use in production if he were building a SaaS or paid product.

The surrounding toolchain also strengthens the case. Cargo handles dependency management, building, testing, and documentation in a single integrated workflow. There is no equivalent of the npm versus yarn versus pnpm decision that JavaScript teams often navigate before they even get to the application itself. Running tests is cargo test. The integration is not a small ergonomic convenience. For teams that have spent years carrying build-system complexity, dependency churn, and fragmented tooling across projects, a coherent toolchain reduces the amount of process overhead attached to the language.

Ciulla sees that as part of why Rust’s reputation can lag behind its current reality. The language still has a learning curve, especially around ownership, lifetimes, and the borrow checker, but the ecosystem around the language has become more integrated rather than more fragmented. For experienced developers willing to learn Rust on its own terms, that changes the adoption equation.

Start the decision with the service

Ciulla’s adoption advice was consistent across our interview. Do not rewrite everything in Rust. Do not introduce it because the internet says it is the future. Do not make the language the strategy. Start with one service.

The right starting point, in his view, is a critical service with a real performance, latency, or memory problem. It might be a login service. It might be an API under heavy load. It might be a component that slows the rest of the system down. The point is to choose the service where Rust’s advantages are tied to an actual constraint rather than a general preference.

That is also where organizational readiness becomes unavoidable. If a team has no one who understands Rust well enough to review AI-generated code or validate a production deployment, the language can become a liability. “Who decides if the AI-generated Rust service is okay to put in production?” Ciulla asks. “You need the validation of an expert.”

That expert does not make Rust special. It makes Rust normal. Every new technology needs someone inside the organization who can tell the difference between code that compiles and code that should be shipped. Without that person, the team is not adopting a better tool. It is creating a new category of production risk.

For teams that do have the right service and the right internal expertise, Ciulla thinks the timing has changed. Rust is no longer only a systems language that backend teams admire from a distance. It is becoming a practical choice for the services where its properties matter most. The mistake is treating that as a universal recommendation. The opportunity is knowing exactly where it applies.

The Rust versus Go decision, then, is not really a Rust versus Go decision at all. It is a question about constraints. If the service needs simplicity, staffing depth, cloud tooling familiarity, and quick delivery, Go may be the better engineering choice. If the service needs predictable latency, low memory overhead, strong runtime control, and a deployment artifact that maps cleanly to containers, Rust deserves serious consideration. The senior engineering move is not to pick a side. It is to know which job each language is being asked to do.

Go deeper with Francesco Ciulla’s book

The Rust Programming Handbook: An End-to-End Guide to Mastering Rust Fundamentals, a practical guide to Rust’s ownership model, memory safety guarantees, concurrency patterns, trait system, and real-world use in systems and web programming.

Explore the book here

In case you missed

🛠️ Tool of the Week

cargo-nextest — a next-generation test runner for Rust that replaces cargo test with faster, more reliable test execution for production Rust projects.

Highlights

• Runs each test in its own process, surfacing hidden state sharing and enabling better parallelism than cargo test’s single-process model.

• Retries flaky tests automatically, reducing noise in CI pipelines without manual intervention.

• Outputs machine-readable JUnit XML, compatible with most CI systems out of the box.

• Version 0.9.137 added a JSON schema for user configuration, standardizing nextest settings across a workspace.

Learn more about cargo-nextest

📎 Tech Briefs

• Docker Desktop 4.77.0 released - Marketplace extensions now install by pinned manifest digest, reducing tag-mutation risk after publication.

• Rust 1.96.0 released - New Copy-compatible range types, assert_matches! macro stabilization, and two Cargo security fixes for third-party registry users.

• Go 1.26.4 released - Security fixes ship for crypto/x509, mime, and net/textproto alongside compiler and runtime bug fixes.

• Docker Desktop security update - Two container-to-host code execution CVEs in the Model Runner inference backend patched, with a Linux kernel backport fixing a container privilege escalation.

• rust-analyzer update - New diagnostics, predicate evaluation, and completion fixes improve daily Rust IDE feedback loops.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Keep building,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company wants to reach senior developers, software engineers, and technical decision-makers, speak to us about partnering with Deep Engineering.

Join this interactive workshop to learn how to turn Claude Code from a session-by-session assistant into a repeatable engineering system, using structured context, reusable skills, scoped rules, hooks, and guardrails that work across real codebases and team workflows.

🗓️ Friday, June 20 · 10:30 AM EDT onwards

Use code DEEPENG50 for 50% off.

Register here

✍️ From the editor’s desk,

Welcome to the 50th issue of Deep Engineering!

Anthropic expanded Project Glasswing on June 2, extending Claude Mythos Preview to approximately 150 new organizations for codebase vulnerability scanning, after initial partners found more than 10,000 high-severity security flaws in production code.

AI can now find vulnerabilities in production systems that were presumably tested, reviewed, and shipped by engineering teams. The gap between code that compiles and code that holds up under real-world pressure is no longer theoretical.

That gap also has a cause. Brian Allbee, Staff Software Engineer at Cleerly and author of Hands-On Software Engineering with Python (Packt), argues that programming focuses on the correctness of the code itself, while software engineering expands that focus to sustainability as change occurs. Too many developers optimize for code that works today, without enough attention to whether that code can be changed, tested, maintained, and handed off tomorrow.

Allbee joined Deep Engineering Live to discuss what closing that gap looks like in practice. Today’s expert insights are based on that conversation, and you can read or watch the full Q&A here.

Let’s get started.

Featured: All the dev content that matters, in one personalized feed

daily.dev is a professional network for developers, built around a personalized feed of the best content from across the dev ecosystem. Millions of developers use it to stay current with their stack, discover new tools and frameworks, and connect with a global community that shares what they’re learning.

Join for free at daily.dev

Expert Insights

Building Better Python Software Is Not About Writing Better Code

by Saqib Jan with Brian Allbee

Most Python developers measure their work by whether the code runs, assuming the job is done once the function returns the right value, the tests pass, and the build is green. Brian Allbee, Staff Software Engineer at Cleerly and author of Hands-On Software Engineering with Python (Packt), thinks that measure is correct but incomplete, noting that the gap between correct code and sustainable software is where most Python developers stop growing without realizing it.

The distinction Allbee draws is precise. Programming, he explained in our live interview, is focused on “the correctness of the code itself,” whereas software engineering “starts expanding out into more of a focus on sustainability as change occurs.” That shift in focus sounds subtle, but it changes almost every decision an engineer makes, from how they structure a module and handle a growing codebase to how they talk about technical debt with the people who control the roadmap.

The discipline that architecture cannot replace

The instinct when a Python codebase starts to grow is to reach for architecture by breaking things into services, introducing abstractions, and redesigning the data model. Allbee’s experience points in a different direction. “I think most of the paths to success in that context, at least the ones that I can think of that I’ve seen, don’t really start with the architecture, but with discipline behind the process,” he shares.

The discipline he describes is specific and unglamorous, emphasizing the need to keep things as simple as possible while wrapping repeated processes into functions or methods, stressing “that teams should agree on documentation standards and stick to them until something unexpected comes up, and that developers must write code with testability in mind from the beginning, even when there is no immediate requirement for tests.” These practices do not require a new framework or a redesign because they rely on consistency, which is often much harder to achieve than architecture.

The reason discipline comes before architecture is that architecture without discipline produces complexity without clarity. Allbee, in our interview, shared a vivid example with the audience from his own experience regarding a system he encountered that had been written in Python by an engineer who came from a C# background, resulting in an architecture where every function and every class had its own isolated module. The functional layers of the system were seven or eight deep depending on the context, creating a project that was “ridiculously huge,” he recalls, and “way more complicated than it needed to be, and it was hard to manage... hard to maintain.”

The problem was not the language or incompetence, but rather a mental model built for a different environment being applied wholesale to Python. Allbee points to a concept from the book Code That Fits In Your Head to explain why this matters, because humans can only keep “five to seven bits of information in the front of their memory at a given point in time.” A system with seven layers of depth saturates that capacity before a developer has even started reasoning about what any individual layer does.

Allbee argues that developers must “keep it simple” and “collapse things down to the point where you don’t have to have 19 different classes and 15 different instances of, you know, all these other classes to deal with something that really should be capable of being managed as a single function.”

Technical debt is a product decision, not a technical one

One of the more practically useful things Allbee explains about managing an evolving Python system is that technical debt is not primarily a technical problem. “Technical debt is one of those product-level priorities,” he reasons, adding that “whoever’s making the prioritization decisions is going to be in control of when those get tackled, if they get tackled.”

That framing shifts where an engineer should focus their energy when technical debt is accumulating, meaning the work is not just to identify the debt but to communicate its consequences clearly enough that the people controlling the roadmap can make an informed decision. “Making sure that you can communicate effectively, here’s what the impact of this technical debt is to your product-level people or whoever’s making those decisions, is gonna be a key thing,” he adds. That requires being able to sit down and say clearly that if the team does not deal with a bug, it is going to lead to cascading issues, and the longer they put it off, the more likely it is to lead to a really significant problem that will take even longer to get past.

The teams that handle technical debt well, in Allbee’s experience, are the ones that treat it as a first-class concern rather than an emergency. The difference between those two approaches is almost entirely about communication, where debt that gets communicated early and framed in terms of product risk gets prioritized, while debt that surfaces as a crisis gets managed badly.

Testing is a design decision

The most common framing of testing in Python projects is that it is something you add to code that already exists, but Allbee’s position is that testability is a property of the code itself, meaning that designing for it from the beginning changes the shape of the code in ways that make it easier to understand, change, and hand off to other engineers.

His testing approach for his own projects is a method that “exercises valid and invalid inputs for all of the parameters of every callable in the project.” He shares, “You combine that with judicious monitoring of missing lines and a code coverage report, that has served really well for me in making sure that the targets of those tests are being both thoroughly and realistically exercised.” The more important principle underneath the practice is that tests are most valuable when they reflect how the system is actually used, not just how the code is structured.

In team contexts, Allbee advocates for explicit agreement about how tests are organized and what tools are involved. “I’ve seen what happens when different engineers who aren’t communicating with each other each go their own way,” he points out, noting that “the tests that result, even if they’re rigorous and well thought out, are oftentimes difficult to follow across different test modules.” The investment in agreement upfront produces a test suite that the whole team can confidently read, maintain, and extend.

On AI-generated code in testing contexts, Allbee recommends defining a test suite that only humans are permitted to modify, making it as rigorous and complete as possible, and then allowing AI to generate implementation code that must pass that suite. He explains the boundary by stating that you can tell the AI to “write all the code you want,” but it “must pass this test suite” and it does “not get to modify that test suite.” That boundary, he reasons, provides about as much coverage as can realistically be achieved when AI is involved in production code.

Concurrency is a design problem first

Python’s performance limitations and its Global Interpreter Lock have been a recurring concern for engineers building high-throughput systems, and CPython’s free-threaded build has stirred interest in what Python might make possible beyond the GIL. Allbee is measured about expectations, highlighting that most Python code is IO-bound rather than CPU-bound, which is where the GIL has its most significant impact, though he is hopeful that the free-threaded model will open doors for more CPU-bound work to be written in Python.

The framing that matters most to many developers is not about the runtime at all. “Concurrency is a design problem before it’s a runtime problem,” he underscores, adding that having better concurrency support in the language really does not eliminate the need to understand how your processes are going to contend against each other, how to deal with data ownership at the scope of the code, or how failures can happen. His practical advice on concurrency reflects this directly by recommending that developers add it sparingly and only when there is an actual benefit that outweighs the overhead of handling errors, data contention, and coordination costs. “Optimize your clarity and correctness first,” he recommends, and “really only reach for concurrency when you understand where the time is actually being spent.”

Cloud readiness is designing for volatility

The question of what makes a Python application cloud-ready is one Allbee addresses in terms of design principles rather than tools or platforms. The containerized application is cloud-ready, he acknowledges, but so are function-as-a-service constructs like AWS Lambda functions, proving that the specific mechanism matters less than the underlying design orientation.

“The key concept that ties almost every cloud-resident system together, containerization, stateless design, any of those, is that they are inherently disposable,” he explains. Because a container can be killed at any time, a Lambda invocation could be terminated before it reaches a successful completion, and Kubernetes pods restarting are probably routine events, designing for that reality means building processes around the expectation that the hardware can disappear at any point in time.

Statelessness in that context is about making failure cheap. There is no state to manage and no need to write code to reacquire that state, meaning a process simply ends and is restarted, making recovery from a failure as simple as starting a new instance. “Statelessness and containerization matter more because they make failure cheap and recovery routine than for any other purpose or reason,” he says, arguing that this principle should sit near the top of the list of factors shaping design decisions for any system built to run in a cloud environment.

What senior engineers actually do

The question of what separates an engineer ready for senior work from one who is not comes back to the same systems-oriented thinking that distinguishes engineering from programming, where the indicator is not technical mastery but curiosity about the system rather than just the isolated function.

“If they started demonstrating that they’re concerned with more than just is the code doing what it’s supposed to do,” he explains, “if there’s a certain amount of curiosity, why are we doing it this way, do they recognize the trade-offs, those are the things that I think start really indicating somebody is actually ready to go beyond just I’ve written this function, and it’s done, and it’s tested, and it works. Done. I’m finished.”

The senior engineers Allbee has tried to emulate and seen do their best work are not defined by the code they write but by the systems they shape and the teams they are enabling. That involves asking questions that guide less senior engineers to ask those same questions on their own, such as why the team is going down a certain road, what the benefits are, and what trade-offs exist. “There are always trade-offs,” he notes, emphasizing, “Always, always, always trade-offs.”

The advice he offers to Python developers trying to grow in an AI-accelerated world collapses to three core principles, which are to “think in systems,” to “design for change,” and to “optimize for your team.” If you come away thinking differently about why you write the code that you are writing and not just how, then that is the shift that matters. Since the language, tools, and expectations placed on Python engineers will inevitably keep growing, the engineers who hold up under those pressures are the ones who stopped measuring their work by whether the code runs and started asking what it will take for the system to survive.

Go deeper with Brian Allbee’s book

Brian Allbee explores these ideas in more depth in Hands-On Software Engineering with Python, a practical guide to building Python systems that are easier to test, maintain, evolve, and hand off.

Explore the book here

In case you missed

🛠️ Tool of the Week

Ruff - fast Python linting and formatting for teams trying to keep quality gates enforceable without slowing every commit.

Highlights

• Consolidates linting, import sorting, upgrade checks, and formatting behind one configuration surface.

• Runs fast enough for pre-commit and CI workflows, which makes quality checks more likely to stay enabled.

• Supports monorepos and hierarchical configuration, helping larger teams avoid one-off project rules.

• Already used across major Python projects, making it a practical default rather than a niche experiment.

Learn more about Ruff

📎 Tech Briefs

• Copilot SDK is now generally available - GitHub made Copilot SDK stable across six languages, letting teams embed agent workflows into internal tools.

• Anthropic expands Project Glasswing - Project Glasswing now extends to 150 organizations, shifting AI vulnerability discovery toward coordinated patching capacity.

• Python pip 26.1.2 - Pip 26.1.2 shipped with Trusted Publishing attestations, tightening provenance for standard Python installation workflows across teams.

• Using uv in GitLab CI/CD - Astral added GitLab CI guidance for uv images and cache pruning, simplifying reproducible Python pipelines outside GitHub.

• Pyright 1.1.410 - Pyright 1.1.410 refreshed the Python wrapper package, keeping CLI and editor type checks aligned automatically.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Keep building,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

With agents discovering and calling APIs at runtime rather than developers hardcoding them at build time, that design problem has become one of the most urgent infrastructure questions in enterprise engineering.

This month’s special issue digs into how APIs built for developers need to become discoverable, understandable, governed, and safe runtime capabilities for agents, with commentary from Erik Wilde, Head of Enterprise Strategy at Jentic; Nandita Giri, Senior Software Engineer at Microsoft; Rohan Gupta, Principal Product Manager at Harness; and Mayank Bhola, Co-Founder and Head of Products at TestMu AI.

Let’s get started.

Special issue — June 2026

Your APIs were built for developers, not agents

“You don’t just look for APIs when you’re writing an app. You kind of look for APIs every time you solve a problem.” — Erik Wilde, Head of Enterprise Strategy at Jentic and OpenAPI Ambassador

Most large enterprises have no idea how many APIs they have. Ask them and the honest answer is usually somewhere between a guess and a shrug. What they do know is that the number is large, the endpoints are larger, and the documentation is somewhere between incomplete and missing. For years that was manageable because the people consuming those APIs could compensate. They had context, experience, and enough judgment to work around the gaps in a poorly written spec or an ambiguous parameter name.

For the better part of two decades, engineering teams designed APIs for a specific kind of consumer, a developer sitting at a keyboard, reading documentation, and making deliberate decisions about which endpoints to call and in what order. That consumer had context, experience, and the judgment to fill in the gaps that a poorly written spec inevitably left open. The API did not need to be perfect because the developer compensated for its imperfections at design time, before a single line of integration code was written.

That assumption breaks down when the consumer is an agent, says Erik Wilde, Head of Enterprise Strategy at Jentic and OpenAPI Ambassador. Agents do not read between the lines, compensate for ambiguous parameter names, or infer the intent behind a generic error response. They act on what the contract says, at runtime, every time they need to solve a problem, and the gap between what most enterprise APIs offer and what agents actually need is where many AI projects begin to fail before they deliver measurable value.

Anthropic’s 2026 State of AI Agents Report found that 46% of engineering teams cite integration with existing systems as their primary challenge when deploying agents, placing it above model capability, prompt quality, and every other factor on the list. The bottleneck is infrastructure, and that infrastructure depends on APIs that were never designed for this kind of consumer.

Masterclass: Building AI-Ready APIs with Agent Skills

Join OpenAPI Ambassadors Erik Wilde and Frank Kilcommins for a hands-on masterclass on building AI-ready APIs with agent skills, covering OpenAPI, Overlay, Arazzo, semantic discovery, deterministic workflows, and governance guardrails for agent-driven integrations.

🗓️ July 1, 2026 · 10:30 AM – 1:30 PM ET · Online

Use code DEEPENG50 for 50% off.

Register here

Agents discover APIs at runtime, developers do not

In our live interview, Wilde gave one of the clearest framings for how agent consumption differs from developer consumption. Developers search for APIs when building an application, make a decision, and hardcode the integration so it stays consistent for the lifetime of the application. Agents search for APIs at runtime, every time they encounter a problem they need to solve, against a catalog that may contain hundreds of thousands of options across a large enterprise.

That changes the API problem from documentation quality to runtime selection. The consumer is no longer a skilled person who can fill in the gaps of a poorly written spec. It is a machine that acts on exactly what the contract says, nothing more and nothing less, and it does so without the accumulated context that a developer brings to the integration process.

Wilde illustrated the scale of this problem by sharing a recent experience with a car manufacturer operating roughly 50,000 APIs and 500,000 endpoints across the organization. The point is not that this number is exceptional. For a large enterprise with decades of accumulated systems and services, it is closer to the normal condition than most teams would like to admit. What changes with agents is the cost of that normal condition. When the consumer needs to find the right capability at runtime, the selection problem alone can make the API landscape effectively unusable without a serious restructuring of how capabilities are described, organized, and exposed.

Agents cannot compensate for spec drift

“Think of the API as a contract with a very literal, very curious machine.” — Nandita Giri, Senior Software Engineer at Microsoft

Nandita Giri, Senior Software Engineer at Microsoft with prior engineering experience at Meta and Amazon, works across agentic AI and automation, and the pattern she observes across organizations working to become AI-ready is consistent and predictable. Teams invest in producing a good OpenAPI specification at launch, treat it as a first-class deliverable at the time of release, and then watch the specification and the actual API behavior silently diverge over the following months as the code evolves faster than the documentation does.

For developer-facing APIs, this drift is a manageable nuisance because developers notice the discrepancy, ask questions in Teams, Slack or a GitHub issue, and someone eventually updates the documentation before the next consumer runs into the same problem. But for agent-facing APIs, spec drift is not a nuisance. It is a silent failure mode that is exceptionally difficult to trace because agents have no mechanism for noticing the discrepancy between what the spec says and how the API actually behaves. They act on what the spec says, encounter failures they cannot interpret without the surrounding context that a developer would have, and either produce incorrect results or abandon the task entirely without surfacing a meaningful error to the system that called them.

The only way to stop that drift from compounding, Giri argues, is to treat the specification as a first-class part of the release process on every change, with CI pipelines that validate spec fidelity against actual runtime behavior before deployment proceeds, not as a quarterly audit task but as a gate that blocks release when the spec and the actual behavior have diverged.

Giri is equally specific about what good specifications actually require for agent consumption, and her examples are concrete enough to apply immediately. A field called status that returns values 1, 2, and 3 is useless to an agent unless the spec also documents that 1 means New, 2 means In Progress, and 3 means Completed, because the agent has no way to infer that mapping from the field name or the values themselves. An endpoint that documents only that it returns a 400 error for bad input, without specifying which input combinations trigger that response, leaves an agent unable to prevalidate its requests or recover gracefully when the error occurs. A rate limit that appears only in external documentation and not in the spec itself is invisible to any agent that has not been specifically trained on that external documentation. These are not edge cases that organizations can deprioritize. They are the normal state of most enterprise API specifications, and they are a primary reason why agents fail in ways that produce poor results without surfacing a clear explanation of what went wrong.

The same distinction applies on the API producer side. Standard linting tools check structure, including whether a description field exists, whether it meets a minimum length, and whether required parameters are present. That structural check is genuinely useful as a first line of defense, but it cannot evaluate whether a description is written in a way that helps an agent understand what the operation is actually for.

A field that passes every linting rule can still be useless to an agent if it describes what the endpoint does technically without explaining the intent a consumer would bring to it. Descriptions need to represent intent, including what somebody would use the operation for, what constraints apply, and how the agent should reason about the result. The gap between a description that passes a linting check and a description that an agent can act on reliably is the gap that most teams are not yet closing, and closing it requires evaluation mechanisms that go beyond pattern matching on the specification itself.

Cross-service inconsistency breaks agent workflows

“It’s not just about connecting the dots for AI agents. It’s about making sure they understand what those dots mean.” — Rohan Gupta, Principal Product Manager at Harness

Rohan Gupta, Principal Product Manager at Harness, approaches the same problem from the perspective of an organization managing APIs across many teams and many services. His concern extends beyond the quality of any individual specification to the consistency of API design across the entire landscape. When agents operate in enterprise environments, they rarely interact with a single service in isolation. They move through workflows that cross multiple services, passing data and decisions from one system to another, and every inconsistency between how different teams have designed their APIs adds friction at the exact points where agents need to reason about how to connect things together.

Gupta’s view is that API specifications must be well-annotated and thoroughly documented so that agents can understand and execute the tasks they are given with accuracy and clarity, and that the design sloppiness which developers could historically compensate for becomes a structural blocker when the consumer is a machine reading a schema as its only source of truth. Missing descriptions, vague parameter names, inconsistent error handling patterns, and exposed implementation quirks that make no sense outside the context of the original development team all force agents into guesswork, and agents that guess tend to fail in ways that are difficult to reproduce and harder to debug than the original error would have been.

The governance problem becomes harder at the cross-service level. If one service in an agent’s workflow provides ambiguous or outdated information, the agent can be misled into triggering actions on a completely separate system in ways that no individual team would have anticipated or authorized. Lifecycle management for APIs that agents consume cannot focus only on backward compatibility within a single service anymore. It has to account for the cross-platform consistency and auditability of changes across every service in every workflow that agents are permitted to traverse, which is a meaningful expansion of what API governance has historically required.

APIs that work for developers fail agents in production

“If you can’t explain why your agent made a decision, you’re not ready to go live.” — Mayank Bhola, Co-Founder and Head of Products at TestMu AI

Mayank Bhola, Co-Founder and Head of Products at TestMu AI, has a practitioner’s view of where the failure patterns actually surface when organizations move from building agentic systems in development to running them in production. The pattern he observes is consistent across teams and organizations. APIs that worked reliably for developer consumption fail at meaningful rates when agents start calling them, and the root cause is almost always constraints and rules that were documented in external guides or tribal knowledge rather than encoded explicitly in the specification itself, leaving agents with no mechanism for knowing those rules exist until they violate them and encounter a failure they cannot interpret.

The fix Bhola advocates for is not simply better documentation, because better documentation that lives outside the machine-readable contract is still invisible to agents. It requires rethinking how APIs surface information about their own behavior, making all constraints explicit within the spec itself and building API surfaces that are structured to reduce the cognitive overhead agents face when trying to understand what an endpoint does, when to call it, and what the consequences of calling it incorrectly might be. For organizations with established API landscapes, he recommends maintaining two parallel layers, with a legacy developer API preserving backward compatibility for existing integrations and an AI-optimized layer built on top of it that flattens nested data structures, makes all constraints and relationships explicit, and exposes capabilities at a level of abstraction that agents can act on without needing to combine multiple lower-level calls to accomplish a single business task.

Bhola believes the industry’s biggest blind spot is assuming that successful API consumption automatically leads to reliable agent behavior. In practice, many failures emerge after the API call succeeds. The agent selects the wrong tool, misinterprets context, follows an invalid reasoning path, or takes an action that technically satisfies the request but violates business intent. This is why validation infrastructure must be designed before deployment rather than after incidents occur.

Testing agentic systems requires teams to evaluate decision quality, tool selection accuracy, reasoning traceability, and behavioral consistency under changing conditions. The goal Bhola highlights is not just to verify outputs, but to understand whether the agent arrived at those outputs for the right reasons.

Too many endpoints, not enough intent

The structural problem underneath all of this is that most enterprise APIs are too fine-grained for agents to use reliably, even when every individual specification is perfectly written and maintained. As Wilde frames it, accomplishing anything meaningful often requires combining many different endpoints in a specific order that encodes implicit business logic which is obvious to a developer who understands the domain but entirely opaque to an agent that has only the API contracts to work from.

When doing something meaningful requires chaining thirty endpoints in the right sequence, agents become confused about how to combine them, inventive in ways that produce incorrect results, or they make errors partway through the sequence that cascade into larger failures that are difficult to unwind. Wilde’s position is that AI readiness requires reducing the number of endpoints agents are exposed to and improving the business alignment and intent-based nature of the APIs that remain, so that a workflow that wants to accomplish a task ideally needs only a single tool call rather than having to orchestrate many lower-level calls in the correct order. The solution he and his colleagues at Jentic are working toward is a workflow layer that sits above the existing fine-grained API landscape, exposing business-level capabilities that are designed for runtime discovery and agent consumption rather than for developer integration at build time.

This pattern already shows up in enterprise partner integrations. Organizations with complex APIs that they expose to partners face a specific version of the fine-grained problem, where a partner integrating with a large API surface has to understand the full landscape even when they only need a small part of it, and the engineering effort of that integration is significant enough to slow or block adoption entirely.

The solution Wilde describes is building purpose-built workflows for specific partners, so that a partner only needs to understand the workflows that were designed for their particular use cases rather than navigating the full API surface independently. The underlying APIs do not change. What changes is the layer of business-level capabilities that sits above them, designed for a specific consumer’s needs rather than for maximum flexibility across all possible consumers. The benefit for agents is the same as the benefit for partners, with fewer options to navigate, clearer intent at each step, and a much lower chance of combining things incorrectly.

The insight that makes this approach worth pursuing beyond its value for agents alone is one that Wilde makes explicit. This improvement is not only valuable for agents. Any developer who currently has to call fifteen underlying APIs to accomplish a task that should conceptually be a single operation would also benefit from a better-designed capability API on top of those underlying services. The investment in agent-readiness is an investment in the overall quality and usability of the API landscape, and the returns compound across every consumer of those APIs whether that consumer is a human developer or an autonomous agent running at runtime.

The API layer is where the next two years are decided

Wilde’s view of API lifecycle management is the right closing frame for this issue. Agents do not consume APIs the way developers do. They discover capabilities at runtime, decide whether a tool looks useful in the moment, and need machine-readable signals about what the API does, what constraints apply, what side effects it may trigger, and whether it is safe to keep using.

That changes how organizations need to think about versioning, deprecation, and governance. The old model assumes that a developer reads the documentation, notices a migration notice, and updates an integration on a schedule the team can manage. Agent-facing APIs need more of that information to be visible at runtime. If an API is being deprecated, if a capability is nearing sunset, or if a safer replacement exists, the consuming system needs a way to discover that signal before it makes a decision.

This is where API lifecycle management needs to move, and organizations that invest in the governance structures to support it now will be better positioned than those that wait for the pressure to become unavoidable. The agents are already in production, and the limiting factors are no longer model capability alone but integration, security, and operational scalability, which means the API layer is where the most consequential infrastructure work of the next two years will happen for most engineering organizations.

The same design assumption that broke enterprise APIs, that the consumer has context, judgment, and the ability to fill in gaps, is present in every other infrastructure layer that agents call at runtime. Wilde’s framing brings the issue back to a practical rule. Agents should not be used to compensate for infrastructure that fails to express intent, constraints, lifecycle state, or safe operating boundaries. The teams that build on infrastructure designed to make those signals explicit will ship more reliable agentic systems than those still working around infrastructure that was never designed for this kind of consumer.

Thank you for reading this special issue of Deep Engineering on why the API layer has become the most consequential infrastructure problem in enterprise AI.

We’ll be back on Thursday with more expert-led content, and next month, on the first Tuesday of July, with another special issue.

Keep building,
Saqib Jan
Editor-in-Chief, Deep Engineering

daily.dev is a professional network for developers, built around a personalized feed of the best content from across the dev ecosystem. Millions of developers use it to stay current with their stack, discover new tools and frameworks, and connect with a global community that shares what they're learning.

Whether you're an early-career engineer levelling up or a senior dev tracking what's next, daily.dev makes sure the signal reaches you - without the noise.

Join for free at daily.dev

✍️ From the editor’s desk,

Welcome to the 49th issue of Deep Engineering!

Earlier this month, CISA and its international cybersecurity partners released Careful Adoption of Agentic AI Services, a guide for organisations adopting AI systems that can plan, use tools, access data, and act across digital environments. That changes the risk model because AI systems operating inside real workflows inherit risk from the surrounding data, permissions, tools, and context.

That risk is becoming easier to understand in practice. On 23 May 2026, Rohan Pandey of DigitalOcean and Archit Bhujang of Arizona State University published Poisoning the Watchtower, which shows how logs, alerts, URLs, payloads, DNS queries, and usernames can carry attacker written instructions into LLM assisted security workflows.

AI systems do not only consume clean prompts from users. They consume context from operational systems, open web sources, documents, logs, tools, and knowledge bases that the model does not control. Once that context includes contradiction, deception, malicious text, or attacker controlled content, relevance alone becomes an unsafe target for retrieval and summarisation.

David Knickerbocker, founder of Verdant Intelligence and author of Network Science with Python (Packt), builds systems for Open Source Intelligence (OSINT) environments where messy and adversarial data is normal. His perspective matters in this issue because the systems he builds separate observing from judging, treat claims as claims rather than facts, and preserve minority signals that simpler retrieval pipelines often discard.

In issue 43, we looked at Knickerbocker’s work on real-time knowledge graphs and AI systems that treat knowledge as a live stream of claims. Today’s issue continues that conversation with what OSINT teaches engineers about messy, adversarial data. You can also watch our interview or read the full Q&A here.

Let’s get started.

Claude Code for Software Engineering

Learn how to structure Claude Code with context, reusable skills, scoped instructions, and guardrails so it works reliably across real codebases and team workflows.

🗓️ Friday, June 20 · 10:30 AM EDT onwards

Use code DEEPENG50 for 50% off.

Register here

Expert Insights

Building AI Systems That Handle Contradiction at Scale

by Saqib Jan with David Knickerbocker

Most engineers building AI systems have never had to question whether their data source is working against them. The data comes, is processed, retrieved, and the system responds. The assumption underneath all of that is that the source is cooperative, that it was created to convey information accurately, stored in a format designed for retrieval, and that what comes back when you query it is at least an honest attempt at an answer. The problem is that assumption is so embedded in how most AI systems are designed that it never gets examined.

David Knickerbocker, founder of Verdant Intelligence and author of Network Science with Python (Packt), builds systems for environments where data is not clean, settled, or cooperative. His AI systems ingest from the open web, across sources that contradict each other, where some information may be misleading, incomplete, or adversarial. The engineering challenge is not only making retrieval accurate. It is making the system useful when the real world refuses to behave like a clean dataset.

The assumption that data is helpful

Engineers who have worked primarily with internal databases, structured APIs, or carefully assembled training sets carry a baseline assumption that data is cooperative. It was created to convey information accurately, stored in a format designed for retrieval, and accessed through interfaces that return what was asked for. The job of the retrieval system is to find the right thing efficiently.

Open-source intelligence does not work this way. When ingesting from the open web at scale, some fraction of what arrives is wrong, some is deliberately misleading, and some represents one side of a contested claim. For Knickerbocker, the ingestion layer is not the right place to decide what is true. “You can have two different groups that are in opposition from each other,” he says. “One group will say this is the truth, and another group will say this is the truth, and they will be in direct conflict with each other.” The system’s job, in that moment, is to capture what is being claimed and preserve enough context for judgment to happen later.

The practical design response is to treat the system as an observer rather than an adjudicator. Knickerbocker draws that line clearly. “My systems do not care who is right or wrong,” he explains. “They just do not. My systems are observers.” The point is not neutrality as a value statement. It is an architectural boundary. The system captures what is being said, keeps competing claims visible, and avoids collapsing observation into judgment too early.

This distinction matters far beyond open-source intelligence. Any AI system that draws on user-generated content, social media, news, or unstructured enterprise data is working with material that was not created to be machine-readable and was not vetted before ingestion. The assumption that the data is trying to help is not just wrong in those environments. It is a liability.

Bigger clusters are not more important than smaller ones

One of the quieter failures in production NLP systems is the treatment of minority signals as noise. A similarity-based retrieval system returns the most representative results, which in practice means the most common results. A clustering pipeline that surfaces the largest groups first will consistently deprioritize small but significant signals. In a world where the interesting thing is often the outlier, that is a serious problem.

In open-source intelligence specifically, this failure mode has consequences. A small cluster of claims pointing toward something dangerous is not less important because it is small. A single source saying something that contradicts the majority view is not less worth capturing because it is in the minority. “Bigger clusters are not more important than smaller clusters,” Knickerbocker observes. “In open source intelligence, everything matters, top to bottom.”

Drawing from his engineering experience building these systems, Knickerbocker ensures his APIs return full context rather than a ranked shortlist. “If you use a tool to do a search to find out something, you are getting a snapshot of time,” he says. His systems are designed to capture what he calls the heartbeat of the internet. “If I use my API... it is going to come back with 10,000 things. My APIs do not return 10. They return full context.” That creates a harder downstream problem because the question is no longer how to retrieve the best few results. It is how to make a large, shifting body of claims usable without discarding the signals that do not look dominant at first.

The parallel for general AI systems is specific and direct. Any retrieval or summarisation pipeline that privileges majority signal is making a judgment call that the most common view is the most relevant one. That judgment call is often wrong, and it is invisible because the discarded minority signal never surfaces.

The difference between a claim and a fact

Engineers trained on factual datasets tend to build systems that treat retrieved content as facts to be combined and presented. The underlying assumption is that if the source is credible and the retrieval is accurate, what comes back is true. In a contested information environment that assumption collapses immediately, and the design has to change with it.

Knickerbocker’s approach separates the task of capturing claims from the task of evaluating them. What a source says is observable. Whether what it says is correct requires judgment that depends on context, corroboration, and often human expertise that the system does not have. Turning that claim into an evaluated fact requires a different layer of judgment, and Knickerbocker is careful not to build that decision into the first act of ingestion. “I do not make that decision, and I do not allow my AI to make the decision what is true or what is false either,” he says. “I am more interested in what people are claiming is what is going on in the world.”

This design choice has a significant downstream consequence. It means the system can handle contradiction without breaking. Two sources saying opposite things about the same event are not a problem to resolve at the retrieval layer. They are two data points, both of which belong in the response. Knickerbocker simply logs these varied claims as parallel ribbons of information. The human or the downstream system that receives them can then apply judgment about which to act on, in what context, and with what confidence.

The verification boundary

One of the hardest design decisions in any AI system that works with real-world data is where to draw the line between surfacing an insight and making an actionable claim. The two feel similar at the output layer but require very different things from the system that produces them.

In our live interview, Knickerbocker was specific about where that line sits. “Everything that I do is intentional,” he shares. His real-time intelligence layer is built for awareness. It captures what is happening and surfaces it without making the final judgment on what should be done next. If a piece of intelligence looks actionable, the system does not automatically act on it. It surfaces the signal so a human or downstream process can decide whether it matters, who should see it, and what level of confidence is appropriate.

In practice, this means that even when a piece of intelligence looks clearly actionable, the system does not act on it. It surfaces it. The routing of that intelligence to the right person or the right downstream process is a separate engineering and organisational problem, and conflating it with the retrieval problem produces systems that are either too conservative to be useful or too confident to be trusted.

This is a principle with broad application. AI systems that are asked to be both the observer and the actor tend to perform neither role well. Keeping the observation layer and the action layer separate, with a clear boundary between them, is one of the most reliable ways to build something that stays trustworthy as it scales.

Entity extraction gets easier but never clean

Entity extraction from clean text is comparatively well understood. The models are good, the cleanup is manageable, and the output is reliable enough for most downstream uses. Entity extraction from the open web at scale is a different challenge, not because the models are worse but because the data has properties that laboratory text does not.

Knickerbocker began this work in 2018, starting with part-of-speech tagging before NER models were mature, moving to spaCy as those models improved, and more recently using LLMs for extraction. The trajectory is one of improving reliability rather than changing fundamentals. “Entity extraction has improved a lot since 2015,” he notes. “I mostly have to just throw away less. I have less cleaning to do, and it gets things right a lot easier.”

What has not changed is the messiness. Natural language processing at scale on real-world text always produces noise. The question is how much noise is acceptable for the downstream use case and how to handle the cleaning efficiently. At the scale he describes from previous work, including entity extraction across internet-scale datasets, the cleaning cannot be purely manual. It has to be part of the pipeline rather than an editorial step applied after the fact.

He also flags a risk in the current extraction approach that is worth understanding. Older NLP models produced visible noise that engineers learned to catch and correct. LLM-based extraction produces outputs that look clean even when they are wrong, because the model is good at generating confident-looking text regardless of underlying accuracy.

The implication for engineers is that moving to LLMs for extraction does not reduce the need for validation. It makes validation harder to remember because the outputs no longer look like they need it.

Building for the world that actually exists

The thread running through Knickerbocker’s work is a commitment to grounding. He builds systems for the world as it is, not a cleaned version of it. That leads to a specific set of design choices: treat data as claims rather than facts, preserve minority signals, separate awareness from judgment, and let the system observe before any person or downstream workflow decides what to do next.

Those principles come from the kinds of environments Knickerbocker has worked in: data operations, cybersecurity, open-source intelligence, and production systems where the cost of getting something wrong is real. “The real world is a messy space,” he says. “Natural language processing is just messy. I have not seen it get really cleaned up yet.”

For engineers who have worked mostly with clean internal systems, that might sound like a warning about a narrow class of hard problems. It is broader than that. Any AI system that deals with content created by people, pulled from the web, generated by users, or routed through operational systems eventually has to confront the same condition. Real-world data is messy by default. The systems that handle it well are intentionally designed for that mess before it becomes a production failure.

In case you missed

🛠️ Tool of the Week

GraphRAG — A graph-based retrieval pipeline for unstructured text

GraphRAG helps teams preserve relationships across messy documents, conflicting claims, and large text collections before asking an LLM to answer.

Highlights:

• Builds knowledge graphs from unstructured text instead of relying only on isolated chunks.

• Links entities, claims, and topics so retrieval can use structure, not just similarity.

• Supports local and global search for both narrow evidence lookup and corpus-level synthesis.

• Gives engineers a practical starting point for testing graph-based RAG patterns.

Learn more about GraphRag

📎 Tech Briefs

• Claude Compliance API Integrations - Compliance API integrations help IT and security teams govern Claude across connected enterprise workflows.

• MCP Events Working Groups - Gateway, transport, registry, and agents groups advanced protocol work around tool-connected AI systems.

• RAGFlow v0.25.6 - Browser agents and RAPTOR AHC mode expand RAGFlow from document retrieval into web-aware ingestion workflows.

• Qdrant v1.18.1 — Vector dimension validation before WAL writes reduces ingestion failure risk during async upserts.

• Weaviate v1.38.0-rc.0 - Nested object filtering and namespace support improve retrieval precision for structured, multi-tenant corpora.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Keep building,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

A hands-on workshop covering how to build production-grade AI agents using Java and LangChain4J.

🗓️ Friday, June 13 · 10:00 AM – 1:30 PM ET · Online

2 for 1 deal is live. Use code DEEPENG50 for 50% off.

Register here

✍️ From the editor’s desk,

Welcome to the 48th issue of Deep Engineering!

Google announced Managed Agents in the Gemini API two days ago at Google I/O, making it possible to spin up an agent that can reason, use tools, execute code, and browse the web with a single API call. The infrastructure work that previously required teams to build and manage sandboxes, scaffolding, and execution environments is being abstracted away. The capability is in public preview and Google is clear that outputs should be reviewed before use in sensitive workflows, but the direction is quite clear. Deploying agents is getting significantly easier.

What is not getting easier at the same pace is making the APIs those agents will call worth calling. APIs designed for actual developers, who can tolerate ambiguous descriptions, infer intent from sparse documentation, and navigate hundreds of operations to find the right one, do not work the same way for agents. Agents are less reliable at resolving ambiguous API semantics, choosing among many overlapping operations, and safely composing actions without machine-readable contracts and guardrails.

Erik Wilde, Head of Enterprise Strategy at Jentic and OpenAPI Ambassador at the OpenAPI Initiative, has spent considerable time on solving for that gap. We spoke with Wilde about what agent-ready actually means in practice, and he explained from his engineering purview why MCP will not fix a poorly designed API foundation, and what platform engineers should start planning for today.

The expert insights in today's issue are based on our recent live interview with Wilde and you can read or watch the full Q&A here.

Let’s get started.

Featured Newsletter: Machine Learning at Scale.

Are you a SWE looking to upskill into ML systems? Get high quality ML system design content delivered to your inbox. Learn how to design and scale Machine Learning Systems.

Subscribe to Machine Learning at Scale

🧠 Expert Insights

Your APIs Are Not Ready for Agents, and MCP Will Not Fix That

by with

The conversation about AI agents in enterprise software dominates engineering mindshare as to how agents will consume APIs and what it actually takes to make that consumption work reliably. Most organisations have taken the shortcut. They have built an MCP server, pointed it at their existing API landscape, and told themselves the agent problem is solved. Erik Wilde, Head of Enterprise Strategy at Jentic and OpenAPI Ambassador at the OpenAPI Initiative, thinks that is the wrong bet, and his reasoning is specific enough to be useful.

“Whatever you invest in better APIs becomes useful for everybody,” Wilde affirms. “If you invest specifically in MCP, that investment is effectively scoped to LLM consumers.” The point is not that MCP is useless. It is that MCP is a delivery mechanism, and delivery mechanisms change. The API foundation underneath it does not change nearly as quickly, and if that foundation is poorly designed for the agents that will eventually consume it, no amount of tooling stacked on top of it will compensate. The organisations that will be in the strongest position in two years are the ones investing in the foundation now, not the ones chasing the current delivery protocol.

The abstraction level problem

The clearest way to understand what makes an API agent-ready is to look at a concrete example, and Wilde in our interview offered one that makes the problem immediately legible. The GitHub REST API currently has around 1,100 operations. That is not unreasonable for a product as complex as GitHub. A developer can navigate 1,100 operations because they bring context, experience, and the ability to read documentation and infer intent. They know roughly what they are looking for and they can work toward it even when the path is not obvious.

An agent does not work that way. “For an agent to work directly with that GitHub API is pretty complex,” Wilde points out, “because a lot of those operations need to be combined in a certain way to result in the workflows that you really want to accomplish on GitHub.” The agent has to figure out not just what each individual operation does but how they compose, in what order, under what conditions, and with what dependencies. With 1,100 operations, the combinatorial space of possible workflows is enormous, and agents navigating it without guidance will produce unreliable results.

Now look at the GitHub MCP server, which has around 70 tools. Each of those tools represents a higher-level workflow, something a developer might actually want to accomplish on GitHub rather than a low-level operation that contributes to that accomplishment. The reduction from 1,100 to 70 is not a loss of capability. It is a gain in usability for the specific class of consumer that is trying to get things done rather than explore a surface. “What I would say,” Wilde argues on this point, “is that if you had a genuinely agent-friendly GitHub API, it might also just have around 70 operations.” The MCP server is not adding something new. It is providing the abstraction level that the underlying API should have provided in the first place.

This is the abstraction level problem, and it is the most important design question for engineering teams building API infrastructure that agents will consume. The APIs that were designed for developer flexibility, with many fine-grained operations that compose in powerful ways, are exactly the wrong shape for agents that need to accomplish specific goals reliably. The discipline of designing for agents is the discipline of asking what a consumer actually wants to accomplish and surfacing that at the API level, rather than exposing every atomic capability and leaving the composition to the consumer.

What agent-ready actually means

The properties that follow from the abstraction level insight are consistent and actionable. An API designed for agent consumption should not be too fine-grained, and its descriptions should be intent-based and written at a level that is meaningful for a language model rather than just technically accurate for a developer who already knows the domain. It should have examples, ideally multiple examples per operation rather than one, because examples are one of the most reliable ways for a model to understand what an operation actually does in practice. Its error messages should be meaningful enough that an agent encountering a failure has enough information to understand what happened and what it might do next.

“If an AI agent looks at a poorly described API and cannot figure out how it works, it will just move on to the next one,” Wilde notes. “It has less context. It has less experience. It does not really know as well as an actual developer what to do.” This is the practical consequence of the abstraction level problem at the description level. A developer reading a sparse API description can fill in the gaps from domain knowledge and engineering experience. An agent cannot do that reliably, and the result is not a helpful error or a clarifying question. It is a silent failure or a wrong action.

Wilde and his team have built a scoring mechanism for API readiness that makes these dimensions concrete. The scoring uses a combination of standard linting, running tools like Spectral and Redocly to check structural conditions, and LLM-based checks that evaluate whether descriptions are written in a way that is genuinely useful for an agent rather than just present. The distinction matters because a description that exists and passes a structural check may still be useless for an agent if it describes what an operation does technically without explaining what a consumer would use it to accomplish. “These descriptions need to represent intent,” Wilde highlights. “What is the intent of somebody who would use this operation?”

Linting though necessary is not sufficient

Linting has become standard practice in well-run API programs, and Wilde endorses it as a first line of defense. The popular tools are capable and in some cases open source, and the practice of defining shared rule sets that teams can discuss, extend, and maintain in version control is genuinely useful. But in our conversation he was clear that linting alone does not get you to agent-ready, and teams that treat it as the complete solution are leaving the most important problems unaddressed.

The structural checks that linting tools perform are exactly that. They can tell you whether a description field exists and whether it meets a minimum length requirement, but they cannot tell you whether the description is written in a way that helps an agent understand what the operation is for. They can flag a missing example but cannot evaluate whether the examples present give a model enough signal to use the operation correctly in a novel context. The gap between what linting checks and what agent readiness requires is the gap between structure and meaning, and closing it requires evaluation mechanisms that go beyond pattern matching on OpenAPI descriptions.

Wilde also makes an important point about rule set governance that is worth taking seriously. “I am not a big fan of just reusing existing rule sets,” he contends. “I would always say start owning this, build up your own in a collaborative fashion.” The Zalando and Adidas rule sets that circulate in the API community are useful references, but they were built for specific contexts and specific quality standards. Adopting them wholesale means inheriting decisions that were made for a different organisation’s constraints. The value of a rule set comes not just from the rules it contains but from the process by which those rules were agreed upon, which is a process that builds shared understanding of what good API design actually means in a particular context.

MCP is a delivery mechanism, not a foundation

MCP has been growing fast. It is now under the Linux Foundation, major model providers support it, and a growing number of enterprise vendors are shipping MCP servers as a standard part of their product offering. For engineers deciding where to invest, it looks like an obvious answer to the question of how to make APIs accessible to agents.

Wilde’s skepticism is not about MCP’s current momentum. It is about what MCP is and what it is not. “MCP is the current delivery mechanism,” he says. “You need a delivery mechanism, but I would not build too many things that are MCP-specific.” At Jentic, the team supports MCP because it is what the market expects right now, but they have deliberately avoided deep investment in MCP-specific infrastructure. If MCP were replaced by something else, the transition would be straightforward because the underlying work, making APIs well-described, well-structured, and semantically rich, would carry over entirely. That work is not MCP-dependent. It is foundational.

The risk for teams that invert this priority is real. Building an MCP server on top of a poorly designed API landscape means the MCP server inherits all of those same problems. Operations that are too fine-grained stay too fine-grained, descriptions that lack intent stay unreadable to a model, and error messages that tell a human nothing tell an agent even less. The wrapper changes the protocol by which those problems reach the agent, not the problems themselves.

Open standards outlast any delivery protocol

One of the clearest threads in Wilde’s thinking is the value of building on open standards rather than specific tools or protocols. This is not an abstract preference for openness. It is a practical argument about optionality. Teams that build their API practices on OpenAPI, Arazzo, and Overlays are building on specifications that are independent of any vendor, any model provider, and any current delivery protocol including MCP. When the next delivery mechanism arrives, or when the current tooling landscape shifts, the foundation remains.

Arazzo is worth understanding in this context. It is a workflow language published by the OpenAPI Initiative that allows you to describe sequences of API interactions in a standardised format. If accomplishing a particular goal requires calling five endpoints in a specific order with specific dependencies, Arazzo is the language for expressing that. For agents, which struggle with exactly this kind of multi-step composition, a well-constructed Arazzo workflow is one of the most useful things an API producer can provide. “Figuring out multi-step workflows is one of the hardest things for agents to do right now,” Wilde says, “and Arazzo is genuinely good at describing those. We just need to make it discoverable.”

Overlays, the third specification from the OpenAPI Initiative, provides a way to express changes to an OpenAPI description in a standardised diff format. “We use overlays,” Wilde shares, “to deliver improvement suggestions alongside API scores. When the scoring mechanism identifies that an API is not well-designed for AI consumption, it also produces an Overlay that shows exactly what would need to change to improve it.” That makes the gap between current state and agent-ready state concrete and actionable rather than a list of abstract recommendations.

The APIs you design today will still be running in two years

The practical implication of everything Wilde argues is a specific recommendation about timing. API landscapes change slowly. Whatever is designed or changed today will likely remain largely unchanged for one to three years. Agents are arriving in enterprise contexts incrementally but consistently. The customer support and HR agents that are already deployed broadly are the early wave, and the business agents with genuine decision-making authority are behind them.

“API landscapes evolve slowly,” Wilde says. “Whatever you design or change today, you will probably have around for a year or two or three before you touch it again.” The teams that start building API readiness for agents now are the teams whose infrastructure will be in the right shape when agents with more capability and more authority arrive. The teams that wait for agents to become mainstream before improving their APIs will find themselves doing expensive remediation work on a landscape that is already in production and already depended upon.

The recommendation is not to stop shipping features or to redesign everything at once. It is to make agent readiness a standard consideration in the decisions that are already being made. When writing a new operation, write the description for an agent as well as for a developer. When adding examples, add enough that a model can generalise. When defining error responses, add enough context that a consumer without domain knowledge can understand what happened. These are not large investments per decision and they compound over time into an API landscape that agents can actually use.

To this end, “All the platform people out there who are building API platforms or doing platform engineering,” Wilde says, “think about how all of this will change if you have more and more agentic actors and consumers in your organisation, and start planning for that today, even if you can say that right now you do not have it this much and it is going to be another year or two. It is going to arrive.”

In case you missed

Here’s the full Q&A with the interview video featuring Erik Wilde.

🛠️ Tool of the Week

Spectral — open-source JSON and YAML linter with built-in support for OpenAPI, Arazzo, and AsyncAPI

• Validates OpenAPI v3.1, v3.0, v2.0, Arazzo v1.0, and AsyncAPI v2.x out of the box.

• Supports fully custom rule sets, letting teams build and own their own governance standards.

• Integrates with VS Code, JetBrains, GitHub Actions, and Azure API Center for shift-left linting.

Learn more about Spectral

📎 Tech Briefs

• Google Managed Agents now in Gemini API - A single API call now provisions an ephemeral Linux sandbox with code execution, web browsing, and tool use built in.

• Kyverno 1.18 released post-CNCF graduation - First post-graduation release patches two SSRF CVEs and adds cleanup policy support to the Kubernetes policy engine.

• OpenAI and Dell bring Codex to on-premises enterprise - The partnership makes the Codex coding agent available in hybrid and air-gapped enterprise environments for the first time.

• A2A protocol underpins Google’s full agent stack - Agents built at any abstraction level can be called as sub-agents across the entire Google Cloud agent platform.

• 43% of AI-generated code fails in production - Survey of 200 SRE leaders finds teams average three production redeploy cycles to verify a single AI-suggested fix.

That’s all for today. Thank you so very much for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

This hands-on workshop teaches you to build reliable, production-ready AI systems using eval-driven development. Taught by Imran Ahmad, Data Scientist and author of 50 Algorithms Every Programmer Should Know.

🗓️ May 30 · 11:00 AM – 3:30 PM ET

Use code EDD50 for 50% off.

Register here

✍️ From the editor’s desk,

Welcome to the 47th issue of Deep Engineering!

Debian’s APT package manager is moving toward the Rust threshold its maintainers set more than six months ago. APT maintainer Julian Andres Klode announced on the Debian developer mailing list that hard Rust dependencies and Rust code would be introduced no earlier than May 2026, citing memory safety and stronger unit testing as reasons to move core parsing and signature-verification paths toward Rust and the Sequoia ecosystem. For a tool that underpins Debian, Ubuntu, and their many derivatives, this is not a marginal adoption story. It is Rust moving into infrastructure that enormous numbers of systems rely on every day.

That kind of decision does not get made because Rust is fashionable. It gets made because the language can shift whole classes of errors from production failures into compile-time constraints. That is precisely the argument Evan Williams, senior software engineer and author of Design Patterns and Best Practices in Rust, makes in this week’s issue. We spoke with Williams about what it actually takes to think in Rust, why the borrow checker is a design tool rather than a compiler obstacle, and why he found it harder to write bad Rust than good Rust when working on the book. You can watch our interview or read the full Q&A here.

Let’s get started.

Featured Newsletter: DevOps Bulletin

If you work across DevOps, Cloud Native, AI and security and want a weekly read that surfaces the most relevant open-source tooling, stories, and insights in the space, DevOps Bulletin is worth adding to your reading list.

Subscribe to DevOps Bulletin

Expert Insights

Rust Makes It Harder to Write Bad Code Than Good Code

by Saqib Jan with Evan Williams

Most engineers who struggle with Rust describe the same experience. The compiler rejects code that would compile without complaint in C++ or Java, and the borrow checker surfaces errors that feel arbitrary until, gradually, they start to feel like something else entirely. Evan Williams, author of Design Patterns and Best Practices in Rust (Packt), has a precise name for what they are. They are design feedback, not feedback about syntax or style, but about the structure of the program itself, the shape of data flow, the discipline of ownership, and the decisions about who controls what and for how long. “Rust is your partner in doing that,” Williams says. “You can still write code with bugs in it, but Rust makes it harder to do that and easier to write code that’s going to be solid.”

That framing changes how to think about the borrow checker, and it changes how to think about what Rust actually is. Most languages make it easy to write code that works in isolation. Rust makes it hard to write code that fails in combination, and the difference matters more as systems grow.

The borrow checker is enforcing design, not syntax

Most engineers who pick up Rust treat borrow checker errors as obstacles to route around. The instinct is understandable because in Java or Python, the path from a failing compiler error to working code runs through adjustment: find what the compiler dislikes, change it, move on. Rust works differently, and engineers who apply the same strategy find that routing around the borrow checker is possible in the short term and damaging in the long term.

“The borrow checker is your friend because it prevents you from making a messy design. It prevents you from making a broken design. It prevents you from writing whole classes of bugs that you will then spend many hours trying to find,” Williams says. “I have found it to be an incredible partner in writing code that allows me to sleep at night.”

The reason the borrow checker behaves this way is structural. Java and Python allow data to be accessed from many places at once, which gives engineers flexibility but leaves the responsibility of managing that access entirely with the programmer. Rust removes that flexibility. A value has one owner. References are either shared and immutable or exclusive and mutable, never both at the same time. This constraint forces the programmer to be explicit about who owns what and when, because the compiler will not let the program proceed otherwise. The practical consequence is that programs which compile in Rust tend to have a quality that programs in other languages achieve only through discipline: their data flows are explicit. You can read a Rust program and understand, without running it, who controls which piece of state, when that control transfers, and what happens at the boundary.

“The principles that the borrow checker forces you to adhere to in Rust are the exact principles that you should be using in every programming language,” Williams reasons. “But you don’t have to. So it’s very easy to not think about those things.”

The object-oriented trap

The single most common mistake engineers make when getting started with Rust is treating it as an object-oriented language. It resembles one superficially, with structs, methods, and something that looks like encapsulation, but it has no inheritance, no abstract base classes, and no shared mutable state by default. An engineer who brings a Java or C++ mental model will find that the things they reach for instinctively are either unavailable or actively counterproductive.

“If you carry with you an object-oriented language mindset, then you’re going to have nothing but trouble,” Williams says. “The more experienced you are, the more years you have doing something in some other language, the more trouble you’re likely to have, because you have patterns of thought that come from those languages that you don’t even realize are there.”

This is a precise observation about how expertise transfers, or fails to. An engineer with ten years in Java has a large inventory of solutions to common problems, and most of those solutions depend on inheritance, shared mutable references, or runtime polymorphism through interfaces. In Rust, none of those approaches work as expected. The patterns are not wrong in their original context. They are wrong for this one, and the difficulty is that the engineer applying them does not recognize the mismatch until the borrow checker makes it unavoidable.

The design patterns that experienced engineers carry into Rust need to be examined before use, not applied by default. Some evolve into new forms because Rust’s enums and advanced generics make several classical patterns either less necessary or unnecessary entirely. Others require fundamental rethinking. The Singleton pattern, useful enough in Java and Python that engineers reach for it without deliberation, tends to become either redundant or actively problematic in Rust. “In Rust, it tends to be either completely unnecessary because other features of the language make it unneeded, or it tends to encourage designs that are really not necessary and where a much better approach could be used,” Williams says.

The replacement for inheritance in most cases is traits, which provide polymorphism without the coupling that comes from sharing a class hierarchy. The discipline required to work with traits well is the same discipline the borrow checker enforces on data: think about the boundaries, be explicit about what crosses them, and design the structure before writing the code.

Ownership as architecture

The ownership model does more than prevent bugs at the function level. It shapes the architecture of the system, because the rules that apply to individual values apply at every level of scale. A program that handles data ownership correctly in a single function has to handle it correctly across modules, across threads, and across the boundaries between components. The borrow checker enforces this at compile time, which means architectural decisions that in other languages can be deferred until the system grows large enough to break start being made from the beginning.

“You need to think about who controls what, how it is controlled, and you need to start from the very beginning thinking about the boundaries of your program and the system architecture, dividing things up into areas of responsibility,” Williams says. “Because unlike Python or Java, you can’t have links going all over the place. The borrow checker is never going to accept that.”

This constraint produces a specific architectural tendency in well-written Rust systems: data flows in one direction. Rather than components that hold references to each other in a web of mutual dependency, Rust systems tend toward chains of ownership that move in one direction and do not loop back. “By saying, I have a chain of ownership that moves down but never moves back up, you are now much more likely to have a system that is going to work,” Williams says. “Data flowing down is something that feels natural and smooth and just works. Data trying to fight the stream back up is going to end up giving you problems because the borrow checker is not going to like you.”

The architectural benefit of this tendency is legibility as much as correctness. A system where data flows in one direction is a system where behavior is predictable from the structure, and debugging does not require reconstructing who might have modified a value and when, because the ownership model makes that history explicit.

Williams illustrates this with an example from his book, a miniature publish-and-subscribe system built to resemble Kafka at a much smaller scale. “Because Rust has move semantics, you know that if something leaves here and goes here, it’s now not here anymore. It’s there. There’s no question about things like having references dangling or anything like that. The clarity of things moving through the system, the clarity of being able to have immutable data in a lot of places and knowing who can and can’t modify any piece of data, it just makes the design of the system so clear and it makes it so much harder to make a system that doesn’t work,” he says.

The typestate pattern

The most underutilized expression of this architectural discipline in Rust is one that Williams returns to with visible enthusiasm. The typestate pattern uses the type system to encode the state of a value at compile time in a way that makes invalid state transitions not just errors but programs that will not compile.

“It’s a way of developing state machines and systems that have state that evolves where invalid state transitions aren’t just errors, they’re impossible to write. The compiler won’t compile them,” Williams says. “It represents a huge advance in the way that such systems are written because now instead of runtime errors, you have a state machine that is guaranteed to work because every transition either is a valid transition or it won’t even compile. That’s an amazing thing.”

The typestate pattern was not invented for Rust, but the language’s ownership system and its handling of types make it practical in a way that other languages do not. The result is that a class of bugs that normally surfaces at runtime, invalid transitions through a state machine, surfaces instead at compile time, before the program runs. For systems where correctness is not optional, this is a material improvement. “Not invented for Rust, but it fits Rust so perfectly, it’s hard to believe it,” Williams says.

What this requires in practice

None of this comes without a cost. The discipline that Rust enforces at compile time is discipline that engineers have to supply at design time, and for teams moving from other languages the transition is genuinely difficult. Williams is specific about where the difficulty concentrates. Velocity drops during the learning period, often enough that teams take it as a signal that the decision was wrong, and it usually is not. “Once the team becomes very well acquainted with Rust, velocity can increase dramatically, but there is a period of time where it seems like things have gotten worse,” he says.

The answer for most teams is to start with a small, non-critical piece of work rather than a rewrite of an existing system, with the goal of building familiarity in a context where the cost of roadblocks is low and then expanding from there. “What you don’t want to do is jump into saying, we’re just going to rewrite our project in Rust now. Pick a small piece, focus on that, gain confidence and mastery of the language, and then use that to build upon it and start bringing in more things,” Williams says.

There are also cases where Rust is the wrong tool. Prototyping benefits from the flexibility that Python provides and that Rust does not. Environments where the tooling is incomplete are not the right place to fight the language and the Rust ecosystem, while growing rapidly, still has gaps where Java or C libraries are well established. User interfaces are the clearest current example. But in systems where failure is expensive, where correctness cannot be approximated, and where the code has to remain understandable as the team around it changes, Rust’s constraints are not a cost. They are the point.

The harder thing to write

The most revealing observation Williams made came not from a question about patterns or architecture but from the experience of writing the early chapters of his book, the ones about what not to do. He went back and tried to write bad Rust deliberately, the kind of code that would illustrate the mistakes he was cautioning against, and it was harder than he expected.

“When I went back and tried to write bad code in Rust, it was much harder than writing the good code,” Williams says. “That’s an interesting perspective that just didn’t even occur to me.”

That observation captures something important about what the language is doing. Rust is not just a language with a strict compiler. Its constraints push code toward a specific shape, one that is explicit about data flow, deliberate about ownership, and structured around clear boundaries of responsibility. The engineers who find Rust most difficult are often the engineers with the most experience, because they have the most deeply held instincts to unlearn. And the engineers who find it most rewarding tend to be the ones who stop treating the borrow checker as an obstacle and start reading it as design feedback. The language is not rejecting their code. It is asking them to think more clearly about what the code is actually doing.

In case you missed

Here’s the full Q&A with the interview video featuring Evan Williams.

🛠️ Tool of the Week

rust-analyzer — Rust language server that provides IDE functionality for writing Rust programs.

Highlights:

• Surfaces Rust diagnostics, including ownership and borrow-checker errors from compiler checks, inline during editing.

• Supports major LSP-compatible editors such as VS Code, Vim, Emacs, and Zed, with regular stable releases.

• Widely used across the Rust ecosystem as the standard Rust IDE backend, including in workflows at organizations that build with Rust.

Learn more about rust-analyzer

📎 Tech Briefs

• Dirty Frag vulnerabilities disclosed in the Linux kernel - Two CVEs in Linux ESP/IPsec and RxRPC components allow unprivileged local users to gain root on affected systems.

• Linux 7.0.5 stable released with partial Dirty Frag fix - Linux 7.0.5 ships a partial XFRM/ESP patch for Dirty Frag, with a second required fix still in development at release time.

• GitHub secret scanning via MCP Server now generally available - Credential scanning is now available in MCP-compatible coding agents before commits or pull requests, requiring GitHub Secret Protection to be enabled on the repository.

• Linux 7.1-rc2 published — KVM selftest renaming drove the unusual patch volume in rc2, with functional work covering driver and networking fixes throughout the tree.

• MySQL 9.7.0 LTS generally available - New MySQL LTS line ships the Hypergraph Optimizer in Community Edition, with Dynamic Data Masking remaining Enterprise-only in this release.

That’s all for today. Thank you so very much for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and help grow our work.

See what’s new for the HubSpot Developer Platform!

Ship faster with AI coding tools like Cursor, Claude Code, and Codex. Build MCP-powered AI connectors, run serverless functions with support for UI extensions, and use date-based versioning to streamline roadmap planning.

Explore Updates

✍️ From the editor’s desk,

Welcome to the 46th issue of Deep Engineering!

This week, InfoQ analyzed what it actually took for Cloudflare to run large language models efficiently on their global network. The team built a custom inference engine called Infire from scratch in Rust, split model processing into two separate hardware stages because a single machine could not handle both efficiently, and compressed model weights by 15 to 22 percent to reduce what GPUs need to load and move during inference. The reason they had to do all of this is the same one that matters to every engineering team building AI systems: the hardware layer is not an abstraction you can ignore. It is the constraint that every other architectural decision is made around.

This pattern, where standard approaches to running AI workloads break down under real production constraints and the fix requires going back to the hardware layer, is one most engineering teams will eventually encounter. The engineers who avoid it are the ones who understood the hardware constraints before they started building, not after they hit them. Cloudflare’s engineering blog post goes into the technical detail for teams who want to dig further.

This week we are featuring Jim Ledin, CEO of Ledin Engineering and author of Modern Computer Architecture and Organization, now in its third edition published by Packt. Ledin has over thirty years of experience working on embedded systems, safety-critical hardware, and cybersecurity. In this issue he breaks down what engineers building AI systems get wrong about the hardware layer and why it costs them.

Let’s get started.

Architecting Production-Ready APIs for Agents

Most API ecosystems were not built for autonomous agent usage. This hands-on masterclass covers governed API design, OpenAPI specifications, and multi-API workflow modelling with Arazzo so your platform stays predictable and safe under automated usage.

2 FOR 1 deal is also live. Bring a colleague free and learn how to design AI-ready, governed APIs

Use code DEEPENG50 for 50% off.

Register here

Expert Insights

Hardware Is Not Someone Else’s Problem

by with

For most software engineers working in the cloud, hardware is an abstraction managed by someone else. You provision compute, write code, deploy, and pay the bill. What happens between the instruction and the silicon is not your concern. That assumption has always had a cost. In an AI-accelerated world, that cost is becoming visible in ways that are harder to ignore.

Jim Ledin, CEO of Ledin Engineering, has been working at the boundary where software meets silicon for over thirty years. His entry point into computer architecture was not a formal computer science curriculum. It was a Commodore 64, a joystick, and a drawing program so slow you could watch it move one pixel at a time.

“That episode really cemented for me how important it is to understand what is going on in the hardware of a system, and not just write what you want to do in your favourite language,” Ledin reflects.

He rewrote the inner loops of that drawing program in 6502 assembly, poking opcodes directly into memory, and the line shot across the screen faster than he could see. The lesson stayed with him across thirty years of embedded systems work, electric vehicle software, and cybersecurity testing on safety-critical hardware. Understanding what the hardware is actually doing is not an optimization exercise. It is the difference between software that works and software that works reliably under real constraints.

That distinction matters more now than it ever has, because the hardware layer is where most AI system performance problems actually live, and most of the engineers building those systems have never had to care about it before.

Where the GPU consensus breaks down

The idea that GPUs are the right architecture for AI workloads has become so widely accepted that most teams treat it as settled. Ledin’s view is more specific, and the specificity matters. For local and personal use, running models on a consumer GPU like an Nvidia RTX 4090, GPUs are the right choice. For large-scale deployments running the largest models, the picture is different.

The distinction comes down to what GPUs were actually designed to do. The “G” in GPU stands for graphics, and consumer GPUs still carry silicon dedicated to real-time video generation and gaming workloads. TPUs, by contrast, are built entirely around the tensor operations that dominate AI model processing. At least 80% of the execution time in a transformer-based model is matrix multiplications, and TPUs concentrate every transistor on exactly that work.

The more pressing constraint, though, is memory bandwidth. “AI workloads are becoming increasingly memory bandwidth limited. That means it is taking more time to bring data into the GPU or TPU memory than it is taking for the computation itself to complete,” Ledin explains.

This is the reason high-end AI systems use high bandwidth memory, or HBM, stacked RAM modules with far higher data rates than anything available on a consumer GPU. “It is also,” Ledin notes, “part of why DDR5 is becoming harder to find. Production capacity for memory is increasingly going into HBM modules for AI infrastructure rather than into consumer components.”

And so, for engineering teams choosing hardware for AI deployments, the implication is concrete: the GPU consensus is correct for a specific part of the problem space, and incomplete for the rest of it.

Data movement is the real cost

The performance conversation in AI engineering tends to focus on compute: cores, clock speed, parallelization. Ledin redirects it toward something that gets less attention and causes more problems.

“Data movement can often be more expensive than the actual computation steps. The latency of moving large data structures across different levels of the memory hierarchy can dominate and leave a lot of compute bandwidth idle,” he emphasizes.

This is not a new insight in systems engineering, but it is one that most application developers have never had to internalize because the abstractions they work with hide it. In a modern PC, reading a single byte from DRAM causes 64 bytes to be transferred into the CPU cache. If the code then bounces to other memory locations, causes those to be loaded into cache, and pushes that first block out, the next access to that original data requires fetching it again from DRAM. The latency compounds across every cache miss, and in AI workloads operating on large data structures, those misses accumulate fast.

The practical recommendation follows directly. Iterating across large data structures multiple times in an algorithm should be avoided wherever possible. Working through memory linearly, in a way that keeps recently accessed data in cache rather than evicting it, is the single most impactful optimization available to most AI system code. It does not require a new framework or a different hardware platform. It requires understanding what the hardware is doing with the data you give it.

In cloud environments, this understanding has a direct financial translation. “You are paying for the usage of the system whether the CPU is actually crunching instructions or sitting idle waiting for a data item to come in from memory,” Ledin warns. This is because inefficient memory access patterns do not just slow down a system. They inflate the bill for it.

When abstraction becomes the problem

Abstractions are one of the most effective tools available to software teams. They accelerate development, limit mistakes, and allow large teams to work on complex systems without every engineer needing to understand every layer. Ledin does not dispute any of this. His concern is more specific: abstractions that obscure hardware costs, in performance-critical applications, are not just unhelpful. They actively create risk.

“Where it becomes dangerous is when abstraction obscures what is happening with the data layout in memory and the execution patterns, basically how the processor is interacting with data as the algorithm proceeds,” he cautions.

The failure mode is not that abstractions break. It is that they make costs invisible until those costs produce an incident. An engineer works within an abstraction layer, the code looks correct at that level, and the performance problem lives underneath it in a layer the abstraction was designed to hide. By the time the problem surfaces in production, the context needed to diagnose it is buried.

Ledin’s recommendation is a two-layer design. Use the most expressive code at the edges of the system, where the abstractions are doing the most valuable work. Use performance-aware code in the core, where the hardware interaction is most consequential. The boundary between those layers is not fixed, and finding it requires benchmarking rather than intuition. But knowing the boundary needs to exist is the starting point. Teams that treat the expressive outer layer as the whole system tend to discover, under load, that the core was never designed for the hardware it runs on.

The CPU versus GPU distinction, for engineers who have never had to care

Most senior software engineers working today have built careers without ever needing to think about the difference between a CPU and a GPU. That is changing, and Ledin’s framing of the distinction is the most useful one available for engineers coming to it for the first time.

A CPU is optimized for low-latency execution of complex branching code. It is built to handle conditional logic, to predict branches and recover when predictions are wrong, and to minimize the latency cost of that work. A GPU is optimized for high-throughput execution of linear code across massively parallel workloads, and it works best when it is running the same instruction across thousands of data streams simultaneously with as little branching as possible.

The implication, therefore, for algorithm design is practical. “The GPU only really becomes attractive when you have enough work for it to do that it can be parallelised, and enough that it will amortise the costs associated with moving data onto the GPU, launching the kernels to execute the code, and doing the management work to transfer data to and from the GPU,” he points out.

That last point is the one most teams miss. A GPU is not a general purpose computer. It cannot run a program on its own. It needs to be started and managed from a CPU, and the overhead of moving data onto the GPU, scheduling the kernels, and moving results back is real. If the workload is not large enough and parallel enough to amortize that overhead, the CPU implementation wins, not because GPUs are slow, but because the cost of using them correctly exceeds the benefit for that specific workload.

Knowing where that line sits, for a specific algorithm running on specific hardware, is the kind of judgment that requires understanding what the hardware is actually doing. It cannot be read off from a benchmark or inferred from a framework’s documentation. It comes from the same place Ledin’s understanding came from: going one level deeper than the abstraction, and learning what happens when the instruction meets the silicon.

In case you missed

Here’s the full Q&A with the interview video featuring Jim Ledin.

If the hardware layer argument resonates, the article below by Lee Peterson, VP of Secure WAN Product Management at Cisco, covers the same constraint from the networking and distributed compute angle.

🛠️ Tool of the Week

vLLM - high-throughput, memory-efficient inference and serving engine for large language models

Cloudflare referenced it as the baseline they benchmarked their custom Infire engine against when building hardware-optimized inference at scale.

Highlights:

• PagedAttention eliminates the memory waste that causes most GPU out-of-memory failures in production inference.

• Continuous batching processes requests in a dynamic stream rather than static batches, keeping GPUs saturated under real load.

• Disaggregated prefill/decode runs compute-bound and memory-bound stages on separate hardware for better throughput.

• Supports tensor parallelism, FP8 and NVFP4 quantization across multi-GPU deployments.

Learn more about vLLM

📎 Tech Briefs

• Inference gives AI chip startups a second chance - Disaggregated inference, splitting prefill and decode across purpose-built silicon, is making GPU-only inference architectures look like the wrong default for large-scale production deployments.

• OpenAI releases MRC for AI training networks - OpenAI’s MRC shows frontier training now depends on failure-tolerant network design, making the interconnect layer a first-class engineering constraint rather than an infrastructure afterthought.

• Anthropic opens Claude Security public beta - Claude Security moves vulnerability scanning closer to code review, triage, and patch creation, shifting security work earlier into the engineering workflow rather than treating it as a downstream audit step.

• Google opens Workspace MCP server preview - Google is turning enterprise agents into a governed API and access-control problem, with MCP making the boundary between agent capability and enterprise data policy the next infrastructure challenge for platform teams.

• vLLM v0.20.1 ships with DeepSeek V4 stabilization and FP4 improvements - The patch release stabilizes DeepSeek V4 serving and improves FP32-to-FP4 conversion speed.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and help grow our work.

See what’s new for the HubSpot Developer Platform!

Ship faster with AI coding tools like Cursor, Claude Code, and Codex. Build MCP-powered AI connectors, run serverless functions with support for UI extensions, and use date-based versioning to streamline roadmap planning.

Explore Updates

✍️ From the editor’s desk,

Welcome to the 45th issue of Deep Engineering!

The TIOBE Index for April 2026 puts Rust at number 16, up from 18 last year. While the community widely expected it to break into the top 10, that momentum has slowed. TIOBE attributes this to adoption friction, noting that broader mainstream uptake has proven harder to achieve than the language’s early trajectory suggested.

The institutional picture, however, tells a different story. The NSA and CISA issued joint guidance in June 2025 urging organisations to adopt memory-safe languages for national security systems and critical infrastructure. Google’s Android security team reported that memory safety vulnerabilities, which accounted for 76% of Android bugs in 2019, fell below 20% for the first time in 2025 after the team prioritised writing new code in memory-safe languages.

The Linux kernel maintainers also made Rust permanent, making it a core part of the kernel. The argument about whether memory-safe systems programming languages belong in production is settled at the institutional level. The harder argument, therefore, is how engineering teams adopt it without the expensive, disruptive projects that give the language an undeserved reputation for being hard to introduce.

Francesco Ciulla, author of The Rust Programming Handbook and head of developer relations at Zerops, directly challenges that perception. His framework starts not with the language’s strengths but with the one service in your system where Rust would make an undeniable, measurable difference. That is what this issue covers.

Let’s get started.

Architecting Production-Ready APIs for Agents

Most API ecosystems were not built for autonomous agent usage. This hands-on masterclass covers governed API design, OpenAPI specifications, and multi-API workflow modelling with Arazzo so your platform stays predictable and safe under automated usage.

2 FOR 1 deal is also live. Bring a colleague free and learn how to design AI-ready, governed APIs

Use code DEEPENG50 for 50% off.

Register here

Expert Insights

Rust Does Not Need to Replace Your Stack to Make It Better

by with

Every engineering team that considers Rust tends to circle the same concerns. The language is difficult to learn, expensive to adopt, and more practically useful to systems programmers than to the teams building and running services at scale.

Francesco Ciulla, author of The Rust Programming Handbook, Docker Captain and head of developer relations at Zerops, says, "I've heard that conversation many times, and my response has always been that the framing is wrong from the start."

Ciulla has been building with Rust since 2022, has spoken about it internationally at conferences, and his perspective on Rust adoption is shaped less by enthusiasm for the language and more by a practitioner’s view of where it actually earns its place in a production system. That starting point matters, he says, because the teams that struggle with Rust adoption tend to start from the wrong question.

A joke that contains a kernel of truth

People in the Rust community have long joked about rewriting everything in Rust, and the memes around it have become something of a cultural shorthand for over-enthusiastic adoption. Like most good jokes, Ciulla acknowledges, it contains a kernel of truth. But the practical lesson is the opposite of what it implies. “The best way to introduce Rust in a big project is to find that hard part that is slowing things down, the bottleneck of all your services, and try to write one single service in Rust.” The rewrite everything instinct is how adoption projects become expensive and difficult to justify. The bottleneck-first instinct is how teams get a proof of concept that demonstrates real value before committing to anything broader.

The practical implication is that the adoption decision is not a language decision at the organizational level. It is an engineering decision at the service level. The question is not whether the organization should adopt Rust. The question is whether there is one service in the system that is slow, resource-intensive, or difficult to keep stable, where the properties Rust offers would make a measurable difference. If that service exists, it is the right place to start. If it does not, the case for introducing Rust at all is weaker than it might appear.

Ciulla’s production experience makes this concrete. Running a Rust web server on his own machine, he shared during our conversation that it used four megabytes of RAM in development and five in production. On a one-gigabyte droplet, that means more than 200 services running simultaneously in idle. That number is the kind of resource profile that changes what is economically viable to deploy, and it is the kind of argument that lands differently with an ops team than a language comparison ever could.

Flat latency is a real engineering argument

One of the most underappreciated technical arguments for Rust in production systems is not about speed in the raw throughput sense. It is about predictability. Languages that rely on garbage collection, including Go, Java, and Node.js, introduce periodic pauses when the collector runs. Those pauses can last hundreds of milliseconds. An HTTP request that arrives during a GC cycle experiences higher latency than one that does not. The user on the receiving end did not do anything differently. They were just unlucky.

Ciulla is candid about what this means in practice. “By not having a garbage collector on the back end side, you basically have flat latency. You don’t rely on luck, or on the user not being the unlucky one. It’s a problem that is removed.” For most web applications running at moderate scale, this distinction is invisible. For services with strict latency requirements, high concurrency, or SLAs that depend on consistent tail latency rather than average response time, it is one of the more significant architectural arguments available.

This connects to a broader point about where Rust earns its place and where it does not. The resource efficiency and latency predictability are not arguments for using Rust everywhere. They are arguments for using Rust in the specific services where those properties matter. A service that scrapes a website once a month does not need flat latency. A service handling a million concurrent users does. Knowing the difference is what separates a good adoption strategy from an expensive experiment.

Rust, when it is the wrong choice

Ciulla is honest about the cases where Rust is not the right tool, which is part of what makes his advocacy for it credible. If a team needs something simple and the deadline is tomorrow, Rust is probably the wrong choice. If a developer needs a working API by the end of the day and has no Rust experience, this is not the moment to start learning the language under delivery pressure. “When you need something simple, and you’re familiar already with Java or JavaScript, why don’t you use it?” The question is not rhetorical. It is the right question to ask before any technology adoption decision.

The ecosystem argument is also honest. Python has better libraries for data science. JavaScript has a larger package ecosystem for certain kinds of web work. Rust integrates well with other languages, but if what a team needs is native to another ecosystem, that is a real constraint rather than a preference. Good engineers use the right tool for the problem. The case for Rust is strongest when the problem involves performance, memory efficiency, or concurrency at a level where other languages start showing their limits.

The shepherd principle

One of the more practical observations Ciulla makes about organizational adoption is about knowledge rather than tooling. The bottleneck to Rust adoption at scale is rarely the language itself. It is whether the organization has someone who knows it deeply enough to validate the work being done in it. He draws the parallel to Docker adoption at the European Space Agency, where he worked and observed the tool move slowly not because of anything wrong with Docker, but because it was not well understood internally. The technology is never the problem, he points out. The knowledge is.

“You need the validation of an expert,” Ciulla says. In the era of AI-accelerated development, this point is sharper than it has ever been. Teams can now generate Rust code with AI assistance far faster than their ability to validate it has grown. That gap between generation speed and validation depth is where production incidents come from. Having at least one engineer on the team who understands the ownership model, the borrow checker, and the concurrency primitives well enough to review what the AI produces is not a nice-to-have. It is the thing that determines whether the Rust service is a genuine improvement or a liability waiting to surface.

Concurrency without the trauma

Most engineers who have worked with concurrency in Java or C++ carry a specific kind of wariness about it. The mental model for concurrency in older languages is that it is an advanced topic requiring extra care, specialized libraries, and a heightened awareness of race conditions and deadlocks. Ciulla describes learning concurrency in Java at university as the final, difficult session of the course, something treated as inherently dangerous and saved for the end of the degree.

His first attempt at a concurrency example in Rust produced the opposite experience. “When I had to teach concurrency in Rust in a YouTube video, I made an example in three minutes. I was done. I say okay, the basic example really lasted like two or three minutes because you just declare a couple of threads and literally done.” That experience reflects something structural about how Rust was designed. The language was created after multi-core processors were already standard. Concurrency was not retrofitted onto a model designed for single-threaded execution. It was built in from the start, and the ownership system that prevents data races at compile time is the same ownership system that governs memory safety everywhere else in the language. There is no separate concurrency model to learn. The properties that make Rust memory safe are the same properties that make concurrent code safe.

For teams building services that need to use available CPU resources efficiently, this is not a minor ergonomic improvement. It means that the gap between writing concurrent code and writing correct concurrent code is substantially smaller in Rust than in the languages most engineers have used before. The cost of concurrency, measured in debugging time and production incidents rather than lines of code, is genuinely lower.

The compiler is the most patient teacher on your team

The reputation Rust has for being difficult to learn is real, and Ciulla does not dismiss it. But his explanation for where the difficulty actually comes from is different from the common framing. The problem is not that the concepts are inherently harder than those in other languages. It is that Rust forces you to unlearn patterns that other languages allowed. Engineers who have spent years in C++ or JavaScript carry assumptions about how memory works, how mutability is managed, and what the runtime will silently fix for them. Rust does not fix those things silently. It surfaces them at compile time and requires you to address them explicitly before the code runs.

That shift in where the pain lands is the key insight. “Rust is not hard to learn,” Ciulla says. “It’s different. And this is how we should advocate for it.” The difficulty is front-loaded by design, because the language makes a deliberate trade of more friction during development in exchange for fewer failures in production. Teams that have spent significant time debugging null pointer exceptions, race conditions, or memory leaks in production understand this trade intuitively. The hours lost to a null pointer exception in production dwarf the hours spent fighting the borrow checker upfront.

The Rust compiler, which is the primary source of that upfront friction, is also the primary teaching tool the language provides. Error messages in Rust are unusually detailed and specific. They do not just tell you that something is wrong. They explain what rule was violated, show the relevant code, and often suggest a fix. Ciulla describes the compiler as a teacher rather than a gatekeeper, one that overcommunicates in the same way a good mentor does. “The errors in Rust are basically tutorials. They are helping you to write better code.” For teams introducing Rust to engineers who have not used it before, this property matters practically. The compiler is doing a significant part of the knowledge transfer work that would otherwise fall on the senior Rust engineer on the team.

Rust in the Linux kernel is paying off

The decision by the Linux kernel maintainers to not only allow Rust in the kernel but to plan for components that require it going forward is the kind of institutional endorsement the language community has been waiting for. Ciulla frames it clearly. Even if the experiment had failed, just the fact that Rust was considered a viable option for kernel-level work would have been a meaningful milestone. The language was competing in a domain that had been exclusively C and C++ territory for decades, and it earned a permanent place there.

For engineering leaders tracking where the industry is moving, this matters beyond the kernel itself. Government systems, military applications, and other security-critical domains are beginning to treat Rust as a default rather than an experiment. “Rust is slowly getting adopted at bigger and bigger levels,” Ciulla says. The adoption curve is not linear, but the direction is consistent. Teams that build internal expertise now are not chasing a trend. They are positioning ahead of a transition that is already underway in the most demanding environments in the industry.

The ecosystem argument is changing

One of the historically valid objections to Rust for web development was tooling maturity. Two years ago, Ciulla would not have committed to shipping a production SaaS product in Rust. Today he would, and the reason is specific rather than general. The Axum framework has matured to the point where it is a production-grade choice for web APIs, and the broader ecosystem around async Rust has improved substantially. “In 2026, I will use it,” he says of building a paid product with Rust as the backend, dropping the qualification he would have applied even a year earlier.

The toolchain story is also one of Rust’s genuine advantages for teams evaluating the full cost of adoption. Cargo handles dependency management, building, testing, and documentation in a single integrated tool. There is no equivalent of the npm versus yarn versus pnpm decision that teams arriving to JavaScript have to navigate before writing a single line of code. Running tests is cargo test. The integration is native to the language, not an ecosystem of competing choices layered on top of it. For teams that have spent time debugging JavaScript build configurations, this is not a small thing.

Rust in the AI accelerated development era

Ciulla makes an argument about Rust and AI that is worth sitting with. The claim is not that Rust is better for writing AI applications, though he has views on that too. The claim is that Rust may be one of the best languages to work in during the current period of AI-assisted development, specifically because of what it requires from the engineer reviewing AI-generated code.

When AI writes Rust code, the engineer validating it still has to understand ownership, borrowing, and the type system well enough to know whether the generated code is correct. The compiler will catch a large category of errors, but the human reviewer still needs to understand why the compiler is happy with a piece of code before shipping it. “If you have no control, either you are useless or you cause a problem. So in both cases, it’s not a good time for you.” That discipline, the requirement to understand what the code actually does rather than just accepting output that compiles, is not a burden unique to Rust. But it is more explicitly enforced by the language than in most alternatives, and that enforcement is valuable at a moment when the volume of AI-generated code is increasing faster than the average team’s ability to review it carefully.

🛠️ Tool of the Week

wrkflw — open-source tool for validating and running GitHub Actions locally

wrkflw lets you validate and execute GitHub Actions workflows on your local machine before pushing, catching configuration errors and pipeline failures before they reach CI. Version 0.8.0 shipped this week. Built in Rust.

• Validates GitHub Actions workflow syntax locally before pushing to CI

• Runs multi-step jobs and matrix builds without cloud dependency

• Fast startup and low resource overhead from Rust’s binary compilation model

• Catches pipeline failures early, reducing the feedback loop between code and CI

Learn more about wrkflw

📎 Tech Briefs

• Linux 7.0 ships with Rust as an official core kernel language - Rust loses its experimental tag in Linux 7.0, reaching full parity with C for kernel development after the Linux Kernel Maintainers Summit decision in December 2025.

• Rust 1.95.0 released - The latest stable release introduces cfg_select!, a compile-time macro for conditional configuration, and removes unstable support for custom target specifications on stable toolchains.

• crates.io opens new Svelte-based frontend for public testing - The Rust team invites the community to test the rebuilt crates.io frontend ahead of the planned production migration.

• Cargo stabilises build.warnings configuration - The build.warnings field is now stable, giving teams a standardised way to configure compiler warning behaviour across workspace builds.

• Zed 1.0 ships - The Rust-built code editor reaches stable release with GPU-accelerated rendering, real-time collaborative editing, Git integration, and native AI assistant support across macOS, Windows, and Linux.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and help grow our work.

Hone your skills to design, implement, and ship an AI-powered internal developer platform from scratch with Ajay Chankramath.

📅 25th and 26th April 2026 | 11:00 AM to 3:00 PM ET

Use code DEEPENG50 for 50% off, or grab the 2-for-1 deal

Register now

Includes access to The Platform Engineer’s Handbook by Ajay Chankramath upon release (eBook worth $35.99)

✍️ From the editor’s desk,

Welcome to the 44th issue of Deep Engineering!

C++ just had its most consequential standard in years finalized. Contracts and static reflection are expected after a decade of committee work, and the async execution model is in. This has the conference circuit energised, blog posts are multiplying, and most production teams are continuing to ship C++17 or C++20 while they wait for compiler support to catch up and patterns to settle.

The ISO C++ Foundation also opened its 2026 annual developer survey this week, the single biggest opportunity the global C++ community gets each year to tell the standards committee and tooling vendors what actually matters in practice. If you write C++ at any scale it is worth ten minutes of your time before it closes next week.

This week we are featuring Sándor Dargó, senior software engineer at Spotify, on what responsible C++26 adoption decisions should look like in production, what most teams get wrong, why fallback plans need to be on every adoption checklist, and what the gap between a finalized standard and a production system actually costs. The feature is based on our live interview with him and you can watch or read the full Q&A here.

Dargó also spoke at length in that session about clean code, cognitive load, and what it actually costs teams to optimize for the wrong things. We have collated those insights alongside our previous conversation with Sam Morley, mathematician and C++ researcher at the University of Oxford, into this separate piece “Clean Code Is a Trap, Decompose Instead for Physics and Performance” published this week.

Let’s get started.

View the latest HubSpot Developer Platform updates in Spring Spotlight

See what’s new for the HubSpot Developer Platform!

Ship faster with AI coding tools like Cursor, Claude Code, and Codex. Build MCP-powered AI connectors, run serverless functions with support for UI extensions, and use date-based versioning to streamline roadmap planning.

Check → HubSpot Developer Platform

Most C++ Teams Get Feature Adoption Wrong, and What the Pragmatic Ones Do Differently

by with Sándor Dargó

C++26 is heading to its final approval ballot. Contracts and static reflection are expected. The feature set is the largest the language has seen in years, and the C++ community is already deep into discussing what it means. And yet, while they wait for compiler support to catch up and patterns to settle, engineering teams are starting to ask how to prepare for when they can actually ship it.

That gap between what the standard says and what teams can actually ship is not a failure of ambition, but the normal condition of C++ in production. It has looked roughly the same across every major standard transition for the past fifteen years. “The engineering teams that navigate it well have learned to keep understanding and adoption on separate timelines,” says Sándor Dargó, senior software engineer at Spotify. “It serves well to invest deeply in knowing a feature years before you need to ship it, because conflating the two is where most adoption mistakes gain foothold.”

Dargó has a different set of questions he likes to ask first. His framework is intentional and straightforward. The feature set is not the starting point. What matters is whether the team can live with the code a year from now, when the engineer who introduced it has moved on and compiler support is still uneven across platforms. Proven and maintainable is not playing it safe. It is what keeps production systems running while the language moves underneath them.

At Spotify, where he works on large-scale C++ systems with a hard requirement that the code stays clean, maintainable, and operable over time. He has observed teams navigate standard transitions from C++11 through C++23, and the pattern he sees repeat is always the same one. “Teams focus on what the standard says, and they underestimate the distance between the standard and the compiler, between the compiler and the toolchain, and between the toolchain and the production system that has to run reliably when all of this settles,” he said. “Everyone is talking about contracts and reflection. That’s going to change everything. I’m not sure about the time scale though. If you look at C++23 support right now, even that is not complete yet, especially if you look at the differences across compilers. You go on cppreference, check what’s implemented on which compiler, and we are simply not there yet.”

That observation shapes everything that follows in Dargó’s approach to feature adoption, and it is worth understanding precisely what he means by it.

The gap between the standard and the production system

The C++ standard finalizes on a schedule. Compiler vendors implement on a different one, and they do not always agree on approach, especially for genuinely new features that require significant infrastructure rather than incremental additions. Small library extensions arrive with almost no implementation lag because compilers already have the machinery to support them. But contracts and reflection are not small library extensions. Both require compilers to build something they have never built before, and different vendors will make different choices along the way, as they did with modules in C++20.

Dargó cited modules in our live interview as the clearest recent example of how this plays out in practice. Vendors diverged, build system integration took years to stabilize, and teams that moved early discovered that the standard’s guarantees did not insulate them from the fragmentation that came from uneven implementation across compilers. “There might be a bigger gap, just like we saw with C++20 modules. I hope it won’t be that problematic. But I don’t think I’ll be able to use those in a production environment in the next one or two years.” Dargó, drawing on his years of engineering experience leading high-velocity teams at Spotify, a company with engineering resources that most teams do not have. If Dargó is not planning to ship contracts or reflection to production in the next two years, the average multi-platform C++ team operating with a mixed compiler pipeline and a long-lived codebase should be thinking in similar terms.

This does not mean the features are not worth understanding. It means the time horizon for understanding and the time horizon for shipping are different, and most adoption conversations conflate the two.

Cautious is not the same as resistant

Dargó’s posture toward new features is not skepticism. It is a more specific kind of discipline that he applies before any adoption decision, and it comes down to two distinct questions that most teams skip. The first is whether the usage pattern is proven in a production environment with real operational constraints, not in a blog post or a conference demo. The second is whether the feature actually solves a problem in the specific codebase, rather than just offering a more modern way to write something that was already working. “If they only bring a different syntax, that’s not really good. If they actually solve a problem in your codebase, in your use case, then it’s good. And let’s discuss why and how that’s the right solution.”

The distinction between a genuinely useful adoption and a syntactic upgrade matters more than most teams want to admit, because every feature adoption comes with a cost that does not appear in the initial pull request. The team has to understand the feature well enough to review code that uses it, to debug it when it breaks, and to maintain it when the engineer who introduced it has moved to a different team. “You cannot just push the new way of coding. You should share the knowledge and discuss what these new features actually bring. Deep down, everyone wants to learn about these things, and we all want to start using them as soon as possible.” Dargó acknowledges that impulse is not wrong. But the enthusiasm of the early adopter on the team and the readiness of the rest of the team to live with the decision are two different things, and adoption decisions that ignore the second tend to create the kind of complexity that shows up as maintenance debt two years later.

No one puts the fallback plan on the checklist

The most operationally concrete principle Dargó applies to feature adoption is also the one that almost no team’s adoption checklist includes. Before shipping code that uses a new compiler feature, make sure you can roll back the compiler version without having to change the code. This constraint is not about being conservative for the sake of it. It is about what happens when a compiler update introduces a regression or a new feature behaves differently than expected on a specific platform, and the fastest path to stability is to revert the compiler version rather than to fix the code.

Dargó walked through the alternative. “Move to a new version, start using concepts from C++20, and then in two weeks they say there’s a problem, we must go back. And then you realize it’s not just updating the compiler version, you actually have to change the code.” At that point the team is no longer choosing between a rollback and a fix. They are choosing between a broken build and a risky code change under pressure, and neither is a good position to be making decisions from. The fallback plan requirement changes how the team thinks about adoption from the start because it sets a concrete constraint on which features can be used and how deeply they can be integrated before the team has sufficient confidence in the compiler support. For single-platform, single-compiler environments, that constraint is loose. For multi-platform pipelines with mixed compiler versions, it is the gate that determines what gets shipped and what gets saved for a later standard cycle.

Making trade-offs explicit before someone else makes them implicit

The broader philosophy Dargó brings to C++ at scale is that complexity is always the enemy, and the job of a senior engineer is to reduce it through the daily decisions that accumulate into the character of a codebase over time. Clean code reduces cognitive load. Smaller binaries reduce operational cost. Eliminating undefined behavior reduces hidden risk. New language standards reduce boilerplate and enable safer abstractions. And all of these things connect, in Dargó’s framing, to the same underlying goal: making large C++ systems more maintainable and more evolvable over the years that teams and requirements will inevitably change.

But maintainability does not happen by default. It requires making trade-offs explicit at the moment they are made, in the code itself, not just in the code review thread that will be forgotten. Dargó has been on the receiving end of what happens when this does not happen. He came into a codebase, saw code that looked wrong, began cleaning it up, and realized too late that the seemingly redundant choice was affecting binary size in a way that mattered operationally. Some pull requests had already merged before the context became clear. “Trade-offs there will be. But make them conscious and share the knowledge.” That principle applies to binary size decisions and it applies equally to feature adoption decisions. The team that adopts contracts without documenting why, which problem it was solving, what compiler version it was verified on, and what the fallback plan is, has introduced complexity that is invisible until the moment it becomes a production incident.

What C++26 actually changes and when it matters

Contracts and reflection are genuinely significant features. Contracts bring the language its first formal precondition and postcondition system, giving teams a way to enforce interface discipline through the language rather than through documentation and convention. Static reflection enables code generation, serialization frameworks, and tooling that the language simply could not support before in any ergonomic way. Both of these things will change how C++ gets written over the next decade, and engineers who understand them deeply before the production window opens will be in a stronger position than those who try to learn them under delivery pressure.

But a decade is closer to the right time scale for thinking about the full impact than a release cycle is. The standard has finalized, but the compilers have not caught up. And the community has not yet established the patterns worth following or the misuse patterns worth avoiding. For engineering leaders building roadmaps, Dargó’s framework points toward a clear separation between understanding and shipping. Invest in understanding contracts and reflection now, run experiments on non-production code, track compiler support as it lands, and build team knowledge before the toolchain is ready. Then, when the support is solid, the patterns are clear, and the fallback plan can be written, adoption becomes a technical decision rather than a leap of faith.

Dargó likes to put it in simple words. “Being among the first adopters is sometimes good. Sometimes it’s better to be in the second line.” For most production C++ systems with multi-platform requirements, long-lived codebases, and teams where knowledge has to be shared rather than siloed, the second line is where proven and maintainable gets built. The first line is where the lessons come from that make the second line possible.

🔍 In case you missed it…

🛠️ Tool of the Week

vcpkg — open-source C and C++ package manager

vcpkg removes the dependency management friction that makes compiler and toolchain transitions harder than they need to be, letting teams verify library support across targets without manually managing platform-specific build configurations.

• Parallel file installation for faster builds

• 2,773 ports in the curated registry, each validated across all major triplets

• Dependabot support for automated dependency vulnerability tracking

• OpenSSL packaging security fix on Windows

Learn more about vcpkg

📎 Tech Briefs

• Spotify publishes how it used Honk, Backstage, and Fleet Management to migrate thousands of datasets - Details how background coding agents reduced the manual overhead of large cross-repository migrations at scale.

• GitHub ships C++ Language Server for Copilot CLI in public preview - Extends semantic C++ code intelligence to the command line, giving Copilot symbol definitions, call hierarchies, and type data beyond what grep returns.

• Clang/LLVM 22.1.4 released - Patch release of Clang 22, the compiler with the most active C++26 feature tracking. Teams on Clang should update.

• MSVC Build Tools 14.51 release candidate ships in Visual Studio 2026 Insiders - Adds expanded C++23 conformance, CWG and LWG issue resolutions, and runtime performance improvements. Stable release expected in May.

• cppreference.com returning to read-write after ISO C++ Foundation takes over infrastructure - Herb Sutter announced the Foundation is taking over hosting and maintenance for cppreference, the primary compiler support reference for the C++ community.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and help grow our work.

Learn ownership, RAII, smart pointers & allocators to eliminate leaks/UB—live hands-on with Patrice Roy—#1 bestselling author of C++ Memory Management and ISO C++ Standards Committee (WG21) member.

📅 Online: Apr 18 & Apr 19. Limited seats.

Register now

✍️ From the editor’s desk,

Welcome to the 43rd issue of Deep Engineering!

On April 15, Neo4j ran NODES AI 2026, a full-day virtual conference dedicated to AI with knowledge graphs, with tracks covering GraphRAG, graph-based agent memory, and context engineering. Interestingly, the opening keynote framed the central challenge directly around how most AI systems behave as if they have never seen the data that the organisations deploying them have spent years accumulating. The gap between what a system has access to and what it can actually use at query time is, in the view of the practitioners at NODES, the defining engineering problem of this moment.

And so, in today’s issue we want to address exactly that gap. David Knickerbocker, founder of Verdant Intelligence and author of Network Science with Python (Packt), has spent years building knowledge graph systems that stay current with a different design constraint than most teams start from: not how to make retrieval faster, but how to make the system aware of what changed a minute ago. His system (Verdant Eye) treats knowledge as a continuous stream of claims rather than a static store of facts, and that distinction has significant consequences for how you handle freshness, hallucination, temporal drift, and testing.

This is the first of two features from our conversation with David, the second will cover what building AI on top of messy, adversarial, real-world data teaches you that clean-dataset engineers never have to confront. You can read the full interview and watch the conversation here.

Let’s get started.

30k+ DevOps Engineers Read EverythingDevOps

The cloud-native ecosystem moves on its own schedule. New tooling, shifting practices, team structures that look nothing like they did two years ago. Therefore, keeping up is not optional when your job is to build and run reliable systems.

EverythingDevOps‘ weekly digest is built specifically for engineers who want the signal without the scroll.

• What’s changing in the ecosystem — and what it actually means for your work

• Career thinking for engineers moving from senior to staff to leadership

• Events, opportunities, and community conversations worth your Frida

Join 30,000+ engineers

AI That Sees the World as It Is, Not as It Was

by with David Knickerbocker

Most AI systems are answering questions about a world that no longer quite exists. They draw from training data frozen at a point in time, from retrieval systems that return whatever was most recently indexed, from pipelines that treat knowledge as a stable object rather than a moving one. For David Knickerbocker, founder of Verdant Intelligence, this is not a limitation to work around, but a design error to avoid from the start.

The problem starts at problem definition

Building a knowledge graph that stays fresh does not require a different database. It requires a different question. For Knickerbocker the starting point is always the same, “what is the system actually trying to know, and at what resolution of time?”

He explains, “If you want to build a world AI and be able to answer questions about things that happened a minute ago, then that is your problem statement. And then you think about how to get that data into the database so that it is there and it is fresh. But then you also have to get AI to be able to use that data. There are two sides to this coin.”

The engineering philosophy he applies is KISS (Keep it Super Simple), and YAGNI (You Aren't Gonna Need It"). Both push in the same direction, he reasons “build the minimum thing that works, validate that it works, and expand from there.” His system (Verdant Eye) maintains data fresh up to a minute old, not by doing something architecturally exotic, but by treating freshness as a first-class constraint from day one rather than an optimization to add later.

“The AI industry feels very shiny and very new, but there is a lot of old school discipline that is still extremely useful. You start with the idea, you go through the ideation, from ideation you create your spec, from the spec you do your project management, you assign tasks and do the work. It feels like vanilla old school engineering to me.”

Awareness before truth

The instinct when building a real-time knowledge system is to treat freshness and accuracy as a tension to manage: newer data is less verified, older data is more reliable, and the system needs some formula for weighing one against the other. But Knickerbocker rejects this frame entirely.

He explains from his engineering experience that why his systems are not trying to determine what is true. But instead trying to capture what is being claimed, and those are fundamentally different engineering problems.

“In the world of open source intelligence, it has less to do with right and wrong. It has less to do with facts. What I am looking for is really claims of what is going on in the world. I do not make that decision, and I do not allow my AI to make the decision about what is true or false either.”

In adversarial open source intelligence environments, two sources in direct conflict with each other are not a problem to resolve. But they are both signal. And what matters is that both claims are captured, clustered, and surfaced. Knickerbocker describes this structure using a frame he developed from years of NLP work: not clusters in the geometric sense, but ribbons, layered bands of related information that emerge from the data.

“You have a whole bunch of information and this top ribbon might be this bad thing happened. The next ribbon might be this event is happening at the library. The next ribbon might be a punk rock show is happening at this nightclub.”

The consequence of this design is that awareness comes before truth, and that is by intention. For the applications the Verdant Eye is built for, knowing what is being said and where is actionable before you know whether any of it is correct. Adjudicating truth is a downstream problem, one that can be handled by human judgment or purpose-built downstream systems, not by the ingestion layer.

Snapshot versus movie

What separates a real-time knowledge graph from an agent that runs searches is not a matter of degree. It is a structural difference in what kind of object the system is.

An agent with search tools takes a snapshot. It queries for something, gets back the most relevant indexed results, and presents them. The system only knows what it was asked to look for, and it only looks when prompted. A knowledge graph that continuously ingests from the open web is more like a movie: it is always recording, and when you query it, you are not taking a new photograph but pulling a frame from a film that has been running the whole time.

“If you use a tool to do a search to find out something, you are getting a snapshot of time. My systems capture the heartbeat of the Internet themselves, and they are always listening. It would be closer to a movie compared to a photograph.”

The practical difference shows up clearly in demos with clients who need urgent situational information. Running a calibration query against the Verdant Eye, asking what is the latest information, returns results anywhere from a few seconds old to a minute and a half old. That behaviour is not possible with a search-tool approach at any level of optimization.

An empty result matters too. If nothing has been reported in the last minute on a given topic, the system returns nothing. That discipline is load-bearing.

“An empty dataset,” Knickerbocker underscores, “is better than a hallucination.”

The rush to use a tool before understanding it

Knickerbocker spent years watching teams fail with graph databases in 2020 and 2021, and he sees the same pattern repeating now with agents and AI. Teams adopt the tool before building the understanding, and then cannot tell whether it is working.

“There is a rush to use agents before even understanding AI. And if the understanding is not there, then it is just wishful. If you do not know how it works, you cannot tell the difference between it ran and it ran correctly. Those are very different things.”

His own relationship with graphs predates the graph database trend by years. At Intel, from 2015 to 2019, he was using graph theory for data flow mapping, tracing inputs to outputs across thousands of scripts and hundreds of servers, using centralities to find important nodes and shortest paths to understand flow. He was never using graph databases. He was using the underlying science.

“I was never invited to the cool kid graph database parties. I was always just doing stuff with graphs and using it to map out data flows and fix production outages. Dead serious stuff.”

That gap between tool and understanding is what drove him to write Network Science with Python. Teams were populating graph databases and then stopping, treating the populated database as the deliverable. The value, in his view, starts the moment the graph is populated and you begin running network science on top of it.

“If you do not know about the basics of network science, then what are you gonna do with the graph database? You have put your data in the graph database. Whereas if you come to graph databases and you have learned about centralities, community detection, shortest paths, simulations, then it can really have an impact on your network thinking.”

Similarity is not the same as identical

The central limitation of vector-based retrieval is one that Knickerbocker has framed the same way since at least 2017. Semantic similarity is a probabilistic measure, not a logical one. Two pieces of text can be highly similar in embedding space while referring to entirely different things. A graph replaces probabilistic similarity with structural traversal.

“Similarity in language is not equal to same. I will say that one more time. Similarity is not equal to same. Similar sounding things can be very, very different from each other. A graph anchors things into a piece of context.”

In a GraphRAG system, the query does not ask what is most like this input. It traverses the graph: if you are asking about jazz events in Portland, Oregon, you are connected to the Oregon node, to the Portland node, to the jazz node. The hallucination space collapses because the answer space is bounded by structure rather than probability.

“In a GraphRAG system, if there is no match then the output is that there is no match. There is no hallucination opportunity. With a similarity-based system, there could be similarity even if it is only a single word in a paragraph.”

This intuition was clear to him during his data operations years at Intel. Log files across hundreds of servers written by dozens of engineers contain enormous amounts of natural language. Working with that language to understand what a production system was doing made it obvious that the structure, the graph, and the meaning, the language, were inseparable.

“Graphs show you where things go,” Knickerbocker adds. “But all of the context about what that node even is is often carried by language itself.”

Deliberate forgetting

Temporal drift is usually framed as a correctness problem: facts become outdated, relationships change, the graph silently goes wrong. The standard response is validation pipelines, contradiction detection, freshness scores. Knickerbocker builds around a different principle.

Sharing from his experience building the Verdant Eye, Knickerbocker explains that the Verdant Eye system is not called the Verdant Brain because the metaphor is not storage. It is perception.

“The Verdant Eye sees, and it does not contain eternal memory, because that is not what an eye does. An eye sees. When the scene changes, the scene changes. Your eyes do not need to be recalibrated. The thing has just changed.”

The design principle is biological: living systems do not maintain eternal memory, and a knowledge system that tracks a living world should not either. The practical implementation mirrors transactional database patterns he worked with throughout his career in data operations. Data that is no longer operationally relevant gets archived. The live layer runs on what is current, and the cost of maintaining it stays proportional to what the system actually needs to know.

“In a transactional database, you operate off of what you need, and data that is not needed eventually gets archived. I want to build AI that does not boil the ocean, that can be bootstrapped by individuals. Infinite memory is not just philosophically wrong for this kind of system. It is economically unworkable.”

The pass butter philosophy

Knickerbocker shares how he currently runs three production GraphRAG systems, each built for a distinct purpose with its own testing criteria. Verdant Intelligence operates at high altitude, tracking events across states and regions. Grooveseeker (which he created to put his AI system to a different and specific kind of use) operates at street level, finding events in specific cities on specific nights. A third system contains thirty years of AI research, useful for tracing the intellectual lineage of ideas when building new things.

The principle connecting all three comes from a scene in Rick and Morty. Rick builds a robot. The robot asks what its purpose is. Rick says: you pass butter.

“There is no testing framework anybody else can give me that is going to be fit for purpose for what I am trying to build, because I am not trying to build general intelligence. I am trying to build intelligence that serves a specific purpose.”

Each system is therefore tested against the specific failures that would make it useless for its purpose. For Grooveseeker, that means verifying the right city, not a city of the same name in a different state; the right date, not a historical recurrence of the same event; and a URL that actually leads somewhere you can buy a ticket.

The final test of the Grooveseeker system was to stop writing articles proving it worked and simply use it. The system returned events in Portland between March 10 and March 13. Knickerbocker even attended one of them, bought a ticket at the door, saw the band, and spoke to one of the musicians.

His AI did not send him to a venue that did not exist. “That is how I know it works,” Knickerbocker affirms.

🔍 In case you missed it…

🛠️ Tool of the Week

Graphiti — open-source temporal knowledge graph engine for AI agents

Highlights:

• Temporal fact management: Facts carry explicit validity windows — invalidated when superseded, never deleted, queryable at any point in time.

• Incremental ingestion: New data integrates immediately without batch recomputation, keeping the graph current as the world changes.

• Hybrid retrieval: Combines semantic search, BM25, and graph traversal in a single query, typically under 100ms.

• MCP-native: Ships an MCP server for direct integration with Claude, Cursor, and other MCP clients.

Learn more about Graphiti

📎 Tech Briefs

• Neo4j NODES AI 2026 closing keynote: From Data to Knowledge to Action - Sudhir Hasbe makes the case that most enterprise AI behaves as if it has never seen the organisation's own data, and how graphs help close that gap.

• Graph memory for LLM agents: the relational blind spots that flat vectors miss - Zep's LongMemEval evaluation shows 18.5% accuracy gains over vector baselines, context tokens dropping from 115,000 to 1,600, and latency falling from 29 seconds to under 3 seconds.

• FalkorDB Browser ships April 2026 update for GraphRAG developers - Adds favourite query saving, one-click connection string copying, enhanced graph statistics, and configurable table views..

• Neo4j Aura Agent Hackathon opens - Developers building knowledge-graph-grounded AI agents can register for cloud credits and managed GraphRAG platform access.

• Four GraphRAG papers accepted at ACL 2026 - PolyU’s Awesome-GraphRAG repo confirmed acceptances including ProbeRAG for retrieval faithfulness, LegalGraphRAG for legal reasoning, and LinearRAG for efficient graph construction.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and support my work.

Hone your skills to design, implement, and ship an AI-powered internal developer platform from scratch with Ajay Chankramath.

📅 25th and 26th April 2026 | 11:00 AM to 3:00 PM ET

Use code DEEPENG50 for 50% off, or grab the 2-for-1 deal

Register now

Includes access to The Platform Engineer’s Handbook by Ajay Chankramath upon release (eBook worth $35.99)

✍️ From the editor’s desk,

Welcome to the 42nd issue of Deep Engineering!

Multi-agent AI is widely being adopted in production, but the engineering reality is messier than the demos suggest. GitHub Engineering published a post by Gwen Davis laying out the most common ways multi-agent workflows fail in practice. Most failures come down to missing structure, not model capability.

Agents make implicit assumptions about state, ordering, and validation at the points where they hand off work to one another, and without explicit data formats, typed interfaces, and defined action schemas at those boundaries, the system breaks in ways that are hard to reproduce and harder to debug.

The architectural decisions you make around the model matter more than the model itself. That is precisely what today’s issue covers. Fitz Nowlan, VP of AI and Architecture at SmartBear, has spent years building agentic systems in production.

Let’s get started.

Featured Newsletter: The Cloud Playbook

If you own reliability, cost, and compliance in production on AWS, The Cloud Playbook newsletter is worth your attention.

It is a weekly read for engineering leaders building platforms where incidents, audits, and surprise AWS bills are real problems. It covers AWS architecture, platform engineering, FinOps, security, observability, and compliance frameworks like FedRAMP, HIPAA, and ISO 27001.

Subscribe to The Cloud Playbook

Multi-Agent Systems Need Rules to Stay Reliable in Production

by with Fitz Nowlan

As agents pass tasks to one another, reason autonomously, and compose solutions on the fly, it is easy to construct a version of multi-agent AI that looks technically sound on a whiteboard. In practice, however, that version tends to fall apart when context gets lost between handoffs, when loops run without terminating, and when open-ended systems become progressively harder to test, debug, or trust.

Fitz Nowlan, VP of AI and Architecture at SmartBear, has spent the past two years building agentic systems in production, and his view, drawn from that engineering experience, is that the reliability of a multi-agent system is determined less by the model you choose and more by the architectural decisions you make around it.

Preserving facts across the handoff boundary

The first place multi-agent systems break down is at the handoff. When agent A passes context to agent B, the question is not just what to pass but how to pass it. Raw string prompts are the path of least resistance, but Nowlan argues they are also the path most likely to introduce errors.

“We are almost always using some form of structured data,” he points out. “It’s generally going to be JSON. What we’ll often try to do on top of that JSON is define, effectively, a domain-specific language, or DSL, or our own schema, to cache, kind of, to preserve truth that we’ve already determined.”

The distinction matters because of what happens when you do not preserve that truth. Suppose an agent is working within the context of a browser and identifies that two elements appear inside the same container on a page. That relationship has been syntactically proven through a screenshot, the HTML, or the DOM. If you pass that context as plain text to the next agent, that agent has to decompose the text and then probabilistically reconstruct whether the relationship exists at all.

“That other agent has to effectively decompose and then probabilistically, i.e., potentially hallucinate, that relationship is there,” Nowlan notes. “So what we try to do is communicate over JSON and lock in to the JSON in our custom schema the facts that we’ve identified or extracted from one context or world to another.”

The practical principle is simple enough. Anything that has been established as a fact should be encoded as a fact, not as prose. Structured contracts at the handoff boundary are not just a formatting preference. They are a mechanism for preventing the downstream agent from having to guess at things the upstream agent already knew.

When MCP outperforms a static API wrapper

Model Context Protocol (MCP) has become a standard topic in agentic architecture discussions, and anyone planning to use it should understand where it actually adds value and where it does not.

“If you knew all five workflows that your customers ever did in your application, well, then you should just statically code them up,” Nowlan reasons. “Make it a single API endpoint, take the inputs at the outset, chain them all together, and spit the outputs back out to your user. That would be highly efficient. You wouldn’t then need MCP or agents at all.”

The benefit of MCP is the emergent use case. That is the workflow your users need that you did not anticipate, in the sequence they need it, composed on the fly from the tools you have exposed. By giving the model a set of tools rather than a fixed pipeline, you allow the system to compose solutions for problems you never explicitly programmed for.

“If you have an application that’s very diverse, with a diverse set of data, with a wide-ranging type or class of data, and maybe even different user roles that are coming in and using your product, that’s where MCP can really be a massive unlock, because the composition artifacts are effectively infinite,” Nowlan observes.

The corollary is equally important. If your application is relatively rigid and your users reliably need the same five things, MCP introduces complexity without adding value. The decision to use it should follow directly from the diversity of your use cases, not from the fact that it is available.

The 80-20 rule for DAG versus autonomous orchestration

One of the more practically useful frameworks Nowlan describes is how his team decides when to use a fixed Directed Acyclic Graph (DAG)-based workflow versus an open-ended MCP-style loop, and how that decision evolves over time.

The starting point is openness. When a new feature or product area is introduced, the team tends to begin with the full MCP loop. The premise is straightforward. Here is the set of tools. Let the system compose whatever the user needs. This preserves maximum flexibility while the team learns how customers actually use the system.

Over time, patterns emerge. Certain workflows appear repeatedly. Those are the candidates for promotion into a DAG.

“There’s kind of an 80-20 rule here, where 80% of the time your customers are looking to solve their problems with 20% of these key workflows that you’ve identified,” Nowlan notes. “Those should then be translated into DAGs, into workflows, into a little bit stricter information flow architecture.”

The payoff is significant, Nowlan affirms. Once a workflow is on a DAG, you can optimise aggressively. You can use a cheaper model for nodes that do not require frontier-level reasoning. You can shed context that is irrelevant to a particular step rather than packing everything into the window. You can bound latency because the execution path is known in advance.

“When you get on that track, that predefined workflow, that’s where you can save cost, you can potentially use a cheaper model for a particular node that you know doesn’t need the most expensive model. And you can also shed information from context in that workflow when you know it’s not necessary for the outcome.”

The open-ended MCP loop remains available for everything else, for the 20% of use cases that do not fit a known pattern. The architecture supports both, and the team actively monitors usage to identify when a new workflow has crossed the threshold and is ready to be promoted. There is no firm cutoff, just the observation that a workflow is appearing often enough to justify the investment in making it faster and cheaper.

The infinite loop problem

Bounding execution is one of the more fundamental challenges in agentic systems, and it is one that the industry is still working through. The early solutions were basic by necessity. At the most fundamental level, Nowlan explains, it comes down to two questions: has the system hit a timeout, and has it exceeded a set number of attempts.

The deeper issue is that the problem did not originate with modern agents. In the early days of GPT-3-era pipelines, a common approach was to ask the model for one action at a time, execute it, and return for the next. The loop would never terminate because the model had no reliable sense of when it was done.

“We quickly realized they would literally go on forever. The loop would just never terminate, would never know that it was done, or it would know that it was done way too early.”

The response was to shift from reactive step-by-step prompting to upfront scoping. Rather than asking the model what to do next after each action, you ask it at the outset to decompose the overall task into a bounded set of subtasks, and then you execute those subtasks in sequence and stop.

“We pivoted from the early days toward putting more bounds on the problem space. What are the things you think I should do? Break this down into a set of tasks, a set of subtasks, and then I’ll take each of those subtasks in turn, but when I get to that last subtask, I’m done. I’m not going back and asking you for more.”

As models have improved, this has relaxed somewhat. The team now allows a final evaluation pass, a double or triple check at the end, before closing the loop. But the underlying principle has not changed. Get the model to scope the work before execution begins, and treat that scope as a constraint rather than a suggestion.

“You want to get the AI to put bounds and scope around the overall work that you’ll be doing to solve or complete some task, and try to stick to that as a guide so that you don’t run off into the infinite space of querying forever.”

Testing non-deterministic flows

Testing a system where the execution path changes based on the model’s output requires a different approach from conventional integration testing. Nowlan and his team lean toward trace-based evaluation, grounded in a close understanding of realistic user inputs.

“All of our evaluations are, we think, we hope, reasonably close to the reality of those inputs that we’re going to get from our end users.”

The logic is that if you understand the domain well enough, you can anticipate the shape of the tasks users will bring to the system even if you cannot predict the exact inputs. In a web testing context, for example, that means understanding that users are going to log in, fill out forms, scroll through lists, and navigate between pages. Those logical actions become the basis for evaluations, and the evaluations can then check whether the agent followed a reasonable path to completion rather than whether it followed an exact predetermined path.

At the same time, Nowlan acknowledges that no evaluation suite fully covers the space of real-world inputs. The complement to pre-built evaluations is comprehensive logging and tracing of every prompt and response the system exchanges with the model.

“We log and trace all of our inputs and outputs that we exchange to the LLMs, and then we can go back and debug those, and examine those, and we can obviously use AI to probabilistically parse and understand those inputs and outputs.”

This creates a feedback loop that pre-built evaluations cannot replicate. When a user reports a bad experience or churns, the team can go back to the traces for that user, examine the prompt and response sequences, and use a model to evaluate whether the quality of the AI outputs degraded at any point. The evaluation happens after the fact, using the actual production inputs rather than synthetic ones.

Nowlan’s broader point is that evaluation in non-deterministic systems is not a gate you run before deployment. It is an ongoing process that runs in parallel with production, using real data to surface quality issues that no pre-deployment test suite would have caught.

🛠️ Tool of the Week

smolagents — Hugging Face’s open-source library for building agents that think in code

Highlights:

• Code-first agents: Generates and executes Python code instead of structured tool calls, which can reduce back-and-forth with the model in some workflows.

• Model-agnostic: Works with local Transformers models, Hugging Face Inference API, and providers via LiteLLM.

• Supports sandboxing: Can run code in environments like Docker, E2B, or Pyodide, but requires proper setup for safety.

• Hub integration: Tools and agents can be shared and reused via the Hugging Face Hub.

Learn more about smolagents

📎 Tech Briefs

• Anthropic moves Claude Mythos into controlled early access under Project Glasswing - Limited rollout to partners including Amazon, Apple, and Microsoft for defensive security work, with early reports pointing to strong capability in identifying software vulnerabilities

• Microsoft Copilot Studio ships multi-agent orchestration to general availability - Now supports A2A protocol for agent-to-agent delegation and cross-app agent reuse via the Microsoft 365 Agents SDK.

• Google releases Gemma 4 under Apache 2.0 - Ranges from lightweight edge models to a 31B-parameter variant with strong performance on reasoning, agentic workflows, and multilingual tasks.

• LangGraph ships async subagents in latest Deep Agents update - Adds non-blocking background subagents alongside type-safe streaming and Pydantic coercion in the new v2 API.

• Microsoft releases open-source Agent Governance Toolkit - A seven-package system that intercepts every agent action before execution at sub-millisecond latency, with native integrations for LangChain, CrewAI, Google ADK, and Microsoft Agent Framework.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and support my work.

Learn ownership, RAII, smart pointers & allocators to eliminate leaks/UB—live hands-on with Patrice Roy—#1 bestselling author of C++ Memory Management and ISO C++ Standards Committee (WG21) member. Online: Sat Apr 11, 10:30 AM–Sun Apr 12, 4:00 PM ET. Limited seats.

Register now

✍️ From the editor’s desk,

Welcome to the 41^st issue of Deep Engineering!

As AI scaling hits the memory wall, system performance is increasingly limited by data movement rather than raw compute power. This shifts the optimization focus toward data locality because adding cores provides no benefit when bandwidth bottlenecks the system. Writing fast code now requires precise control over memory layout and concurrency, yet these techniques often introduce complexity that becomes unmanageable as codebases grow.

That specific tension defines this week’s feature with Sam Morley, Research Software Engineer and Mathematician at the University of Oxford and author of The C++ Programmer’s Mindset.

Building on our earlier discussion in Part 1 around decomposition and abstraction costs, today's issue digs deeper into the guiding principles for scaling C++ systems. It covers how teams can keep complexity under control as codebases grow, why template metaprogramming deserves extreme caution, and why thinking with the machine is non-negotiable when optimising cache layout.

Let’s get started.

Stop Building Vault

Secrets, PKI, & PAM in one platform. Postgres-backed. No custom orchestration. Flexible deployment.

Start for free today →

Adopting the C++ Programmer’s Mindset (Part 2) with Sam Morley

Before listing any specific practices, Morley introduces a concept he returns to repeatedly. He calls it the future you.

“Future you is your future self and for all intents and purposes this is a different person,” Morley says. “When you’re writing some code, you understand things the way they are in the context of what you’re doing at the moment. Future you will have lost this context. So, when you come back to your code in a month, six months, a year’s time and you look at it and you think what was I thinking — almost surely the answer to that is: I don’t know.”

The practical implication is easy to skip under deadline pressure, but it could be costly to ignore. Comments are not just for teammates. They are messages to yourself, written at the moment of maximum understanding, for a future reader who no longer has that understanding. Morley is explicit that this does not mean narrating the obvious. What he does instead, particularly on mathematically intricate work, is write large block comments that describe where the process is, how the next section works, and what the algorithm is intended to achieve. “These comments save me so much pain when I jump off the project for a week and then go back and have to remember exactly what I was trying to do.”

Beyond comments, Morley points to three C++ specific practices that matter most for scalability. The first is a strict separation of concerns. Numerical computation should not live in the same place as user-facing code. Components should be modular enough to be tested in isolation. This also pays dividends when distributing computation across clusters, because reusable tight-loop routines can be dropped into different distribution mechanisms without rework.

The second practice is thinking about thread safety earlier than you think you need to. Designing class members with that future in mind costs almost nothing early and can be enormously expensive to retrofit later. The third is keeping your build system clean. Morley uses CMake and is emphatic about this. “Having a broken build system is far worse than having broken code. It’s much harder to figure out what exactly has gone wrong if your build system is broken.” Build systems accumulate debt quietly and the consequences arrive at the worst possible moment, usually when you need to extract a component into its own library under time pressure.

When metaprogramming becomes the problem

“I’ve seen some horrendous template metaprogramming in my life,” Morley reflects. “I’ve written some horrendous template metaprogramming in my life. I’m going to be the first one to admit that it’s never worth it.”

The target here is the elaborate kind: deeply nested type machinery and SFINAE-heavy enable_if chains. Instantiating a complex template metaprogram can easily double compile time for a single translation unit. At scale, across thousands of files, the cost becomes structural. Morley notes this is exactly why Google kept metaprogramming to an absolute minimum when writing Abseil. Modern C++ has reduced the genuine need for TMP significantly. Concepts and constexpr functions cover much of what engineers used to reach for TMP to solve, with readable syntax and without the compile time penalty.

Lambdas are a different conversation. Used correctly, Morley thinks they are one of the most readability-enhancing tools in the language. Used carelessly, they introduce exactly the kind of invisible coupling that makes the future you miserable. His specific warning is about lambdas that capture and modify values defined far away from where the lambda is used. “Every time you think what is this lambda doing, it’s modifying something that you’ve not looked at for a long time because your screen has been further down the page.” In cases where a lambda is the only place that reads or writes a particular value, Morley argues the value almost certainly belongs inside a class, where the ownership and mutation path are explicit.

Thinking with the machine

Morley uses a road analogy to introduce hardware-aware programming. If you are driving down an unfamiliar road in the dark, you slow down because you cannot see what is ahead. Knowing the road means you can go faster without risk. The system you are running the code on is the road, the code you are writing is the car, and understanding the road conditions means you can drive faster with confidence.

The most practically important piece of hardware to understand is the cache hierarchy. Cache behavior is invisible in the code but determines a large part of actual runtime performance. The canonical example comes from the games industry: the debate between Array of Structs (AoS) and Struct of Arrays (SoA).

If you represent each game entity as a large struct and store those structs in a vector, then iterating over position data means striding across the full width of each struct to reach the next position value. “Big strides are bad for the cache. What you really want is all of the position data to be close together.”

The alternative is to separate data by type, storing all position values together and all velocity values together, so that each field is laid out contiguously in memory rather than interleaved across structs. As Morley puts it: “This transformation basically doubles or quadruples your throughput because now you don’t have to step over all of the useless data in order to update your position.”

Matrix multiplication illustrates the same principle at a more concrete level. In one direction, data access is sequential and cache-friendly. In the other direction, moving between rows in a column means jumping across memory in steps as wide as the entire matrix, which is exactly what the cache is designed to avoid. The standard mitigation is tiling: load a small tile of the matrix, do as much computation as possible on that tile, then move to the next one. Morley walks through a tiled implementation in the book and notes roughly a four-times improvement over the naive approach from tiling alone, before any SIMD or pipelining optimisations are applied.

Branch prediction and SIMD matter too, but Morley is careful about context. For general-purpose application code, the compiler will usually make reasonable decisions, and the latency bottleneck is more often a network call or a disk write than an instruction pipeline stall. These details start to matter seriously when throughput is the primary constraint: training large models, running physics simulations, anything where a microsecond saved per operation compounds into meaningful wall-clock time across billions of invocations. “Taking an extra microsecond to do a calculation is devastating when you have to do that a billion times.”

Concurrency: two models, two risk profiles

Morley distinguishes two fundamentally different multi-threaded scenarios that require different mental models.

The first is data parallelism. A large dataset split across threads, each thread operating on its own independent range with no shared state. “There’s never any overlap. Each thread goes away, does its work, and the results are put in the buffer.” This is relatively safe territory. Parallel algorithms and OpenMP handle much of the machinery, and as long as the data partition is clean and there is no self-referential access, data races are structurally prevented.

The second scenario is harder. Multiple worker threads operating on shared state within a larger system. A dispatch queue is the typical pattern, where a main thread stacks work items and worker threads pull from the queue. Here, ownership discipline becomes non-negotiable.

Morley’s design goal is simple to state and difficult to achieve: only one thread, one function, one whatever should be able to modify a value at any given time. There are two ways to achieve this. The first is architectural: design the program so that each thread exclusively owns its data and never touches another thread’s values. The second is synchronisation: use atomics, mutex-locked values, or other thread-safe mechanisms to control access.

Morley urges engineers to read the synchronisation documentation carefully before reaching for any of these tools. Deadlocks happen when engineers use them without fully understanding their semantics. The architectural solution is always preferable when achievable: if you can design the program so that each thread never touches another thread’s data, you eliminate the entire class of problem. When you cannot, synchronisation primitives are available, but the burden of correctness falls entirely on the engineer.

Memory safety and the Rust argument

The issue closes on a topic that provokes strong reactions in C++ communities. With around 70% of serious security vulnerabilities attributed to memory safety failures in C and C++ code, the question of whether the language itself is the problem is not academic.

Morley’s answer is direct. “Go and learn some Rust.”

He anticipates the objection. “A lot of C++ programmers turn their nose up when Rust is mentioned. Generally, the feeling is: we don’t need Rust, we can do all of this in C++. But that’s not the point.” The point is that the Rust compiler forces a specific kind of thinking about ownership and lifetime that C++ leaves entirely to the developer. Rust’s sync and send traits enforce thread safety at the type system level. Unsafe code is marked explicitly, which means the developer makes a deliberate choice to step outside safe boundaries rather than doing so accidentally. “Learning a bit of Rust will make you better at writing safe C++. The reverse is not true.”

For engineers who are not yet ready to invest in Rust, Morley's immediate C++ recommendations are practical and specific. Prefer std::array over C-style arrays. Use smart pointers instead of manual memory management; writing operator new directly in application code is an antipattern at this point. Use std::span rather than raw pointers when passing data around. Avoid the C standard library IO functions entirely: gets, puts, sprintf, and their relatives have documented vulnerabilities and safer equivalents exist.

“The best-case scenario for a bad memory access is a crash. If it goes unnoticed, it could happen for months before you notice that this has been producing garbage the entire time. By which point you’ve wasted months.”

The throughline from Part 1 holds here. Whether the subject is decomposition, abstraction, maintainability, or memory safety, Morley keeps returning to the same discipline: be conscious of what you are doing, understand the costs of your choices, write for the person who will read the code next, and treat the machine as a collaborator rather than a black box.

🔍 In case you missed it…

The complete Chapter 1 from Sam Morley’s book, The C++ Programmer’s Mindset 👇

🛠️ Tool of the Week

Google Benchmark — A microbenchmark support library for C++

Google Benchmark is an open-source framework for measuring the performance of isolated C++ code so you can test cache-aware designs and tight-loop optimizations with high precision.

Learn more about Google Benchmark

📎 Tech Briefs

• C++26 Finalizes Technical Work — The ISO committee completed the C++26 draft in London, locking in features like compile-time reflection, memory safety improvements, and contracts ahead of final approval.

• Rust 1.94.1 Shipped — A point release that fixes compiler regressions, resolves clippy false positives on match arms, and updates Cargo dependencies to address critical CVEs.

• RustRover 2026.1 Released — JetBrains introduced cargo-nextest integration, a call hierarchy view, LLDB 21 upgrades, and improved DWARF indexing for procedural macros.

• CppCon 2026 Opens Registration — Registration is open for the Colorado event, with sessions covering AI infrastructure, low-latency design, and hardware-aware performance.

• Linux Kernel 7.0 Approaches — After Linux 6.19, the merge window for 7.0 is now open, pointing to a major version release expected later this April.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

A full-lifecycle workshop on designing, securing, evaluating, and optimizing agent systems that hold up in production.

Online: March 29 | 10:30 AM - 4:00 PM EST

✓ 2-for-1 deal: Bring a colleague, get 2 passes for the price of 1.

Register now

✍️ From the editor’s desk,

Welcome to the 40th issue of Deep Engineering!

The conversation around AI infrastructure is heating up. Nvidia this week pointed to more than $1 trillion in AI chip revenue opportunity by 2027, explicitly tying that outlook to growing demand for inference. And when inference at scale becomes a boardroom number, model efficiency stops being a niche engineering concern and starts shaping cost control, product design, and build versus buy decisions. That is the context in which small language models are becoming harder to ignore.

The case is not that general-purpose LLMs are going away – it is that a large share of production workloads, classification, structured extraction, and domain-specific reasoning, do not need an LLM. Serving a 7-billion parameter SLM runs 10 to 30 times cheaper than a 70-billion parameter LLM, and that gap compounds across millions of requests. For engineering leaders, the question is no longer whether SLMs belong in the stack – it is how to train them to reason well, how to route across multiple specialized models at runtime, and how to govern correctness when AI is accelerating the pace of change.

This week’s feature is based on a conversation with Karun Thankachan, Senior Data Scientist at Walmart. We dig deep into how to treat reasoning as a budgeted resource, when routing across specialized models beats relying on a single large one, and why context engineering is winning over fine-tuning for many teams right now.

And we also have a thought leadership piece by Lee Peterson, VP of Secure WAN Product Management at Cisco, on why the network layer underneath agentic systems matters just as much as the models running on top of it.

Let’s get started.

Stop Building Vault

Secrets, PKI, & PAM in one platform. Postgres-backed. No custom orchestration. Flexible deployment.

Start for free today →

The Case for Small Language Models in Real Systems

by with Karun Thankachan.

Karun Thankachan’s conversation with us keeps circling back to one production truth:

The “best” model is the one that meets your accuracy target inside your cost and latency budget.

In early phases of a product, teams often use large general-purpose LLMs to explore what’s possible. But once the user experience stabilizes, the engineering focus shifts – toward predictable latency, controllable spend, and repeatable quality. That shift is where small language models (SLMs) start to matter more than giant LLMs, because many real systems don’t need universal reasoning – they need reliable, narrow reasoning at scale.

Recent research and releases support this trajectory. For example, Zihao An et al. (Jan 20, 2026) show that a 0.6B-parameter reasoning model (ReasonLite) can reach 75.2% accuracy on AIME 2024, and they attribute it to distillation and training recipe design rather than sheer parameter count. This is exactly the kind of result that makes SLMs a serious production option once you know what task you need to solve.

Cost-effective reasoning is a budget problem

Thankachan’s core argument is not that LLMs are bad. It’s that general-purpose reasoning is expensive overkill for many high-volume workflows (customer support classification, catalog normalization, routing, anomaly triage, internal copilots for one domain). When a feature goes from thousands of calls to millions, the economics change.

A key point from the research community is that reasoning often becomes expensive because teams implicitly pay for long reasoning traces even when a request is simple. Kun Liang et al. (Jan 13, 2026) describe this problem directly: long-form chain-of-thought reasoning can improve accuracy, but applying “overlong reasoning” uniformly at inference time creates “substantial and often unnecessary computational cost.” They propose ORBIT, which aims to make reasoning behavior controllable across multiple “budgets.”

That maps cleanly to production reality: you don’t want a single mode of intelligence. You want a system that can do “fast and cheap” when it can, and “slow and deep” only when it must.

Teaching SLMs to reason without bloating latency

The approach Thankachan describes in the interview treats SLM training like an engineering workflow: combine distillation techniques, program-aided verification, and evaluation probes to close the gap between a teacher model and a smaller student.

What’s notable is that the same idea is showing up in recent public work: reasoning performance in small models is increasingly a function of distillation data quality, training curriculum, and trace control, not only scale.

In the AMD ReasonLite write-up, the authors describe a two-stage distillation curriculum: first fine-tune on short chain-of-thought for efficiency, then fine-tune on long chain-of-thought for peak accuracy. They also report that the “Turbo” variant tends to generate shorter outputs and offers a better efficiency/accuracy balance, while the long-trace variant reaches higher peak performance. This lines up with the production emphasis Thankachan describes: if your model is trained to be verbose, it will be slow and costly; if you want predictable serving, you must treat thinking length as a first-class knob.

A practical pattern for senior engineers is to separate training-time compute from serving-time compute:

• Use heavier techniques (like generating multiple traces, voting, or deeper teacher reasoning) during dataset creation and training, where batching and offline pipelines reduce the pain.

• Enforce strict budgets at inference (max reasoning tokens, max latency, max retries) and treat budget violations as signals to improve training data or shorten traces.

The leadership takeaway: budgeting belongs in the contract, not just in infrastructure dashboards after costs explode.

Orchestrating many small models beats one giant model

The SLM-Fusion framing Thankachan describes – routing across specialized models, merging where it helps, and wrapping the whole thing behind an OpenAI-compatible gateway – reflects a broader trend:

Modern systems are becoming mixtures, not monoliths.

Recent work in retrieval-heavy QA shows why. Yasaman Zarrinkia, Venkatesh Srinivasan, and Alex Thomo (Mar 18, 2026; v2) report a striking result when evaluating a Graph-RAG system: 77%–91% of questions contain the gold answer in the retrieved context, yet end-to-end accuracy is only 35%–78%, and 73%–84% of errors are reasoning failures.

That finding matters for architecture decisions. It implies that “better retrieval” alone isn’t enough; you need better reasoning structure and better context shaping. The same paper proposes two augmentations – structured prompting plus graph-walk compression – and then shows something that should make engineering leaders pay attention: with question-type routing, an open-weight Llama-8B configuration can match or exceed an unaugmented Llama-70B baseline across multiple benchmarks at about 12× lower cost.

This is the pragmatic case for SLM-centric production:

• Route by intent/domain and cost budget, not just by “best model available.”

• Specialize models for stable tasks (where fine-tuning or distillation pays off).

• Escalate to a larger model only when uncertainty is high or stakes demand it.

Why context engineering is beating fine-tuning right now

Thankachan argues that industry attention is shifting toward RAG and context engineering because it can be cheaper and easier to operationalize than constant fine-tuning – especially when domain knowledge changes frequently.

The Graph-RAG results above reinforce this. Even when the answer is present in retrieved context, the failure mode is often reasoning over context, not retrieval itself. That pushes teams toward two concrete engineering priorities:

• Structure the prompt so the model reasons in a way that matches the data representation (for Graph-RAG, they use SPARQL-style decomposition).

• Compress context so the model sees less noise and wastes fewer tokens, while still preserving the minimal information needed to answer.

For architects, the key decision lens is: fine-tuning changes the model; context engineering changes the problem presented to the model. When requirements or source documents change weekly, the second approach often has a faster operational loop.

What’s next – budget-aware reasoning and diffusion models

Thankachan flags diffusion models as a trend to watch. The key reason is not hype – it’s that diffusion-based language models may offer new efficiency controls that differ from token-by-token autoregressive decoding.

A concrete example: Vittorio Rossi et al. (Mar 6, 2026) argue that diffusion language models (DLMs) can waste compute because they run denoising steps over a fixed maximum length even when the desired response is short. They propose a zero-shot mechanism to estimate required output length and crop the context window before generation, reporting large FLOP reductions without statistically significant performance loss (and improvements in 2 of 4 benchmarks).

This matters because “right-sized reasoning” is becoming an explicit research goal across paradigms – whether you do it with controllable budgets in autoregressive models (as in ORBIT) or with length control in diffusion models.

Finally, distillation itself is getting more modular. Shaoxiong Yang et al. (Feb 1, 2026) propose FutureMind, which combines adaptive distillation with a multi-stage reasoning pipeline and retrieval guidance – explicitly acknowledging that SLMs are attractive for low-latency settings but often struggle on knowledge-intensive tasks without structured reasoning and retrieval help.

Key Takeaways

• Treat reasoning as a budgeted resource. Put token limits and latency targets into your API contract, not just your monitoring dashboards, and design “fast vs deep” modes intentionally.

• Distillation makes small models competitive when the recipe is right. Evidence from subbillion models shows that curriculum design and trace supervision can close surprising amounts of the gap.

• RAG failures are often reasoning failures, not retrieval failures. Even when the answer is in-context, models may fail to use it – so prompt structure and context compression matter.

• Routing is a cost lever. With the right augmentations and routing, smaller open-weight models can rival much larger baselines at large cost reductions.

• Watch diffusion LMs for new efficiency primitives. Length-aware diffusion decoding suggests a different path to controlling wasted compute, which may matter for production workloads dominated by short answers.

🔍 In case you missed it…

💡 Industry Perspective

Agentic AI Is Redefining Edge Infrastructure

Thought leadership piece by with Lee Peterson, VP of Secure WAN Product Management at Cisco.

As agentic systems move closer to the edge, the network underneath them matters just as much as the models running on top. Peterson makes the case that organizations still designing around centralized control will hit a wall at exactly the wrong moment, and that getting edge compute and networking right is not an upgrade but a rethink from the ground up.

Packt Deep Engineering

Agentic AI Is Redefining Edge Infrastructure

Artificial intelligence is entering a new phase with agentic AI, where autonomous systems perceive, decide, act, and learn without constant human oversight, operating independently across distributed environments while collaborating with other agents in real time…

Read more

3 months ago · Saqib Jan

🛠️ Tool of the Week

DistillKit — open-source knowledge distillation toolkit for language models

Highlights:

• Two distillation methods in one toolkit: Supports both logit-based distillation and hidden states-based distillation, which aligns intermediate layer representations and allows distillation across different model architectures.

• Offline distillation at scale: An advanced logit compression system using polynomial approximation, error-diffusion quantization, and bit-packing makes it practical to train from pre-captured teacher outputs without running the teacher model live during every training step.

• Composable loss functions: Mix and match KL divergence, JSD, TVD, ranking losses, and hidden state alignment to control exactly what the student model learns and how aggressively it is pushed toward the teacher’s behaviour.

Learn more about Distillkit

📎 Tech Briefs

• Hugging Face Spring 2026 open source report released: The platform now has 13 million users and over 2 million models, with Chinese models accounting for 41% of downloads and robotics datasets growing 23x to become the largest category.

• vLLM 0.18.0 released with 30.8% throughput improvement and new Realtime API: The release ships async scheduling with Pipeline Parallelism delivering a 30.8% end-to-end throughput gain, a WebSocket-based Realtime API for streaming audio, and expanded hardware support across NVIDIA, AMD, Intel, and TPU.

• OpenAI launches GPT-5.4 mini and GPT-5.4 nano: Two smaller models optimized for speed and efficiency in coding, subagents, and high-volume tool use, continuing the industry shift toward task-specific models over general-purpose frontier inference.

• NVIDIA’s AI Grid reference design cuts edge inference costs by 76% in early benchmarks: The architecture distributes AI compute across telco network nodes, with Comcast benchmarks showing sub-500ms latency at P99 burst traffic and 80.9% throughput gains over a centralized deployment.

• Anthropic announces the Anthropic Institute: A new research body focused on the economic, societal, and security impacts of advanced AI, signalling that frontier labs are formalizing impact analysis as a structured part of model development and evaluation.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

A full-lifecycle workshop on designing, securing, evaluating, and optimizing agent systems that hold up in production.

Online: March 29 | 10:30 AM - 4:00 PM EST

✓ 2-for-1 deal: Bring a colleague, get 2 passes for the price of 1.

Register now

✍️ From the editor’s desk,

Welcome to the 39th issue of Deep Engineering!

In Issue #34 we spoke with José Dimas Luján Castillo and Ron Veen, authors of Kotlin for Java Developers, to unpack practical decision lenses for modernizing JVM systems. In the past week, JetBrains released Kotlin 2.3.20 with a simpler Maven setup and compiler-plugin improvements, and OpenAI added GPT-5.4 mini to Codex as a faster, lighter model for coding tasks. The language stack is evolving, and so are the AI tools teams now use to change production code at speed.

But how can a large Java codebase be migrated without turning modernization into a rewrite, a slowdown, or a source of subtle semantic risk? We are collaborating again with Ron Veen, whose article answers that by treating migration as a controlled, layer-by-layer transformation—one where Kotlin’s gains in null safety and expressiveness matter, but where the harder engineering work lies in framework compatibility, review discipline, and keeping humans accountable while AI accelerates the mechanics.

Ron Veen is a JVM veteran with over 20 years of experience across enterprise systems, from mainframes to modern microservices. Veen is an Oracle Certified Java Programmer and certified Sun Business Component Developer. As a Lead Developer at Team Rockstars IT and a regular international speaker, Veen brings both hands-on experience and architectural perspective.

Let’s get started

Mike Scientific Repo: Tons (> 1200) of FREE Math, AI, Machine Learning Books, All reviews (almost 600), presentations and more learning resources: https://github.com/merlihson/scientific-resources

Science and AI with Mike: Telegram channel - fresh deep learning paper reviews and free books: https://t.me/science_and_ai_with_mike_english/

From Java to Kotlin at Scale – with AI in the Loop

by with

Most large engineering organizations are sitting on a paradox.

Their core systems – often millions of lines of Java – are stable, battle-tested, and deeply valuable. But they’re also increasingly out of step with how modern teams want to write software: safer by default, more expressive, and easier to reason about. Kotlin promises exactly that. The problem is not why to migrate, it’s how to do it without freezing product velocity or introducing subtle, system-wide risk.

This is where Ron Veen’s migration story becomes relevant. Instead of treating Java-to-Kotlin as a one-shot rewrite or a tooling exercise, he approaches it as a controlled, layer-by-layer transformation – reconstructed on an OpenMRS codebase to mirror real-world enterprise constraints. The twist: AI is not just assisting but actively accelerating the migration – while humans remain accountable for every semantic decision.

What emerges is not just a migration playbook, but a shift in how we think about large-scale code evolution. The bottleneck is no longer writing code – it’s governing correctness at speed. And in that world, success depends less on conversion tools and more on how teams handle semantics, frameworks, and oversight under AI-assisted velocity.

To this end, today’s issue focuses on three action points for senior engineers and engineering leaders:

• Kotlin migrations succeed when you treat them as a semantic change (types, nullability, and idioms), not a mechanical translation.

• Framework constraints (Spring/Hibernate/JPA) are where migrations usually get stuck – unless you plan for Kotlin’s “final by default” stance and use the right compiler plugins.

• AI can compress weeks of conversion work into days, but it shifts the hard work to governance: review strategy, testing cadence, and repeatable patterns.

1: Treat migration as a semantic refactor, not a syntax swap

One of the easiest mistakes in a Java-to-Kotlin migration is to treat it as a tooling problem. Run the IDE converter, clean up a few warnings, and move on. Veen’s work makes it clear why that approach fails: working Kotlin isn’t the goal – idiomatic Kotlin is. And the gap between the two is where most of the risk, and value, lies.

Veen’s core point is easy to miss if you’ve only used IDE “convert Java to Kotlin” tools: working Kotlin is not the goal – idiomatic Kotlin is. The difference shows up most sharply in nullability, because Kotlin forces you to say what can be null and what cannot.

That’s also why Kotlin’s Java interop rules matter for incremental migration. Kotlin can call existing Java “naturally,” but when Kotlin reads Java types, it must assume any reference might be null. Kotlin handles this by using platform types, which relax compile-time null checks and can still fail at runtime if your assumptions are wrong. For leaders, the practical implication is: your migration policy must include a nullability strategy (what becomes T, what becomes T?, and when you tighten types).

Veen’s entity conversion example shows why this is a worthwhile discipline:

Java Version

@Entity 
@Table(name = "location_tag") 
@Audited 
@AttributeOverride(name = "name", column = @Column(name = "name", nullable = false, length = 50)) 
public class LocationTag extends BaseChangeableOpenmrsMetadata { 
 
    private static final long serialVersionUID = 7654L; 
    private Integer locationTagId; 
    
    public LocationTag() { 
    } 
 
    public LocationTag(Integer locationTagId) { 
        this.locationTagId = locationTagId; 
    } 
 
    public LocationTag(String name, String description) { 
        setName(name); 
        setDescription(description); 
    } 
 
    @Id 
    @GeneratedValue(strategy = GenerationType.IDENTITY) 
    @Column(name = "location_tag_id", nullable = false) 
    public Integer getLocationTagId() { 
        return this.locationTagId; 
    } 
 
    public void setLocationTagId(Integer locationTagId) { 
        this.locationTagId = locationTagId; 
    } 
 
    @Override 
    public Integer getId() { 
        return getLocationTagId(); 
    } 
 
    @Override 
    public void setId(Integer id) { 
        setLocationTagId(id); 
    } 
 
    @Override 
    public String toString() { 
        return getName() != null ? getName() : ""; 
    } 
} 

Kotlin Version

@Audited 
@Entity 
@Table(name = "location_tag") 
@AttributeOverride(name = "name", column = Column(name = "name", nullable = false, length = 50)) 
class LocationTag() : BaseChangeableOpenmrsMetadata() { 
 
    @Id 
    @GeneratedValue(strategy = GenerationType.IDENTITY) 
    @Column(name = "location_tag_id", nullable = false) 
    var locationTagId: Int? = null 
 
    constructor(locationTagId: Int?) : this() { 
        this.locationTagId = locationTagId 
    } 
 
    constructor(name: String?, description: String?) : this() { 
        this.name = name 
        this.description = description 
    } 
 
    override fun toString(): String = name ?: "" 
 
    override fun getId(): Int? = locationTagId 
 
    override fun setId(id: Int?) { 
        locationTagId = id 
    } 
 
    companion object { 
        private const val serialVersionUID = 7654L 
    } 
} 

With this approach, boilerplate shrinks, but more importantly, intent becomes visible in the type system.

For senior reviewers, the takeaway is that “semantic migration” is reviewable. You can ask: Do the nullability markings reflect reality? Did we remove boilerplate without losing invariants? Are we leaning on Kotlin’s strengths (properties, expression bodies, Elvis operator) in ways that reduce bug surface?

2: Framework constraints are the hidden migration backlog

Most Java-to-Kotlin migration pain doesn’t come from language syntax. It comes from frameworks that depend on bytecode tricks: proxies, reflection, and runtime enhancement.

Two Kotlin compiler plugins exist precisely to reduce this friction:

• The all-open plugin addresses Kotlin’s default of making classes and members final. Kotlin’s docs explicitly call out that this default is inconvenient for frameworks such as Spring AOP, which require classes to be open so proxies can be created.

• The no-arg plugin generates a synthetic zero-argument constructor for annotated classes, which helps frameworks like JPA instantiate entities via reflection even when Kotlin doesn’t define a zero-arg constructor explicitly.

These are leadership-level concerns because they determine whether your migration scales. If teams manually sprinkle open and constructors across hundreds of files with inconsistent rules, you get a “looks migrated, runs broken” outcome.

Veen shows this in his service implementation example:

@Service("medicationDispenseService") 
@Transactional 
open class MedicationDispenseServiceImpl : BaseOpenmrsService(), MedicationDispenseService { 
 
    @Autowired 
    lateinit var dao: MedicationDispenseDAO 
 
    @Transactional(readOnly = true) 
    override fun getMedicationDispense(id: Int?): MedicationDispense? = 
        dao.getMedicationDispense(id) 
 
    override fun saveMedicationDispense(dispense: MedicationDispense): MedicationDispense { 
        require(dispense != null) { "MedicationDispense cannot be null" } 
        return dao.saveMedicationDispense(dispense) 
    } 
 
    @Transactional(readOnly = true) 
    override fun getMedicationDispenses( 
        patient: Patient, 
        drug: Drug?, 
        dateRangeParam: DateRangeParam?, 
        startIndex: Int?, 
        limit: Int? 
    ): List<MedicationDispense> { 
        require(patient != null) { "Patient cannot be null" } 
        return dao.getMedicationDispenses(patient, drug, dateRangeParam, startIndex, limit) 
    } 
} 

In the preceding code, the open keyword is a runtime requirement, not a stylistic choice, when frameworks proxy your classes for transactions.

A practical leadership move here is to standardize how these requirements are met: prefer compiler plugins (where feasible) over manual edits. Kotlin’s docs note that you can use a Spring-specific wrapper (kotlin-spring) on top of all-open, and that Spring project templates typically enable it by default. Likewise, Kotlin documents that kotlin-jpa is a wrapper on top of no-arg and automatically targets common JPA annotations.

3: AI accelerates conversion, but governance determines quality

Veen’s migration stance is not “AI replaces engineers.” It’s closer to: AI replaces repetitive conversion work, so humans can spend time on correctness and architecture.

Recent developments reinforce why that governance burden is growing. Anthropic positions Claude Opus 4.6 as improved for coding tasks that involve longer planning, more reliable operation in larger codebases, and stronger debugging and code review – with a very large context window intended for long, multi-step work. In practice, Anthropic’s telemetry-based research suggests users are already letting Claude Code run autonomously for longer on the most complex work: the 99.9th percentile “turn duration” in Claude Code nearly doubled from under 25 minutes to over 45 minutes over a three-month span, and experienced users enable auto-approve more often while interrupting when needed.

That pattern maps directly onto migration risk. The faster you transform files, the easier it is to:

• Propagate a wrong nullability decision across a layer.

• Preserve a Java anti-pattern in Kotlin clothing.

• Break a proxy/reflection expectation in subtle ways.

For senior engineers and architects, the emphasis shifts to controlling the blast radius. Veen’s own best practices (small batches, frequent commits, tests between batches, and explicit pattern guides) are not process overload – they are how you turn “AI speed” into “organizational safety.”

Even Kotlin itself nudges teams toward explicitness where Java might have relied on convention. For example, Kotlin treats Java’s checked exceptions as unchecked – the compiler won’t force catches – so interoperability requires deliberate documentation patterns (for example, @Throws for Java callers, as Veen shows).

A smaller but telling example from Veen’s validator conversion shows how Kotlin idioms can reduce branching and clarify intent – if reviewers insist on it:

@Component 
@Handler(supports = [EncounterRole::class]) 
class EncounterRoleValidator : RequireNameValidator() { 
 
    @Autowired 
    private lateinit var encounterService: EncounterService 
 
    override fun validate(obj: Any, errors: Errors) { 
        super.validate(obj, errors) 
        if (errors.hasErrors()) return 
 
        val encounterRole = obj as EncounterRole 
        val name = encounterRole.name?.trim() 
 
        if (!name.isNullOrBlank()) { 
            val duplicate = encounterService.getEncounterRoleByName(name) 
            if (duplicate != null && encounterRole.uuid != duplicate.uuid) { 
                errors.rejectValue( 
                    "name", 
                    "EncounterRole.duplicate.name", 
                    "Specified Encounter Role name already exists" 
                ) 
            } 
        } 
    } 
} 

Key Takeaways

• Migration success depends on semantic decisions, not just conversion volume. Lock down your nullability rules early, and review them as architecture, not style.

• Use Kotlin’s compiler plugins to meet framework requirements at scale. Kotlin is final by default; all-open (and wrappers like kotlin-spring) exist because frameworks such as Spring AOP need open classes.

• Plan for reflection and JPA. If your entities need zero-arg constructors for runtime instantiation, no-arg (and wrappers like kotlin-jpa) can generate them safely.

• AI makes migration fast, but it increases the need for governance. Long-context models are designed for sustained, multi-step coding work.

• Adopt “human-in-the-loop” guardrails that match how agents are used in practice – users increasingly auto-approve and let agents run longer, intervening when needed.

🧠 Expert Insight

🔍 In case you missed it…

🛠️ Tool of the Week

Exposed — Kotlin’s type-safe SQL framework, now at 1.0

Exposed is JetBrains’ open-source SQL framework for Kotlin, hitting 1.0 this week with R2DBC support for reactive database access and a guaranteed stable API, making it a production-ready database layer for teams adopting Kotlin.

📎 Tech Briefs

• Java 26 released — Java 26 ships ten JEPs including HTTP/3 for the HTTP Client API, lazy constants in second preview, and an AOT cache that now works with any garbage collector including ZGC.

• Kotlin 2.3.20 released — The release removes significant manual Maven setup by automating source root configuration and stdlib inclusion, and adds name-based destructuring declarations to the language.

• Java 26 support lands in IntelliJ IDEA — IntelliJ IDEA adds full Java 26 support this week, including inspections and quick-fixes for lazy constants, primitive patterns, and the extended ahead-of-time cache for ZGC.

• Kotlin Foundation joins Google Summer of Code 2026 — The programme is accepting applications for projects spanning Swift-to-Kotlin interop, tail call support in Kotlin/Wasm, and a Kotlin education landscape research report, with the deadline on March 31.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

AI can refactor your code in seconds, but it can introduce subtle breakages just as quickly. Learn how to combine AI-assisted reasoning with AST-based structural guarantees to refactor code safely.

Online: March 14 | 10:00 AM EST

✓ 2-for-1 deal: Bring a colleague, get 2 passes for $48
✓ Includes 300+ page Mastering ast-grep ebook

Register now

✍️ From the editor’s desk,

Welcome to the 38th issue of Deep Engineering!

The rise of AI-generated code is putting pressure on a familiar engineering constraint: review does not scale as easily as output. Anthropic’s own recent research on Claude Code points in the same direction. It found that longer autonomous coding sessions are becoming more common, and that experienced users increasingly shift from step-by-step approval to letting the tool run and intervening when needed. But, if more code is being written with less direct review, how do engineers stop poor design from shipping in the first place?

Anthropic’s new Code Review system for Claude Code is one answer at the downstream end. It runs five specialized reviewers in parallel, scores findings on a confidence scale, and only posts high-confidence comments back to GitHub. That is a useful safeguard. But it also clarifies the limit of review itself. The more teams rely on layered automation to inspect pull requests, the more important the original design decisions become. Review can catch defects. It cannot compensate for weak boundaries, poor abstractions, or code that was hard to reason about from the start.

That is the context for this week’s Expert Insight from Steven F. Lott. Lott has been programming since the era when computers were large, expensive, and rare, and has worked with Python since the 1990s. He is the co-author of the newly published fifth edition of Python Object-Oriented Programming, updated for Python 3.13 with added material on areas such as type hints, testing, and professional software engineering practice. In this issue, his piece looks at object-oriented design as a practical engineering discipline: dependency injection, protocols, and duck typing as choices that shape coupling, change, and maintainability in production systems. We have also included Chapter 1, “Object-Oriented Design,” from the new edition for readers who want the full foundation.

Let’s get started.

PulseMCP Newsletter

Our friends at PulseMCP recently wrote up a great post on the rise of agentic engineering. It highlights both the very public, high profile case studies of top tier engineering organizations adopting the practice, and gets into the weeds of what’s working in the PulseMCP team’s personal coaching of engineering teams trying to make the transition from “AI pair programming” to “autonomous agents”.

Check out PulseMCP →

🧠Expert Insight

Part 2: Getting Started with Object-Oriented Programming in Python

by

Python provides three primary ways to structure immutable data: dataclass with frozen=True, NamedTuple, and TypedDict. Each carries different trade-offs that matter in production systems.

A dataclass with frozen=True provides immutability through runtime enforcement. When you attempt to modify a frozen dataclass instance, Python raises an exception. This makes frozen dataclasses useful when you need immutability guarantees but also need features like inheritance, custom methods, or post-initialization processing. The runtime check adds minimal overhead and catches mutation attempts immediately in development and testing.

NamedTuple provides immutability through tuple semantics. Because NamedTuple instances are actual tuples under the hood, they inherit tuple’s immutability at the language level. This makes NamedTuple marginally faster than frozen dataclass and ensures that mutation is structurally impossible rather than runtime-prevented. NamedTuple works well for simple data containers that do not require inheritance or complex initialization logic.

TypedDict does not provide immutability at all. TypedDict exists purely for type checking and documentation. At runtime, a TypedDict is just a regular dictionary with no special behavior or constraints. This makes TypedDict appropriate when you need to type-hint dictionary structures for API contracts or configuration, but you do not control the actual dictionary creation or mutation.

The choice between these three depends on what guarantees your system actually requires. If you need inheritance or custom methods, use frozen dataclass. If you need maximum performance for simple data structures, use NamedTuple. If you need to type-hint existing dictionary-based interfaces without changing runtime behavior, use TypedDict.

Hashability and when it matters

Immutable objects in Python can be hashable, which means they can be used as dictionary keys or stored in sets. This is not just a convenience feature. Hashability enables certain architectural patterns that would otherwise require significantly more complex code.

When you need to deduplicate objects, track which objects have been processed, or use objects as cache keys, hashability becomes a structural requirement rather than a nice-to-have feature. A frozen dataclass or NamedTuple containing only hashable fields is automatically hashable. A regular mutable dataclass is not hashable because its contents could change after being added to a set or used as a dictionary key, which would break the fundamental contract of hash-based collections.

SOLID principles in Python

The SOLID principles originated in statically-typed object-oriented languages like Java and C++, but they translate to Python with some adjustments for Python’s dynamic nature and duck typing.

Single Responsibility Principle states that a class should have one reason to change. In Python, this often means keeping data structures separate from business logic, keeping validation logic separate from transformation logic, and keeping I/O operations separate from computation. When a class starts accumulating methods that could logically belong to different concerns, it is time to split it.

Open/Closed Principle states that software entities should be open for extension but closed for modification. In Python, this is typically achieved through composition and protocols rather than deep inheritance hierarchies. Instead of modifying existing classes to add behavior, you compose new classes that delegate to existing ones or define protocol-based interfaces that new classes can implement without touching existing code.

Liskov Substitution Principle states that subtypes must be substitutable for their base types. In Python, this means that if your code expects a file-like object, any object that implements the file protocol (read, write, close) should work correctly. The principle prevents surprising behavior when substituting one implementation for another.

Interface Segregation Principle states that clients should not depend on interfaces they do not use. In Python, this translates to keeping protocols narrow and focused rather than creating large monolithic base classes with dozens of methods that most implementations leave empty or raise NotImplementedError.

Dependency Inversion Principle states that high-level modules should not depend on low-level modules; both should depend on abstractions. In Python, this is achieved through dependency injection and protocols. Instead of instantiating concrete dependencies inside a class, you pass dependencies in as constructor arguments, and instead of depending on concrete types, you depend on protocols that define the interface you actually need.

Dependency injection without frameworks

Dependency injection in Python does not require a framework. The pattern is simple: instead of creating dependencies inside a class, pass them in as constructor arguments.

python

# Without dependency injection
class OrderProcessor:
    def __init__(self):
        self.payment_gateway = StripeGateway()  # Hard dependency
        self.email_service = SendGridService()  # Hard dependency
    
    def process(self, order):
        self.payment_gateway.charge(order.amount)
        self.email_service.send_confirmation(order.email)

# With dependency injection
class OrderProcessor:
    def __init__(self, payment_gateway, email_service):
        self.payment_gateway = payment_gateway
        self.email_service = email_service
    
    def process(self, order):
        self.payment_gateway.charge(order.amount)
        self.email_service.send_confirmation(order.email)

The second version allows you to swap payment gateways or email services without modifying OrderProcessor. It allows you to test OrderProcessor with mock implementations that do not actually charge credit cards or send emails. It makes the dependencies explicit in the constructor signature rather than hidden inside the implementation.

Protocols and duck typing

Python’s protocols allow you to define interfaces without requiring inheritance. A protocol specifies what methods an object must have without requiring the object to inherit from a specific base class.

python

from typing import Protocol

class PaymentGateway(Protocol):
    def charge(self, amount: float) -> bool: ...

class StripeGateway:
    def charge(self, amount: float) -> bool:
        # Stripe implementation
        return True

class PayPalGateway:
    def charge(self, amount: float) -> bool:
        # PayPal implementation
        return True

Both StripeGateway and PayPalGateway satisfy the PaymentGateway protocol without inheriting from it. Type checkers like mypy will verify that any object passed where a PaymentGateway is expected actually implements the charge method with the correct signature.

This is duck typing with type safety. You get the flexibility of duck typing (if it walks like a duck and quacks like a duck, it is a duck) combined with static verification that the duck actually has the methods you are going to call on it.

Continue Reading

Chapter 1: Object-Oriented Design

The complete foundation chapter from Object-Oriented Python by Steven Lott, from Python Object-Oriented Programming (Fourth Edition) that teaches how to build robust and maintainable object-oriented Python applications and libraries.

Chapter 1 Object Oriented Design (from Python Object Oriented Programming Build Robust And Maintainable Object Oriented Python Applications And Libraries , Fourth Edition)

1.05MB ∙ PDF file

Download

Download

🔍In case you missed it…

🛠️ Tool of the Week

mypy — static type checker for Python

Highlights:

• Catches type errors before runtime: Analyzes Python code statically to detect type inconsistencies, function signature mismatches, and attribute errors before the code ever runs, preventing entire classes of bugs from reaching production.

• First-class support for OOP patterns: Understands inheritance hierarchies, protocols, generics, and abstract base classes, making it especially valuable for object-oriented codebases where polymorphism and interface contracts matter.

• Gradual typing for existing projects: Works alongside untyped code, allowing you to add type hints incrementally to legacy projects without requiring a full rewrite, and can be configured to enforce strict typing only where needed.

Learn more about mypy

📎 Tech Briefs

• Python 3.15.0a7 (March 10, 2026) — The seventh alpha for Python 3.15 gives developers an early look at changes that matter to Python design work, including a built-in frozendict, typed extra items for TypedDict, TypeForm, and further JIT performance gains.

• Python 3.12.13, 3.11.15, and 3.10.20 security releases (March 3, 2026) — Python shipped source-only security releases for 3.10–3.12 that fix header-injection issues, control-character handling bugs in HTTP and cookies, and bundled libexpat vulnerabilities.

• Copilot code review now runs on an agentic architecture (March 5, 2026) — GitHub moved Copilot code review to an agentic tool-calling architecture that pulls broader repository context so review comments are higher-signal, lower-noise, and more grounded in architectural intent.

• 60 million Copilot code reviews and counting (March 5, 2026) — GitHub says Copilot code review usage has grown 10x to more than one in five code reviews on GitHub, and says its current system is optimized around accuracy, signal, and speed rather than comment volume.

• Request Copilot code review from GitHub CLI (March 11, 2026) — GitHub now lets developers request Copilot review directly from the terminal with gh pr edit and gh pr create, pulling AI review more tightly into the existing pull-request workflow.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Back by demand (3rd run). Learn ownership, RAII, smart pointers & allocators to eliminate leaks/UB—live hands-on with Patrice Roy—#1 bestselling author of C++ Memory Management and ISO C++ Standards Committee (WG21) member. Online: Sat Apr 11, 10:30 AM–Sun Apr 12, 4:00 PM ET. Limited seats.

Register now

✍️ From the editor’s desk,

Hello, It’s a privilege to take over as the new Editor-in-Chief of Deep Engineering, which already has a strong foundation and readership.

With support from colleagues at Packt, I also plan to expand coverage into system design, cloud native applications, platform engineering, and other areas of modern software development where engineering leaders have strong interest.

Thank you.

Welcome to the 37th issue of Deep Engineering!

This week’s Android Security Bulletin includes a familiar warning for anyone who builds systems close to the metal: “There are indications that CVE-2026-21385 may be under limited, targeted exploitation.” Whatever the specific exploit chain looks like, memory corruption remains a production-grade failure mode, not a theoretical one.

This week’s Expert Insight comes from Shreyans Doshi (ex Flow Traders, Morgan Stanley), a software engineer who has spent the last several years building trading infrastructure in C++ and Rust. His article, From NIC to P99: Engineering Low-Latency C++ Trading Systems in 2026, walks through the full stack of decisions required to hit a p95 wire-to-wire latency under 50 microseconds. It covers kernel bypass and memory pool design, NUMA affinity, hardware clock selection, and observability at nanosecond granularity.

This issue answers three questions that recur in performance-critical systems (trading and otherwise):

• Where the latency budget actually goes once you include the NIC, kernel boundary, CPU topology, caches, and clocks—not just business logic.

• What belongs on the critical path vs. off it, using a clean pipeline model (ingestion → checks → pricing → strategy → OMS).

• How to measure regressions early, using stage-level histograms and system-level tracing that exposes jitter sources you won’t see in application profiles.

Let’s get started.

OpenClaw Masterclass Workshop

Learn how to turn AI into a safe and cost-effective operational backbone with live case studies from experts who are running their business

Register

Before we dive in…

On the morning of August 1, 2012, Knight Capital Group opened for trading as one of the largest market makers in the US, handling roughly 10% of all equity trading volume on American exchanges. By 10:15 AM, 45 minutes after the market opened, the firm had lost $440 million and was effectively finished as an independent company.

The cause was a software deployment error. A new version of Knight’s order routing system had been pushed to seven of its eight production servers. The eighth server, missed during deployment, continued running a dormant piece of code that had not been used since 2003. When the market opened, that server began firing child orders into the market at full speed to fill just 212 customer orders, and kept going. The system sent more than 4 million orders before anyone intervened. Ninety-seven automated alert emails had been generated referencing the error before the market even opened. And nobody even acted on them. (SEC enforcement order, October 2013)

Knight Capital did not fail because its engineers were careless or inexperienced. It failed because the engineering discipline required to run software at trading-system speeds was not matched to the speed at which the system could cause damage. That discipline covers deterministic deployment, observable behavior, and a tested kill path. In a system capable of sending thousands of orders per second, the gap between “something is wrong” and “the damage is done” is measured in minutes, not days.

This is what Shreyans Doshi addresses in his article.

🧠Expert Insight

From NIC to P99: Engineering Low-Latency C++ Trading Systems in 2026

by Shreyans Doshi

Trading systems have been evolving for decades, moving from manual execution based on charts and indicators to automated strategies that fire within tens of nanoseconds using FPGA and ASIC hardware. Both approaches still coexist today, but most advanced and systematic trading has moved into the sub-100 microsecond range. In this article I want to focus on the hardware and software aspects you need to understand in order to build a trading system with p95 wire-to-wire latency under 50 microseconds.

Traditional technology stacks like Java, JavaScript, TypeScript, and C# offer a lot of flexibility and scalability for large-scale applications, but their inherently nondeterministic nature makes it very difficult to get below a p50 latency of 500 microseconds. While there are ways to optimize garbage collection and other Java components for latency, most standard Java libraries do not focus on low-level concepts like aligning data to cache lines or reducing cache line misses. C and C++ applications optimize for a specific platform at the cost of portability, but that trade-off is exactly what gives you access to manual memory management and the ability to avoid multithreading pitfalls like false sharing. When the goal is to reduce latency, you need lower-level languages like C, C++, or Rust on critical paths. I will not cover pure hardware implementations like FPGAs or ASICs here since they are not required for our 50-microsecond latency target.

Important definitions

Before diving into the architecture and hardware optimizations, it will be helpful to establish exactly how performance is measured in a low-latency environment.

Wire-to-wire latency

Wire-to-wire latency is measured from the moment a market data packet is received on the NIC from the exchange to the moment an order is sent back to the exchange on the same NIC. You compute it as order send timestamp minus market data receive timestamp. The most precise measurements require NIC cards with hardware timestamping enabled, so the NIC stamps every inbound and outbound packet with a hardware timestamp that you can later use to analyze the latency graph. Exanic cards provide APIs for retrieving these hardware timestamps.

Assumptions about the system

To hit the latency target of under 50 microseconds, you need to optimize both hardware and server setup alongside your software, and the following assumptions underpin everything in this article.

• Colocated host: Most stock exchanges in traditional finance provide colocation services where your servers sit physically adjacent to the exchange’s servers, removing extra network jitter from the equation.

• UDP market data access: Most stock exchanges distribute market data via UDP multicast to all subscribed parties, and this is the fastest way to consume and react to it.

• Kernel bypass NIC: The system runs on NIC cards enabled for kernel bypass and tuned for low-latency operation.

• Single venue: These systems are single-venue, meaning at any point they process market data to generate orders for one stock exchange only. For multi-venue setups, the assumption is one colocated host per venue with no shared critical-path processing.

Workload model

Most trading systems follow this model of data processing in abstract terms.

Every strategy engine takes market data from the exchange as its primary input and produces orders sent to the exchange as its primary output, moving through multiple stages as described below. Exchange responses are a secondary input to the engine but are not on the hot path for latency calculations.

Market data ingestion

The first layer is responsible for ingesting the market data (raw or normalized) from the network card. This layer routinely processes terabytes of data per day and can exceed a petabyte on busy days at venues like CME or Eurex. On an average day at CME you can expect 20 to 30 million trades and 100 to 150 million open interest events in a single segment. Processing a full L1 feed for one segment means receiving traffic in the range of a few terabytes to hundreds of terabytes per day, so both software and hardware need to be extremely optimized.

Filtering and checks

Filtering and checks come next, cleaning up incoming data and validating it before it reaches downstream stages. This layer handles packet drops, invalid market data, trading phase transitions like moving from an active session to a halted one, and anomalous price changes that might indicate a software bug. When something goes wrong at this stage, the right response is often to instruct strategies to stop rather than pass bad data forward.

Pricing engine

The pricing engine receives valid and normalized market data and can range from a simple no-op or currency conversion all the way to a full ML-powered model. Lightweight pricing fits comfortably on the critical path. Complex ML models that require large amounts of memory and processing do not, because they would blow the latency budget. In practice, ML-powered engines in low-latency applications split into two processes: time-consuming training and inference model preparation happen off the critical path, and a lightweight inference step runs on the critical path using small linearized models fed configuration parameters from the heavier ML output. This keeps the heavy lifting off the hot path while still giving the strategy access to model-driven signals. Colocation also constrains available compute, so you want to reserve it for work that directly impacts critical path performance.

Strategy evaluation

Strategy evaluation compares transformed or theoretical prices against real prices and fires an order when it identifies an opportunity. Strategies generally fall into three categories.

• Maker: Maker strategies keep quotes on both the buy and sell sides to provide liquidity and take advantage of the bid-ask spread. Exchanges often provide incentives to HFT firms that add liquidity this way.

• Taker: Taker strategies identify mispricing and fire IOC or FOK orders for immediate execution.

• Hedging: Hedging strategies reduce risk or book profit by executing the opposite side of an existing position.

Order aggregation and routing (OMS)

Order aggregation and routing (OMS) can be combined with strategy evaluation in simple setups, but should be separated as complexity grows. It serves three purposes: aggregating orders from multiple strategies to reduce the number of orders sent to the exchange, managing exchange responses and passing normalized results back to the strategy, and handling the routing logic that grows more complex as your strategy count increases.

Latency map: where the latency lies

Now that the high-level components are clear, it is worth mapping out where the latency actually sits so you can allocate your budget with precision. Beyond the business logic itself, your latency profile depends on CPU model and generation, NIC type and model, operating system, software architecture, and business logic complexity.

Beyond the business logic stages, several factors contribute latency that stays invisible if you only look at application code. These factors come from hardware, OS and kernel processing, network behavior, and I/O, and they play an outsized role in ultra-low-latency systems. Even if they occur infrequently, they introduce jitter and latency spikes that directly hurt your p99. Although many of these latency sources depend on a multitude of factors, you can work with approximate cost ranges that provide a useful benchmark.

Continue reading

🛠️Tool of the Week

bpftrace — open-source high-level tracing language for Linux systems, built on eBPF

Highlights:

• Instant latency visibility where it leaks: Attach probes to kernel/user-space events (kprobes/tracepoints/uprobes) to pinpoint tail-latency sources—scheduler delays, IRQ/softirq behavior, and “surprise” syscalls—without changing your C++ code.

• Built-in distribution tooling (histograms by default): First-class aggregations like hist() and quantize() make it trivial to generate p50/p95/p99-style views (not just averages), which is exactly what you need for jitter and p99 debugging.

• Ready-to-run scripts for common low-latency failure modes: The bundled tools/ library covers practical staples like syscall counts, run-queue latency, and retransmits—fast to deploy during an investigation, easy to keep as regression checks.

Learn more about bpftrace

📎 Tech Briefs

• PostgreSQL 18.3 / 17.9 / 16.13 / 15.17 / 14.22 released: PostgreSQL shipped an out-of-cycle patch set to all supported branches to fix regressions (including fixes touching standby behavior, substring() encoding edge cases related to a CVE fix, pg_trgm stability, and restoring json*_strip_nulls() immutability for index use).

• GitHub Actions: upload/download artifacts without forced ZIP: GitHub Actions added support to upload and download non-zipped artifacts (via actions/upload-artifact@v7 with archive: false and actions/download-artifact@v8), reducing friction for single-file downloads, in-browser viewing, and “double-zip” workflows.

• DPDK Governing Board minutes: quality-first 26.03 cycle + AI-assisted review gate): DPDK’s Feb 17 governing-board update highlights a code-hardening 26.03 cycle (roughly “half the commits” focused on fixes/backports) and describes introducing AI-assisted code reviews hosted by the UNH lab as an additional quality gate after standard CI passes.

• Node.js 25.8.0 (Current) released: Node.js 25.8.0 landed with notable changes including SQLite DatabaseSync additions, expanded diagnostics channel support, a new --permission-audit flag, and dependency updates (including an npm upgrade).

• Dependabot alert assignees GA: GitHub made Dependabot alert assignees generally available, enabling explicit ownership of dependency vulnerability alerts (UI + REST API + webhooks) and visibility across repo/org/enterprise alert views.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,

Saqib Jan

Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.

Leave a comment

Give AI agents the knowledge of your most experienced engineers. Unblocked builds context from your team’s code, PR history, conversations, documentation, planning tools, and runtime signals to surface what matters for the task at hand – while resolving conflicts and respecting permissions. Ship faster with AI outputs that reflect how your system actually works.

See how it works

✍️From the editor’s desk,

Dear reader,

It has been a privilege to serve you as we built Deep Engineering over the past year. This is my final issue, and next month the newsletter will continue under new editorial leadership. I’m glad to introduce as the new Editor-in-Chief of Deep Engineering.

Deep Engineering remains committed to serving the software community with timely, expert-led insights. Thank you for reading and for being part of this journey with me.

Welcome to the 36th issue of Deep Engineering!

Cloudflare’s Q4 2025 DDoS Threat Report described a record 31.4 Tbps attack and hyper-volumetric HTTP floods exceeding 200 million requests per second—a useful reminder that production systems run in a hostile environment. Today’s issue is grounded in my conversation with Archit Agarwal, Principal Member of Technical Staff at Oracle working on ultra-low-latency authorization services in Go, and the author of The Weekly Golang Journal. As Agarwal says, we must assume the system will fail and the system will be attacked, then work backward into controls that keep the blast radius small.

In this issue, we explore what makes a system design hold up in production once cost, failure, and security constraints become non-negotiable? Agarwal’s guidance is consistent across topics: delay microservices until “splitting signals” are visible, treat performance per dollar as a design requirement, cap autoscaling to prevent runaway spend during attacks, and keep dependency risk contained through abstraction, version discipline, and unified observability. Agarwal also offers battle-tested tips on cracking the system design interview.

You can watch the complete interview and read the transcript here, or read on for distilled insights.

Build an Internal Developer Platform from scratch (2-day live workshop)

Most IDPs fail when teams build tools, not products. This 2-day live, hands-on workshop shows how to design and build a production-ready IDP with platform-as-a-product patterns, adoption, governance, and reliability. Sat, Apr 25, 2026 — 11:00 AM ET.

Reserve your seat →

System Design That Holds Up in Production with Archit Agarwal

A system design diagram is only useful if it reflects the constraints the system will live under. Archit Agarwal’s answer to most design questions is the same: make the trade-offs explicit, and choose them deliberately.

When microservices are worth it

It is now generally established best practice to begin with a monolith when requirements are still changing and the domain is still unfamiliar. Agarwal recommends keeping module boundaries clean so that separation is possible later, but he delays distribution until there is clear evidence it will help.

He looks for “splitting signals”:

• Deployment friction shows up: Releases get larger, slower, and harder to roll back.

• The blast radius grows: A defect in one area increasingly breaks unrelated user paths.

• Independent scaling becomes necessary: One workload needs more capacity than others during predictable peaks.

• Technology needs diverge: A subsystem truly benefits from different runtimes, storage, or processing tools.

Microservices add team autonomy, but they also add coordination overhead and network latency. According to Agarwal, splitting makes sense when the monolith’s pain is observable, repeatable, and clearly outweighs the operational tax of moving to services.

Performance per dollar as a design requirement

Cost is part of the architecture. Agarwal evaluates choices as “performance per dollar,” so teams weigh latency gains against the full cost of running and operating the system.

A cost-aware loop can:

• Make spend visible through dashboards, so teams connect architecture to real numbers.

• Put cost into design reviews alongside latency, availability, and security trade-offs.

• Build for horizontal scaling, then scale only when demand and reliability targets prove the need.

• Manage data by lifecycle. Keep frequently accessed data in hot tiers and move cold data to cheaper storage.

Design for a hostile environment

It is also important to design around two expectations: systems fail, and systems get attacked. Planning for both keeps failure domains small and recovery predictable.

Recent threat telemetry supports the posture. Cloudflare’s Q4 2025 DDoS report describes a record 31.4 Tbps attack and a subsequent campaign with hyper-volumetric HTTP floods exceeding 200 million requests per second. The same report says DDoS attacks surged 121% in 2025, averaging 5,376 mitigations per hour on Cloudflare’s network.

Agarwal recommends a practical resilience toolkit that ties these risks to concrete controls:

• Add layered defenses at the edge before the application spends CPU.

• Rate-limit meaningfully by user, token, IP, and geography, and tune limits using real traffic.

• Cap autoscaling so an attack cannot trigger runaway capacity and spend.

• Separate failure domains with multi–availability zone deployments, and reserve multi-region for the paths that must stay up.

• Keep observability complete, because incident response depends on fast visibility.

Supply chain exposure belongs in the same threat model. Sonatype reported 394,877 new malicious open-source packages in Q4 2025 alone, a 476% jump compared to the prior three quarters combined, with 99.8% of the malware it saw in that quarter originating from npm. ReversingLabs’ 2026 software supply chain report also points to npm as the primary hotspot, reporting 10,819 malicious packages detected there in 2025 (nearly 90% of its detections across OSS platforms) and describing a first-of-its-kind self-replicating registry-native worm dubbed Shai-hulud that compromised close to 1,000 npm packages across two campaigns.

Design implications follow naturally: pin and review dependency updates, validate new versions before promotion, avoid automatic upgrades on production-critical paths, and enforce least privilege and deny-by-default authorization so that a compromised component has limited reach.

Depend on services without letting dependencies own you

Modern systems rely on external services and SaaS APIs. Agarwal recommends keeping those dependencies from becoming single points of failure.

His approach is pragmatic:

• Put an abstraction layer in front of major external services, so the application talks to an interface rather than a vendor-specific client library.

• Prefer portable standards where they fit, such as containers and unified telemetry.

• Replicate only what is mission-critical across regions or providers, and keep observability in one place.

Ultra-low latency without sacrificing the whole codebase

Agarwal recommends treating latency like a budget: every hop and allocation spends from it. Ultra-low-latency work, in his view, is mostly subtraction.

Replicable strategies:

• Put compute closer to users when geography is on the critical path.

• Remove network hops from hot paths, and keep the performance-critical code lean.

• Reuse connections and choose efficient service-to-service protocols when calls are unavoidable.

• Match hardware and runtime to the goal, because software choices cannot fully offset weak infrastructure.

He also draws a boundary that protects maintainability: only a small part of most systems needs extreme optimization. Keep the rest readable and easy to change.

How to communicate system design under pressure

Agarwal’s interview advice doubles as a template for real design discussions: keep the design aligned with requirements, and keep your reasoning easy to follow.

His recommended structure for cracking the system design interview:

• Align on functional requirements first.

• Capture nonfunctional requirements next, because they drive architecture.

• Present a high-level design, then zoom in one component at a time.

• State trade-offs as you choose, including what you gain and what you give up.

When constraints change midstream, he recommends restating the change, identifying the affected parts, and updating only what needs updating. That approach shows control, adaptability, and a clear “commit history” of decisions.

🧠Expert Insight

🛠️Tool of the Week

Envoy Proxy: a high-performance L7 proxy that standardizes service-to-service and edge traffic management (routing, resilience, security, and telemetry) across distributed systems.

Highlights:

• Advanced traffic routing & load balancing: dynamic routing rules, weighted traffic splitting, and multiple load-balancing policies.

• Resilience controls built in: timeouts, retries, circuit breaking, and outlier detection to contain failures.

• Security + observability hooks: mTLS/TLS termination, external authorization integration, and first-class metrics/logging/tracing support.

Learn more about Envoy Proxy

📎Tech Briefs

• Cloudflare outage (BYOIP route withdrawal): Explains how a change in BYOIP prefix management unintentionally triggered BGP route withdrawals for some customers, causing timeouts and connectivity failures, and documents the mitigations and follow-up controls.

• Post-quantum encryption across Cloudflare One (SASE): Cloudflare has rolled out standards-based post-quantum hybrid ML-KEM across Zero Trust and WAN on-ramps/off-ramps (including TLS/MASQUE/IPsec paths) without requiring specialized hardware changes.

• Private incident + maintenance notifications (SSO & Slack) — Fastly:
  Fastly has introduced private, security-sensitive incident and maintenance notifications with integrations like SSO and Slack to improve operational transparency for customers without exposing details publicly.

• apt.postgresql.org operational updates: The PostgreSQL project’s apt repository announces automatic apt-retrievable changelogs, colocated build logs, and repository changes as Ubuntu 26.04 work begins and Ubuntu 25.04 reaches EOL.

• PostgreSQL Anonymizer 3.0: Dalibo’s PostgreSQL Anonymizer 3.0 adds parallel static masking via background workers and JSON import/export for masking rules, alongside security fixes and breaking-change notes.

That is all from me today. Thank you for reading this issue of Deep Engineering.

We will be back next week with more expert-led content.

Signing off one last time,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

(Don’t forget to stay awesome!)

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Leave a comment

Move MCP from demos to production. In this 5-hour online workshop you’ll cover OAuth 2.1 auth pitfalls, OpenTelemetry observability, scaling persistent connections, and managing context growth/cost. Led by Saxo Bank Principal Dev Peder Holdgaard Pedersen. When: Mar 29, 10:30am–3:30pm ET / 7:30am–12:30pm PT.

Get tickets on Eventbrite

✍️From the editor’s desk,

Welcome to the 35th issue of Deep Engineering!

This is the second installment in Deep Engineering’s ongoing series adapted from chapter 2 of The IT Strategy Playbook by Scott Millett (Packt; publishing May 2026). Millett is a long-time IT leader and architect who has served in CIO/CTO roles and led large-scale transformation work across multiple sectors (as you saw in Part 1’s “100 Day Plan” boardroom story).

Today’s adaptation focuses on a practical problem: how technical realities become executive decisions. Millett lays out a repeatable approach for communicating strategy—clarifying the single objective, framing the narrative with SCQA (Situation–Complication–Question–Answer), and supporting the recommendation using the Pyramid Principle—so boards and executives can evaluate trade-offs and commit to a direction.

This is well aligned with what US technology leaders are navigating this week: Gartner’s Feb 17 Q&A piece argues that fragmented AI regulation is accelerating spend on AI governance platforms as organizations try to manage compliance and operational risk. At the same time, a multi-industry coalition urged the Commerce Department to preserve the core of the NIST AI Risk Management Framework, a signal that many enterprises are already using it as a shared language for oversight and internal controls. And CISA’s Feb 13 announcement of CIRCIA rulemaking town halls reinforces that incident reporting expectations—and the need to communicate risk and response clearly—remain active, moving targets.

Let’s get started.

Safely Refactor Large Production Codebases with AI

Safely refactor large production codebases with AI. Learn a workflow that pairs Claude Code with ast-grep’s AST-based rewrites for structural guarantees—migrations, guardrails, and best-practice enforcement. Ticket includes Mastering ast-grep + prompt templates. Mar 14, 10–11:30am ET / 7–8:30am PT.

Get tickets

🧠Expert Insight:

The IT Strategy Playbook (Part 2): Closing the Story Gap by Scott Millett

“The most powerful person in the world is the storyteller.” — Steve Jobs

In the early 2000s, Nokia dominated mobile phones. Its strategy, built on exceptional hardware engineering and global supply-chain execution, worked flawlessly until 2007. The company didn’t fall because its engineers lacked insight. It fell because the organization couldn’t convert critical technical knowledge into strategic intent. The signals were present; the translation mechanisms were not.

Nokia’s software teams had long recognized that Symbian’s architecture was reaching its limits. It was fragmented, costly to evolve, and increasingly misaligned with the emerging touch-driven experience. Alternative platforms such as Maemo and later MeeGo were being explored, and the technical risks were well understood within engineering. But these concerns were filtered through a hardware-first culture that interpreted platform limitations as minor technical defects, not as threats to the business model. The communication path from “this architecture will constrain what the business can do” to “this requires a strategic shift” simply wasn’t there.

At the same time, senior leadership continued to optimize short-term device metrics. They still viewed the competitive landscape as a handset battle rather than an ecosystem battle, and the implications of Apple’s and Google’s platform strategies never translated into a unified architectural or strategic response. Nokia’s organizational structure created a communication gradient: technical truths lost meaning as they travelled upward, and strategic signals failed to reorient investment, priorities, or direction.

Within five years, the world’s leading mobile manufacturer was forced to exit the handset business, not because it lacked capability, but because it lacked the ability to translate insight into coherent action. This chapter focuses on the story gap: how to communicate strategic needs clearly, how to ensure signals travel across the organization intact, and how to avoid the silent failures that undermine strategy long before the market ever does.

In this chapter, you’ll learn how to:

• Communicate strategic intent with clarity, ensuring technical insight translates into decisions rather than confusion

• Structure a coherent strategic narrative using proven frameworks to move from context to conclusion

• Develop presentation material that supports decision-making, not just information sharing

• Deliver your message effectively, guiding senior stakeholders from understanding to commitment

You will use these skills to close the story gap: translating strategy, architectural intent, and plans into a narrative that enables decisions. Not more slides, but a clear line from context to conclusion, so leaders understand the trade-offs, commit to a direction, and act.

How to Be An Effective Communicator

Delivering a technology strategy is not simply an act of presenting information; it is an act of leadership and influence. The best strategy in the world will fail if the communicator lacks clarity of purpose or the ability to authentically connect with the audience.

Clarify your objective

Before you open a slide deck or write a single email, you must answer one question with absolute, non-negotiable clarity: What is the one thing I need the audience to believe or do when I stop speaking or reading? This is the foundation of strategic communication. Without a single, clearly defined objective, your message will lack focus, and your audience will leave confused about the desired next step. Your objective should never be passive, such as “to inform the Board” or “to update the team.” It must be an action verb tied to a measurable outcome. Consider the examples in table 2-1 where we take a passive goal and turn it into an active objective.

When you define your objective down to the ‘one thing,’ you achieve two crucial results:

1. Elimination of Noise: Every slide, chart, and word you use must directly serve that single objective. If a piece of data doesn’t help you achieve the objective, it belongs in the appendix, or nowhere at all.

2. Forcing the Answer: By knowing the desired action, you structure your entire presentation to lead logically and inevitably to that conclusion. Your presentation becomes a persuasive roadmap, not a rambling document review.

Master the three modes of Persuasion

As an IT leader, you must consciously deploy all three modes to secure buy-in. Relying solely on data (Logos) is rarely enough; you must also establish your credibility (Ethos) and connect your strategy to the organization’s goals and fears (Pathos). The following subsections detail how to weave each of these essential appeals into your strategic narrative to maximize influence and drive action.

Continue reading...

🔍In case you missed it…

🛠️Tool of the Week

Mermaid — Open-source “diagrams as code” for flowcharts, sequence diagrams, and architecture visuals (rendered from plain text)

Mermaid (mermaid-js/mermaid) is a text-based diagramming tool that turns Markdown-like syntax into diagrams you can embed in docs, RFCs, and strategy decks—useful when you want visuals that stay versionable and easy to review.

More about Mermaid

📎Tech Briefs

• CIRCIA town halls / reporting requirements (US critical infrastructure CIRCIA rulemaking: DHS/CISA scheduled sector-specific virtual town halls (plus two general sessions) to gather additional stakeholder input on refining the scope and burden of the proposed cyber incident reporting rule.

• GitHub Agentic Workflows (technical preview): GitHub introduced Markdown-authored “agentic” automations that compile into standard Actions workflows, aiming to make repo automation more natural-language and security-constrained by default.

• Attack-surface and app-sec bundling: Cloudflare and Mastercard announced a partnership to combine attack surface monitoring/risk scoring with Cloudflare’s application security controls for faster remediation across internet-facing assets.

• CISA KEV update: CISA added four vulnerabilities to its Known Exploited Vulnerabilities Catalog (evidence of active exploitation), reinforcing KEV-driven prioritization for vulnerability management.

• VictoriaMetrics LTS release: VictoriaMetrics shipped LTS release v1.122.15 (released 2026-02-13) with bugfixes intended for production observability deployments.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend

Code review is stuck. It's slow (manual), noisy (tools), or misses things. Qodo 2.0 is different. With highest precision & recall, it finds critical issues other review tools miss, while limiting noise. Learning from your codebase & PR history, Qodo gives fixes developers actually use.

Try Qodo for Free

✍️From the editor’s desk,

Welcome to the 34th issue of Deep Engineering!

On February 10, 2026, the OpenJDK Quality Group warned that JDK 27 will remove ThreadPoolExecutor.finalize(), a source-incompatible change that can surface as compile errors in code that calls or overrides it. It is a useful reminder that JVM modernization happens under load while production systems keep running. Today’s issue looks at what “practical modernization” means after the Kotlin hype cycle, grounded in my conversation with José Dimas Luján Castillo and Ron Veen, co-authors of Kotlin for Java Developers.

Luján Castillo brings a mobile-first view shaped by 15 years building Android, iOS, and Flutter applications and leading teams globally, including work across 500+ mobile apps, plus experience as a teacher and author. Veen brings an enterprise JVM lens from 20+ years in the Java ecosystem, spanning mainframes to microservices, with certifications including Oracle Certified Java Programmer and Sun Business Component Developer, and hands-on work around Jakarta EE migrations and modern concurrency.

You can watch the complete interview and read the transcript here, or read on for distilled insights.

Today’s issue also includes the complete Chapter 15: Coroutines from their book in addition to our regular Tool of the Week pick and Tech Briefs.

Let’s get started.

Sponsored:

Webinar | How to Build Faster with AI Agents: Learn how full‑stack developers boost productivity by 50% with AI agents that automate layout, styling, and component generation through RAG and LLM pipelines. See how orchestration and spec‑driven workflows keep you in control of quality and consistency. Save your seat!

Kotlin and Java after the Hype Cycle with José Dimas Luján Castillo and Ron Veen

A language switch succeeds or fails in the space between code and people. Teams can adopt Kotlin fast and still ship Kotlin that behaves like Java. The most useful guidance is keeping modernization practical while systems keep running.

Kotlin’s first gains: safety and continuity

Luján Castillo says Kotlin’s benefits were easy to see on Android because “Java was too verbose.” He names one feature that changed everyday work immediately: “null safety. It’s automatic.” He also describes the adoption detail that matters most in mature codebases: Kotlin’s “interoperability with Java.” “Maybe I can do the next steps in my applications with Kotlin, but I don’t need to fight with the legacy code even if it’s in another language,” he says, and he reminds teams that “the main problems are still there even if you change the language.”

Veen’s enterprise view focuses on maintenance and defects. “There will be less code—and less code is good (because it means) less code to maintain,” he says, and he expects Kotlin to help because it “tends to be less error-prone,” with null handling as a steady source of bugs.

The migration trap: translating Java line by line

Both warn that Kotlin adoption can stall at Java-flavored Kotlin. Veen calls it a natural reflex: teams start Kotlin while still thinking in the patterns they already know, because there is still a project to ship. “It’s not a drop-in replacement,” he says. “You really have to be willing to learn.”

Luján Castillo describes the common first step: “99% of developers start by just translating the code.” Translation helps people learn syntax, but it does not deliver the major improvements teams expect. “If you are translating each line, you will see you don’t have less code—you have the same code in a different way,” he says. He looks for a shift where developers start asking whether Kotlin makes a line unnecessary.

Veen gives a concrete way to test that shift. He rewrote an implementation and reduced it “from 250” lines “down to under 100,” not because shorter is always better, but because it forced him to ask how to “leverage Kotlin’s native way of doing things.” Luján Castillo describes “immutability first” as a sign that developers have started to think in Kotlin’s defaults rather than Java’s habits.

Concurrency decisions: coroutines, virtual threads, and structure

Luján Castillo frames concurrency as choosing a model you can work with under pressure. For coroutines, he highlights the structure: “You have cancellation, you have propagation, you have control of the whole cycle.” For virtual threads, he emphasizes familiarity. They are “very traditional,” and he sees them as attractive when a system already has “a very complex structure with layers.” He avoids declaring a winner: “It’s not a model about who is better.”

Veen names the shared goal behind both approaches: async work that can “appear sequentially in the code,” so developers can reason about flow and avoid “callback hell.” OpenJDK’s current structured concurrency work follows that reasoning-first intent. JEP 525 describes subtasks whose lifetimes are “confined to the syntactic block” of their parent task so they can be managed “as a unit,” and it calls structured concurrency “a great match for virtual threads.”

Modernization without regret: boundaries first, then change

Luján Castillo argues that “microservices, in my experience, is not the goal, OK? It’s the consequence for that.” Before a team breaks apart a monolith, he asks, “Are we prepared for paying the real cost to use microservices—not just the money?” and “are we prepared to use this architecture or not?” He also rejects the idea that monoliths are obsolete: “you can use monoliths without problems for huge systems.”

Veen explains where the cost shows up. Microservices require “a lot more overhead” in monitoring, and debugging grows harder when incidents span “dozens or hundreds of services.” His default is a “modular monolith,” where modules have clear boundaries and must use a “predefined API” to interact. He then watches for signals that justify extraction, such as a module that forces frequent redeployments or has distinct scaling needs.

Both speakers treat adoption as a leadership problem. Veen recommends bringing skeptics into the work early and using mixed teams and code reviews to build confidence. Luján Castillo recommends “small limits” and warns leaders away from shallow metrics: “Don’t try to read productivity… as: ‘If you have more lines of code or less lines of code.’”

JetBrains lists Kotlin 2.3.10 as released on February 5, 2026, which supports the kind of incremental upgrades both speakers describe

🧠Expert Insight:

1. The complete Deep Engineering interview with José Dimas Luján Castillo and Ron Veen

2. The complete Chapter 15: Coroutines from Kotlin for Java Developers by José Dimas Luján Castillo and Ron Veen (Packt, 2025)

🛠️Tool of the Week

OpenTelemetry Java Agent — Open-source, zero-code auto-instrumentation for JVM services (traces, metrics, and more)

The OpenTelemetry Java Agent (from open-telemetry/opentelemetry-java-instrumentation) is a Java agent JAR you attach at startup to capture telemetry from popular Java libraries and frameworks without changing application code.

Highlights:

• Zero-code startup integration: Attach the agent with the JVM -javaagent flag to “gather telemetry data from a Java application without code changes.”

• Collector-first by default, configurable when needed: The agent’s default path uses OTLP to send telemetry to an OpenTelemetry Collector endpoint (and you can configure it via system properties or environment variables).

• Open source, production-oriented ecosystem, recently updated

More about OpenTelemetry Java Agent

📎Tech Briefs

• Kotlin 2.3.10 Released: Kotlin 2.3.10 is a new Kotlin/JVM release, giving teams a concrete upgrade point for Kotlin codebases that need to stay current with the Java ecosystem.

• OpenJDK JDK 26 Release-Candidate Builds (Build 34): DK 26’s RC builds are now available, which is useful for teams that want to validate JVM and runtime behavior changes (including concurrency-related changes) against real workloads before GA.

• Spring Boot 4.0 Migration Guide: Spring Boot’s 4.0 migration guide captures the practical baseline shifts (Java 17+, Spring Framework 7.x, Jakarta EE 11, Kotlin version requirements) that enterprise JVM teams need to account for in upgrades.

• Quality Outreach Heads-up (JDK 27): Removal of ThreadPoolExecutor.finalize(): This heads-up flags a source-incompatible change where removing ThreadPoolExecutor.finalize() can trigger compile errors in code that directly calls or overrides it.

• Optimizing Java Class Metadata in Project Valhalla: A Project Valhalla update digs into how the JVM is evolving class metadata to support value classes and flattening/inlining, which matters for performance-sensitive JVM designs over the long term.

That’s all for today. Thank you for reading this issue of Deep Engineering.

We’ll be back next week with more expert-led content.

Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering

If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.

Refer a friend