⚡ Forward Deployed Engineering Workshop
Scope a 90-day agent deployment for a regulated customer, then defend it in a CISO hot seat. Two live sessions with Keith Bourne, Forward Deployed AI Engineer at Tribe AI, and Tanya Dixit, Forward Deployed Engineer at Google.
🗓️ 19 and 20 September · Early bird 40% off
✍️ From the editor’s desk
Welcome to the 62nd issue of Deep Engineering!
The Agentic AI Foundation has officially accepted Agent2Agent as a Growth Stage project, joining MCP, AGENTS.md, Goose and agentgateway under Linux Foundation governance. That means two major interoperability protocols now share a home, each keeping its own maintainers and release schedule.
Governance settling is useful up to a point, but it does not tell you how to build. A specification describes how an agent may call a tool or talk to another agent. It does not describe what happens on your side of that exchange, which is where nearly all of the engineering actually is.
That is our focus in today’s issue, featuring James Ward, who works on agent experience at AWS, represents Amazon on the AAIF technical committee, and created WebJars.
You can read the complete deep dive, where he builds the integration layer in Java with Spring AI, starting from a model that cannot tell you the time.
Let’s get started.
Moyai - Monitor your agents for failures
Stop guessing whether your agents are failing. Moyai surfaces behavioral anomalies in your agent traces and classifies each failure with root cause analysis and remediation steps.
Works on your existing observability stack, no new SDK required.
🧠 Practical Deep Dive
Agents Are Just LLMs With Integrations, Running in a Loop
by James Ward, Edited by Saqib Jan
Let me start with the thing that makes all of this necessary.
Go to your agent and ask it what the current weather is. It will tell you it does not have access to real-time weather data or your location, and suggest you look out of the window. Ask it what time it is and you get the same shape of answer, because it does not have a clock. That is not a failure. By default, LLMs have no access to external things. They are just a model.
The way I think about what an LLM actually is: a knowledgeable translator. It translates natural language to natural language, natural language to an image, an image to natural language, natural language to structured data, structured data to structured data, natural language to a programming language. That is all it does. Everything else you want from an agent has to be built around it.
An agent, then, is not complicated. You take an environment, usually a history of messages plus whatever else you want to carry. You take tools, the things you want to give the model access to. You add a system prompt to give it a goal or a personality. And then the agent runs in a loop until it decides it has achieved what the user asked, or that it cannot.
All the code is at github.com/jamesward/agent-integration-demo, and I am using Spring AI and Java throughout.
Inference is the easy part, and it is worth seeing what is on the wire
Spring AI hit 1.0 a little over a year ago, and it gives you one abstraction across model providers. I am using AWS Bedrock with the Converse API and the Nova Pro model, but there are around 250 models on Bedrock and you could just as easily point this at Ollama running locally. The provider is a config change.
In the application you inject a ChatClient.Builder rather than a concrete client, and the reason matters. In a real system you will want several chat clients, one on a large model and one on something faster and cheaper, and your agentic architecture will route between them.
The basic call is a user prompt, .call(), and .content(). Ask it to say hello and it says hello.
More useful is structured output. Define a Java record, annotate a field with @JsonPropertyDescription("most popular food"), and ask for a List<City> back using .entity() with a ParameterizedTypeReference because of how Java generics are reified. What happens underneath is that the record’s metadata gets sent to the model so it knows how to shape a response that will deserialize cleanly. When you are building agentic applications, it very often makes more sense to interact with the model through structured data than through free-form text.
I want to demystify what is actually happening on those calls, because the API is concise and what goes over the wire is not. Run it in debug mode and you see the request carries the message, the media slots for images, the max tokens from your settings, the model name, a pile of defaults and the system message. The response carries metadata about rate limits, prompt tokens in, completion tokens out, total tracked, and one field worth knowing by name.
finish_reason. On a simple call it comes back as end_turn, which is the model saying it has done what you asked. Hold on to that, because the entire agentic loop turns on it becoming something else.
Spring AI wires token metadata into Actuator and Micrometer, so you can push those metrics wherever you already send metrics.
Streaming is a one-word change. Swap .call() for .stream() and you get a Flux of chunks instead of waiting for the whole response to assemble. And a system prompt gives the whole interaction a personality or some grounding. Mine was “you are a Wookiee from Star Wars,” and it growled at me. One caution: you cannot rely on system prompts to protect a system from being used for things you did not intend. More on that later.
Tool calling is a four-step conversation, and the model never touches your tool
Now ask what time it is, with no tools defined. The model tells you it has no access to a clock. This is the wall.
Here is what actually happens when you get past it.
You send the user message to the model, and alongside it you send metadata describing the tools you have. Just the name, the parameters, the description. The tool itself is not on the model’s side. It is on your application’s side. You are saying, here is what the user wants, and here are some things I can do if you decide you need them.
The model looks at the request and responds. And the finish_reason this time is not end_turn, it is tool_use. The model is telling you which tool it needs and what parameters to pass. For a weather question that is get_weather with a city name.
Your application invokes the tool. No model involvement at all in this step.
Then you call the model again with the whole history: the original question, the fact that it asked for a tool, and the result you got back. Now it assembles a real answer and comes back with end_turn.
The key thing to hold on to is that the LLM never invokes anything. You define and call the tools. The model only tells you which ones it wants.
In Spring AI this is short. Annotate a method with @Tool and a description, then pass the containing object to .defaultTools() on your chat client. The description is doing real work, because that is what the model reads when deciding whether the tool is relevant.
@Tool(description = "Get the current date and time in the user's time zone")
String getCurrentDateTime() {
return LocalDateTime.now()
.atZone(LocaleContextHolder.getTimeZone().toZoneId())
.toString();
}The API for calling this is identical to the basic inference call. Same .call(), same .content(). Spring AI uses the same API for a single inference as for a full agentic loop, and the loop just keeps going until it reaches end_turn, making however many tool calls it needs on the way.
MCP is the microservices version of the same thing
MCP has become the standard way to do tool calling, and you have probably used it in a code assistant with a local server. Plenty of businesses now publish their services as MCP too.
It works over HTTP, so servers can be remote. Underneath it is JSON-RPC carrying a method, a tool name and arguments. That is genuinely all it is. MCP is remoting for the tool calling I just described.
In Spring AI, configure the client with the streamable HTTP protocol and a URL. I pointed mine at a JavaDocs MCP server I built, which gives access to all the Javadocs on Maven Central. Then inject a ToolCallbackProvider, hand it to .defaultTools() alongside your local tools, and the wiring is done. Ask for the latest version of a library and it calls the tool rather than guessing from training data, which is the difference between a correct version number and a plausible one.
You can build MCP servers with Spring AI as easily as you consume them. But I would push back on wrapping everything. Most real architectures are a mix of MCP tools and local ones, and there are good reasons to keep tools local. Something like getting the current date, or doing arithmetic, belongs in the same process as your agent. My general approach to architecture is to start with a monolith, build it so it can become microservices, and only break things out when you need to. The same reasoning applies here exactly.
The portability is straightforward when you do need it. Take a tool written in Spring, pull it into a separate project, expose it as an MCP server, and the code barely changes. Swap @Tool for @McpTool if you want the MCP-specific features, though @Tool will work as it is. And the security model stays the same, so user identity flows to MCP tools the same way it flows to local ones, which is usually the painful part of that kind of migration and here is not.
Too many tools is a token problem and a reasoning problem
Here is what breaks at scale. Every request to the model carries the metadata for every tool you have. With a hundred tools that is a lot of tokens spent describing capabilities before the model has done anything. And it gets harder for the model to pick the right one as the list grows.
And it gets harder for the model to pick the right one as the list grows.
🛠️ Tool of the Week
Embabel — an agent framework on the JVM, built on Spring AI
Embabel is the layer above Spring AI for teams that want more than a foundation, and it takes a different route through the problems in today’s issue.
Created by Rod Johnson, who wrote the Spring Framework, so the idioms will be familiar to any Spring team
Unfolding tools load domains of tools progressively, an alternative to semantic tool search for teams with a lot of tools
DICE, domain integrated context engineering, gives a structured approach to what goes into the context window
Sits on top of Spring AI rather than replacing it, so an existing Spring AI application is the starting point
📎 Tech Briefs
Claude Platform adds Fable 5.1 tool-call constraints - Claude Fable 5.1 now rejects
anyandtoolchoices, pushing tool-call guarantees toward stricter interfaces.Content exclusions reach Copilot app and CLI - Copilot agents now respect repository, organization, and enterprise exclusions before using files as context automatically.
MuleSoft MCP Server expands governance tools - New lineage, classification, model-wallet, and vault tools make MCP-discovered services easier to govern and budget.
CrowdStrike launches Verified Agent certification - Falcon partners get a defined validation path before publishing agent integrations to CrowdStrike Marketplace for buyers.
Copilot in VS Code improves agent session handling - Agent sessions gain side chats, portable plugins, transcript navigation, and cross-window continuation in VS Code.
That’s all for today. Thank you for reading this issue of Deep Engineering.
We’ll be back next week with more expert-led content.
Keep building,
Editor-in-Chief, Deep Engineering
Partner with Deep Engineering
If your company wants to reach senior developers, software engineers, and technical decision-makers, speak to us about partnering with Deep Engineering.