Deep Engineering #27: Amar Akshat on Agentic Architecture and Trustworthy AI
How Paysafe’s SVP of Architecture puts AI in the payments path—using cells, prompts-as-code, and confidence routing.
Agentic AI Frontier Summit 2025 (Online): From Single Models to Autonomous Systems
Join us on Dec 13, 2025 · 9am–2pm ET / 6am–11am PT to discover how AG2, autonomous research agents, and real-world multi-agent systems are reshaping science, engineering, and AI-native companies. Get 40% off with code DEEPENG40 and secure your spot at the frontier of agentic AI.
✍️From the editor’s desk,
“You don’t need to train models; you need to design systems that can survive their uncertainty.” — Amar Akshat, SVP of Architecture at Paysafe Group
Last Friday, Bank of America told investors it will channel $4 billion of its $13 billion tech budget into AI, with its “Erica” assistant already logging 3 billion client interactions—work the bank says would otherwise require about 11,000 employees. And nearly 47% of US banking decision-makers now say their institutions have fully rolled out generative AI—up from just 10% in 2023
Against that backdrop, today’s issue narrows in on two core questions engineers and architects working in fintech and critical systems are grappling with:
How do you put AI in the critical path of payments and wallet flows without giving up determinism, auditability, and regulatory control?
What architectural patterns—cells, prompts-as-code, confidence routing, and vendor-neutral orchestration—can scale in production, instead of becoming yet another fragile AI sidecar?
To help answer these, we’ve collaborated with Amar Akshat, SVP of Architecture at Paysafe Group and author of the forthcoming book Decode the Compiler (Packt, 2026). At Paysafe, Amar leads AI-native modernization across payments, wallets, and compliance platforms. Earlier in his career, he helped shape the architectural foundations of Apple Pay and contributed to wallet and tokenization frameworks at Apple. Before that, at Morgan Stanley, he worked deep in build and integration systems and compiler workflows. His work sits exactly at the intersection our readers care about most: compiler-level thinking, distributed systems, and architecture as a living, observable system.
In today’s issue:
Feature Article – “Agentic Architecture: Designing AI Systems with Guardrails for Trust”: A focused synthesis of Amar’s approach to agentic architecture, including:
Cell-based boundaries
Treating prompts as code in CI/CD
Confidence-based routing and “trust deltas”
Prompt manifests as “Dockerfiles for AI”
Our complete interview with Amar Akshat: You can watch the complete conversation below or read the refined Q&A article here.
Today’s issue is about making AI structural to your architecture while staying inside the constraints that regulators, customers, and your own risk teams will require.
Let’s get started.
AI-Powered Development with Cursor Online Workshop: Ship Production Apps Fast
Nov 29, 2025 · 8:30am–12:30pm ET / 5:30am–9:30am PT · Online. Learn battle-tested Cursor workflows to ship production-grade full-stack apps faster. Get 40% off with code CURSOR40 and unlock 2 free eBooks via this follow-up form after purchase.
Architecting AI-Native Systems: Agentic Platforms, Guardrails, and Practical Patterns with Amar Akshat
According to Cloudera’s The Evolution of AI: The State of Enterprise AI and Data Architecture report, 96% of enterprises have at least some AI integrated into core processes (and 54% report “significant” integration). In domains like finance and payments, however, turning AI’s newfound capabilities into resilient, auditable, production-grade systems remains a formidable challenge. Architects must reconcile the non-deterministic nature of large models with the deterministic guarantees required for correctness, security, and compliance.
AI in the Architectural Loop: Agentic Systems and Human Judgment
Akshat’s current focus is what he calls:
“agentic architecture” – “designing systems that can reason about themselves.”
At Paysafe, this materializes in internal AI assistants (MCPX and ArchX) embedded into the development lifecycle. These agents codify architectural know-how:
“We have trained internal AI agents to analyze architectural decision records (ADRs) and suggest reusable design patterns, effectively learning from the scars of every project before it,” Akshat says.
Human judgment, however, remains non-negotiable.
“Human judgment still owns context, risk appetite, regulatory nuance, and product trade-offs… The AI can propose, but humans prioritize.”
He offers a rule of thumb:
“If a task mainly benefits from pattern recognition, that’s where putting AI in the loop makes sense. If it instead requires legal, financial, or compliance certainty, we keep it deterministic,” Akshat notes.
Patterns for AI-Infused Architecture: Data, Guardrails, and “Cell” Boundaries
Designing a reliable AI-heavy system starts with clear architectural boundaries and contracts. Akshat outlines three baseline elements his teams define up front:
Data Postulate: What is the current state of data that the AI can access?
Guardrail Postulate: What is allowed or not allowed for the AI to do? “Do you want to completely ground the system? Do you want purely deterministic responses or use an existing LLM?,” he says.
System Prompt Package: The system’s initial prompts and configuration – how the AI is instructed and what output format is expected. “For example, do you want more deterministic performance or do you want more accuracy? More transparency or more speed? These are the kinds of trade-offs that we encode into that system prompt package,” Akshat explains. His team creates a “prompt manifest” for each AI service – “like a Dockerfile for AI,” as Akshat describes it.
With these elements in place, Akshat’s architecture uses a “cell-based” structure to isolate AI components.
“All of our AI deployments are [like] cells – the tiniest unit of life – with their own regulatory nuances within themselves,” he says. “The critical path of analysis never leaves the cell boundary, and it only leaves the cell for audit and storage purposes. That keeps us safe – first of all – fast, and then deterministic.”
Guardrails and Reliability: Building Trust in Non-Deterministic Systems
Even with good design patterns, AI systems can fail in unpredictable ways. Akshat’s approach is to engineer for reliability upfront, with multiple layers of guardrails and fallbacks:
Confidence-Based Routing: “We route by confidence, really,” Akshat says. If the model’s confidence is below a threshold, the system escalates to a deterministic path or human approval.
Guardrails and Schema Validation: “Prompt validation and schema enforcement… we use Pydantic plus semantics and sanity checks,” he notes. “A human only steps in for logic failures, not syntax,” Akshat says.
Continuous Testing for Drift: “Basically, you have to look at AI prompts as code. That’s it,” Akshat says.
Auditability and Traceability
“We log every inference as an ‘architectural replay,’ which includes things like model ID, prompt version, and data snapshot,” says Akshat. Auditability is “incredibly important” in payments because “this is real data you’re dealing with… sometimes real transactions.”
Operationalizing AI Systems: Cost, Vendor Neutrality, and Skills
Cost Control – “Cache, Batch, Distill”: Akshat’s mantra for cost-efficiency is: “cache, batch, distill.” Caching stores and reuses AI results wherever possible. Batching aggregates requests to process many at once. Distilling means using smaller, cheaper models for simpler tasks.
Multi-Provider Strategy – Avoiding Lock-In: “We try to stick to the protocols the market is standardizing on – for example, the chat completions APIs and MCP,” Akshat says. His team built MCPX, an internal orchestrator that routes requests to different AI backends – OpenAI, Azure’s hosted models, Anthropic, or their own on-prem LLMs – all through the same interface.
Skills and Process for the AI Era: In Akshat’s words, “You don’t need to train models; you need to design systems that can survive their uncertainty.” His organization introduced “AI Behavior Reviews” in their design reviews and has identified an “AI SRE” role – someone who can discern if an incident was caused by model drift versus a code bug.
“First principles still apply,” Akshat reminds us. CAP theorem trade-offs, Twelve-Factor app practices, least privilege security – these remain the north star. AI can help augment the architect – even act as a junior designer or reviewer – but it doesn’t replace the need for sound architectural thinking.
🧠Expert Insight
In the full Q&A that follows Akshat walks through MCPX and ArchX, cell-based boundaries, prompts-as-code, confidence routing, and concrete payments and wallet scenarios, showing exactly how his teams operationalize these principles under real regulatory and reliability constraints.
Architecting AI-Native Platforms in the Real World: A Conversation with Amar Akshat
AI is already in the loop for writing code, reviewing changes, and even drafting architecture diagrams—but turning those capabilities into resilient, auditable, production-grade systems in regulated domains is still hard. In payments and financial services especially, architects have to reconcile non-deterministic models with deterministic guarantees ar…
🛠️Tool of the Week
Guardrails: Open-source guardrails for production LLM applications
Guardrails is an open source Python framework for building reliable LLM applications by running configurable input/output guards around models and enforcing structured, validated outputs.
Highlights:
Production-focused architecture: Ships both a Python library and a standalone Guardrails Server, with first-class docs on deploying the client/server model to production environments (including AWS), so teams can centralize safety policies without redesigning their apps.
Typed, structured outputs: Uses JSON/Pydantic-style schemas plus a library of reusable “validators” in Guardrails Hub to turn free-form LLM responses into strongly-typed, semantically validated outputs, with automatic correction and redaction when checks fail.
Actively maintained and ecosystem-friendly: Apache-2.0 licensed, with multiple releases in 2025 (latest v0.6.7) and integrations for LangChain, LlamaIndex, and the OpenAI SDK, making it straightforward to drop into existing LLM stacks.
📎Tech Briefs
Innovation Session | Introducing Microsoft Agent 365: the Control Plane for AI Agents: Focuses on how to onboard, train, and run autonomous AI agents securely at scale, with a strong emphasis on Agent 365 and Microsoft’s emerging “agent estate” model. It walks you through patterns for making agents adaptive to changing business needs while keeping them inside guardrails for data access, regulatory compliance, and threat protection, effectively positioning security/governance as part of the agent development lifecycle.
Introducing Google Antigravity, a New Era in AI-Assisted Software Development: Google has launched Antigravity, an AI-first IDE where multiple agents can directly operate the editor, terminal, and an embedded browser to write, test, and verify code autonomously. It’s built around Gemini 3 Pro but also supports other models like Claude Sonnet 4.5 and GPT-OSS, and introduces “Artifacts” (task lists, plans, screenshots, browser recordings) as first-class objects to audit and review what agents did.
GitHub Annotation Toolkit: open-source Figma library for accessible design handoff: GitHub has open-sourced its Annotation Toolkit, a Figma library of structured annotations that lets designers embed intent and accessibility behavior directly into their design files, instead of scattering it across comments and Slack threads. Built by GitHub’s accessibility design team after they found that nearly half of their audit issues could have been prevented with better upfront documentation, the toolkit gives teams reusable “stamps” to describe interactivity, responsive behavior, and WCAG-aligned requirements, and ships with checklists, tutorials, and a GitHub repo for contributions.
Salesforce eVerse: Simulation environments for enterprise agents:
Salesforce AI Research has launched eVerse, a simulation environment for training voice and text agents using synthetic data generation, stress-testing, and reinforcement learning before they ever see real customers. The goal is to fix “jagged intelligence” in enterprise agents—brilliant on benchmark tasks but brittle with messy, real-world inputs like noisy calls, frustrated tone, or off-script requests—by running them through lifelike, high-variance scenarios at scale.Forbes Tech Council – “How MCP Gateways Can Mitigate Hidden MCP Security Risks”: Suman Sharma, Head of PAM Engineering at Ping Identity argues that MCP gateways (sitting between agents and servers) are becoming essential to defend against malicious or misconfigured MCP servers, focusing on policy enforcement, monitoring, and isolation to contain prompt injection and data-exfiltration risks.
That’s all for today. Thank you for reading this issue of Deep Engineering. We’re just getting started, and your feedback will help shape what comes next. Do take a moment to fill out this short survey we run monthly—as a thank-you, we’ll add one Packt credit to your account, redeemable for any book of your choice.
We’ll be back next week with more expert-led content.
Stay awesome,
Divya Anne Selvaraj
Editor-in-Chief, Deep Engineering
If your company is interested in reaching an audience of developers, software engineers, and tech decision makers, you may want to advertise with us.
 |
|