Deep Engineering #43: David Knickerbocker on Building AI That Sees the World as It Is, Not as It Was
Real-time knowledge graphs, awareness before truth, and why an empty dataset is better than a hallucination
C++ Memory Management Masterclass (Live) — Back for the 3rd Run
Learn ownership, RAII, smart pointers & allocators to eliminate leaks/UB—live hands-on with Patrice Roy—#1 bestselling author of C++ Memory Management and ISO C++ Standards Committee (WG21) member.
📅 Online: Apr 18 & Apr 19. Limited seats.
✍️ From the editor’s desk,
Welcome to the 43rd issue of Deep Engineering!
On April 15, Neo4j ran NODES AI 2026, a full-day virtual conference dedicated to AI with knowledge graphs, with tracks covering GraphRAG, graph-based agent memory, and context engineering. Interestingly, the opening keynote framed the central challenge directly around how most AI systems behave as if they have never seen the data that the organisations deploying them have spent years accumulating. The gap between what a system has access to and what it can actually use at query time is, in the view of the practitioners at NODES, the defining engineering problem of this moment.
And so, in today’s issue we want to address exactly that gap. David Knickerbocker, founder of Verdant Intelligence and author of Network Science with Python (Packt), has spent years building knowledge graph systems that stay current with a different design constraint than most teams start from: not how to make retrieval faster, but how to make the system aware of what changed a minute ago. His system (Verdant Eye) treats knowledge as a continuous stream of claims rather than a static store of facts, and that distinction has significant consequences for how you handle freshness, hallucination, temporal drift, and testing.
This is the first of two features from our conversation with David, the second will cover what building AI on top of messy, adversarial, real-world data teaches you that clean-dataset engineers never have to confront. You can read the full interview and watch the conversation here.
Let’s get started.
30k+ DevOps Engineers Read EverythingDevOps
The cloud-native ecosystem moves on its own schedule. New tooling, shifting practices, team structures that look nothing like they did two years ago. Therefore, keeping up is not optional when your job is to build and run reliable systems.
EverythingDevOps‘ weekly digest is built specifically for engineers who want the signal without the scroll.
What’s changing in the ecosystem — and what it actually means for your work
Career thinking for engineers moving from senior to staff to leadership
Events, opportunities, and community conversations worth your Frida
AI That Sees the World as It Is, Not as It Was
by Saqib Jan with David Knickerbocker
Most AI systems are answering questions about a world that no longer quite exists. They draw from training data frozen at a point in time, from retrieval systems that return whatever was most recently indexed, from pipelines that treat knowledge as a stable object rather than a moving one. For David Knickerbocker, founder of Verdant Intelligence, this is not a limitation to work around, but a design error to avoid from the start.
The problem starts at problem definition
Building a knowledge graph that stays fresh does not require a different database. It requires a different question. For Knickerbocker the starting point is always the same, “what is the system actually trying to know, and at what resolution of time?”
He explains, “If you want to build a world AI and be able to answer questions about things that happened a minute ago, then that is your problem statement. And then you think about how to get that data into the database so that it is there and it is fresh. But then you also have to get AI to be able to use that data. There are two sides to this coin.”
The engineering philosophy he applies is KISS (Keep it Super Simple), and YAGNI (You Aren't Gonna Need It"). Both push in the same direction, he reasons “build the minimum thing that works, validate that it works, and expand from there.” His system (Verdant Eye) maintains data fresh up to a minute old, not by doing something architecturally exotic, but by treating freshness as a first-class constraint from day one rather than an optimization to add later.
“The AI industry feels very shiny and very new, but there is a lot of old school discipline that is still extremely useful. You start with the idea, you go through the ideation, from ideation you create your spec, from the spec you do your project management, you assign tasks and do the work. It feels like vanilla old school engineering to me.”
Awareness before truth
The instinct when building a real-time knowledge system is to treat freshness and accuracy as a tension to manage: newer data is less verified, older data is more reliable, and the system needs some formula for weighing one against the other. But Knickerbocker rejects this frame entirely.
He explains from his engineering experience that why his systems are not trying to determine what is true. But instead trying to capture what is being claimed, and those are fundamentally different engineering problems.
“In the world of open source intelligence, it has less to do with right and wrong. It has less to do with facts. What I am looking for is really claims of what is going on in the world. I do not make that decision, and I do not allow my AI to make the decision about what is true or false either.”
In adversarial open source intelligence environments, two sources in direct conflict with each other are not a problem to resolve. But they are both signal. And what matters is that both claims are captured, clustered, and surfaced. Knickerbocker describes this structure using a frame he developed from years of NLP work: not clusters in the geometric sense, but ribbons, layered bands of related information that emerge from the data.
“You have a whole bunch of information and this top ribbon might be this bad thing happened. The next ribbon might be this event is happening at the library. The next ribbon might be a punk rock show is happening at this nightclub.”
The consequence of this design is that awareness comes before truth, and that is by intention. For the applications the Verdant Eye is built for, knowing what is being said and where is actionable before you know whether any of it is correct. Adjudicating truth is a downstream problem, one that can be handled by human judgment or purpose-built downstream systems, not by the ingestion layer.
Snapshot versus movie
What separates a real-time knowledge graph from an agent that runs searches is not a matter of degree. It is a structural difference in what kind of object the system is.
An agent with search tools takes a snapshot. It queries for something, gets back the most relevant indexed results, and presents them. The system only knows what it was asked to look for, and it only looks when prompted. A knowledge graph that continuously ingests from the open web is more like a movie: it is always recording, and when you query it, you are not taking a new photograph but pulling a frame from a film that has been running the whole time.
“If you use a tool to do a search to find out something, you are getting a snapshot of time. My systems capture the heartbeat of the Internet themselves, and they are always listening. It would be closer to a movie compared to a photograph.”
The practical difference shows up clearly in demos with clients who need urgent situational information. Running a calibration query against the Verdant Eye, asking what is the latest information, returns results anywhere from a few seconds old to a minute and a half old. That behaviour is not possible with a search-tool approach at any level of optimization.
An empty result matters too. If nothing has been reported in the last minute on a given topic, the system returns nothing. That discipline is load-bearing.
“An empty dataset,” Knickerbocker underscores, “is better than a hallucination.”
The rush to use a tool before understanding it
Knickerbocker spent years watching teams fail with graph databases in 2020 and 2021, and he sees the same pattern repeating now with agents and AI. Teams adopt the tool before building the understanding, and then cannot tell whether it is working.
“There is a rush to use agents before even understanding AI. And if the understanding is not there, then it is just wishful. If you do not know how it works, you cannot tell the difference between it ran and it ran correctly. Those are very different things.”
His own relationship with graphs predates the graph database trend by years. At Intel, from 2015 to 2019, he was using graph theory for data flow mapping, tracing inputs to outputs across thousands of scripts and hundreds of servers, using centralities to find important nodes and shortest paths to understand flow. He was never using graph databases. He was using the underlying science.
“I was never invited to the cool kid graph database parties. I was always just doing stuff with graphs and using it to map out data flows and fix production outages. Dead serious stuff.”
That gap between tool and understanding is what drove him to write Network Science with Python. Teams were populating graph databases and then stopping, treating the populated database as the deliverable. The value, in his view, starts the moment the graph is populated and you begin running network science on top of it.
“If you do not know about the basics of network science, then what are you gonna do with the graph database? You have put your data in the graph database. Whereas if you come to graph databases and you have learned about centralities, community detection, shortest paths, simulations, then it can really have an impact on your network thinking.”
Similarity is not the same as identical
The central limitation of vector-based retrieval is one that Knickerbocker has framed the same way since at least 2017. Semantic similarity is a probabilistic measure, not a logical one. Two pieces of text can be highly similar in embedding space while referring to entirely different things. A graph replaces probabilistic similarity with structural traversal.
“Similarity in language is not equal to same. I will say that one more time. Similarity is not equal to same. Similar sounding things can be very, very different from each other. A graph anchors things into a piece of context.”
In a GraphRAG system, the query does not ask what is most like this input. It traverses the graph: if you are asking about jazz events in Portland, Oregon, you are connected to the Oregon node, to the Portland node, to the jazz node. The hallucination space collapses because the answer space is bounded by structure rather than probability.
“In a GraphRAG system, if there is no match then the output is that there is no match. There is no hallucination opportunity. With a similarity-based system, there could be similarity even if it is only a single word in a paragraph.”
This intuition was clear to him during his data operations years at Intel. Log files across hundreds of servers written by dozens of engineers contain enormous amounts of natural language. Working with that language to understand what a production system was doing made it obvious that the structure, the graph, and the meaning, the language, were inseparable.
“Graphs show you where things go,” Knickerbocker adds. “But all of the context about what that node even is is often carried by language itself.”
Deliberate forgetting
Temporal drift is usually framed as a correctness problem: facts become outdated, relationships change, the graph silently goes wrong. The standard response is validation pipelines, contradiction detection, freshness scores. Knickerbocker builds around a different principle.
Sharing from his experience building the Verdant Eye, Knickerbocker explains that the Verdant Eye system is not called the Verdant Brain because the metaphor is not storage. It is perception.
“The Verdant Eye sees, and it does not contain eternal memory, because that is not what an eye does. An eye sees. When the scene changes, the scene changes. Your eyes do not need to be recalibrated. The thing has just changed.”
The design principle is biological: living systems do not maintain eternal memory, and a knowledge system that tracks a living world should not either. The practical implementation mirrors transactional database patterns he worked with throughout his career in data operations. Data that is no longer operationally relevant gets archived. The live layer runs on what is current, and the cost of maintaining it stays proportional to what the system actually needs to know.
“In a transactional database, you operate off of what you need, and data that is not needed eventually gets archived. I want to build AI that does not boil the ocean, that can be bootstrapped by individuals. Infinite memory is not just philosophically wrong for this kind of system. It is economically unworkable.”
The pass butter philosophy
Knickerbocker shares how he currently runs three production GraphRAG systems, each built for a distinct purpose with its own testing criteria. Verdant Intelligence operates at high altitude, tracking events across states and regions. Grooveseeker (which he created to put his AI system to a different and specific kind of use) operates at street level, finding events in specific cities on specific nights. A third system contains thirty years of AI research, useful for tracing the intellectual lineage of ideas when building new things.
The principle connecting all three comes from a scene in Rick and Morty. Rick builds a robot. The robot asks what its purpose is. Rick says: you pass butter.
“There is no testing framework anybody else can give me that is going to be fit for purpose for what I am trying to build, because I am not trying to build general intelligence. I am trying to build intelligence that serves a specific purpose.”
Each system is therefore tested against the specific failures that would make it useless for its purpose. For Grooveseeker, that means verifying the right city, not a city of the same name in a different state; the right date, not a historical recurrence of the same event; and a URL that actually leads somewhere you can buy a ticket.
The final test of the Grooveseeker system was to stop writing articles proving it worked and simply use it. The system returned events in Portland between March 10 and March 13. Knickerbocker even attended one of them, bought a ticket at the door, saw the band, and spoke to one of the musicians.
His AI did not send him to a venue that did not exist. “That is how I know it works,” Knickerbocker affirms.
🔍 In case you missed it…
Knowledge Graphs, GraphRAG, and Real-Time AI in Production with David Knickerbocker
This conversation with David Knickerbocker keeps returning to a single conviction: the best engineering starts with intentional problem definition, and most AI failures happen when teams rush to use a tool before understanding what they are actually trying to build.
🛠️ Tool of the Week
Graphiti — open-source temporal knowledge graph engine for AI agents
Highlights:
Temporal fact management: Facts carry explicit validity windows — invalidated when superseded, never deleted, queryable at any point in time.
Incremental ingestion: New data integrates immediately without batch recomputation, keeping the graph current as the world changes.
Hybrid retrieval: Combines semantic search, BM25, and graph traversal in a single query, typically under 100ms.
MCP-native: Ships an MCP server for direct integration with Claude, Cursor, and other MCP clients.
📎 Tech Briefs
Neo4j NODES AI 2026 closing keynote: From Data to Knowledge to Action - Sudhir Hasbe makes the case that most enterprise AI behaves as if it has never seen the organisation's own data, and how graphs help close that gap.
Graph memory for LLM agents: the relational blind spots that flat vectors miss - Zep's LongMemEval evaluation shows 18.5% accuracy gains over vector baselines, context tokens dropping from 115,000 to 1,600, and latency falling from 29 seconds to under 3 seconds.
FalkorDB Browser ships April 2026 update for GraphRAG developers - Adds favourite query saving, one-click connection string copying, enhanced graph statistics, and configurable table views..
Neo4j Aura Agent Hackathon opens - Developers building knowledge-graph-grounded AI agents can register for cloud credits and managed GraphRAG platform access.
Four GraphRAG papers accepted at ACL 2026 - PolyU’s Awesome-GraphRAG repo confirmed acceptances including ProbeRAG for retrieval faithfulness, LegalGraphRAG for legal reasoning, and LinearRAG for efficient graph construction.
That’s all for today. Thank you for reading this issue of Deep Engineering.
We’ll be back next week with more expert-led content.
Stay awesome,
Saqib Jan
Editor-in-Chief, Deep Engineering
If your company is interested in reaching an audience of senior developers, software engineers, and technical decision-makers, you may want to advertise with us.
Thanks for reading Packt Deep Engineering! Subscribe for free to receive new posts and support my work.
most "knowledge bases" are really just yesterday's claims frozen as facts, and nobody notices until someone asks about something that changed last tuesday.