Context Is an Architectural Imperative

Why the enterprise data stack we built for people has to be re-architected for AI agents

The uncomfortable truth of this cycle is that the models are ready and the data architecture is not. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027 — citing escalating costs, unclear business value, and inadequate risk controls. MIT’s 2025 NANDA study found that 95% of enterprise GenAI pilots deliver no measurable financial return, and RAND put the broader enterprise-AI failure rate at roughly 80%. None of those are model-quality problems. They are architecture problems. And they surface at exactly the moment an organization tries to go from one clever demo to a hundred production agents.

We built our systems for humans. Agents are a different kind of user.

For thirty years we designed enterprise data systems around a single, reasonable assumption: a person is on the other end. A human runs a report, reads the dashboard, notices the number looks off, and applies judgment before acting. The human is the reasoning engine, the error-correction layer, and the accountability boundary. Our architectures could afford to be fragmented because a person silently reassembled the context in their head.

An AI agent removes that human from the loop, and with it every assumption we quietly relied on. Consider what changes:

• Access pattern. A person opens a handful of records; an agent may traverse thousands of entities and relationships in a single reasoning step, then do it again on the next turn.
• Tolerance for ambiguity. A human sees two conflicting customer records and knows which one is real. An agent will confidently pick one — or average them — and act.
• Latency and volume. Humans query occasionally; agents query constantly, in loops, across many concurrent sessions. Retrieval cost and consistency become first-order concerns, not footnotes.
• Explainability. A human can explain why they approved something. An agent needs the relationships and provenance made explicit and machine-readable, or the decision is unauditable.

This is the core shift. Systems built for humans optimized for storage, retrieval, and presentation. Systems built for agents have to optimize for reasoning and trust — the machine has to understand how the business fits together before it acts, because there is no longer a person to catch the mistake.

The stack we already own was built for a different job

Most enterprises did not arrive here empty-handed. They have decades of well-engineered infrastructure: ETL and ELT pipelines, systems of record, data warehouses, data lakes, lakehouses, and mature ML platforms. That stack is excellent at what it was designed to do — move data reliably, store it cheaply, and answer known questions at scale. The warehouse answers “what were Q3 refunds by region.” The lakehouse trains a churn model on last year’s features. These are batch-oriented, schema-on-read or schema-on-write, human-in-the-loop analytics patterns.

The problem is not that this infrastructure is wrong. It is that it was never asked to supply live, connected, trustworthy business context to an autonomous reasoner. A refund agent does not need a quarterly aggregate. It needs to know, right now, that this customer, on this contract, with this open dispute, tied to this shipment and this policy exception, is or is not eligible — and to show its work. That is a graph traversal over current, resolved, governed relationships, not a table scan.

What AI actually adds on top

The first wave of enterprise AI answered a real question: how do we get relevant information into the model at runtime? Retrieval-Augmented Generation, vector search, GraphRAG, and hybrid retrieval were genuine advances, and they moved the industry forward. But each also revealed the next constraint.

Retrieval finds information. It does not guarantee the information is coherent, current, or safe to act on. Stanford’s RegLab found that production legal RAG systems — from vendors who do this for a living — still hallucinated on 17% to 33% of queries. Vector similarity captures semantic closeness, not relationships or truth; Databricks reported a 15-point jump in retrieval accuracy simply by adding reranking, which tells you how much the raw signal was missing. The lesson is not “RAG is bad.” It is that retrieval alone answers where is the information, while production AI also has to answer how does it fit together, and can I trust it enough to act.

Answering the second question introduces requirements the legacy stack never had to meet: entity resolution so the agent sees one customer instead of five near-duplicates; explicit relationships so it can reason across customers, contracts, assets, and policies; temporal awareness so it knows what was true at the moment it acted; provenance so every answer is traceable; and continuous, incremental updates so context reflects the business as it changes rather than a nightly snapshot. Add graph, vector, document, and search into the mix and you are no longer bolting a feature onto the warehouse — you are describing a different architecture.

This is a systems problem, not a tools problem

Here is where I see the most expensive mistakes. Faced with these new requirements, teams reach for tools: a vector database here, a graph library there, an orchestration framework, a reranker, a pipeline to keep them all in sync. Each tool solves its slice. Together they create a distributed systems problem that nobody owns. Teams often assume the answer is to add a graph database alongside a vector database. Production AI requires a unified contextual architecture that brings together relationships, semantic retrieval, documents, search, time, provenance, and governance.

This webinar, How to Build Reliable Context-Aware AI at Enterprise Scale captures it well: stitch together SQL, NoSQL, vector, graph, and search — each scaling differently, each with its own API, consistency model, and failure modes — and you are operating twenty-odd technologies just to keep context alive. Every new AI initiative then adds another retrieval pipeline, another graph, another vector index, another way of reconstructing the business. This is why data teams are drowning in plumbing: a 2025 Matillion survey found 64% of organizations report their data teams spend more than half their time on repetitive, manual work, and the share of engineering time consumed by AI-related data wiring nearly doubled from 19% to 37% in two years. That effort produces no differentiated business value. It is the tax you pay for treating an architectural problem as a shopping list of tools.

The question isn’t whether AI needs context. It’s where that context should live, how it’s maintained, and whether the enterprise reconstructs it every time AI runs — or maintains one shared, trusted understanding of the business.

— Ravi Marwaha, COO & CPTO, Arango

At scale, the bill comes due

When an organization runs one agent, the weaknesses hide. When it runs a hundred, they compound. The concerns that were afterthoughts in the pilot become the entire conversation in production:

• Governance and access control. An agent that traverses relationships can just as easily traverse a permission boundary. Row- and record-level entitlements have to travel with the context, not sit in an application above it.
• Observability and reliability. When two agents reach different conclusions about the same customer because they each reassembled context differently, you need to see why — across retrieval, relationships, and versions of the truth.
• Maintainability and upgradability. N pipelines feeding N agents means N things to migrate every time a source system changes. Shared context means you fix it once.
• Cost. Redundant indexes, repeated graph rebuilds, and ballooning token consumption from over-retrieval are the quiet line items that turn a promising project into a canceled one — which is precisely the failure mode Gartner is describing.

Every one of these gets harder if context is reconstructed per application, and dramatically easier if it is maintained once as shared enterprise infrastructure — persistent, governed, observable, and reusable across every agent.

Solve one problem fast, or build to solve many

There is a real tension here, and it is worth naming honestly. Any capable team can wire up a single impressive AI use case in a few weeks. The trap is that the architecture that ships one demo fast is rarely the architecture that supports the tenth, fiftieth, or hundredth use case. Speed on the first problem quietly borrows against every problem after it.

The strategic choice is not “build vs. buy” — it is where you spend your scarce engineering judgment. Your differentiation is your domain: your customers, your fraud patterns, your supply chain, your clinical or financial or operational expertise. It is almost never knowledge-graph modeling, index synchronization, retrieval-strategy tuning, or entity resolution. Those are hard, general problems that a platform should solve once, well, for everyone — the same way you don’t build a separate customer database for every application, you shouldn’t rebuild business context for every AI system.

Fourteen years in the making: from ArangoDB.com to Arango.ai

I’ll allow myself a moment of pride here, because this is the part of the story I care about most. When we founded ArangoDB in 2014, no one was talking about agents, GraphRAG, or contextual data layers. The problem we set out to solve was quieter and, as it turns out, far more durable: enterprises were accumulating more applications, databases, documents, and systems every year, and the hard part was never storing the data — it was understanding how it connected. We made an unusual bet for the time. Instead of yet another single-model store, we built ArangoDB as a native multimodel database that held graph, document, key-value, search, and eventually vector data in one engine — because meaning doesn’t live inside individual records. It lives in the relationships between them.

That bet is what ArangoDB.com quietly powered for more than a decade: fraud detection, cybersecurity, digital twins, supply chain optimization, network and asset management, investigations, and knowledge graphs — mission-critical systems where understanding relationships wasn’t a nice-to-have, it was the entire point. We were solving the context problem years before the industry had a name for it.

Then agentic AI arrived, and the gap came into focus. First-generation agentic workflows reconstruct context on every run — a new pipeline, a new graph, a new vector index for every use case — and treat the connective tissue of the business as an afterthought. That is precisely the problem we had spent fourteen years learning how to solve. Our evolution from a core database at ArangoDB.com to the Arango Contextual Data Platform — Arango 2.0 — isn’t a pivot. It’s the natural next step of the same mission, arriving at the exact moment the whole industry has caught up to it.

This is Arango’s moment, and I don’t say that lightly. The foundation we built for connected data is precisely the foundation autonomous AI now depends on. We didn’t have to reinvent ourselves to meet it — we had to extend what we’ve believed from the beginning. After fourteen years, watching the industry rediscover, from the AI direction, the same architectural truth we started with is the proudest kind of validation.

Context is not an afterthought. It’s the architecture.

So when I say context should be a first-class architectural layer, it isn’t theory — it’s what the Contextual Data Platform now automates: turning structured and unstructured enterprise data into a living, governed context that AI can search, retrieve, and reason over, kept continuously current through automated graph construction, adaptive retrieval, incremental updates, and time-aware data. Every one of those capabilities exists to remove the undifferentiated heavy lifting so your teams build the applications that actually move your business.

The organizations that win the agentic era won’t be the ones with the cleverest prompt or the largest context window. They’ll be the ones who decided early that context is a first-class architectural layer — maintained once, trusted everywhere — and who let a partner own that data problem so they could own their domain. The models are ready. The question is whether your architecture is.

Focus on your use case. Let the platform own the context. That’s not a convenience — at agentic scale, it’s the difference between a project that ships and one that gets canceled.

— Ravi Marwaha, COO & CPTO, Arango