Introduction
Businesses need solutions that are accurate, cost-effective, and scalable to get meaningful insights from massive amounts of data. Retrieval-Augmented Generation (RAG) is an advantageous technique that can raise the game for large language models by integrating them with a retrieval system. This white paper compares two RAG implementations, (i) using vector databases and (ii) ArangoDB GraphRAG with knowledge graphs, focusing on four main criteria:
- Completeness & accuracy of response
- Cost of response
- Scalability
- Efficiency & contextual understanding
This whitepaper offers a detailed analysis – backed up by real-world examples and an in-depth costing comparison – of how GraphRAG with ArangoDB is superior to RAG performed with a vector database in all four criteria.
Why This Comparison Matters to Businesses
Understanding the differences between these approaches is important if data teams want to optimize their data and analytic solutions. Two very relevant use cases illustrate the importance of this comparison:
- Customer 360: Combining data from various sources to provide a complete view of customer interactions, preferences, and behaviors. Accurate and comprehensive data retrieval will improve customer satisfaction and loyalty.
- Fraud Detection: Identifying patterns and anomalies in transactional data to detect and prevent fraudulent activities.
Efficient and accurate data retrieval helps mitigate risks and reduce losses from fraud.
Vector Databases: Definition and Description
A vector database stores, indexes, and queries high-dimensionality vector data. Vectors are derived from machine learning models that convert data into numerical representations. Vector databases are optimized for similarity search, using operations like nearest neighbor search and cosine similarity, applicable in recommendation systems, image retrieval, document search, anomaly detection, and natural language processing (NLP).
Steps for Implementing RAG with Vector Databases
- Data Preparation: Convert text data into vector embeddings using models like ChatGPT.
- Vector Storage: Store the embeddings in a vector database such as FAISS.
- Retrieval: Retrieve relevant vectors based on similarity to the query.
- Generation: Use the retrieved vectors to augment the response generation by a large language model.
ArangoDB GraphRAG with Knowledge Graphs: Definition and Description
ArangoDB GraphRAG with knowledge graphs uses ArangoDB’s multimodel capabilities, focusing on graph databases to benefit response generation. Data is represented as nodes and edges, reflecting complex relationships. Queries retrieve relevant data by traversing the graph, which improves the richness of the generated responses and avoids dangerous hallucinations.
Key Components
- ArangoDB: Supports document, key-value, and graph data models.
- Knowledge Graph: Nodes represent entities; edges represent relationships.
- GraphRAG: Combines retrieved data with language models to generate detailed responses.
Steps for Implementing GraphRAG with ArangoDB Knowledge Graphs
- Data Preparation: Structure data into nodes and edges to form a knowledge graph.
- Graph Storage: Store the graph in ArangoDB.
- Retrieval: Query the graph to retrieve relevant and interconnected data.
- Generation: Use the retrieved graph data to augment the response generation provided by a large language model.
Multi-Model Capabilities of ArangoDB
ArangoDB’s multi-model capabilities enhance GraphRAG in several very important ways:
- JSON Document Support: Stores unstructured data, enabling flexible schema design and easy integration of multiple data sources and formats.
- Integrated Full-Text Search: Easily queries textual data efficiently and as part of the core database platform, improving the relevance and accuracy of search results.
- Geospatial Support: Allows for the incorporation of location-based data to support advanced queries and analytics based on geographical information.
These multi-model capabilities make ArangoDB a far superior platform for implementing GraphRAG, offering a unified solution for handling various data types and complex queries. And of course, far superior to Vector Store RAG.
Use Cases and Relevance
Customer 360
Scenario 1: Personalized Marketing
A business wants to send personalized marketing messages to customers based on their purchase history, preferences, and behavior.
- Vector Databases: Convert customer data into vector embeddings. Use similarity search to find customers with similar behaviors and generate marketing messages based on common patterns.
- GraphRAG with ArangoDB Knowledge Graphs: Use a knowledge graph to connect customer data, including purchase history, preferences, interactions, and behaviors. Traverse the graph to identify patterns and relationships. Generate personalized marketing messages using the interconnected data. Augment this customer data analysis with a blend of document, search, graph, key/value, and geospatial data in a single LLM-to-graph query.
Knowledge graphs provide a more comprehensive and contextual view of customer data, allowing for more accurate and personalized marketing efforts. This leads to higher customer engagement and satisfaction.
Scenario 2: Customer Support
Imagine a business that aims to improve customer support by providing accurate and very fast responses to customer inquiries, complaints, and other interactions.
- Vector Databases: Convert support tickets and knowledge base articles into vector embeddings. Use similarity search to find relevant articles and generate responses based on the retrieved data.
- GraphRAG with ArangoDB Knowledge Graphs: Use a knowledge graph to connect support tickets, knowledge base articles, customer interactions, and issue/complaint resolutions. Traverse the graph to find relevant information and generate highly accurate and very details responses without hallucinations.
Knowledge graphs ensure that responses are not only accurate but also contextually rich, considering the full history and context of customer interactions. This improves customer satisfaction and improves resolution times.
Fraud Detection
Scenario 1: Transaction Monitoring
Imagine a financial institution needing to monitor transactions for potential fraud by identifying patterns and anomalous items and behaviors.
- Vector Databases: Convert transaction data into vector embeddings. Use similarity search to identify transactions that deviate from normal patterns and generate alerts that are meaningful and not just noise.
- GraphRAG with ArangoDB Knowledge Graphs: Use a knowledge graph to connect transaction data, account histories, user behaviors, and external risk indicators. Traverse the graph to find suspicious patterns and generate fraud alerts that also include detailed contextual information, which helps to prevent further fraud by identifying the perpetrators quickly.
Knowledge graphs provide a more complete and detailed view of transactions and their context, allowing for more accurate and complete fraud detection alerts and remedies. This reduces false positives and hallucinations, and gives fraud teams even better tools and techniques to address fraud quickly.
Scenario 2: Identity Verification
Imagine a company that needs to verify the identity of users during account creation and transactions to prevent identity theft.
- Vector Databases: Convert user data into vector embeddings. Use similarity search to find similar profiles and identify potential identity theft.
- GraphRAG with ArangoDB Knowledge Graphs: Use a knowledge graph to connect user data, transaction histories, device information, and behavioral patterns. Traverse the graph to verify identities and identify discrepancies but with more context and richness of information delivered.
Knowledge graphs allow for a deeper and more accurate ID verification process by taking into account the full context of user data and behaviors. This beefs up security and reduces the risk of identity theft.
Comparison Criteria
Completeness and Accuracy of Response
- RAG with Vector Databases: Responses depend on the quality of retrieved documents and the language model’s generation capabilities. Vector DB Rag almost always lacks detailed context due to the disjoint nature of vector embeddings.
- ArangoDB GraphRAG with Knowledge Graphs: Provides comprehensive and accurate responses by leveraging interconnected data and relationships in the knowledge graph. This approach captures complex nuances and semantics, ensuring richer context and detailed answers.
Real-World Example
Let’s use a real-world example data set that analyzes and reports on violence in developing world countries. Government officials and organizational analysts can supply simple, natural-language questions to the system to understand events quickly and accurately.
We include for reference actual images of queries run in a Jupyter notebook to establish the real-world nature of this comparison.
Vector Store Example
You will note the sparse amount of information provided in this Vector Store RAG example. It’s basically 3 high level sentences without a lot of detail.
Let’s recap the key part of this response for visual clarity:
“Yes, there has been violence against citizens. For instance, students protested against the government agenda at Haramaya university and police forces responded nine deaths. There were also instances where civilians were killed or injured in various incidents such as bombings, attacks, and roadside explosions.”
ArangoDB GraphRAG Example
Let’s also highlight the key part of this response for visual clarity:
Yes, there have been several instances of violence against citizens. Here are some examples:
- On January 5, 1997, 5 citizens were beheaded in Douaouda, Tipaza.
- On January 2, 1997, two citizens were beheaded in Hassasna.
- On January 3, 1997, two citizens were killed in a raid on the village of Hassi EI Abd. Several televisions were destroyed and the school of the village was burned.
- On January 4, 1997, 16 citizens were murdered in the village of Benachour, Blida by masked men a few hundred meters away from a military camp.
- On January 5, 1997, 18 citizens were killed in the Oliviers district of Douaouda, Tipaza. Among the victims were 3 children and 6 women.
- On January 6, 1997, 23 citizens were horribly mutilated and killed in Hadjout, Tipaza by an armed group.
- On January 5, 1997, the president of the chamber of bailiffs of the Jiiel court was killed by armed men.
Why are the results so much better? It has to do with the inherent connectedness of the data and the ability to traverse the graph to find details, and rich context provided by ArangoDB’s multi-model capabilities that can combine graph, document, full-text search, and geospatial all in a single query, which would be completely impossible in a Vector Store situation. Moreover, there is no additional cost for Embeddings generation, which also saves a tremendous amount of time. Query cost is extremely predictable because you can specify and throttle JSON tokenization in the response generation using AQL.
Cost of Response
- RAG with Vector Databases: High initial and ongoing costs due to embedding generation, storage, and indexing. Computational resources are substantial, especially for large datasets.
- ArangoDB GraphRAG with Knowledge Graphs: More cost-effective due to optimized graph traversal and lower storage requirements. Graph databases handle complex queries efficiently, reducing computational overhead.
Scalability
- RAG with Vector Databases: Scalability is challenging due to high computational and storage costs for embedding generation and retrieval processes. As data volume increases, maintaining performance becomes more resource-intensive.
- ArangoDB GraphRAG with Knowledge Graphs: Highly scalable due to efficient graph traversal and relationship indexing. ArangoDB can handle large datasets with complex interconnections, maintaining performance and reducing the need for extensive computational resources.
Efficiency and Contextual Understanding
- RAG with Vector Databases: While efficient in similarity searches, it may miss out on complex contextual relationships due to the limitations of vector embeddings.
- ArangoDB GraphRAG with Knowledge Graphs: Excels in contextual understanding by capturing and utilizing complex relationships, providing richer and more nuanced responses.
TCO Comparison / Calculator
Here’s how a Total Cost of Ownership (TCO) model and a costing calculator for comparing Vector Stores for RAG (Retriever-Augmented Generation) vs. GraphRAG (Graph-based Retriever-Augmented Generation) with Knowledge Graphs might look.
Model Inputs
- Data Size (GB/TB)
- Query Volume (queries per second/minute/hour)
- Compute Costs (cost per vCPU hour)
- Memory Costs (cost per GB hour)
- Storage Costs (cost per GB/month)
- Data Processing Costs (cost per query/operation)
- Implementation Costs (one-time setup and integration costs)
- Maintenance Costs (annual support and maintenance)
Example Calculator
Vector Stores for RAG
- Data Size: 1 TB
- Query Volume: 10,000 queries/day
- Compute Costs: $0.05 per vCPU hour
- Memory Costs: $0.01 per GB hour
- Storage Costs: $0.02 per GB/month
- Data Processing Costs: $0.001 per query
- Implementation Costs: $50,000
- Maintenance Costs: $20,000/year
Vector Stores for RAG
- Data Size: 1 TB
- Query Volume: 10,000 queries/day
- Compute Costs: $0.03 per vCPU hour
- Memory Costs: $0.005 per GB hour
- Storage Costs: $0.01 per GB/month
- Data Processing Costs: $0.0005 per query
- Implementation Costs: $40,000
- Maintenance Costs: $15,000/year
TCO Calculation for One Year
Vector Stores for RAG
- Storage Costs: 1 TB * $0.02 * 12 months = $240
- Compute Costs: 10 vCPUs * $0.05 * 24 hours * 365 days = $43,800
- Memory Costs: 1 TB * $0.01 * 24 hours * 365 days = $87,600
- Data Processing Costs: 10,000 queries/day * 365 days * $0.001 = $3,650
- Implementation Costs: $50,000
- Maintenance Costs: $20,000
- Total Cost: $205,290
Vector Stores for RAG
- Storage Costs: 1 TB * $0.01 * 12 months = $120
- Compute Costs: 10 vCPUs * $0.03 * 24 hours * 365 days = $26,280
- Memory Costs: 1 TB * $0.005 * 24 hours * 365 days = $43,800
- Data Processing Costs: 10,000 queries/day * 365 days * $0.0005 =$1,825
- Implementation Costs: $40,000
- Maintenance Costs: $15,000
- Total Cost: $127,025
Savings with GraphRAG: $205,290 – $127,025 = $78,265
Costing Justification
It’s important to back up these estimates with some explanation as to the values in the model.
Storage Costs
- Vector Stores for RAG: Storage costs are determined based on the volume of data that needs to be stored. For Vector Stores, the cost is relatively high because the entire dataset must be kept in memory for fast retrieval, leading to significant storage expenses. At $0.02 per GB per month, storing 1 TB of data incurs a cost of $240 annually.
- GraphRAG with Knowledge Graph: GraphRAG systems typically require less immediate data storage in high- speed memory due to the efficient querying of structured data. This leads to reduced storage costs of $0.01 per GB per month, amounting to $120 annually for 1 TB of data.
Compute Costs
- Vector Stores for RAG: High compute costs are associated with Vector Stores due to the necessity of extensive parallel processing to manage large-scale data and ensure rapid query responses. This results in $0.05 per vCPU hour, totaling $43,800 annually for the required compute resources. Furthermore, there are potentially very high costs for the initial (and ongoing) generation of Embeddings.
- GraphRAG with Knowledge Graph: GraphRAG systems benefit from more efficient data retrieval mechanisms inherent to graph databases, which reduces the computational load. With compute costs at $0.03 per vCPU hour, the annual expenditure amounts to $26,280, reflecting the efficiency of graph-based queries.
Memory Costs
- Vector Stores for RAG: Vector Stores require significant memory to store and process large datasets in-memory, ensuring quick data retrieval. This leads to substantial memory costs of $0.01 per GB hour, resulting in $87,600 annually for 1 TB of data stored continuously.
- GraphRAG with Knowledge Graph: GraphRAG systems typically leverage optimized data storage techniques, reducing the need for extensive in-memory data, thereby lowering memory costs. With a rate of $0.005 per GB hour, the annual cost for 1 TB of data is $43,800, demonstrating the efficiency of knowledge graph structures.
Data Processing Costs
- Vector Stores for RAG: Data processing costs for Vector Stores are higher due to the complexity of handling and retrieving data from dense vector representations. At $0.001 per query for 10,000 queries per day, the annual cost amounts to $3,650.
- GraphRAG with Knowledge Graph: GraphRAG systems, leveraging structured data in knowledge graphs, enable more efficient data processing. This results in lower costs of $0.0005 per query, translating to $1,825 annually for the same query volume, showcasing the cost efficiency of graph-based systems.
Implementation Costs
- Vector Stores for RAG: The implementation costs for Vector Stores include the initial setup, integration of vector databases, and deployment of RAG systems, which tend to be high due to the complexity of these systems. This one-time cost is estimated at $50,000.
- GraphRAG with Knowledge Graph: GraphRAG implementation costs are slightly lower due to the maturity and integration capabilities of graph databases with existing systems. The initial setup and integration cost is estimated at $40,000, reflecting lower complexity and streamlined deployment processes.
Maintenance Costs
- Vector Stores for RAG: Annual maintenance for Vector Stores includes ongoing support, updates, and optimizations, which are costly due to the system’s complexity and the need for high availability and performance. This is estimated at $20,000 per year. Additional maintenance costs needed to regenerate vector embeddings are unknown and impossible to predict. Deploying a solution with unknown costs is not advised.
- GraphRAG with Knowledge Graph: GraphRAG systems generally require less intensive maintenance due to the robust nature of graph databases and efficient query handling. Annual maintenance costs are estimated at $15,000, indicating lower ongoing support requirements. Because there are no steps or costs related to maintaining vector embeddings, the cost profile of GraphRAG with Knowledge Graphs is far simpler and 100% predictable.
Conclusion
ArangoDB RAG with Knowledge Graphs offers superior completeness and accuracy of responses, cost efficiency, and scalability compared to RAG systems relying on vector databases. By leveraging detailed, interconnected data and efficient graph traversal, ArangoDB provides a more viable solution for large-scale, data-intensive applications. This makes it very valuable for use cases like Customer 360 and Fraud Detection, where very complete and contextual data retrieval is crucial.
The multi-model capabilities of ArangoDB, including support for JSON documents, integrated full-text search, graph, and geospatial data, further differentiate its effectiveness, making it a powerful and versatile platform for modern data and analytics applications.
And probably most importantly, because GraphRAG with Knowledge Graphs does not need any creation or maintenance of embeddings, the simplicity of the overall system is far greater, and the cost profile far lower and more predictable.
