From Tickets, Topology, and Runbooks to an AutoGraph Incident Agent on Arango

Provisioning Arango and Picking Your Dataset

This section walks through the environment setup and the dataset choices that make the rest of the tutorial work. By the end, you will have your environment variables set and a clear understanding of why each data source was picked, or ruled out.

Prerequisites

You need three things before any code runs:

1. Access to the Arango Contextual Data Platform with the Agentic AI Suite enabled (AutoGraph, the Retriever, and Ada are enterprise-tier capabilities; request access via the demo page). You’ll run this on your own Arango deployment, either on the Arango Managed Platform (AMP) or on your own self-managed install (see Ways to run it). To set up a self-managed one, see Get started with the Contextual Data Platform. To use an access token instead of your password, see Access tokens.
2. Python 3.11+
3. An OpenAI API key (or any OpenAI-compatible provider of your choice)

Install the dependencies:

python3 -m venv venv_dir
source venv_dir/bin/activate
pip install -r requirements.txt

The file contains the packages that cover the Arango driver, OpenAI calls, dataset loading, environment config, REST calls, graph visualization, and results formatting; see requirements.txt for the full list.

Export your credentials as environment variables, or put them in a .env file (see .env.example in the repo for an example):

ARANGO_HOST=https://<your-deployment>.arango.ai
ARANGO_DB=incident_demo
ARANGO_USER=<user>
ARANGO_PASSWORD=<password>
OPENAI_API_KEY=<your-key>
GRAPHRAG_DB=incident_demo
GRAPHRAG_PROJECT=incidents-runbook-autograph

Test your credentials by running check_all_creds.py in your repo:

~$  python3 check_all_creds.py
[OK] Arango: connected to https://<your-deployment>.arango.ai (ArangoDB 3.12.11)
[OK] OpenAI: API key valid; text-embedding-3-small and gpt-5-mini available
All credentials look good.

Choosing Your Dataset

The tutorial uses 6StringNinja/synthetic-servicenow-incidents (MIT, 500 rows) as the default incident corpus. The reason is specific: its short_description, description, and resolution fields are real free-text, which is exactly what the vector index needs. Volume matters less than embeddable content.

The dataset is not used as-is, though. The raw data is flat tickets with generic ServiceNow categories and no service map, which is not representative of real-world service data. Three changes shape it into something more useful: a hand-authored service topology (12 services, 13 dependency edges, 5 on-call teams) is overlaid so the graph traversal has something to walk; each ticket’s generic category is remapped to a concrete service so incidents belong to real nodes in that graph; and resolution is stored but deliberately not embedded so the known fix cannot leak into retrieval. The result is a deliberate simulation spanning P1 to P3 across infrastructure and application services, mirroring the case study shape rather than generic disconnected tickets. Swap in a real ServiceNow export and the same pipeline runs unchanged.

The Runbook Corpus

The structured ticket data pairs with unstructured runbooks. The tutorial uses 11 synthetic runbooks that ship with the repo under data/runbooks/. If you want to extend the corpus, danluu/post-mortems links to around 200 real-world post-mortems from engineering blogs; it’s a list of links rather than the post-mortem bodies, so you’d need to fetch those separately (budget 10 to 20 percent dead links).

Modeling and Ingesting the Three Data Shapes

The multimodel store is the foundation that everything else builds on. Before any code runs, it helps to know exactly what you are building: document collections for incidents, alerts, and teams; a service_health collection with a current status per service; an edge collection describing which services depend on which; a named graph over those edges; and vector indexes on the embedded text fields of the incidents and alerts collections. Runbooks get a different treatment; they go into AutoGraph, which builds a knowledge graph from them rather than storing them as flat documents.

Defining the Schema

The schema is created and torn down by src/ingest.py, which is idempotent: safe to re-run. The core structure looks like this:

incidents = db.create_collection("incidents")
services = db.create_collection("services")
teams    = db.create_collection("teams")
alerts   = db.create_collection("alerts")
service_health = db.create_collection("service_health")
graph    = db.create_graph("service_topology")
graph.create_edge_definition(
  edge_collection="service_depends_on",
  from_vertex_collections=["services"],
  to_vertex_collections=["services"],
)

One named graph service_topology sits over the service_depends_on edge collection. That graph is what the AQL traversal walks at query time to compute blast radius.

Embedding Configuration

Both vector indexes, on incidents and alerts, share the same model (text-embedding-3-small, 1536 dimensions, cosine similarity). Keeping them identical matters: the query layer hops between them without translation, so a dimension mismatch between ingest and query time causes an insert failure. Pin the model once and reuse it everywhere.

Ingesting Tickets

The 500 synthetic incidents load at runtime. Each record is embedded on short_description + description; the resolution field is stored but deliberately not embedded, so the answer does not leak into retrieval. The vector index is created after the full load so it trains on the complete set:

texts = [f'{r["short_description"]} {r["description"]}' for r in ds]
vectors = embed(oai, texts)
incidents.insert_many(
    [
        {
            "_key": r["number"],
            "short_description": r["short_description"],
            "description": r["description"],
            "resolution": r["resolution"],
            "category": r["category"],
            "assignment_group": r["assignment_group"],
            "urgency": r["urgency"],
            "impact": r["impact"],
            "service": cat2svc.get(r["category"], "onboarding-api"),
            "embedding": v,
        }
        for r, v in zip(ds, vectors)
    ]
)

# Vector index is created after the data loads so it trains on the full set.
incidents.add_index(
    {
        "name": "incidents_vec",
        "type": "vector",
        "fields": ["embedding"],
        "params": {"metric": "cosine", "dimension": EMBED_DIM, "nLists": 10},
    }
)

The dataset does not ship a CMDB (a configuration management database that maps your services and their dependencies), so the service topology is hand-authored in data/topology.json: 12 services, 5 teams, 13 dependency edges. In production, you would derive edges from your real service registry.

Ingesting Alerts

Each alert is embedded and stored alongside its payload in the alerts collection; the embed() helper in src/ingest.py batches calls to stay within provider rate limits.

Service Health

A service_health collection gives the reasoning layer a real-time signal to check against: one document per service with a current status. At demo time, document-service is marked degraded, which is what triggers the root cause pivot in the sample alert. In production, this collection would be populated from your monitoring system rather than hand-authored.

Ingesting Runbooks via AutoGraph

Runbooks take a different path. Rather than flat document storage, they go into AutoGraph, which reads the corpus, extracts entities and relations, and builds a knowledge graph. AutoGraph is what separates this from a plain vector search pipeline: instead of storing runbooks as chunks to retrieve by similarity, it understands the relationships between services, symptoms, root causes, and resolution steps, and makes those relationships queryable. The full pipeline, import, corpus build, RAG strategizer, is scriptable via src/graphrag_ingest.py.

You can also confirm visually in the graph viewer: the platform’s built-in graph explorer that lets you inspect the nodes and edges AutoGraph extracted from your runbooks, so you can see the knowledge graph structure before querying it:

This is the service_topology graph rendered in Arango’s graph explorer. Each node is a service, and each edge is a dependency: onboarding-api depends on document-service, merchant-service, and notification-service, which in turn depend on databases and gateways further downstream. This is the structure the AQL traversal walks when an alert comes in: starting at the alerting service and fanning out hop by hop to find everything that could be affected. Verifying it visually before running the resolver confirms that the graph was built correctly and the dependencies are wired the way you expect.

For deeper reference: AutoGraph quickstart, vector indexes, and APPROX_NEAR_COSINE.

Resolving an Incident Across Document, Graph, and Vector

This is the section everything else has been building toward. The data layer is in place: incidents embedded and indexed, services wired into a dependency graph, runbooks extracted into a knowledge graph by AutoGraph. Now we fire a real alert through all three layers simultaneously. The goal is to show what a multimodel foundation enables that a vector-only pipeline cannot: finding the true root cause rather than just the most similar past incident. A single AQL query pulls similar incidents, walks the dependency graph with live health signals, and feeds the evidence to an LLM judge that decides the true root cause before the Retriever is called for the cited answer.

The Marquee AQL Query

The query combines three multimodel operations without leaving the database:

LET similar = (
  FOR i IN incidents
    LET sim = APPROX_NEAR_COSINE(i.embedding, @vec)
    SORT sim DESC
    LIMIT 3
    RETURN {
      number: i._key,
      short_description: i.short_description,
      resolution: i.resolution,
      service: i.service,
      similarity: sim
    }
)
LET root = @service != null ? @service : FIRST(similar).service
LET affected = (
  FOR v, e, p IN 0..3 OUTBOUND CONCAT("services/", root) GRAPH "service_topology"
    LET health = DOCUMENT("service_health", v._key)
    COLLECT service = v._key, name = v.name, status = health.status, note = health.note
      AGGREGATE depth = MIN(LENGTH(p.edges))
    SORT depth
    RETURN {service, name, depth, status, note}
)
LET team = DOCUMENT("teams", DOCUMENT("services", root).team)
RETURN {
  root_service: root,
  similar_incidents: similar,
  affected_services: affected,
  on_call: {team: team.name, contact: team.oncall}
}

Three moves in one round trip: APPROX_NEAR_COSINE over the incidents collection (vector), OUTBOUND traversal over service_topology with a health status lookup per service (graph), and DOCUMENT() lookups for the on-call team (key-value). The full query lives in src/resolver.py as MARQUEE.

You can also paste it directly into the Arango query editor to run it interactively. Either way, one database call gets you back everything you need: similar incidents, blast radius, and who to page. No stitching together results from separate systems.

The Cited Answer

The structured payload tells you what happened. The agent then calls the Retriever to get a cited, runbook-grounded resolution step. The resolver sends the payload to the Retriever using Unified Search (query_type: 3), a retrieval mode that searches across the full knowledge graph and returns a cited, grounded answer.

response = requests.post(
    f"{retriever_url(jwt=jwt)}/graphrag-query",
    headers={"Authorization": f"Bearer {jwt}", "Content-Type": "application/json"},
    json={"query": query, "query_type": 3,
          "show_citations": True, "include_metadata": True, "use_cache": False},
    timeout=180,
)

The agent has already pivoted the root cause to document-service before this call is made. Here is a trimmed excerpt of what comes back (the full response is captured in incident_resolution.ipynb):

## Resolution Steps
1. **Stale Permission Cache**: Invalidate the merchant's permission cache entry and retry the e-sign step. The cache TTL is 15 minutes but manual invalidation resolves it immediately [1].
2. **Document-Service Degraded**: Check for PDF render worker saturation and scale the render workers if necessary [2][3].
## First On-Call Step
Page the Onboarding On-Call team, scale document-service, and follow the document-service runbook [1][2].
## Verification
- A test merchant should be able to complete the e-sign step end to end.
- `onboarding_documents_failed_total` returns to baseline and document-service p99 latency back under 500ms [1][2].
[1] (primary) document-service-degraded.md
[2] default_0_a:17478240291931595361_1

The full response continues with a first diagnostic step and a verification checklist, all cited back to the same runbook. The corroboration check separately confirms this agrees with the graph traversal: {'agree': True, 'matched_services': ['document-service', 'merchant-db', 'merchant-service', 'network-gateway', 'notification-service', 'onboarding-api', 'payment-gateway']}

Firing a Real Alert

Run the resolver against the sample alert:

python src/resolver.py data/alert.sample.json

The alert payload:

{
  "service": "onboarding-api",
  "severity": "P1",
  "region": "us-east-1",
  "text": "Merchants unable to e-sign the Merchant Agreement; onboarding stuck at the document step with a permission error in the UI."
}

The structured AQL response arrives in a median of 149ms. The LLM judge adds roughly 2 to 5 seconds on top, and the cited Retriever answer adds another 4 to 8 seconds. Three separate steps, three honest numbers.

Output:

{
  "structured": {
    "root_service": "document-service",
    "similar_incidents": [
      {
        "number": "INC00311",
        "short_description": "Merchant unable to e-sign agreement",
        "resolution": "The network team identified a routing issue causing the intermittent connections and resolved it by optimizing the network configuration to ensure stable and uninterrupted service.",
        "service": "auth-service",
        "similarity": 0.8412749767303467
      },
      {
        "number": "INC00469",
        "short_description": "Access Rights Restriction",
        "resolution": "Development team identified a bug in the service configuration module that was blocking the setup. A code fix was deployed, allowing Sales Agent to configure services for the additional location.",
        "service": "onboarding-api",
        "similarity": 0.8315266966819763
      },
      {
        "number": "INC00354",
        "short_description": "Equipment selection dropdown missing options",
        "resolution": "User permissions for the Sales Agent were reviewed and updated to ensure proper access to merchant data. The system cache was cleared to reflect the correct merchant ID for the new location.",
        "service": "onboarding-api",
        "similarity": 0.80251145362854
      }
    ],
    "affected_services": [ … ], // 8 services from onboarding-api outward, depth 0–2; document-service marked degraded
    "on_call": { … }, // Onboarding On-Call, onboarding-oncall@example.com
    "reasoning": "The onboarding-api is healthy but a depth-1 dependency, document-service, is degraded with 'high latency on /generate-envelope', which directly matches e-signature/document generation failures and permission errors in the UI. This degraded dependency plausibly and directly explains the symptom.",
    "confidence": "high",
    "method": "llm-judge",
    "verify": "kubectl top pods -l app=document-service && kubectl logs -l app=document-service --tail=50"
  }
}

Look closely at the top vector match. INC00311 scores highest on similarity (0.841), but notice its service field: auth-service, not onboarding-api. Its resolution was a network routing fix, nothing to do with the e-sign permission error in this alert. Vector search matched on the words in the description (“unable to e-sign,” “connections,” “configuration”) without understanding which service actually broke. If the resolver relied on vector search alone, it would have handed an engineer the wrong playbook and paged the wrong team.

This is where the reasoning layer steps in. The AQL query has already gathered the evidence: the alert symptom, the top similar past incidents, and the health status of every service in the blast radius. The reason() function packages all of that into a single OpenAI call. The model reads the evidence and decides: which service in the blast radius best explains the symptom? In this case, it sees that onboarding-api is healthy but document-service at depth 1 is degraded with “high latency on /generate-envelope”: a signal that directly matches an e-sign failure. It pivots the root cause, updates the on-call routing, and writes its own reasoning sentence. In this demo the pivoted root happens to map to the same team, since document-service is also owned by the Onboarding On-Call team, but the code re-points it regardless.

The second and third similar incidents (INC00469, INC00354) both belong to onboarding-api and describe the right symptom; INC00311 is the noise a vector-only resolver would have acted on. A hardcoded rule would have pivoted to whichever degraded service appeared first in the list. The model reads the symptom and picks the one that actually fits.

Here is a trimmed excerpt of the core of the reason() function in src/resolver.py, the evidence packet and the OpenAI call:

def reason(payload, alert_text=None, db=None):
    affected = payload["affected_services"]

    # Package the evidence the AQL query already gathered
    evidence = {
        "alert": {
            "service": payload["root_service"],
            "symptom": alert_text or payload["similar_incidents"][0].get("short_description", ""),
        },
        "similar_incidents": [
            {k: inc[k] for k in ("number", "short_description", "service", "similarity")}
            for inc in payload.get("similar_incidents", [])
        ],
        "affected_services": affected,  # includes status and note per service
    }

    # One LLM call judges the evidence and returns root_service, confidence, reasoning
    response = OpenAI().chat.completions.create(
        model=REASONING_MODEL,
        reasoning_effort="minimal",
        messages=[
            {"role": "system", "content": REASONING_PROMPT},
            {"role": "user", "content": json.dumps(evidence)},
        ],
    )
    decision = json.loads(response.choices[0].message.content)
    # → {"root_service": "document-service", "confidence": "high", "reasoning": "..."}

To see why the LLM judge matters, consider a scenario with two degraded dependencies: document-service showing high latency on /generate-envelope and notification-service showing an email delivery queue backlog. A hardcoded rule would pivot to whichever degraded service appeared first in the list and report high confidence. The wrong answer at full confidence. The judge reads the symptom (“merchants unable to e-sign”) and picks document-service because the envelope generation failure directly explains an e-sign error, while an email backlog does not. That is the moment the model earns its seat: not just finding that something is degraded, but understanding which degradation fits the symptom.

The corroboration check is an independent signal: it verifies that the cited runbooks reference the root service or an affected service, confirming that the two surfaces (AQL and Retriever) converged on the same incident. Across all 8 demo alerts the result is 8/8 grounded on the correct runbook, 8/8 corroborated, at a median AQL query time of 149ms.

For deeper reference on the AQL side: AQL graph traversals, AQL vector functions, and Vector Search in Arango.

Ask Ada in Plain English

If you would rather not write AQL by hand, ask Ada, Arango’s AI assistant. You describe what you want in plain English, and Ada, scoped to your incident_demo database and running on the LLM you configure, generates the query and runs it, returning rows or a chart, not a draft you still have to execute. It will not replace the audited resolver, but for ad-hoc questions mid-incident, it beats opening the query editor.

Before asking anything that requires a join, prime Ada with a schema question first: “Show me a sample document from the services and teams collections.” This matters because the display name (“Web Portal”) and the _key used in joins (“web-portal”) are different fields, and the team relationship is a foreign key. Priming first means Ada’s later queries traverse the schema correctly instead of guessing.

With the schema primed, ask the questions an on-call engineer would actually type. First, the graph traversal:

“If the Onboarding API goes down, what else is affected?”

Ada lists its plan (find the service, query the dependency edges), runs it, and on request renders a dependency visualization as a generated artifact.

Then the vector search:

“Which past incidents look like this onboarding-api outage, and how were they fixed?”

Ada finds the closest matching incidents and their resolutions without you writing a single line of AQL.

Your engineers query the same contextual layer in plain English. No AQL required.