RAG Runs

Retrieval pipelines are production systems.

Production systems require observability.

Most AI platforms hide ingestion and retrieval behind a simple:

Upload Document
      ↓
Knowledge Base Updated

When something goes wrong, teams are left asking:

• Was the file processed?
• Were embeddings generated?
• Was the vector database updated?
• Which embedding model was used?
• Why isn’t the agent retrieving the document?

Karyam solves this with RAG Runs.

---

What Are RAG Runs?

RAG Runs provide a complete execution history of retrieval ingestion pipelines.

Every ingestion event creates a new RAG Run.

Examples include:

• File uploads
• Embedding generation
• Vector storage operations
• Parsing failures
• Retry operations

This makes retrieval pipelines observable and debuggable.

---

Why RAG Runs Matter

Without observability:

Upload File
      ↓
???
      ↓
Agent Doesn't Find Document

With RAG Runs:

Upload File
      ↓
Parsing
      ↓
Chunking
      ↓
Embedding Generation
      ↓
Vector Storage
      ↓
Success

Every step becomes visible.

---

The RAG Pipeline

A typical ingestion pipeline looks like this:

File Uploaded
      ↓
Embedding Listener Triggered
      ↓
Document Parsing
      ↓
Chunk Generation
      ↓
Embedding Generation
      ↓
Vector Storage
      ↓
RAG Run Completed

Each stage is recorded independently.

---

Example Run

Example:

Run ID:
rag_01HXYZABC123

Type:
Ingestion

Status:
Completed

Duration:
3.4 seconds

---

Example Execution Timeline

10:01:03 File Uploaded
10:01:04 Listener Triggered
10:01:04 Parsing Started
10:01:05 32 Chunks Generated
10:01:06 32 Embeddings Created
10:01:06 Stored In Qdrant
10:01:06 Run Completed

This provides complete visibility into processing behavior.

---

Information Captured

A RAG Run records:

Source Information

• File Path
• Workspace
• Upload Location
• File Type

---

Processing Information

• Parsing Status
• Chunk Count
• Chunk Size
• Processing Duration

---

Embedding Information

• Embedding Model
• Vector Dimensions
• Number of Embeddings Generated
• Embedding Provider

---

Storage Information

• Vector Database Provider
• Collection or Storage Name
• Number of Stored Vectors

---

Error Information

• Parsing failures
• Embedding failures
• Storage failures
• Retry attempts

---

Run Statuses

RAG Runs move through several states.

Status	Description
Pending	Waiting for processing
Running	Pipeline currently executing
Completed	Processing completed successfully
Failed	Processing failed
Retrying	Automatic retry in progress

---

Example Failure

Example:

vpn-policy.pdf
      ↓
Parsing Completed
      ↓
Chunking Completed
      ↓
Embedding Failed

Failure reason:

Embedding model unavailable.

Without RAG Runs this issue would be extremely difficult to diagnose.

---

Retry Support

Certain failures can be retried automatically.

Examples:

• Temporary provider failures
• Network interruptions
• Vector database unavailability

This improves ingestion reliability.

---

Debugging Retrieval Problems

When an agent fails to retrieve information, RAG Runs should be the first place to investigate.

Typical questions include:

Was the file processed?

Check:

Status = Completed

---

Were embeddings generated?

Check:

Embeddings Created = 32

---

Were vectors stored?

Check:

Vector Storage = Success

---

Which embedding model was used?

Check:

Embedding Model:
gemini-embedding-2

---

Was the correct listener triggered?

Check:

Listener:
IT Knowledge Listener

---

Example Investigation

Employee asks:

Can I renew my VPN access?

Agent responds:

I cannot find information regarding VPN renewals.

Investigation:

RAG Run
↓
Embedding Failed
↓
OpenAI API Key Invalid

Issue identified immediately.

---

Production Visibility

RAG Runs expose:

• Processing duration
• Throughput
• Failure rates
• Retry counts
• Provider behavior

This makes retrieval infrastructure observable in production environments.

---

Relationship With Other Logs

Karyam provides multiple layers of observability:

Agent Logs
    ↓
Workflow Logs
    ↓
RAG Runs
    ↓
Audit Logs

Together they provide complete visibility into AI execution.

---

Example Production Scenario

New VPN Policy Uploaded
        ↓
Embedding Listener Triggered
        ↓
RAG Run Created
        ↓
35 Embeddings Generated
        ↓
Stored In PGVector
        ↓
Agent Can Retrieve Policy

No redeployment.

No retraining.

No manual synchronization.

---

The Karyam Philosophy

Enterprise AI systems require more than retrieval.

They require retrieval that is:

• Observable
• Reliable
• Traceable
• Governed
• Production-ready

RAG Runs provide that visibility.

---

Next Steps

You now understand the complete retrieval pipeline:

Files
    ↓
Chunking
    ↓
Embeddings
    ↓
Vector Databases
    ↓
Retrieval
    ↓
RAG Runs

These components form the foundation of enterprise AI systems in Karyam.

---