Building RAG on Panorama: A Vision for Intelligent Search with Astra DB Vector
Building RAG on Panorama: A Vision for Intelligent Search with Astra DB Vector
After successfully migrating from DSE to Astra DB, we find ourselves sitting on an untapped capability: Astra DB Vector. With vector search already enabled on our cluster, the natural next question as a Principal Engineer is: how do we evolve this data platform into an intelligent platform?
This post outlines my technical vision and roadmap for integrating Retrieval-Augmented Generation into the Panorama platform - the use cases that justify it, the architecture that enables it, and the path to get there.
Why RAG on an Enterprise Data Platform?
Traditional search on data platforms relies on exact keyword matching - querying table names, column names, or pre-defined tags. This breaks down when:
- Engineers ask questions in natural language: "Which tables feed the daily risk report?"
- Incident responders need context: "Have we seen this timeout pattern before?"
- New team members need onboarding: "How does the settlement flow work?"
RAG bridges this gap by combining semantic search (finding relevant context via vector similarity) with LLM generation (synthesising a coherent answer from retrieved documents). As the platform matures, this is the logical next evolution.
The Key Advantage: Zero New Infrastructure
Unlike many RAG implementations that require standing up a separate vector database (Pinecone, Weaviate, Milvus), this architecture would leverage what we already have:
| Component | Already In Place | Role in RAG |
|---|---|---|
| Astra DB | ✅ Post-migration | Vector store + metadata store |
| Databricks/Spark | ✅ Data processing | Embedding generation pipeline |
| Splunk | ✅ Observability | Source documents (incidents, logs) |
| Azure Private Link | ✅ Security | Secure LLM API connectivity |
Proposed Architecture
Ingestion Pipeline
The proposed Spark-based ingestion pipeline would handle document processing:
Source Documents → Chunking → Embedding → Astra DB Vector Table
Sources to ingest:
- Platform runbooks and operational documentation (Confluence export)
- Incident history and post-mortems (Splunk + ServiceNow)
- Data catalog metadata (table schemas, column descriptions, lineage)
- Migration documentation and architecture decision records
Chunking strategy:
- Structured data (schemas, configs): preserve as atomic units
- Long-form docs: recursive splitting at 512 tokens with 50-token overlap
- Incident reports: split by section (summary, timeline, root cause, remediation)
Embedding Model
The plan is to use a self-hosted embedding model within our Azure tenant to meet security requirements:
- Model:
text-embedding-3-small(OpenAI via Azure OpenAI Service) - Dimensions: 1536
- Connectivity: Azure Private Link (no public internet exposure)
- Batch processing: Spark UDF for bulk embedding generation
Astra DB Vector Table Schema
The proposed schema leverages SAI for vector indexing:
CREATE TABLE panorama_rag.knowledge_base (
doc_id UUID,
chunk_id INT,
content TEXT,
embedding VECTOR
CREATE CUSTOM INDEX ON panorama_rag.knowledge_base(embedding) USING 'StorageAttachedIndex';
The Storage Attached Index (SAI) on the vector column would enable ANN (Approximate Nearest Neighbour) search directly within Astra DB - no external index required.Retrieval & Generation
Simplified retrieval flow
query_embedding = embed(user_question)Vector search in Astra DB
results = session.execute(""" SELECT content, source_type, source_ref, similarity_cosine(embedding, ?) AS score FROM panorama_rag.knowledge_base ORDER BY embedding ANN OF ? LIMIT 5 """, [query_embedding, query_embedding])Construct prompt with retrieved context
context = "\n---\n".join([r.content for r in results]) prompt = f"""Based on the following context, answer the question. Context: {context} Question: {user_question}"""answer = llm.generate(prompt)
Envisioned Use Cases
1. Platform Operations Assistant
Problem: Engineers spend 15-30 minutes searching runbooks and Slack history during incidents.
Proposed solution: A chat interface that retrieves relevant runbook sections and past incident resolutions.
- Most common cause: compaction backlog after batch migration job
- Immediate action: Check pending compactions via nodetool
- Resolution: Throttle migration Spark job during peak hours
- Runbook: [Settlement Cluster Operations §4.2]"
2. Semantic Data Discovery
Problem: Finding relevant tables requires knowing exact names or navigating a complex catalog UI.
Proposed solution: Natural language search over schema metadata and column descriptions.
- risk_engine.customer_scores (similarity: 0.92)
- settlement.risk_parameters (similarity: 0.87)
- reporting.daily_risk_aggregates (similarity: 0.84)
3. Onboarding Accelerator
Problem: New engineers take 4-6 weeks to understand platform architecture and data flows.
Proposed solution: RAG over architecture decision records, design docs, and migration history.
4. Alert Context Enrichment
Problem: Splunk alerts fire with technical metrics but lack business context.
Proposed solution: When an alert triggers, automatically retrieve related incidents and runbook sections, attaching context to the alert notification.
Keeping Vectors Fresh
Stale embeddings would degrade RAG quality. The proposed refresh strategy:
| Source | Refresh Frequency | Trigger |
|---|---|---|
| Schema metadata | Daily | Spark scheduled job |
| Runbooks | On change | Confluence webhook → Spark job |
| Incidents | Real-time | Splunk event → streaming ingest |
| ADRs | On merge | Git webhook → Spark job |
Security & Governance
Enterprise RAG demands strict controls. The design must address:
- Data classification: Only embed documents classified as Internal or below - no Restricted/Confidential content enters the vector store
- Access control: Astra DB role-based access limits which teams can query which source types
- PII filtering: Pre-processing pipeline strips PII before embedding (regex + NER model)
- Audit trail: All queries logged with user identity, retrieved chunks, and generated responses
- Network isolation: LLM API accessed via Azure Private Link - no data leaves the private network
Expected Outcomes & Roadmap
Based on industry benchmarks and our platform's characteristics, the projected impact:
- 70% reduction in mean time to find relevant runbook content during incidents
- 3x faster onboarding for new engineers on platform operations
- Zero additional infrastructure cost - all running on existing Astra DB + Databricks
Roadmap:
| Phase | Timeline | Scope |
|---|---|---|
| Phase 1 | Q3 2026 | Proof of concept - Ops Assistant with runbook ingestion |
| Phase 2 | Q4 2026 | Data catalog semantic search + incident history |
| Phase 3 | Q1 2027 | Alert enrichment + onboarding assistant |
| Phase 4 | Q2 2027 | Agentic patterns - LLM executes read-only platform queries |
Key Takeaways
- Activate what you have - if your database supports vectors, you don't need a separate vector DB
- Start with internal ops - lower risk, immediate value, builds confidence
- Chunking matters more than model choice - garbage in, garbage out
- Security first - enterprise RAG without access control is a data leak waiting to happen
- Keep vectors fresh - stale context is worse than no context
The migration to Astra DB wasn't just about cost reduction - it positioned us to build intelligent capabilities on top of our data platform without adding complexity. RAG is the first step in a vision to make Panorama a truly intelligent platform.