RAG Document QA

End-to-end RAG: chunked corpus, FAISS retrieval, FastAPI /ask with cited sources, and an embedding/index contract enforced at startup so serving cannot silently drift from the indexed model.

Interactive demo on this site posts to the public API. If the browser blocks the request, use API docs or curl — often CORS or API key on the server.

Python · FastAPI

FAISS · sentence-transformers

Docker · VPS

OpenAI / Ollama

Limitations & scope

Fixed corpus only; the system does not browse the open web.
Retrieval quality constrains answer quality — weak matches need human review.
LLM behaviour still needs inspection even when sources are returned.

Overview

Procedures, SLAs, and policies often live in PDFs and tickets. Keyword search misses paraphrases; this repo implements semantic retrieval over a fixed corpus, then an OpenAI-compatible chat model answers from retrieved passages only. Every response includes document, page, and chunk id for auditability.

The shipped demo indexes eight synthetic policy documents (billing, SLA, refunds, incident response, support workflow, onboarding, pricing, internal FAQ). Production work scopes your corpus; the patterns stay the same.

Workflow

① Ingest & chunk

② Embed & index

③ Retrieve top-k

④ Answer + sources via POST /ask

Offline indexing is separate from the online path. One-time CLI sequence: python -m src.ingestion.run_extraction → python -m src.ingestion.run_chunking → run_indexing_pipeline() in src.pipeline.indexing_pipeline. At startup the API loads the FAISS index and chunk metadata from disk and enforces the embedding model contract before serving traffic.

Demo corpus & API contract

Reviewers can trace behaviour from repo files to HTTP responses — not a black-box widget.

Corpus — eight .txt files in data/raw_docs/ (e.g. service_level_policy.txt, incident_response_manual.txt, billing_policy.txt).
Chunking — size 1200, overlap 200 (src/ingestion/chunk_documents.py); outputs artifacts/chunks/chunked_documents.csv.
Embeddings & index — sentence-transformers/all-MiniLM-L6-v2, normalized vectors, FAISS IndexFlatIP (src/embeddings/embed_chunks.py).
Request — POST /ask with {"question": "…"}.
Response — answer, sources[] with document, page, chunk_id; optional retrieved_chunks for inspection (see OpenAPI).

Example response shape

{

  "answer": "For P1 incidents, response is within 15 minutes.",

  "sources": [{

    "document": "incident_response_manual.txt",

    "page": 1,

    "chunk_id": "incident_response_manual_p1_0"

  }]

}

What it demonstrates

Grounded answers — generation uses retrieved context, not unconstrained model prior on corpus facts.
Source citations — each hit lists document, page, and chunk id.
API-first — FastAPI + OpenAPI; integrate from browsers, scripts, or internal tools.
Deployable stack — Docker image, VPS + reverse proxy, GET /health with a ready flag before trusting /ask.

Quality & ops

Retrieval eval — Hit@1, Hit@3, Hit@5 on data/eval/eval_questions.csv after the index is built.
Tests — pytest covers extraction, chunking, prompt assembly, and API contracts; slow embedding tests skippable with -m "not slow".
Production knobs — optional RAG_API_KEY, rate limiting, and RETRIEVAL_MIN_SCORE (fallback answer when retrieval is weak, chunks still returned).

Streamlit demo ships in-repo (src/demo/streamlit_app.py); this static site adds an overview plus portfolio-demo.html for browser Q&A against the live API.

Deployment / live interface

API host for this build: https://rag-qa.vahdetkaratas.com/ · OpenAPI at /docs. Static pages deploy on rag.vahdetkaratas.com (this recruiter build).

Architecture diagram and retrieval flow: reports/figures/ARCHITECTURE.md in the repository.

Why this project

It shows a full retrieve → prompt → answer path behind HTTP with explicit sources — not a notebook-only sketch. The same patterns apply wherever teams need accountable answers from an agreed document set.

Vahdettin Karataş

Location:

Technical focus

For hiring review

ML systems

Data tools