Project Roadmap

Upgrade /identify-temple to handle all Indian art. Upload image → Claude Vision identifies dynasty/style/period → searches indian_visual_gallery for matching training images → searches pallava_knowledge for relevant PDF passages → synthesized scholarly response with sources.

After pdf_ingester.py saves page PNGs: run registration script to add image entries into image_registry.json from saved pdf_pages/ folder. Links PDF scholarly text with visual images.

Upload box in gallery.html → POST /identify → results panel showing: dynasty/style card, matched training images grid, relevant PDF passages, full synthesis. Works for any uploaded photo.

Add "Pallava Sources Only" mode to Ask + Research dropdowns. ChromaDB already stores tags as a comma-separated string per chunk (field: tags, written by knowledge_store.py on every ingest). Implementation: add a where={"tags": {"$contains": "pallava"}} filter to the collection.query() call in rag/retriever.py. UI: a mode toggle on the Ask panel (index.html) and Research page. No re-ingestion needed — tags are already in ChromaDB metadata for all existing chunks.

Assign authority ratings per source. Add a weight field (1.0–5.0) to source_library.py catalog entries. Editable via a star-rating control in Library UI (library.html). At query time, rag/retriever.py multiplies each chunk's similarity score by its source weight before ranking — same pattern as the existing author_precedence.json boost. Weight defaults to 3.0 (neutral) for all existing sources. Requires the Retroactive ChromaDB patch below to propagate weights to existing chunks.

After star weights are assigned, backfill weight and tags into all existing ChromaDB chunks — no re-ingestion, no API cost. ChromaDB supports in-place metadata updates via collection.update(ids=[...], metadatas=[...]). Script: load source_library/catalog.json, group chunk IDs by fingerprint (already stored in chunk ID as know_{fingerprint}_chunk_N), call collection.update() in batches. Run once after weights are assigned; all future ingests automatically include the weight at index time.

Flag when sources disagree with a visible warning; present both views

Add Chola, Gupta, Chalukya, Hoysala, Pala galleries using the same crawl infrastructure. Dynasty is a first-class tag — gallery and /identify handle all dynasties automatically.

Rate each book 1–5 based on authority and Pallava relevance

e.g. Art & Architecture section always prefers Jouveau-Dubreuil

Backlog #6: update pallava_inscriptions to intfloat/multilingual-e5-large. Delete old collection, run py -3 vector_store.py --build. Only 9 entries — fast.

Use improved retriever with weights + tag filter for better quality

e.g. How Pallava sculptures relate to Bhagavatam stories

For any gallery image, return related passages from pallava_knowledge ChromaDB. Clicking an image in gallery shows matching scholarly text from PDFs alongside it.

Extract keyframes from uploaded video → run each frame through POST /identify → deduplicate → return annotated timeline. Same pipeline, no new Vision logic needed.

Currently video ingestion captures keyframe PNGs to corpus/video_frames/{fingerprint}/frame_NNNN.png and stores scholarly descriptions as text chunks in ChromaDB — but the research paper image system ({IMAGE:label:page} markers) only knows how to render PDF pages. Video frames never appear as images in generated papers.

What to build: A new API endpoint GET /research/video-frame?source_id=&frame=NNNN that serves PNGs from corpus/video_frames/, alongside a new marker format {VIDEOFRAME:source_label:frame_index} embedded in [KEYFRAME] chunks at ingest time. The research UI renders both marker types — PDF pages and video frames — inline in the paper.

Why Phase 3: Only meaningful once multiple videos are ingested. A single video with thin corpus adds limited value — with 5–10 videos, keyframe images significantly enrich research paper output.

Wire up the evidence_count field in corpus/pallava_pua_chart.json by scanning corpus/corpus.json inscription entries and counting how many inscriptions each Pallava character is attested in. A character with ≥3 attestations becomes "Confirmed" on the Unicode Initiative page. Currently all 56 characters show evidence_count=0 — the field exists but was never populated. Required for the Unicode submission to the Unicode Technical Committee. Run after inscription corpus is substantially complete.

Comprehensive scholarly papers from complete corpus — text + images + citations

Themed presentations with gallery images + scholarly captions from KB

Final research papers with embedded images and full references

Export generated research papers as downloadable PDFs directly from the Research UI.

Current state: Word (.docx) and PowerPoint (.pptx) export already work. docx2pdf is already in requirements.txt — it converts the generated .docx to PDF via Microsoft Word's COM interface.

Windows (dev machine): Use existing docx2pdf — calls Microsoft Word COM, requires Word to be installed. Works on Windows only.

⚠️ EC2 Linux limitation: docx2pdf will not work on Linux — there is no Microsoft Word COM interface. On EC2, replace with either:
  • LibreOffice headless — libreoffice --headless --convert-to pdf file.docx (free, good fidelity)
  • weasyprint — generates PDF directly from HTML/CSS (no Word dependency, pip installable)

Recommended approach on EC2: Generate PDF directly from the paper HTML using weasyprint — avoids the Word → PDF conversion step entirely and works cross-platform.

Defer until: Corpus is ~90%+ complete and research output is polished enough to share externally. Word export covers interim needs.

Backlog #7: add copyright_status field to source_library.py. Required before public website launch. Values: owned / licensed / public_domain / attributed_third_party / copyright_unknown.

Long-term goal per CLAUDE.md: all third-party gallery images to be replaced with researcher's own photos. Track replacement status per image in registry.

Why: Currently all videos >20 minutes are transcribed using Whisper base model (140 MB) on CPU. This is a speed compromise — base is ~3× faster on CPU than small, but produces noticeably weaker accuracy for multilingual narration (Tamil, Telugu, Sanskrit) which is common in Pallava archaeology lectures.

When: After EC2 migration. Use a g4dn.xlarge spot instance (NVIDIA T4, CUDA 12) — no need for a permanent GPU server. Spin it up for the batch, run all re-transcriptions, then terminate. Cost: ~$0.22/hr on spot (~$1–2 for a full batch of 10 videos). While stopped, you only pay ~$12/month EBS storage. On GPU, Whisper small transcribes a 1-hour video in ~15–20 min vs 2+ hours on CPU.

Code change (one line): In ingest/video_ingester.py, change:
    WHISPER_MODEL_LONG = "base" → WHISPER_MODEL_LONG = "small"

Re-ingest all videos: Go to Source Library → filter by type: video → for each video source, call POST /admin/retry/{id}. This replaces existing transcript chunks with higher-quality ones. Keyframe manifests (corpus/video_frames/{fingerprint}/manifest.json) are preserved — no Vision API cost is incurred, only Whisper re-runs.

Estimated effort: One-time batch operation. ~15–20 min per 1-hour video on GPU (vs 2+ hours on CPU). If you have 10 videos, plan for a 3–4 hour background job. No manual review needed — chunks automatically replace old ones in ChromaDB.

Write 50+ natural language questions covering all 7 dynasties, inscription types, cross-dynasty comparisons, corpus gap probes, and contradiction probes. This is 1–2 hours of manual work and must be done by you — the query agent fires these questions, so the quality of the test set determines the quality of the validation.

Query Agent fires all test questions sequentially to the History Agent via the multi-agent system. Review Agent independently evaluates each answer against corpus chunks — checking citation presence, citation accuracy, hallucination, and ideological bias. Neither agent evaluates its own output. Review Agent produces a per-question scorecard (pass / flag / fail + reason).

Round 1: fire all N questions, collect scorecards. Between rounds: review flagged/failed questions, fill corpus gaps or refine prompts. Rounds 2–3: re-run only failed questions. Exit when Review Agent passes ≥90% of questions or max 3 rounds reached — whichever comes first. Force-exit after round 3 regardless, with a gap report.

Review the consolidated validation report. If positive: loop ends, system is production-ready. If negative with specific feedback: one final manual round of corpus gap-filling or prompt adjustment, then re-run. All findings, gaps, improvements, and remaining limitations documented as the Phase 5 exit deliverable.