Problem
Document indexing uses the self-hosted `retrieva-ollama` sidecar (CPU-only, single slot, `bge-m3:latest`). Two compounding problems:
- Speed: ~17s per chunk on CPU. A 297-chunk PDF takes ~85 minutes to index.
- Mutual starvation: single ollama slot is shared between indexing AND RAG query embedding. One user uploading a large PDF freezes "Ask AI" for every user on the platform.
Observed in prod: uploading `Azure-SOC-2-Type-II.pdf` blocked all chat for ~85 min.
Root cause
`config/embeddingProvider.js` defaults to `EMBEDDING_PROVIDER=ollama` → self-hosted sidecar. `fileIngestionService.js` uses the same embedding provider as the RAG runtime — no separation.
Proposed fix
Option A — Separate embedding providers by purpose (recommended)
- Indexing path (`fileIngestionService`): `EMBEDDING_PROVIDER_INDEXING=openai` → `text-embedding-3-small` (~500ms/batch, already wired as fallback)
- RAG query path: keep as-is or also move to OpenAI
Option B — Move all embedding to OpenAI globally
Set `EMBEDDING_PROVIDER=openai`. Already implemented and tested.
Expected outcome
- Indexing time: 85 min → <1 min
- Multiple users can index simultaneously without blocking each other or blocking chat
- Removes single point of failure for all AI features
Problem
Document indexing uses the self-hosted `retrieva-ollama` sidecar (CPU-only, single slot, `bge-m3:latest`). Two compounding problems:
Observed in prod: uploading `Azure-SOC-2-Type-II.pdf` blocked all chat for ~85 min.
Root cause
`config/embeddingProvider.js` defaults to `EMBEDDING_PROVIDER=ollama` → self-hosted sidecar. `fileIngestionService.js` uses the same embedding provider as the RAG runtime — no separation.
Proposed fix
Option A — Separate embedding providers by purpose (recommended)
Option B — Move all embedding to OpenAI globally
Set `EMBEDDING_PROVIDER=openai`. Already implemented and tested.
Expected outcome