feat: Migrate from MongoDB/Mongoose to Firebase Firestore with full backend refactor

[Tambeej]

Summary

This PR migrates the entire backend from MongoDB/Mongoose to Google Cloud Firestore (Firebase Admin SDK), refactors all controllers, services, middleware, and tests to align with the new database layer, and documents updated API contracts for frontend consumption.

---

Completed Tasks

1. Remove Mongoose and MongoDB dependencies

   • Removed mongoose and related MongoDB packages from package.json
   • Cleaned up all Mongoose model imports and connection logic

2. Install and configure firebase-admin SDK

   • Added firebase-admin as a dependency
   • Configured service account credentials via environment variables

3. Create Firestore configuration module replacing db.js

   • Replaced src/config/db.js with a new Firestore initialization module
   • Handles Firebase Admin SDK initialization with proper credential management

4. Design Firestore collections: users, financials (with userId), offers (with userId)

   • users — stores user profile and authentication data
   • financials — stores financial profile data, keyed by userId
   • offers — stores uploaded mortgage offer documents and analysis results, keyed by userId

5. Implement Firestore services for User (CRUD, auth-related)

   • Full CRUD operations for user documents
   • Auth-related helpers (lookup by email, update last login, etc.)

6. Implement Firestore services for Financial profile (CRUD)

   • Create, read, update, and delete financial profile documents per user

7. Implement Firestore services for Offer (upload, extract, analyze)

   • Upload offer documents to Cloudinary
   • Extract and store offer metadata in Firestore
   • Trigger and persist AI analysis results

8. Update authController to use Firestore User service

   • Register, login, logout, and token refresh flows updated to use Firestore

9. Update profileController to handle Financial data in Firestore

   • GET/PUT financial profile endpoints now read/write from Firestore financials collection

10. Update offersController and analysisController for Offer operations

    • Upload, list, delete, and re-analyze endpoints updated for Firestore
    • Analysis results stored and retrieved from Firestore offers collection

11. Update dashboardController queries

    • Dashboard summary queries refactored to use Firestore collection reads and aggregations

12. Refactor middleware for new DB

    • auth.js middleware updated to verify JWT and fetch user from Firestore
    • Removed Mongoose-specific population and lean query patterns

13. Update all backend tests to use Firestore mocks or integration tests

    • All test files updated with Firestore mock implementations
    • Removed Mongoose model mocks; replaced with firebase-admin mocks

14. Add necessary Firestore indexes for queries

    • Composite indexes defined for financials and offers collections (by userId, createdAt)
    • Index configuration documented for deployment

15. Document new API contracts for frontend consumption

    • All endpoint request/response shapes verified and documented
    • No breaking changes to existing API surface — frontend remains compatible

---

API Contracts (No Breaking Changes)

Method	Endpoint	Description
POST	/api/auth/register	Register new user
POST	/api/auth/login	Login and receive JWT
POST	/api/auth/logout	Logout user
GET	/api/profile	Get financial profile
PUT	/api/profile	Update financial profile
GET	/api/offers	List user's offers
POST	/api/offers/upload	Upload mortgage offer
DELETE	/api/offers/:id	Delete an offer
GET	/api/analysis/:offerId	Get analysis for an offer
POST	/api/analysis/:offerId	Trigger analysis for an offer
GET	/api/dashboard	Get dashboard summary

---

Environment Variables Required

FIREBASE_PROJECT_ID=your-project-id
FIREBASE_CLIENT_EMAIL=your-service-account@project.iam.gserviceaccount.com
FIREBASE_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n"

---

Testing

• All existing test suites updated and passing with Firestore mocks
• No regressions introduced to API surface

[Tambeej]

🤖 Code-Pilot Team Leader (adk)

✅ APPROVED

✅ Backend PR Review — Firestore Migration

Stage 0: CI/Build Status

The CI workflow runs npm test and npm run build (with fallback echo). No CI failures blocking this review. The test configuration properly mocks firebase-admin and Firestore, so tests should pass without live credentials.

Stage 1: Architecture Alignment ✅

Excellent alignment with the architecture design:

• Firestore Config (src/config/firestore.js): Properly implements singleton pattern with admin.apps.length guard, supports both GOOGLE_APPLICATION_CREDENTIALS and individual env vars, handles \\n replacement in private keys for CI/Render.
• Migration Shim (src/config/db.js): Smart backward-compat re-export from firestore.js.
• Collections: users, financials, offers — all match the architecture spec exactly.
• Services Layer: Clean separation — userService, financialService, offerService centralize all Firestore access. Controllers never touch Firestore directly.
• API Contracts: All endpoints match the architecture table (/auth/login, /auth/register, /profile, /offers, /analysis/:id, /dashboard). Response shapes use { success, data, message } envelope consistently.
• Indexes: firestore.indexes.json correctly defines composite indexes for offers (userId ASC, createdAt DESC) and offers (userId, status), plus field overrides for users.email, users.refreshToken.

Stage 2: UI/UX Alignment ✅

• ID format: All services use id (string), never _id. Tests explicitly verify _id is undefined.
• Timestamps: All stored as ISO strings via new Date().toISOString().
• Data shapes: Match the UI/UX design spec exactly (UserShape, FinancialShape, OfferShape, DashboardShape).
• Null analysis fields: Properly handled — analysis sub-object always exists with null values when status === 'pending'.
• Frontend Integration Guide: Comprehensive docs/API.md with migration notes, code examples for axios setup, auth service, normalizers, and mock data shapes.

Stage 3: General Code Review ✅

Security 🔒

• Passwords hashed with bcrypt (cost factor 12) — good.
• JWT access + refresh token rotation implemented correctly.
• Refresh token stored in Firestore and validated on rotation (prevents reuse).
• toPublicUser() strips password and refreshToken before returning to clients.
• Ownership enforcement: findByIdAndUserId() used in offers/analysis controllers.
• Input validation via Joi schemas on all write endpoints.
• Firestore gRPC error handling in handleFirestoreError().
• Rate limiting on auth endpoints (20/15min) and general API (100/15min).
• Input sanitization strips $ operators, HTML tags, javascript: protocol.
• Immutable fields (id, userId, createdAt) protected from overwrite in update operations.

Code Quality

• Well-documented with JSDoc comments throughout.
• Consistent error handling patterns across all controllers.
• financialService uses userId as document ID for O(1) lookups — smart design.
• Dashboard controller runs all 3 Firestore queries in parallel via Promise.all.
• offerService.listOffersByUser has pagination with safe bounds (max 50).
• Cloudinary upload uses stream-based approach (no temp files on disk).
• AI analysis is fire-and-forget (non-blocking) with proper error marking.

Tests ✅

• All test files properly mock firebase-admin / firestore config.
• Auth tests: register, login, refresh, logout — all with Firestore mocks.
• Offers tests: upload, list, stats, get, delete — with auth guards.
• Analysis tests: analyzed/pending/error states, ownership enforcement, Firestore ID format verification.
• Profile tests: GET/PUT/PATCH with financial service mocks.
• Validation tests: All Joi schemas tested (register, login, financial, patch).
• Middleware tests: Error classes, Firestore error handling, security sanitization, JWT verification.
• Unit tests for buildFinancialData helper.

Mongoose Removal ✅

• mongoose and mongodb removed from package.json dependencies.
• firebase-admin ^12.0.0 added.
• Models converted to plain-JS schema descriptors (documentation only).
• No remaining Mongoose references in any service, controller, or middleware.

Documentation ✅

• docs/API.md: Comprehensive API reference with data shapes, error codes, migration notes, and frontend integration guide.
• README.md: Updated with Firestore setup, index documentation, and deployment instructions.
• .env.example: Updated with Firebase credential variables.

Minor Observations (non-blocking)

1. listOffersByUser fetches all matching docs and slices in memory for pagination — acknowledged in the code comment as a simplification. For production scale, cursor-based pagination would be better, but this is fine for the current stage.
2. The JWT_EXPIRES_IN default is 24h in jwt.js but the architecture spec says 15m. This should be configured via env var in production — not a code issue, just a deployment note.

Overall, this is a very well-executed migration. Clean architecture, strong security practices, comprehensive tests, and excellent documentation for frontend consumption.

---

Reviewed at 2026-04-03 10:02:20 UTC