Added - Persona-Driven Document Intelligence for Offline PDF Analysis

Domains/AI-ML/MiniProjects/PDF Intelligence/.dockerignore

@@ -0,0 +1,10 @@
+venv/
+__pycache__/
+local_model/
+nltk_data/
+test_pdfs/*.pdf
+output/*.json
+!test_pdfs/SouthofFranceCities.pdf
+!test_pdfs/SouthofFranceCuisine.pdf
+!test_pdfs/SouthofFranceHistory.pdf
+!test_pdfs/SouthofFranceRestaurantsandHotels.pdf

Domains/AI-ML/MiniProjects/PDF Intelligence/.gitignore

@@ -0,0 +1,40 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual Environment
+venv/
+ENV/
+
+# IDE
+.vscode/
+.idea/
+
+# --- UPDATED PROJECT-SPECIFIC RULES ---
+# Ignore all PDF and JSON files by default
+output/*.json
+test_pdfs/*.pdf
+
+# But DO NOT ignore these specific sample PDFs needed for the Docker demo
+!test_pdfs/SouthofFranceCities.pdf
+!test_pdfs/SouthofFranceCuisine.pdf
+!test_pdfs/SouthofFranceHistory.pdf
+!test_pdfs/SouthofFranceRestaurantsandHotels.pdf

Domains/AI-ML/MiniProjects/PDF Intelligence/Dockerfile

@@ -0,0 +1,52 @@
+# ---- Stage 1: Build & Data Preparation ----
+# Use a slim Python image to build dependencies and download data.
+# Note: This used python:3.10-slim, not debian:bullseye-slim
+FROM python:3.10-slim as builder
+
+# Set the working directory
+WORKDIR /app
+
+# Create and activate a venv
+ENV VIRTUAL_ENV=/app/venv
+RUN python -m venv $VIRTUAL_ENV
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
+# Copy only requirements for caching
+COPY requirements.txt .
+
+# Install CPU-only PyTorch and other deps. Added --timeout 1000 here to help with stability.
+RUN pip install --timeout 1000 --no-cache-dir torch --index-url https://download.pytorch.org/whl/cpu && \
+    pip install --timeout 1000 --no-cache-dir -r requirements.txt
+
+# Download the models and data into a dedicated folder.
+RUN mkdir -p /app/model_data/local_model && \
+    python3 -c "from sentence_transformers import SentenceTransformer; model = SentenceTransformer('all-MiniLM-L6-v2'); model.save('/app/model_data/local_model')"
+
+RUN mkdir -p /app/model_data/nltk_data && \
+    python3 -c "import nltk; nltk.download(['punkt', 'punkt_tab'], download_dir='/app/model_data/nltk_data')"
+
+
+# ---- Stage 2: Final Production Image ----
+# Start from a fresh, clean base image for the final submission.
+# Note: This used python:3.10-slim, not gcr.io/distroless/python3-debian11
+FROM python:3.10-slim
+
+# Set the working directory
+WORKDIR /app
+
+# Copy the virtual environment with all the installed packages from the builder stage.
+COPY --from=builder /app/venv /app/venv
+
+# Copy ONLY the prepared model and NLTK data from the builder stage.
+COPY --from=builder /app/model_data/local_model/ /app/local_model/
+COPY --from=builder /app/model_data/nltk_data/ /app/nltk_data/
+
+# Copy the rest of your application source code.
+COPY . .
+
+# Set the environment variables needed for your application to run.
+ENV PATH="/app/venv/bin:$PATH"
+ENV NLTK_DATA=/app/nltk_data
+
+# Define the default command to run your application.
+CMD ["python", "main.py", "--pdfs", "test_pdfs/SouthofFranceCities.pdf", "test_pdfs/SouthofFranceCuisine.pdf", "test_pdfs/SouthofFranceHistory.pdf", "test_pdfs/SouthofFranceRestaurantsandHotels.pdf",  "--persona", "travel_planner", "--job", "France Travel", "--output", "output/final_test.json"]

Domains/AI-ML/MiniProjects/PDF Intelligence/README.md

@@ -0,0 +1,87 @@
+# PDF Section Extraction
+
+**Contributor:** sujal-pawar
+
+**Persona-Driven Document Intelligence for Offline PDF Analysis**
+
+Extract and rank relevant sections from PDFs based on user persona and job-to-be-done using semantic search and NLP.
+
+**Note:** Local models are NOT included in the repository. The Docker build process downloads required models (~120MB). For manual setup, run `pip install -r requirements.txt` and download models as specified in the setup section.
+
+## Features
+
+- Persona-driven section extraction and relevance ranking
+- Semantic search using sentence-transformers
+- Granular subsection analysis with relevance scoring
+- 100% offline operation (no API calls)
+- Dockerized for reproducible deployment
+- Standardized JSON output format
+
+## Tech Stack
+
+Python, PyMuPDF, sentence-transformers, NLTK, PyTorch, Docker
+
+## Hackathon Build & Run
+
+```bash
+# Build Docker image (amd64 platform)
+docker build --platform linux/amd64 -t ps1b-submission .
+
+# Run container (offline mode)
+docker run --rm \
+    --platform linux/amd64 \
+    --network none \
+    -v "$(pwd)/output:/app/output" \
+    ps1b-submission
+```
+
+Output JSON will be generated in `output/final_test.json`
+
+## Performance
+
+Meets Adobe Round 1B constraints for CPU-only execution:
+- Model size: ~120MB (within 1GB limit)
+- Processing time: 3-5 PDFs in under 60 seconds
+  - 2 PDFs: ~19s ✅
+  - 3 PDFs: ~20s ✅
+  - 5 PDFs: ~21s ✅
+
+## Manual Setup
+
+```bash
+# Clone and setup
+git clone https://github.com/yourusername/pdf_section_extraction.git
+cd pdf_section_extraction
+python -m venv venv
+source venv/bin/activate  # On Windows: .\venv\Scripts\activate
+
+# Install dependencies and download models
+pip install -r requirements.txt
+python -c "import nltk; nltk.download(['punkt', 'punkt_tab'], download_dir='./nltk_data')"
+python -c "from sentence_transformers import SentenceTransformer; model = SentenceTransformer('all-MiniLM-L6-v2'); model.save('./local_model')"
+
+# Run
+python main.py --pdfs test_pdfs/*.pdf --persona "travel planner" --job "France Travel" --output output/final_test.json
+```
+
+## Project Structure
+
+```
+├── Dockerfile              # Container configuration
+├── main.py                 # Main execution script
+├── parser.py               # PDF parsing and section extraction
+├── ranker.py               # Relevance ranking
+├── refiner.py              # Subsection analysis
+├── embedder.py             # Text embedding generation
+├── test_pdfs/              # Input PDFs
+├── output/                 # Generated JSON results
+└── requirements.txt        # Dependencies
+```
+
+## License
+
+MIT License
+
+***
+
+Built with sentence-transformers, NLTK, PyMuPDF, and PyTorch

Domains/AI-ML/MiniProjects/PDF Intelligence/embedder.py

@@ -0,0 +1,13 @@
+# embedder.py
+from sentence_transformers import SentenceTransformer
+import numpy as np
+
+class Embedder:
+    def __init__(self, model_name="local_model"):
+        self.model = SentenceTransformer(model_name, device='cpu')
+
+    def embed(self, texts):
+        """
+        Batch-encode list of texts into embeddings array.
+        """
+        return np.array(self.model.encode(texts, convert_to_numpy=True, show_progress_bar=False))

Domains/AI-ML/MiniProjects/PDF Intelligence/main.py

@@ -0,0 +1,96 @@
+import argparse
+import json
+import time
+import os  # --- 1. Import the 'os' module ---
+from datetime import datetime
+from parser import extract_sections
+from embedder import Embedder
+from ranker import rank_sections
+from refiner import refine_subsections
+
+def convert_rank_to_relevance_score(rank, max_rank=5):
+    """Convert rank (1=best) to relevance score (1.0=best)"""
+    return round(1.0 - ((rank - 1) / max_rank), 2)
+
+def main(pdf_list, persona, job, output_json):
+    start = time.time()
+
+    # --- 2. Add code to ensure the output directory exists ---
+    output_dir = os.path.dirname(output_json)
+    if output_dir:
+        os.makedirs(output_dir, exist_ok=True)
+    # --- End of new code ---
+
+    embedder = Embedder()
+    prompt = f"{persona} {job}"
+    prompt_emb = embedder.embed([prompt])[0]
+
+    all_sections = []
+    for pdf in pdf_list:
+        secs = extract_sections(pdf)
+        all_sections.extend(secs)
+
+    extracted_sections = []
+    subsection_analysis = []
+
+    top_k_per_pdf = 5
+
+    for pdf in pdf_list:
+        try:
+            pdf_sections = [sec for sec in all_sections if sec["document"] == pdf]
+            if not pdf_sections:
+                continue
+
+            ranked_secs = rank_sections(pdf_sections, prompt_emb, embedder, top_k=top_k_per_pdf)
+
+            for idx, sec in enumerate(ranked_secs, start=1):
+                extracted_sections.append({
+                    "document": sec["document"],
+                    "page": sec["page"],
+                    "section_title": sec["title"],
+                    "importance_rank": idx
+                })
+
+                subs = refine_subsections(sec, prompt_emb, embedder)
+
+                for sub in subs:
+                    subsection_analysis.append({
+                        "document": sec["document"],
+                        "page": sec["page"],
+                        "refined_text": sub["refined_text"],
+                        "relevance_score": convert_rank_to_relevance_score(sub["rank"])
+                    })
+        except Exception as e:
+            safe_pdf_path = pdf.encode('utf-8', 'ignore').decode('utf-8')
+            print(f"⚠️ [WARNING] Failed to process document '{safe_pdf_path}'. Error: {e}. Skipping this document.")
+            continue
+
+    metadata = {
+        "input_documents": pdf_list,
+        "persona": persona,
+        "job_to_be_done": job,
+        "processing_timestamp": datetime.now().isoformat() + "Z"
+    }
+
+    output = {
+        "metadata": metadata,
+        "extracted_sections": extracted_sections,
+        "subsection_analysis": subsection_analysis
+    }
+
+    with open(output_json, "w", encoding="utf-8") as f:
+        json.dump(output, f, indent=2, ensure_ascii=False)
+
+    print(f"[INFO] Processing completed in {time.time() - start:.2f}s")
+    print(f"[INFO] Output saved to {output_json}")
+    print(f"[INFO] Extracted {len(extracted_sections)} sections and {len(subsection_analysis)} subsections")
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="PDF Section Relevance Extraction")
+    parser.add_argument("--pdfs", nargs="+", required=True, help="List of PDF files")
+    parser.add_argument("--persona", type=str, required=True, help="User persona description")
+    parser.add_argument("--job", type=str, required=True, help="Job to be done")
+    parser.add_argument("--output", type=str, required=True, help="Output JSON path")
+    args = parser.parse_args()
+
+    main(args.pdfs, args.persona, args.job, args.output)