👋 @MatiPinto03 — context for OpenRegEval, your regulatory-label evaluation dashboard.
1. Basic context
Drug labels (FDA SPL / DailyMed, EMA SmPC) are the authoritative source of indications, dosing, warnings, and contraindications. OpenRegEval is an interactive comparator that scores outputs/variants against label sections side-by-side — useful for evaluating AI-generated summaries against ground-truth regulatory text.
2. Key challenges in this field
- Structured parsing of semi-structured SPL/XML label sections.
- Ground-truth & hallucination — verifying claims against the exact label section; catching fabricated dosing/warnings.
- Section alignment — labels differ in structure across drugs and agencies.
- Scoring rubric design — turning "is this faithful?" into reproducible metrics.
3. Research value — why this matters
Regulatory text is high-stakes: an incorrect contraindication or dose can harm patients. Reliable, auditable evaluation of AI outputs against labels is foundational to trustworthy pharma AI.
4. What you can deliver
A Streamlit comparator with a model/variant selector, label-section filters, and side-by-side scorecards (faithfulness, completeness, coverage) with the supporting label snippet shown for each judgment.
5. Functions / scripts that will be popular
parse_spl(label_xml) → structured sections; fetch_label(drug) from DailyMed/openFDA.- A faithfulness scorer (claim ↔ source-snippet matching) and a diff/highlight renderer.
6. Data that might be helpful
openFDA Drugs@FDA & label API, DailyMed SPL, EMA SmPC, FDA Orange Book; a small hand-labeled gold set for calibration.
7. How to use AI to self-learn and go deeper
- Ask AI to explain label section semantics (e.g. boxed warning vs. warnings/precautions).
- Have it draft evaluation rubrics, then red-team them by generating tricky near-miss cases.
- Use it as a second grader and compare against your scorer to find disagreements.
Reply here with your own answers to these prompts — especially #4 (what you'll deliver) and #7 (how you're using AI to learn). Treat this as your project's living design doc.
👋 @MatiPinto03 — context for OpenRegEval, your regulatory-label evaluation dashboard.
1. Basic context
Drug labels (FDA SPL / DailyMed, EMA SmPC) are the authoritative source of indications, dosing, warnings, and contraindications. OpenRegEval is an interactive comparator that scores outputs/variants against label sections side-by-side — useful for evaluating AI-generated summaries against ground-truth regulatory text.
2. Key challenges in this field
3. Research value — why this matters
Regulatory text is high-stakes: an incorrect contraindication or dose can harm patients. Reliable, auditable evaluation of AI outputs against labels is foundational to trustworthy pharma AI.
4. What you can deliver
A Streamlit comparator with a model/variant selector, label-section filters, and side-by-side scorecards (faithfulness, completeness, coverage) with the supporting label snippet shown for each judgment.
5. Functions / scripts that will be popular
parse_spl(label_xml)→ structured sections;fetch_label(drug)from DailyMed/openFDA.6. Data that might be helpful
openFDA Drugs@FDA & label API, DailyMed SPL, EMA SmPC, FDA Orange Book; a small hand-labeled gold set for calibration.
7. How to use AI to self-learn and go deeper
Reply here with your own answers to these prompts — especially #4 (what you'll deliver) and #7 (how you're using AI to learn). Treat this as your project's living design doc.