Skip to content

Mini-review: how evaluating AI against regulatory text has progressed #3

Description

@ds4cabs

@MatiPinto03 — a short history of NLP/evaluation on regulatory drug labels.

TL;DR: From rule-based label NLP → biomedical transformers → LLMs (capable but hallucination-prone) → today's evaluation science (faithfulness, groundedness, LLM-as-judge). OpenRegEval is an evaluation tool in that last wave.

1. Origins

Drug labels were standardized as SPL (Structured Product Labeling) XML and surfaced via DailyMed/openFDA. Early mining turned labels into structured knowledge — e.g. SIDER side-effect resource (Kuhn, 2016).

2. Key milestones

  • Dictionary/rule NLP → ML → biomedical transformers: BioBERT (2019), PubMedBERT.
  • LLMs (GPT-3/4, 2020–2023) raised capability and hallucination risk on high-stakes text.
  • Evaluation science matured: groundedness/faithfulness metrics, RAG (Lewis 2020), FActScore (2023), and LLM-as-judge patterns.
  • Regulators engaged: FDA's AI/ML SaMD action plan (2021) and subsequent draft guidances on AI in drug development (2023–2025).

3. Current state

The active problem is benchmarking LLM outputs against authoritative label text — faithfulness scoring with the supporting snippet shown, human-in-the-loop adjudication. That's exactly OpenRegEval's scorecard model.

4. Open challenges & where it's heading

Reliable ground truth, automated hallucination detection, structural variability across labels/agencies, and auditability acceptable to regulators. Frontier: standardized eval frameworks and regulator-grade traceability.


This is a short orientation review, not exhaustive — years/citations are approximate, so verify primary sources before quoting. Reply with papers or milestones you think belong here; this can grow into your project's background/related-work section.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions