A three-month launch that reduced regulatory data-review time by 99% across more than 50,000 FDA product records.
Researchers spent three working days reviewing 50,000+ records. A full-LLM approach added hours of latency and unsustainable API cost, while early deterministic rules produced too many false positives.
Research managers reconciling monthly FDA Orange Book updates across fragmented source systems.
Six discovery sessions shaped a tiered engine that standardized 25–30 attributes, resolved 95% of records deterministically, and routed only ambiguous cases to an LLM and review queue.
incorrect data enters downstream research
tighten agreement thresholds
ambiguous recordrules cannot defend a match
route to the LLM and review state
source conflictautomation cannot resolve evidence
preserve source fields for a researcher
A sanitized reconstruction of the interface where researchers inspect evidence, resolve ambiguity, and export matched records.
Resolve stable cases with deterministic rules and route only the remaining 5%. The tiered architecture reduced latency, token consumption, and audit risk.
Alternative considered: Send all 50K+ records through an LLM or keep the workflow fully manual.
Trade-off: Explicit rules required calibration and maintenance, but cut token consumption by 90%.
Tighten agreement thresholds after false-positive testing. Incorrect matches were more costly than leaving an ambiguous record for review.
Alternative considered: Accept the first-pass rule set or add broader fuzzy matching.
Trade-off: Stricter rules preserved a small review queue but made automated decisions more defensible.
Regulatory data-review time fell by 99% during the first 30 days after launch.
Pipeline execution dropped from two–three hours to under two minutes across 50K+ records.
Routing only ambiguous cases reduced LLM token consumption by 90%.
Definitions and sources are shown so every result can be examined in an interview.
Measured during the first 30 days after launch against the prior three-working-day review workflow.
Share of 50K+ FDA records resolved by calibrated deterministic rules before LLM routing.
Observed full-pipeline runtime, reduced from a two–three-hour baseline.
Measured reduction after replacing full-LLM processing with ambiguity-only routing.
The highest-value AI decision was defining where not to use AI: deterministic rules handled scale, while the LLM handled ambiguity.