Back to selected work
Case 01 /FDA reconciliation · AI product judgment
Launched practicum product

FDA reconciliation with human review.

A deterministic-first reconciliation product: rules for clear records, evidence and human review for the exceptions.

Organization
UC Law Center for Innovation
Role & period
Product Manager · Graduate Practicum
Sep 2025 — Jun 2026
Collaborators
Engineering, research and sponsor stakeholders
95%
deterministic coverage

Share of processed records resolved by rules before LLM routing.

Definition & measurement context ↓
01 / Context

The work behind the problem.

For whom

Researchers comparing monthly FDA Orange Book updates across fragmented sources.

Manual spreadsheet comparisons made recurring updates slow and hard to inspect. Ambiguous records needed judgment, while clear records did not need a model call.

My contribution

  • Led discovery, product strategy, PRD, roadmap and acceptance criteria.
  • Defined deterministic-first routing, discrepancy detection and human exception review with engineering and research stakeholders.
  • Coordinated launch, secure Google OAuth and analytics instrumentation.
02 / Discovery

Clear records and uncertain records need different paths.

In six discovery sessions, I mapped how researchers compared spreadsheet sources and handled records that deterministic matching could not resolve. I used those findings to define the PRD, acceptance criteria and exception-review workflow.

I separated the high-volume matching path from the cases that needed interpretation. We standardized source attributes, applied deterministic logic to clear records, and routed unresolved cases to an LLM with a human exception-review path.

03 / Product judgment

The product decision.

01

Reserve model calls for ambiguous records

Use rules for clear cases and reserve model reasoning for unresolved records. Stable cases did not benefit from probabilistic reasoning; ambiguous cases needed evidence-aware interpretation.

Alternative
Expand the manual spreadsheet workflow, add more rules indefinitely, or send every record to an LLM.
Tradeoff
Rules need maintenance, and a smaller exception queue still requires judgment. Coverage alone cannot establish quality.
04 / Product experience

Review an unresolved conflict.

I used rules for clear records and routed ambiguity to model-assisted human review.

Interface reconstructionIllustrative data · Not a production screenshot
Reconciliation reviewUnlinked
Proposed link / Same product presentation

A strength mismatch
blocks this link.

The proposal would treat these as the same ingredient, strength and dosage form across files. The visible strength disagreement prevents acceptance.

Update record AExample compound

Tablet · Oral

10 mg
Reference record BExample compound

Tablet · Oral

20 mg

Conflict detectedA review note can explain a decision. It cannot correct a source field.

Use a note for investigation context. Any length of note leaves the conflict unresolved.

A 10 mg record and a 20 mg record cannot be accepted as the same product presentation. Reject the visible mismatch or inspect its provenance.

New portfolio demo · No acceptance or override while strength conflicts

This newly designed portfolio demo illustrates the human-review boundary. Reject the mismatch immediately or inspect the local sample extracts and escalate it. No note can resolve the conflict or accept the link. It does not reproduce verified historical production controls.

05 / Evaluation

Outcomes and measurement context.

Project-reported outcomes are separated from discovery counts and program scale. Definitions and measurement limits are included below.

95%

deterministic coverage

Share of processed records resolved by rules before LLM routing.

Measurement context
Baseline
Not supplied
Population / sample
50K+ FDA records
Period
Dated observation window not supplied
Method
Project-reported routing coverage.
Limits
Coverage is not matching accuracy. Precision, recall, evaluation sample and dated routing logs were not supplied.
<2 min

pipeline runtime

Automated processing time, separate from human exception review.

Measurement context
Baseline
2–3 hours
Population / sample
50K+ records
Period
Dated observation window not supplied
Method
Reported processing-time comparison.
Limits
Hardware, run count and workload details unavailable. Does not establish end-to-end human review time.
90%

lower LLM token use

Reported token-consumption reduction after deterministic-first routing.

Measurement context
Baseline
Prior LLM processing approach
Population / sample
Exact sample not supplied
Period
Dated observation window not supplied
Method
Project-reported comparison, not a controlled experiment.
Limits
Token totals, matched workload and measurement dates unavailable. Not a dollar-cost claim.

End-to-end review time must include human exception work. It is not established by automated runtime, and deterministic coverage does not establish matching accuracy.

06 / What remains

The next evidence that matters.

The next validation priority is a reproducible matching-quality evaluation and an end-to-end time study that includes human exception review. Pipeline speed and deterministic coverage answer different questions.

Next / Enterprise workflows · Commercial leadership

A shared roadmap for buyers and daily users.