Creative Mantra
DentalAuto — the AI operator's email dashboard rendered on a laptop in a Krest One Dental office context, teal category badges visible in the Emails list.
Krest One Dental · Case Study

The AI operator for a dental practice. Both halves.

Thirty-five mailboxes on one side. A fine-tuned caries model on the other. Every decision routed to a clinician for the final call. Krest One Dental's paperwork and imaging, on one operator, with a human between the model and the money.

35+
Mailboxes monitored
0.86
YOLO mAP50 · 332 test images
6
Email categories · Claude Haiku 4.5
2,505
Training images · patient-safe split
The Project

A multi-clinic dental group. One operator sitting behind every inbox and every intraoral photo.

Krest One Dental coordinates accounts-receivable across multiple clinics from a single receiver. Before DentalAuto, that meant a human triaging insurance-payment notifications, portal MFA codes, claim-status letters, and patient billing emails out of 35+ office inboxes. And it meant intraoral photos reviewed one screen at a time, no annotation layer beneath them.

Client
Krest One Dental
Multi-clinic group practice. Primary tenant krestonedental.ca (Azure AD). Canada.
Scope
Two halves, one operator
Email triage across 35+ office mailboxes; caries detection on occlusal-view intraoral photographs.
Role
Product, ML, platform
Discovery, product design, model fine-tuning, and platform engineering — all in-house at Creative Mantra.
Stage
In production
Email half deployed on Vercel. Vision half deployable via Ollama on Apple Silicon M4 or cloud CUDA.
The DentalAuto Emails list view — every inbound message across Krest One Dental's 35+ Microsoft 365 mailboxes classified into six categories with per-row forwarding status.
Real dashboard · Emails list · live product
The Two Problems

Not one problem. Two, sitting side by side.

The paperwork problem

Thirty-five inboxes, one coordinator downstream.

  • 35+ office inboxes, one AR coordinator receiving everything.
  • Insurance carriers all send in different formats — every message looks different.
  • Missed claim-status letters cost time and cashflow.
The imaging problem

Intraoral photos reviewed one screen at a time.

  • Caries are small, low-contrast, easy to miss on a tired afternoon.
  • No annotation layer beneath the image — no second look, no rationale on file.
  • Nothing on the screen to remember what the dentist saw last time.
One operator. Two problems. Two halves.
Discovery

We watched the AR coordinator triage a morning of inbox. Then we watched the dentist review a chart.

Before we wrote a line of code, we sat with the people who would use it. Two roles. Two workflows. The product came out of the conversations, not the other way around.

01
The AR triage
02
The dentist review
03
The threshold conversation

Six categories cover the vast majority of what actually lands.

Watching a morning of inbox flow past the AR coordinator's screen, six categories emerged naturally — insurance payments, portal logins, claim status, patient billing, office admin, and spam. Anything more granular would fragment the routing. Anything less would put money and access in the same bucket.

The dentist didn't want a verdict. They wanted a second pair of eyes that leaves a note.

Sitting behind the operatory chair, watching one occlusal photograph after another get reviewed, the request was consistent. Not an AI that decides. Not a second opinion that overrides. A second reader that annotates — and leaves the verdict in the room.

The AR coordinator asked for two thresholds, not one.

Auto-forward what the model is sure about. Queue the maybes. Do not hide the low-confidence ones. That was the request, in three sentences. It is why the routing has two numbers instead of one — and why nothing that reads as low-confidence gets silently dropped.

DentalAuto is one AI operator with two halves. The email half sorts. The vision half annotates. A human clicks the button that matters.

Krest One Dental · in production
Architecture

Two halves. No cross-talk. One operator.

The email half runs on Vercel. The vision half runs on Ollama, local or cloud. They share a client, a design, and a promise: nothing moves without human review. They do not share a database.

DentalAuto solution architecture — two halves A diptych: email half on the left flows from Microsoft Graph webhooks through the Claude Haiku 4.5 classifier and a confidence router into a HITL dashboard. Vision half on the right flows from an intraoral image through YOLO11m into a LoRA-tuned MedGemma rationale composer and the clinician view. A shared Supabase foundation sits beneath both. There is no cross-half API call. Email half 35+ mailboxes · one webhook contract Vision half YOLO11m + fine-tuned MedGemma Microsoft Graph webhooks ChangeNotifications · auto-renewed subscriptions delta-sync on downtime · zero polling Mailbox pool 35+ M365 office inboxes · dedup on message ID emails table · body_preview cap ~200 chars Claude Haiku 4.5 classifier 6-category structured JSON · confidence 0.00–1.00 400 max tokens · temperature 0 · one round-trip Confidence router ≥0.70 auto-forward · 0.50–0.70 review · <0.50 flag forwarding_rules · per-office · editable HITL dashboard Supabase Realtime · inbox · review queue · rules RLS reads · service-role writes · AR coordinator Intraoral image occlusal view · mandibular or maxillary uploaded from operatory · PIL preprocessing YOLO11m detector bounding boxes · class d · confidence ≥ 0.25 best.pt · 77 MB · mAP50 = 0.860 LoRA-tuned MedGemma 1.5 4B-IT per-detection crop · verdict + one-sentence rationale adapter 131 MB · rank 16 · α 32 · QLoRA 4-bit Rationale composer per-box clinical note · lesion, position, confidence analyze.py · PNG panel · boxes + text overlay Clinician view image + YOLO boxes + MedGemma rationale on-screen confirm · dismiss · never autonomous Shared foundation Supabase Postgres · Realtime · RLS · service-role writes only append-only activity_log · auth + admin dashboard · one design language · one HITL rule parallel systems · no cross-call nothing moves without a human click one operator
The Mailbox Fleet

Thirty-five Microsoft 365 mailboxes. One webhook contract. Zero polling.

Every Krest One Dental office mailbox is subscribed to Microsoft Graph's ChangeNotifications. Inbound emails trigger the webhook — the platform never polls. Subscriptions auto-renew before expiry; after any downtime, a delta link fills the gap so no message is lost.

35+ mailbox fan-in topology the mailbox fleet · fan-in topology ofc-billing-01 ofc-claims-02 ofc-front-03 ofc-payments-04 ofc-portal-05 ofc-admin-06 ofc-billing-07 ofc-claims-08 ofc-front-09 ofc-portal-10 ofc-admin-11 ofc-payments-12 ofc-claims-13 ofc-front-14 ofc-portal-15 ofc-billing-16 ofc-admin-17 ofc-payments-18 ofc-claims-19 ofc-front-20 ofc-portal-21 ofc-billing-22 ofc-admin-23 ofc-claims-24 ofc-payments-25 ofc-front-26 ofc-portal-27 ofc-billing-28 ofc-admin-29 ofc-claims-30 ofc-payments-31 ofc-front-32 ofc-portal-33 ofc-billing-34 ofc-admin-35 + subscriptions grow /classify Claude Haiku 4.5 6-category classifier structured JSON · 400 tok 35+mailboxes · one endpoint one webhook contractPOST /webhook · dedup on message IDauto-renewed subscriptions · delta_link recoverylatency mailbox → classified row · < 2s typical

The receiver deduplicates on message ID — webhooks fire twice sometimes; that is fine. Then it inserts into the emails table and hands the message off to the classifier. Latency from mailbox to classified row is typically under two seconds.

POST /webhook · 35+ active subscriptions · auto-renewed
35+
Mailboxes
1
Webhook endpoint
< 2s
End-to-end
0
Polling loops
The Classifier

Six categories. Claude Haiku 4.5. Structured JSON. Nothing more.

Every inbound email lands on a single system prompt with six labels and a bias rule: false negatives on money are worse than false positives. The model responds with category, confidence, should_forward, and reasoning — four hundred tokens, one round-trip, per message.

Six-category email classifier the six categories · one hub Claude Haiku 4.5classifier6 categories INSURANCE_PAYMENTEOBs · payment notifications from carriersteal · money INSURANCE_PORTALlogin credentials · MFA codes · registrationcobalt · access CLAIM_STATUSapprovals · denials · pre-auth · adjustmentscobalt · workflow PATIENT_BILLINGreceipts · remittance · e-transfers · depositsgreen · patient-side OFFICE_ADMINscheduling · supplier orders · HR · appointmentsgray · logistics SPAM_OTHERcold sales · marketing · phishing · off-topicmuted · noise
model: anthropic/claude-haiku-4-5 · max_tokens: 400 · temperature: 0
The Threshold

Confidence is a real number, and it does real work.

Two thresholds per forwarding rule. Both editable. The defaults were tuned with Krest One's AR coordinator: 0.70 for auto-forward, 0.50 for the review queue floor. Anything below 0.50 is held, flagged, and not silently dropped.

Confidence-based routing confidence router · 0.50 & 0.70 thresholds 0.00 0.50review floor 0.70auto-forward 1.00 flagged · heldbelow the review floor. surfaced in the low-confidence view.nothing is silently dropped.< 0.50 confidence review queuea humanapproves or rejects.0.50 ≤ c < 0.70 auto-forwardsends to the AR coordinator inbox.logged. reversible.c ≥ 0.70
0.00 0.25 0.50 0.70 0.85 1.00
< 0.50 · flagged
Held for human review. Nothing dropped silently.
0.50 – 0.70 · review
Queued. Authorized staff approve or reject.
≥ 0.70 · auto-forward
Sent to the AR coordinator. Logged. Reversible.
forwarding_rules: {min_confidence, review_threshold, categories[], office_id, ar_email}
The Dashboard

Every event, in real time, on one screen.

The dashboard subscribes to two Postgres publications via Supabase Realtime — emails and activity_log. New rows appear without refresh. Row-level security enforces read-only for authenticated staff; writes come through service-role API routes only. The receiver never bypasses the audit log.

HITL dashboard schematic hitl dashboard · supabase realtime DentalAuto · email operatorRealtime · connected Mailboxes ofc-billing-01142 new ofc-claims-0286 new ofc-front-0334 new ofc-payments-0457 new ofc-portal-0511 new ofc-admin-0622 new ofc-billing-0761 new + 28 more Subject / fromCategoryConfidence EOB · Claim #4472891 paidnotifications@carrier-a · 2 minINSURANCE_PAYMENT0.94 Adjustment notice · pre-auth #A-9821claims@carrier-b · 6 minCLAIM_STATUS0.88 Payment received · e-transfer #ET-882no-reply@bank · 11 min · needs reviewPATIENT_BILLING0.62 MFA code · portal login expires 05:00security@carrier-c · 18 minINSURANCE_PORTAL0.91 Supplier order · consumables restockaccounts@supplier-x · 22 minOFFICE_ADMIN0.79 Boost your practice with AI marketingdeals@unknown · 31 min · flaggedSPAM_OTHER0.41 Denial letter · Claim #4472902claims@carrier-d · 44 minCLAIM_STATUS0.83 Activity logappend-only auto-forwarded · row 1ar-coordinator · conf 0.9414:02 classified · row 2CLAIM_STATUS · conf 0.8814:04 queued for review · row 3conf 0.62 < auto threshold14:11 reviewed · approvereviewed_by · ar-0114:13 flagged · low-confSPAM_OTHER · held14:33 subscription renewedofc-billing-01 · +48h14:44 rls reads · service-role writes only
The DentalAuto review dashboard's Emails view — real-time list of classified inbound emails with category, forwarding status, and destination coordinator address.
01 · INBOX — sortable by category, status, date · body preview capped at ~200 chars · real product screenshot
Supabase Realtime · RLS on all reads · service-role writes only · activity_log append-only
The Vision Half

We didn't call an image API. We tuned Google's medical model on caries.

The vision half uses two models running in tandem. YOLO11m does the detection — a small, fast object detector, trained on 1,846 patient-safe occlusal-view intraoral photographs to localize dental caries with bounding boxes.

MedGemma 1.5 4B-IT is Google Health AI's medical-image language model. We fine-tuned it in-house with LoRA — rank 16, α 32, dropout 0.05, three epochs on Colab A100. Adapter is 131 MB. It runs alongside YOLO, not on top of it.

The result is a two-stage pipeline: YOLO decides where to look, MedGemma writes what it sees. Both outputs land on the clinician's screen. Neither has the final say.

MedGemma fine-tuning and deployment pipeline MedGemma 1.5 4B-IT · fine-tune · deploy 01 · BASEMedGemma1.5 4B-ITGoogle Health AIvision + language4B paramsbf16 weightsfrozen vision tower 02 · LoRAAdaptertraininglanguage towerrank 16 · α 32dropout 0.053 epochsQLoRA · NF4 03 · MERGEMergedmodelbase + adapterexported safetensors131 MBadapter_model.safetensorsHF safetensors format 04 · QUANTGGUFQ4_K_Mllama.cpp convert~1.5 GB4-bit block quant4× smaller−0.4 pt loss 05 · RUNOllamaruntimelocal deploymentModelfile.carieson-prem or cloudM4 · CUDA linuxsingle command load training: bf16 · lr 1e-4 cosine · 3% warmup · batch 1 × 8 grad-accum · Colab A100-40GB · scripts /train · /convert · /quantize Dataset provenance · patient-safe splits 2,505occlusal imagesafter view-type filtering Split1,846 train327 val332 testpatient-level partition · no mouth appears in more than one split · 850 GT caries in testsource: Zenodo dental caries corpus · script: /split_patient_level
0.860
mAP50 = 0.860 — YOLO11m on 332 held-out test images
precision = 0.835 · recall = 0.898 · F1 = 0.865
The Pipeline

YOLO detects. MedGemma reasons. The clinician decides.

On a single intraoral photograph, YOLO11m outputs a list of bounding boxes, each with a confidence score. Every box is cropped to a small region and fed through MedGemma, which returns a verdict — confirm, uncertain, or reject — and a short rationale, usually one sentence describing the lesion in clinical terms.

Two-stage inference pipeline two-stage inference · per intraoral image stage 1 · detect stage 2 · reason & present 01 · INPUTIntraoral image occlusal view · single framePIL preprocessing · RGB 02 · DETECTYOLO11m 0.840.910.72 bounding boxes + confidence per boxbest.pt · class d · IoU ≥ 0.5 03 · REASONMedGemma rationale crop 1 · 0.91 crop 2 · 0.84 crop 3 · 0.72 per-box verdict + one-sentence note/inference · Ollama · caries-medgemma 04 · PRESENTClinician view 3 caries detectedconfirm · dismiss buttons clinician decidesnothing is autonomous analyze.py — 1 image · full panel   ···   two_stage.py — batch runner · stdout verdicts   ···   latency ~1s YOLO · ~4s MedGemma per crop on M4
Side-by-side comparison on a mandibular occlusal intraoral photograph. Left half: ground-truth annotations (green boxes). Right half: YOLO11m model predictions (blue boxes) with per-detection confidence scores. Ground truth Model prediction
ground truth annotations YOLO11m predictions intraoral occlusal · mandibular 3 caries detected · 1 restoration ignored
analyze.py — 1 image, full panel  ·  two_stage.py — batch runner, stdout verdicts
The Eval

Four numbers. 332 test images we never trained on. Held-out means held-out.

Every number below was measured on a patient-safe held-out test set — 332 intraoral photographs, 850 ground-truth caries, at IoU ≥ 0.5. The training pipeline never saw these mouths.

YOLO11m eval dashboard the eval · held-out test set · IoU ≥ 0.5 0.860mAP50mean average precision 0.835precisionTP / (TP + FP) 0.898recallTP / (TP + FN) 0.865F1harmonic mean · headline metric test set: 332 images · 850 GT caries · TP 763 · FP 151 · FN 87 · confidence ≥ 0.25 · eval script: /eval_two_stage
Pipeline TP FP FN Precision Recall F1
YOLO alone 763 151 87 0.835 0.898 0.865
YOLO + base MedGemma veto 568 79 282 0.878 0.668 0.759
YOLO + fine-tuned MedGemma veto 599 108 251 0.847 0.705 0.769
F1 across pipelines — horizontal bar chart. YOLO alone: 0.865 (highlighted). YOLO + base MedGemma veto: 0.759. YOLO + fine-tuned MedGemma veto: 0.769.
test set: 332 images · 850 lesions · IoU ≥ 0.5 · confidence ≥ 0.25 · eval script: /eval_two_stage
Patient-Safe Splits

We split at the patient, not the image.

The Zenodo dental caries corpus contains multiple photographs per patient — different angles, different visits, sometimes the same lesion twice. Splitting at the image level would leak the same mouth into both training and test. We split at the patient level, then dropped everything that wasn't a mandibular or maxillary occlusal view. What remained: 1,846 train, 327 val, 332 test — a clean partition, publishable metrics, no wishful thinking.

1,846 train
327 val
332 test
Training · 73.7% of the corpus · patient-level partition
Validation · 13.05% · hyperparameter tuning only
Test · 13.25% · touched only for the final eval
2,505
Total occlusal images
1,846 / 327 / 332
Train / Val / Test
850
Ground-truth caries in test
The Annotator

MedGemma Rationale — every detection gets an explanation.

MedGemma runs on every YOLO detection. It receives the cropped region, and it returns a short clinical rationale — usually a single sentence describing the lesion in the language a dentist would use. Position on the arch. Character of the lesion. A note about confidence.

The rationale sits on the clinician's screen next to the YOLO box. The verdict stays with the clinician. The dentist reads the boxes. The dentist reads the notes. The dentist decides.

That is the shipping configuration: YOLO detects, MedGemma annotates, a human confirms. No autonomous rejection. No black-box gating. Every finding surfaces to the room where the decision is made.

Pipeline output panel — maxillary occlusal photograph with five confirmed caries lesions annotated by YOLO and validated by MedGemma with per-lesion clinical rationale.
01_best_confirm — five confirmed lesions with per-box rationales · the working configuration
Pipeline output panel — complex real-world case with three lesion detections and mixed verdicts, each carrying an individual MedGemma clinical note beneath the annotated image.
05_complex_mixed — a harder case, still fully annotated · verdict left to the clinician
Footnote · why MedGemma annotates instead of vetoes. An earlier configuration ran MedGemma as a veto filter on top of YOLO. On the held-out test set that configuration saved 28.5% of false positives but killed 21.5% of true positives, dropping F1 from 0.865 to 0.769. In a caries-detection setting, cautious rejection meant missed cavities, so MedGemma now ships as an annotation layer only — the recommended runtime.
We ship what works. We name what didn't.
Local Deployment

The whole vision model fits in a gigabyte and a half.

Fine-tuned MedGemma is exported from Hugging Face safetensors, converted to GGUF via llama.cpp's convert_hf_to_gguf.py, then quantized to Q4_K_M — four-bit block quantization that trades a fraction of a percentage point for a 4× reduction in memory. The final artifact is about 1.5 GB at runtime and imports into Ollama with a single command from a Modelfile.

best.pt
77 MB
YOLO11m detector weights
adapter_model.safetensors
131 MB
LoRA adapter
Q4_K_M.gguf
~1.5 GB
Runtime · Ollama-ready
$ ollama create caries-medgemma -f Modelfile.caries

Tested on Apple Silicon M4 (MPS device), Colab A100-40GB for training, local CUDA Linux for cloud deploy. A Krest One clinic can, in principle, run the whole detector on the same Mac mini that runs their reception software.

The Bundle

Two halves. No shared database. One operator.

DentalAuto is bundled because the practice runs bundled. The AR coordinator and the dentist are, on most days, seven meters apart. The two halves of the platform serve the two halves of the practice — the front office and the operatory — and both were built with the same discipline: nothing moves without a human click.

The bundle architecture the bundle · two halves, one operator one operator email halfPaperwork operatorRuns on Vercel. Ingests via Microsoft Graph. 35+ mailboxes · one webhook endpoint 6-category classifier · Claude Haiku 4.5 confidence router · 0.50 & 0.70 thresholds HITL dashboard · AR coordinator reviews stack: next.js · supabase realtime · vercel no cross-callparallel systems vision halfImaging operatorRuns on Ollama. Local M4 or cloud CUDA. YOLO11m detector · mAP50 0.860 LoRA-tuned MedGemma · 131 MB adapter two-stage: YOLO detects · MedGemma reasons clinician view · boxes + rationale on-screen stack: pytorch · ultralytics · llama.cpp · ollama Shared foundation Supabase auth + databaseRLS reads · service-role writes Append-only audit logevery event · nothing deleted Admin dashboard · design languageone operator UX · one HITL rule · nothing moves without a click
What they share
The client. The design language. The HITL rule. The Supabase foundation for auth and audit.
What they don't share
A database of business events. A network path. A code path. A model.
Why
Parallel systems fail independently. That is the point — one half can be under maintenance while the other operates untouched.
Healthcare-Adjacency

We didn't buy a compliance badge. We wrote the architecture that behaves like one.

DentalAuto is not marketed as HIPAA-certified. It is marketed as a platform whose architecture matches the constraints of a practice that handles patient data. The distinction matters — one is a checkbox; the other is code.

Body preview cap

No email body stored beyond about 200 characters. Full body fetched from Graph on demand only, and never persisted beyond the classification round-trip.

RLS reads, service-role writes

Authenticated staff can only see their own tenant's rows. Writes never come from the browser — they route through service-role API handlers so the audit story is complete.

Append-only audit log

Every event — received, classified, forwarded, reviewed, errored — has a row. Nothing is ever deleted. The log is the single source of truth for what the operator did.

Patient-safe splits

The vision model was trained with patient-level isolation. Ground truth for evaluation cannot contaminate the training set — a rare thing to enforce, and the reason the eval numbers stand up.

DentalAuto's architecture is HIPAA-adjacent. Formal certification is scoped for the next phase.
The Stack

Two halves. Two stacks. One design language.

Each half runs on the toolchain it was built for. The email half is web-native, deployed to Vercel. The vision half is Python-native, deployed via Ollama on M4 or CUDA Linux. They share only what they need to share.

Email half · email automation
Web-native
Next.js 16 · React 19 · Tailwind 4 · shadcn/uiWeb surface for the review dashboard and forwarding-rule editor.
Microsoft Graph API 3.0.7Webhook subscriptions with delta sync for zero-poll ingestion.
Claude Haiku 4.5 via OpenRouterStructured JSON output · 400 max tokens · temperature 0.
Supabase Postgres + Realtime + RLSService-role API routes for writes · RLS on all reads.
Vercel · Vercel CronDeployment and subscription-renewal scheduling.
Vision half · dental imaging
Python-native
YOLO11m via UltralyticsPatient-safe occlusal-only training set.
MedGemma 1.5 4B-IT (Google Health AI)PEFT LoRA adapter · rank 16, α 32, dropout 0.05.
PyTorch ≥ 2.11 · transformers ≥ 5.5 · peft ≥ 0.19Training and inference toolchain.
QLoRA training (4-bit NF4, bf16)On Colab A100-40GB · 3 epochs · cosine LR.
llama.cpp GGUF · Q4_K_M · OllamaLocal serving on Apple Silicon M4 or CUDA Linux.
total artifact size · best.pt 77 MB · adapter 131 MB · GGUF runtime ~1.5 GB
The Rollout

A day at Krest One Dental, with DentalAuto in the loop.

08:00
First office mailbox wakes. Overnight insurance-payment notifications trigger webhooks; the classifier processes them in seconds; ≥ 0.70-confidence rows auto-forward to the AR coordinator's inbox before opening.
10:15
A mid-confidence PATIENT_BILLING email lands in the review queue. The AR coordinator clicks approve. The reviewed_by and reviewed_at timestamps hit the audit log.
13:40
A dentist uploads an intraoral photo from operatory 2. YOLO finds three candidates in under a second on the clinic's Mac mini; MedGemma writes a rationale for each. The panel opens on-screen next to the patient chart. The dentist confirms two, dismisses one.
17:30
End-of-day review. The AR coordinator scans the day's activity log. Everything auto-forwarded is receipted. Everything reviewed is stamped. Nothing was silently dropped.
What We Learned

Three things we would tell the next team.

01

Threshold conversations first.

Every HITL product lives or dies by whether the operator trusts the threshold. Have the conversation before you have the classifier. Ship a slider you can move, not a number you have to explain.

02

Patient-level splits, or your metrics are theatre.

In dental imaging — and probably most medical imaging — the same mouth in train and test will inflate your F1 by ten points and lie to your face. Split at the patient. Publish only what survives.

03

Ship what worked. Name what didn't.

Configuring MedGemma as an annotator instead of a veto was the honest choice. Saying so in the case study made the pipeline more trustworthy, not less. Restraint is a marketing asset.

Work With Us

Have a paperwork problem, an imaging problem, or both?

We build AI operators that route decisions to humans, ship the models we tuned ourselves, and publish the metrics we measured on data we never trained on.