Triaging low-confidence verifications

Most verifications are decisive — is_compliant=true with a confidence score above 0.95. The rest are the ones humans should look at: a model saying "probably bad parking, but I'm only 78% sure." VerifyAI ships an exception queue for these. Every low-confidence non-compliant verification automatically becomes an exception, and your operations team works it from a single dashboard.

By the end of this guide you'll know how:

Exceptions are auto-created and why duplicates are impossible.
The open → assigned → resolved | dismissed lifecycle works.
To work the queue from the dashboard or from server-side code.
The audit log records every transition.

1. Auto-trigger

Exceptions are inserted by an AFTER INSERT trigger (trg_verify_ai_open_exception) on verify_ai_verifications. The trigger function is verify_ai_open_exception_for_verification() in migration 20260531_verify_ai_exceptions.sql. It fires when all of these hold:

plaintext

NEW.is_compliant = false
AND NEW.confidence IS NOT NULL
AND NEW.confidence < 0.85
AND NEW.customer_id IS NOT NULL

The 0.85 threshold is a constant inside the trigger function; a follow-up migration can move it to a per-customer config table when product needs differ. Compliant results don't queue (a confident "yes" is fine), and ultra-low confidence non-compliant results queue along with the rest so a human makes the final call.

The exceptions table has a UNIQUE (verification_id) constraint and the trigger uses ON CONFLICT (verification_id) DO NOTHING, so re-running the trigger (e.g. a verification gets re-scored) does not create a duplicate. You get at most one exception row per verification, forever.

2. The lifecycle

The status column is constrained to one of these four values:

| State | Meaning | | ----------- | ---------------------------------------------------------------------- | | open | Auto-created. No one has claimed it. (Default for new rows.) | | assigned | A team member owns it. Set by assignException. | | resolved | A human decided. resolution is free-form text describing the call. | | dismissed | Not a real problem (test data, duplicate, etc.). Also set by the same resolveException action with the dismissed flag. |

resolved and dismissed are terminal (resolved_at is stamped on both). To reopen a case, create a follow-up exception manually or re-verify the image.

3. Working the queue (dashboard)

Operations members open Dashboard → Operations → Exceptions to see their queue. Each row links to the original verification, shows the violation reasons, the model's confidence, and any prior assignee notes. Filtering by site, policy, and assignee is built in — site managers see only their assigned sites (see Organizing teams with sites and roles).

Three primary actions:

Assign — claim the case or assign to a teammate.
Resolve — pick compliant or non_compliant, leave a note, and close the case. The result feeds back into model evaluation.
Dismiss — close the case without a verdict (test photos, spam-like duplicates).

4. Working the queue (server-side)

For automation — e.g. auto-assigning by site rota, or escalating cases older than 24h — call the server actions in app/actions/operations.ts:

import {
  listExceptions,
  assignException,
  resolveException,
} from "@/app/actions/operations";
 
// 1. List open exceptions for one site
const { rows, total } = await listExceptions({
  status: "open",
  siteId: "9b1f...e2",
  limit: 50,
});
 
// 2. Claim the oldest one (positional args: exceptionId, memberId)
await assignException(rows[0].id, currentMemberId);
 
// 3. Resolve it. Signature:
//    resolveException(exceptionId, resolution, notes?, dismissed?)
await resolveException(
  rows[0].id,
  "non_compliant",
  "Bike on tactile paving — confirmed via second photo.",
);
 
// To dismiss instead, pass `true` as the fourth arg:
// await resolveException(id, "duplicate", "Same trip as #abc", true);

Every action checks caps.can.resolveExceptions (site_manager or org-level role) and writes a row to verify_ai_audit_log with the transition details.

5. What gets audited

For each exception event, the audit log records:

| Field | Example | | --------------- | ------------------------------------------------------------ | | action | exception.assign, exception.resolve, exception.dismiss | | resource_type | verify_ai_exception | | resource_id | The exception UUID | | metadata | { memberId } on assign; { resolution } on resolve / dismiss | | actor_email | Whoever clicked the button |

You can search this from Dashboard → Operations → Audit log or straight from SQL — see Audit log and SOC 2 readiness.

6. Tuning the threshold

The 0.85 default is hard-coded in the verify_ai_open_exception_for_verification() trigger function. If you're seeing too many open exceptions on a particular policy (a sign the threshold is too aggressive for that domain), the path forward is a follow-up migration that either edits the trigger or moves the threshold to a per-customer / per-policy config table. The trigger is the only place this decision is made today.

What's next

Audit log and SOC 2 readiness — the paper trail for every transition.
Organizing teams with sites and roles — scope the queue to the right people.
Sending verifications to claims platforms — resolved exceptions can downstream to claims systems too.

Get in Touch