Detecting and grading damage

VerifyAI ships a damage-intelligence pipeline that runs on top of the normal verification flow. When a policy has damageMode: true, every successful verification is annotated with:

a list of damage_findings — one entry per visible damage spot,
a panel_inventory — the panels actually visible in the photo,
an overall_severity — the worst severity across all findings,
a list of aiag_codes — finished-vehicle logistics codes, and
a k_grade — K1..K5 finished-vehicle condition grade.

The first three come straight from the cloud VLM. The last two are derived deterministically by pure server-side transforms — see AIAG codes and K-grades for the breakdown.

What this is not. VerifyAI v1 does damage detection and grading, not body-shop estimating. Repair-cost estimates are a separate preview feature that requires a Mitchell partner contract — see Estimating repair cost via Mitchell. On-device damage detection is not in v1 either; damage runs in the cloud VLM only.

1. Enable damage mode on a policy

Damage mode is opt-in per policy. Set damageMode: true on the policy config and the server-side prompt builder appends a "Damage Reporting" section that asks the VLM to enumerate findings and the panel inventory.

// PolicyConfig (lib/verify-ai/types.ts)
{
  mode: "structured",
  categories: [/* ... */],
  criteria: [/* ... */],
  damageMode: true   // ← turn it on
}

The damage section appends a fixed schema to the prompt: per-finding fields (finding_id, panel, damage_type, severity, severity_score, bbox, area_pct, confidence) plus the top-level panel_inventory array. The schema is enforced by Zod when the response comes back — malformed damage payloads fall back to the legacy schema so a verification still returns a verdict, but the damage columns will be empty.

2. Submit a verification

The damage policy uses the same POST /v1/verify endpoint as any other policy — see the Verify reference. There are no new request fields.

curl -X POST https://verify.switchlabs.dev/api/v1/verify \
-H "X-API-Key: vai_your_api_key" \
-F "image=@vehicle_intake.jpg" \
-F "policy=pol_fleet_damage" \
-F 'metadata={"vehicle_id":"VIN1234","inspection_slot":"checkin"}'

3. Read the response

The verification returns the standard fields (id, is_compliant, confidence, category, feedback, violation_reasons, image_url). The damage fields are persisted to the verify_ai_verifications row — damage_findings, panel_inventory, overall_severity, aiag_codes, k_grade — and surfaced in the operations dashboard and via direct DB / webhook payloads. A typical stored row looks like this:

json

{
  "id": "ver_8x92m4k9",
  "is_compliant": false,
  "category": "damaged",
  "confidence": 0.88,
  "panel_inventory": [
    "car_hood",
    "car_door_fl",
    "car_door_fr",
    "car_fender_fl",
    "car_front_bumper"
  ],
  "damage_findings": [
    {
      "finding_id": "f1",
      "panel": "car_door_fl",
      "damage_type": "dent",
      "severity": "medium",
      "severity_score": 0.62,
      "bbox": [0.31, 0.42, 0.44, 0.55],
      "area_pct": 0.07,
      "confidence": 0.86
    },
    {
      "finding_id": "f2",
      "panel": "car_front_bumper",
      "damage_type": "scratch",
      "severity": "light",
      "severity_score": 0.28,
      "bbox": [0.55, 0.62, 0.74, 0.66],
      "area_pct": 0.03,
      "confidence": 0.81
    }
  ],
  "overall_severity": "medium",
  "aiag_codes": ["BF-SC-1", "DFL-DN-2"],
  "k_grade": "K3"
}

Field reference

| Field | Type | Notes | | ------------------ | ------------------------------- | ---------------------------------------------------------------------------------------------- | | damage_findings | DamageFinding[] | Empty when damage mode is off. See the per-finding shape below. | | panel_inventory | string[] | Panel names the VLM could see. Used to suppress angle-based false positives in delta detection. | | overall_severity | "none" \\| "light" \\| "medium" \\| "severe" | Highest severity across all findings. null if damage mode is off. | | aiag_codes | string[] | One stable code per non-none finding, deduplicated and sorted. | | k_grade | "K1" \\| "K2" \\| "K3" \\| "K4" \\| "K5" | Finished-vehicle grade derived from the findings. |

The per-finding shape (DamageFinding, defined in lib/verify-ai/ml/schema.ts):

| Field | Type | Notes | | ---------------- | ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | | finding_id | string | Short ID, unique within the response (e.g. "f1"). | | panel | string | Ontology panel name — e.g. car_door_fl, car_front_bumper. | | damage_type | "scratch" \\| "dent" \\| "paint_chip" \\| "crack" \\| "broken" \\| "missing" \\| "rust" \\| "tear" \\| "stain" \\| "glass_damage" \\| "other" | Closed set — the prompt forbids inventing new types. | | severity | "none" \\| "light" \\| "medium" \\| "severe" | "none" means "inspected, found nothing" and rarely appears. | | severity_score | number [0,1] | Graded score the VLM produces alongside the severity bucket. | | bbox | [x1, y1, x2, y2] | Normalized to [0,1] against the original frame. | | area_pct | number [0,1] | Fraction of the panel area covered by the damage. | | confidence | number [0,1] | The VLM's confidence in the finding. |

4. Branch on the damage results

A practical app uses overall_severity for routing and the findings list for the audit trail:

const stored = await db
  .from("verify_ai_verifications")
  .select("id, overall_severity, k_grade, aiag_codes, damage_findings")
  .eq("id", verificationId)
  .single();
 
switch (stored.data?.overall_severity) {
  case "none":
    // Clear vehicle — no action needed.
    closeInspection(stored.data.id);
    break;
  case "light":
    // Cosmetic only — log and continue.
    flagForRecord(stored.data.id);
    break;
  case "medium":
  case "severe":
    // Hold for review or repair routing.
    routeToReview(stored.data.id, stored.data.aiag_codes);
    break;
}

If you need a single rollup number for downstream systems, use k_grade — it collapses the per-finding list to a five-bucket grade that maps cleanly to "deliver / hold / repair" decisions. The AIAG and K-grade guide covers the mapping rules.

5. Combining two captures into a delta

A single verification answers "what damage does this photo show". A pair of verifications — one at checkout and one at check-in — answers the more useful question: "what damage is new since the vehicle left our lot?" See the Before/after delta guide for the pairing workflow and how panel_inventory is used to keep the diff honest.

Honest limitations in v1

Cloud VLM only. Damage detection runs in the cloud VLM (OpenAI / Anthropic / Gemini). The on-device pipeline does not have a damage model yet — panel_inventory and damage_findings are always populated by the cloud path.
One frame at a time. Each verification looks at exactly one image. Multi-angle aggregation (e.g. 4-corner walk-around) is the caller's responsibility — capture each angle separately and union the findings.
severity: "none" is rare. The prompt explicitly tells the model to omit undamaged panels rather than emit "none" findings. Don't expect a finding per panel inspected.
Repair cost is preview-only. See Estimating repair cost via Mitchell.

What's next

AIAG codes and K1–K5 grading — how the codes and grade are computed.
Before/after rental inspection delta — pair a checkout and a check-in to extract new damage.
Estimating repair cost via Mitchell — preview integration for body-shop pricing.

Get in Touch