How We Measure What We Claim
Accuracy, latency, and cost-comparison numbers are only useful if you can see how they were produced. This page documents exactly how VerifyAI measures accuracy and latency, and how our competitor cost comparisons are sourced — including where competitor prices come from, which assumptions the cost calculator uses, and how to reproduce or request the underlying numbers.
Tied to a version
Every number references a specific model bundle and gold eval dataset. No floating claims.
Per-policy, not aggregate
Results are reported per policy and per condition, because a blended average hides the cases that matter.
Vendor-published prices only
We cite a competitor price only when the vendor publishes it, and we name the source. No estimates, no third-party figures.
How we measure accuracy
Accuracy is measured against held-out gold eval sets, reported as precision and recall per policy, and sliced by the field conditions that actually stress the models. We publish both error directions and the confusion matrix — not a single feel-good percentage.
Gold eval sets per policy
Accuracy is never measured on aggregate. Each policy — scooter_parking, bike_parking, forest_green_bay, forest_designated_bay — has its own held-out gold eval set: real captures, hand-labeled by two reviewers, with a third resolving disagreements. Gold sets are versioned in verify_ai_ml_gold_eval and frozen at the moment a model is promoted, so a score is always tied to a specific dataset and model bundle.
Precision, recall, and F1 — not just "accuracy"
A single accuracy percentage hides the trade-off that matters operationally. We report precision (of the captures we flagged compliant, how many actually were) and recall (of the truly compliant captures, how many we caught) per policy, plus F1 as the balanced summary. For parking verification, false-accepts and false-rejects have very different costs, so we publish both error directions rather than collapsing them.
Per-policy, per-condition slicing
Headline numbers are sliced by the conditions that actually break models in the field: low light, rain and glare, partial occlusion, unusual angles, and edge geographies. A model that scores 99% in daylight and 78% at dusk is not a 99% model. Slices are part of the same gold eval run, not a separate after-the-fact report.
Confusion matrix and threshold transparency
Every gold eval run produces a full confusion matrix at the production confidence threshold (the same threshold that opens an exception in the live triage queue). We show where errors cluster — which violation category gets confused with which — instead of only the top-line rate.
How we measure latency
VerifyAI has two pipelines — on-device ML and cloud image analysis — with different latency profiles. We measure them separately, report percentiles instead of averages, and state exactly where the clock starts and stops on every number.
Two pipelines, measured separately
VerifyAI runs an on-device ML pipeline and a cloud image analysis pipeline. They have fundamentally different latency profiles, so we never blend them into one number. On-device inference is timed from frame-ready to result on representative consumer hardware; cloud verification is timed server-side from request receipt to response written, excluding the client's own network round-trip.
Percentiles, not averages
Averages hide tail latency, and the tail is what users feel. We report p50, p95, and p99 for each pipeline. The <200ms figure you'll see in marketing refers to the on-device pipeline's typical (p50) inference time on target hardware — not an average across all requests, and not the cloud path.
Where the clock starts and stops
On-device: from the moment a frame is handed to the model to the moment a compliant/non-compliant result is available, excluding camera capture and UI render. Cloud: from API request receipt to response serialization, measured at the edge, excluding the device's upload time (which depends on the customer's image size and connection). We state the boundary on every number so it can be compared apples-to-apples.
Representative hardware and load
On-device numbers are measured on mid-tier devices, not flagship-only — because field operators run on whatever phone the rider or driver has. Cloud numbers are measured under realistic concurrent load, not single-request best case.
How the cost comparisons are built
Our own price — $0.008 per single-photo Standard check, no contract, no minimums — is a fact we control. Competitor prices appear only when the vendor publishes them itself, cited to that source and last checked October 2026. Here is the rule and how the comparisons are built.
Our price is a published fact
VerifyAI standard pricing is $0.008 per single-photo Standard check — no annual contract, no minimums, no per-seat fees — with lower rates in waterfall bands at higher monthly volume. Vehicle appraisals are a separate product at $2.00 per completed appraisal, including up to 16 photos and detailed analysis. We always state our price per unit (per photo check, per appraisal, per handoff inspection), never as a monthly or annual total. Everything below is about the numbers we don't control: competitor pricing.
We cite only prices a vendor publishes itself
A competitor price appears on this site only when the vendor publishes it on its own website or in an official listing it controls, such as its AWS Marketplace listing, and we name that source. We do not use estimates, reported or third-party figures, prices from programs that have ended, or numbers we derived ourselves. When a vendor publishes no price, we say so: Captur, for example, does not publish prices; its site describes committed consumption-based pricing with a paid pilot for new customers.
Published prices are shown as the vendor states them
Where a vendor does publish a price, we quote it in the vendor's own terms and unit. Ravin AI's AWS Marketplace listing, for example, is its Vehicle Inspection Enterprise Package at $84,000 for 12 months, up to 20,000 inspections a year; we do not convert it into a per-inspection figure of our own. Our side is always a per-unit rate, so the reader can see both pricing models without us filling in the gaps.
Assumptions used in the ROI calculator
The cost calculator prices VerifyAI on the published single-photo ladder, blended across your monthly volume into an effective rate per check. It contains no competitor figures: the current-vendor fields stay blank until you enter the annual fee and per-check rate from your own contract. Your annual fee is divided by twelve months of your volume and added to your per-check rate, so both sides are compared per check. It does not assume integration or services fees on either side.
Competitor pricing disclosure
Vendor-published onlyWe cite a competitor price only when the vendor publishes it on its own website or an official listing it controls, and we name the source. Vendors that publish no price, including Captur, Luna, UVeye, and Drover AI, are described as not publishing one. Last checked October 2026. Seen a vendor change or remove a price we cite? Send it to us and we will check the vendor's source and correct it.
How to reproduce or request the numbers
None of this is meant to be taken on faith. You can run the cost comparison on your own volume, request the underlying per-policy eval results, or bring your own captures for a blind evaluation before you commit to anything.
Run the calculator with your own volume
The cost calculator is interactive. Plug in your actual monthly verification volume and the annual fee and per-check rate from your current contract, and it recomputes the per-check comparison for your situation rather than ours.
Request the underlying eval numbers
We'll share the per-policy gold eval results — precision, recall, F1, confusion matrix, and condition slices — for the policies relevant to your deployment, along with the model bundle and dataset versions they were measured against. Technical buyers and procurement teams can request these under NDA via the contact form.
Bring your own gold set
The most honest benchmark is your own data. We can run a blind evaluation on a held-out set of your captures so you see precision and recall on your environment, not ours. This is the recommended path before any production rollout.
Tell us when a figure is stale
Competitor pricing moves. If a vendor changes or removes a price we cite, send us the link — we'll check the vendor's own source and correct or remove it. The last-checked date below tells you when we last went through those sources.
What the headline figures actually mean
When you see <200ms, 99.2%, or 50M+ elsewhere on this site: the latency figure refers to typical (p50) on-device inference on target hardware, the accuracy figure is a per-policy result against a versioned gold eval set, and the volume figure reflects cumulative verifications processed. The slices, percentiles, and dataset versions behind each are available on the benchmarks page or on request.
Dig into the numbers
The benchmarks, the calculator, the head-to-head comparison, and a direct line to the underlying data.
Captur vs VerifyAI cost calculator
Interactive: enter your volume and see the effective per-verification comparison.
ViewRequest the raw numbers
Ask for the underlying eval data, or run a blind eval on your own captures.
Contact salesEvaluate it on your own data
Start a free sandbox with a $5 credit — no card required — and run your own captures, or ask us to run a blind eval against your gold set before you switch.