Reading a Dealership AI Vendor's Guarantee (the Methodology Test)

Every dealer has heard the pitch: a bold guarantee on the slide, a percentage lift promised, a confident rep saying you have nothing to lose. Most of those guarantees are real in the way a coupon is real. Technically honorable, structured so you almost never collect. The difference between a guarantee you can hold someone to and a line designed to close you is not the size of the number. It is whether there is an inspectable methodology underneath it. I sell cars for a living, so here is how I would read a vendor's guarantee before I let it move me.

Why does a vendor guarantee usually mean less than it looks like it means?

Because a guarantee is a promise about an outcome, and an outcome is the one thing a vendor does not fully control. Your store changes constantly. Inventory shifts, the market moves, your staff turns over, your other tools get swapped. A guarantee tied to a result sitting inside all that noise is easy to promise and nearly impossible to hold anyone to, unless the methodology for measuring it was agreed on up front.

Here is the move that should make you slow down. The confident number on the slide is doing emotional work, not analytical work. It is there to make you feel that the risk is on the vendor's side. But read it closely and the guarantee almost never says how the number will be measured, against what starting point, or who decides whether it was met. All of that gets settled later, informally, in the vendor's favor, precisely because it was never written down. The guarantee felt like protection. It was actually the absence of protection, dressed as a promise.

A guarantee with no inspectable methodology is not a promise. It is a feeling the vendor rented you until the contract was signed.

What are the five questions the methodology test actually asks?

Five, in order: what is measured, what the baseline is, how the result is attributed, who owns the number, and what the exit terms are. Ask each one out loud in the demo and watch what happens to the conversation. A real guarantee gets sharper under the questions. A marketing line gets vaguer.

What is measured

→

The baseline

→

Attribution

→

Who owns the number

→

Exit terms

Here is what each one is really testing:

• What is measured. Not a category like "more engagement" or "better follow-up." A specific number you already track, defined the same way you define it today. If the metric is one the vendor invented and only the vendor can compute, you cannot verify the guarantee, and a guarantee you cannot verify is not one.
• The baseline. The starting point the result is measured against. Who set it, from what data, over what window. A guarantee of a lift means nothing until you know a lift from what. A vendor who sets the baseline low, or who never writes it down, has quietly made the guarantee unlosable for themselves.
• Attribution. How the result gets credited to the tool rather than to everything else that changed while it ran. This is the hardest one and the one that exposes the most, so it gets its own section below.
• Who owns the number. Which system the settling metric is read from. If it lives only in the vendor's dashboard, the vendor is the player and the scorekeeper at once. Ask that the number be readable from your CRM or DMS, the systems you already trust.
• Exit terms. What happens, in writing, when the number misses. A defined window, a defined threshold, a defined consequence. No exit term means no guarantee, only a hope with a logo on it.

How do you tell an outcome guarantee from a methodology you can actually inspect?

An outcome guarantee promises a result. A methodology shows you the machinery that would let you prove the result true or false. The tell is simple: ask the vendor to walk you from the raw event in your systems to the number in the guarantee, step by step. If they can, you have a methodology. If they redirect to features, you have a slogan.

Say a vendor guarantees more of your leads get worked. Good. Now walk it. Which event in your CRM counts as a lead being worked, and does that definition match how your BDC already logs it? Over what window is it counted? What was the number before they arrived? A vendor with a real methodology answers each step without flinching, because the machinery exists and they built the guarantee on top of it. A vendor selling an outcome starts talking about how smart the AI is, which is a lovely subject that has nothing to do with whether you can ever collect. The drift from your number to their features is the entire tell, and once you have heard it you cannot unhear it.

This matters more in a dealership than almost anywhere, because your data rarely agrees with itself across systems, which makes any single reported number fragile.

When more than half of dealers say the same fact looks different in two systems, a guarantee measured only in the vendor's dashboard is standing on sand. The inspectable version reads the settling number from a system you already trust, so the guarantee is anchored to a fact you can pull yourself. That is the same reason a dashboard is not an intelligence layer: a screen full of the vendor's numbers is not the same as a number you own. Before you weigh any guarantee, it helps to have already mapped where your customers get dropped between systems, so you know which metric actually matters and where its data really lives.

Why is attribution the question that exposes the most?

Because a store is never running one experiment at a time, so almost any result can be claimed by almost any tool. The month the vendor's AI went live, you also ran a sale, hired two salespeople, and the market moved. When appointments go up, whose win is it? A guarantee with weak attribution lets the vendor claim the lift and quietly hand you the blame for any miss.

Attribution is also where the vendor's reported number and your reality drift furthest apart, because the events that feed the number are the same events that go missing between your systems. If a lead never lands in the CRM cleanly, it never enters the count, and the guarantee gets measured on a population that was already filtered in the vendor's favor before anyone did the math.

That is the trap under a lot of attribution. If one in eight in-market leads never makes it into the system of record, then the number the guarantee rests on was never the whole truth, and the vendor gets to define which slice of it counts. The honest version of attribution names, in advance, what belongs to the tool and what belongs to everything else, and it measures against a baseline that includes the leads that usually go uncounted. Ask the vendor how their guarantee handles the leads that never hit the CRM. The answer tells you whether they have thought about your reality or only their own dashboard.

Isn't a strong guarantee a sign the vendor believes in the product?

Sometimes. But conviction and collectability are different things, and only one of them protects you. Plenty of vendors genuinely believe in their tool and still write guarantees you could never actually cash, not out of malice but because they never had to make the methodology inspectable to close deals. The industry data is on the vendor's side here: response speed alone is known to move outcomes dramatically, so a tool that touches speed can point to real published lift and still leave your specific guarantee unmeasurable.

Consider the most cited finding in lead handling. Harvard Business Review's study of online sales leads found that firms contacting a prospect within an hour were nearly seven times more likely to have a meaningful qualifying conversation than those who waited even one hour longer, and more than sixty times more likely than firms that waited a day. That is real, and a vendor can wave at it honestly. But notice what it does not tell you: whether their tool, in your store, against your baseline, measured in your CRM, will move your number by the amount they guaranteed. Industry lift is not your lift. The methodology test is what closes that gap, by forcing the general promise down onto your specific, measurable, owned number. A vendor who believes in the product should welcome that, because it is the fastest path to proving they were right.

The guarantee is only as real as the methodology you can inspect

A guarantee you cannot measure, cannot baseline, cannot attribute, and cannot exit is not a guarantee. It is a well-designed reason to sign. And the fix costs nothing: five questions, asked before the contract, in language plain enough to settle a dispute six months later. What is measured. From what baseline. Attributed how. Owned by whom. With what exit if it misses.

Run every guarantee through those five and the good vendors get sharper while the rest get vaguer, and the difference is visible inside one conversation. That is the whole methodology test, and it turns you from someone being reassured into someone doing diligence. If you want a second set of operator eyes on a guarantee before you sign, that is exactly the kind of thing we walk through on a diagnostic call: read the guarantee together, find the five answers or the five silences, and tell you plainly whether the promise is inspectable or just confident. Worth a look before you sign anything?

Frequently asked questions

How do you stress-test a dealership AI vendor's guarantee?

Run the guarantee through five questions before you sign. What exactly is measured, in a number you already track. What is the baseline it is measured against, and who set it. How is the result attributed to the tool rather than to everything else changing in your store. Who owns the number that decides whether the guarantee was met. And what happens if it misses, stated as exit terms, not sentiment. If the vendor can answer all five in plain language, the guarantee is real. If any answer turns into a feature tour, the guarantee is a marketing line.

What is the difference between an outcome guarantee and an inspectable methodology?

An outcome guarantee is a promise about a result, such as a lift in appointments or a percentage more leads worked. You have to take it on faith unless you can see how it will be measured. An inspectable methodology is the machinery underneath: the exact metric, the baseline, the attribution logic, and who reads the number. A guarantee with no inspectable methodology is a slogan. A methodology you can inspect line by line is a promise you can hold someone to, because you can see how it would ever be proven true or false.

Who should own the number that decides whether an AI guarantee was met?

You should, or at minimum a source you control and can audit. If the only record of whether the guarantee was hit lives in the vendor's own dashboard, the vendor is both the player and the scorekeeper. Ask that the metric be readable from your CRM or DMS, the systems you already trust, so the number that settles the guarantee is one you can pull yourself. When the vendor owns the scoreboard, the guarantee is worth exactly as much as your willingness to never check it.

What exit terms should a dealer look for in an AI vendor contract?

A real guarantee names what happens when the number misses, in writing, before you sign. Look for a defined measurement window, a clear threshold that counts as a miss, and a concrete consequence: a refund, a credit, or a clean exit with no penalty. Vague language like best efforts or continued optimization is not an exit term, it is a way to keep you paying while the miss gets reframed. If the contract describes the win in detail but goes quiet on the miss, the guarantee was never meant to be collected on.

Sources

• Cox Automotive, "Power of Data Study" (2024). coxautoinc.com
• Foureyes, "2024 Automotive Dealer Benchmarks Report." foureyes.io
• Harvard Business Review, "The Short Life of Online Sales Leads" (2011). hbr.org