How do I tell which AI vendors' claims are real?
It depends on whether the vendor can show it working on your own data. A claim is real when the vendor can name the one job the AI does, run it on your actual documents, photos or records, show you how its mistakes are caught, and tell you who on your side checks the results. A vendor who can only show a polished demo on their own examples has not shown you anything yet.
Evidence
The yardstick I use is what has actually worked in the AI systems I have built. Each one does one narrow job, on real inputs, with a check on the results and a person in the loop.
FoodCoach reads a photo of a meal, identifies the food and its approximate portion size, and matches it against the USDA FoodData Central database to work out the nutrition. The job is narrow, the input is a real phone photo, and the result is matched against standard data rather than trusted on the model's word.
BuyCAN reads a photo of a product's front label, identifies the product and the company that makes it, then rates where its ingredients are sourced, where it is made and where the company is based, and explains its reasoning so a person can see why it gave that rating.
GEM drafts and enhances content from a bank of real writing examples, and a person edits every piece before it is published. It runs this site's Articles section. EduGen drafts scripts, finds and ranks stock footage, voices and assembles training videos, while the person editing picks the clips and approves the result.
None of them replaces a person's judgement, and none was sold on a general promise of saving money. Each was built for a specific task, tried on real material, and kept honest by a check. That is what to ask a vendor to show you.
Read the FoodCoach case study, the BuyCAN case study, the GEM case study and the EduGen case study. For the places in a professional services business where AI usually pays off, and why starting small with human oversight matters, see Using AI to Boost Productivity in a Professional Services Business.
What it takes
A specific task. Ask each vendor which one job their AI does for you, in your words, not theirs. "Saves money" and "changes everything" are not tasks.
Your own samples. Give the vendor a handful of your real documents, photos or records, including the messy ones, and ask them to run their product on those in front of you.
A way to check the results. Ask how wrong answers are caught: matching against your data, rules, or a person reviewing uncertain cases. A vendor who says it does not make mistakes is the one to walk away from.
Someone on your side who owns it. A person who knows the work has to judge the results, and keep judging them after the pilot ends.
The honest limits: even a good pilot only proves the narrow job it tested. Whether it pays for itself depends on how often you do that job and what a mistake costs, and that is something only your own numbers can answer.
Next step
Bring me the vendor proposals on your desk. The AI Audit looks at your business, sorts the claims that fit your work from the ones that do not, and tells you plainly where AI would pay off for you.