A seller messaged me at eleven at night with three browser tabs open. Three AI assistants, three free trials, three pricing pages she had read four times. "Just tell me which one," she said.
I didn't answer that. I asked her to paste one real message into each demo — the exact thing her customers send her: "hi is this still available in 38? need it by thursday".
Two of them replied with a menu. Press 1 for sizes, press 2 for shipping. The third asked which of two similar boots she meant, then said the 38 was gone until Sunday and the 39 runs small.
That was the whole evaluation. Ninety seconds. What follows is just a longer version of those ninety seconds, so you don't buy the wrong thing and then decide AI doesn't work for your shop.
Test it with your ugliest message, not their happy path
Every demo on every website is built around a customer who behaves. Types full sentences. Asks one thing at a time. Spells the product name correctly. You have never met this customer.
The messages that decide your month look nothing like that. So before you read a single feature list, go pull five real DMs from last week — copy them exactly, typos and all — and paste them into every tool you're considering.
- The three-in-one: "price? do you ship to Manchester? is it real leather"
- The screenshot with no words: a photo of your own post, followed by "how much"
- The wrong name: the product called what customers call it, not what your catalogue calls it
- The one it shouldn't answer: "my order was supposed to arrive yesterday, where is it"
- The haggle: "can you do a better price if I take two"
You're not watching for perfect answers. You're watching what happens at the edge of what it knows. Does it ask a clarifying question, or does it guess with total confidence? Guessing with confidence is the expensive failure, and it explains most of why sellers quietly switch their assistant off after two weeks.
Knowledge first, features second
Feature lists are almost interchangeable now. Everyone does auto-reply. Everyone does comment-to-DM. What separates a tool that earns money from one that embarrasses you is how it gets your knowledge in, and how it behaves once that knowledge runs out.
Ask three blunt questions on the demo call:
- How does it learn my catalogue? A file I upload once, a link to my shop that stays in sync, or a text box where I type answers by hand? The third one is a keyword bot in a nice costume, and you'll abandon it in your first busy week.
- What happens when I change a price at 2pm? If the assistant is still quoting the old price at 2:15, you now own a machine that starts arguments with customers.
- What does it say when it doesn't know? The right answer is a short, human "let me check that and come straight back to you", followed by a real handoff. The wrong answer is an invented delivery date.
Ask for the transcripts too. If a vendor can't show you a plain log of every conversation the AI had, you're being asked to trust a black box with your reputation. Sellers whose assistants get better month over month all do the same unglamorous thing: they read the logs and patch the gaps. That habit is what turns a bot into an assistant you raise over time.
"Supports Persian" is a checkbox. Speaking it is not.
If your customers write in Persian, Arabic, Turkish, or a mix of their language and Latin letters, this is where most tools fall apart, and the marketing page will tell you nothing about it.
Test it with the real thing: slang, no punctuation, half-typed words, Finglish, numbers written both ways. Then read the reply out loud. Does it sound like a person from that market, or like English wearing a translation? Customers can feel a translated sentence instantly, and it reads as a foreign call centre instead of the shop they follow.
Check the register while you're there. A jewellery shop and a supplement shop should not sound the same. If you can't set the tone — warm and short, or formal and thorough — you're renting someone else's voice.
One brain, or four bots wearing your logo
A customer sees your story on Instagram, asks a question there, then messages your WhatsApp two days later to actually order. If your assistant treats those as two strangers, you've automated confusion.
So ask specifically: is the knowledge shared across channels, or do I set it up again per channel? Do Instagram comments turn into DMs? Is there a widget for my site? If you sell through a Telegram channel, does it handle the follow-up there?
The honest version of this question is simpler: when my return policy changes, how many places do I have to update? If the answer is more than one, that policy will go stale, and stale answers cost sales.
The handoff is the product
Most of the DMs landing in your inbox are not buyers. They're browsers, greetings, and the same twenty questions you've answered a thousand times. Automation is worth paying for because it clears that noise. But the moment a real buyer shows up, everything depends on the handoff.
Run this test during your trial. Have a friend message you and ask for something the assistant can't do. Then watch:
- Does the customer get told a human is coming, or just silence?
- Does the notification reach you somewhere you'd actually see it at 9pm?
- When you start typing, does the AI shut up? A tool that keeps replying over you is a liability.
- Once you're done, does it pick the conversation back up, or is that chat manual forever?
Most tools do the first half well and the second half not at all. That gap is exactly where the unanswered questions that quietly kill your sales live.
Read the pricing like a seller, not like a shopper
Pricing pages are designed to look cheap at the size you are today. Do the arithmetic for the size you'll be during your busiest week instead.
Say you get 40 DMs a day. That's roughly 1,200 conversations a month, and a plan capped at 1,000 will break on exactly the day a post does well. So find out: what's the unit — messages, conversations, or contacts? What happens when you go over: a bigger bill, or a bot that goes silent mid-sale? Is the trial long enough to cover one campaign day, or does it expire during your quiet week?
Then compare the price against the thing it replaces, not against zero. That means the cost of an extra pair of hands, which is the calculation in the math worth doing before you hire another support person. An assistant that costs about three hours of staff time a month and answers 300 messages is not expensive. A cheap one that mishandles two buyers is.
Red flags I'd walk away from
- Screenshots instead of a live demo. If they won't let you type into it yourself, right now, there's a reason.
- "99% accurate" with no account of the other 1%. Accurate at what? Measured how? Ask, and watch the answer wobble.
- No conversation log. You can't improve what you can't read.
- Setup that means writing every answer yourself. That's a rules engine sold at AI prices.
- An annual contract pushed before your first real day. A tool confident in itself will let you survive one busy Thursday first.
- No off switch per conversation. You need to take over instantly, without hunting through settings.
A 30-minute test you can run this week
- Copy five real DMs from your inbox, exactly as customers wrote them.
- Load your actual catalogue — the messy one, with the two products that look alike.
- Paste all five messages. Note one thing per reply: would I have sent this?
- Change one price, wait five minutes, ask about it again.
- Force a handoff and time how long it takes before you're notified.
- Read the transcripts afterwards. If you're bored, that's a good sign. Boring means nothing went wrong.
Do that with two tools and the decision usually makes itself. Whatever you pick, pick the one that fails politely, because every assistant eventually meets a question it can't answer, and how it behaves in that moment is the part your customer remembers.
If you want to run that test against something built for exactly this — sellers on Instagram, WhatsApp, Telegram and their own site, one shared knowledge base, a human handoff when it matters — that's what we built Vardast to be. Bring your five ugliest DMs.