Skip to main content

My AI agent is giving wrong answers

The three ways an AI agent goes wrong (stale facts, improvised answers, unwanted behavior) and the fix for each.

When your AI agent says something wrong on a call, the fix depends on which kind of wrong it is. There are three kinds, and they have three different fixes. Diagnose first, then fix.

Which kind of wrong is it?

What happened

The type

The fix

It said your old store hours or an old policy

Stale knowledge

Update the source, then re-check

It made something up when it could not find an answer

Missing guardrails

Add behavior guidelines

It answered with real data, but not the way you want

Behavior preference

Request a configuration change

Type 1: stale knowledge

The agent's facts come from your connected store, your website, and the knowledge base sources in your workspace. If a fact changed recently and the agent has not caught up, the source it reads has not refreshed yet.

Knowledge base

What to do:

  1. Find which source holds the fact (store settings, website page, or a custom text entry)

  2. Update it at the source

  3. Make a test call and ask the exact question a customer would

Tip: for facts that change often, a custom text entry in the knowledge base beats relying on a website page, because you control it directly.

Type 2: improvised answers

This is the risky one. A customer asks something the agent cannot look up, and instead of saying so, it guesses. Real example: a customer read out a confirmation number, the agent could not match it, and it speculated that the order might be fraud. That turned a routine call into a scare.

The fix is behavior guidelines: plain instructions that fence off risky ground. Open your agent and add guidelines like these:

  • "If you cannot find an order, never speculate about why. Say you will have a teammate check and arrange a callback."

  • "Never discuss fraud, chargebacks, or bank disputes. Offer to transfer to a person."

  • "Never give medical or health claims about products."

  • "If you are not sure, say you are not sure and offer a follow-up by text."

Agent settings

Type 3: right data, unwanted behavior

Sometimes the agent is technically correct, but you want it handled differently. Example: recommending products that are out of stock. These are configuration requests. Tell us the behavior you want and we will set it up.

Always finish with a test call

After any change, call your own number and ask the question that went wrong. Thirty seconds of testing beats finding out from a customer.

When the agent gets your hours wrong

If the agent tells callers you are closed when you are open, or refuses a transfer inside its configured window, check all three places, in this order:

  • Workspace business hours (Settings, Business hours). This is the one people miss. The agent reads these, so they can override what your knowledge base and guidelines say.

  • Knowledge base sources for a conflicting or outdated set of hours

  • Behavioral guidelines for transfer-window instructions that disagree with the settings

Checking only the knowledge base and guidelines is the common dead end. If those look right and the behaviour continues, the workspace setting is usually the cause.

When the agent quotes the wrong price

Wrong prices are the most damaging kind of wrong answer, because a customer can read it as bait and switch. Prices come from your knowledge base and your synced catalog, so a price that changed on your side but not in your sources will keep being quoted. When a price changes, update the knowledge base the same day, then place a test call and ask for that exact product.

Related articles

Did this answer your question?