Why your Arabic assistant keeps handing over to a human

The complaint arrives in almost identical words every time. “We built an Arabic assistant. It works in the demo. In production it hands half the conversations to a human.” The instinct is to blame the model, and the model is rarely the problem. The problem is that Arabic is not one language, and most systems are built as though it were.

The short answer

Your assistant was almost certainly built and tested against Modern Standard Arabic — the Arabic of newspapers, contracts and school textbooks. Your customers write in Kuwaiti dialect, which differs from MSA in vocabulary, spelling, sentence shape and even in which letters get typed at all. The gap between those two is where handovers come from. It is an engineering gap, not a model failure, and it is closeable.

What actually differs between MSA and how people write

It helps to be specific, because “dialect” sounds like an accent and it is much more than that.

Vocabulary is different, not just pronounced differently. The everyday word a Kuwaiti customer uses for “I want”, “now”, “how much” or “good” is frequently not the MSA word at all. A retrieval system matching on MSA terms simply will not find the right passage.

Spelling is unstable. There is no single correct written form of a dialect. The same word arrives with different long vowels, with or without hamza, with ه where ة belongs. A system doing exact matching sees four different words.

Latin-script Arabic is common. A substantial share of messages arrive as Arabizi — Arabic typed in Latin letters with digits standing in for sounds, as in “shlonik” or “3ndkom”. To a system expecting Arabic script, that input is not Arabic at all.

Code-switching is the norm, not the exception. One message will carry an Arabic sentence, an English product name, a price in Western digits and an emoji. Each of those is a separate handling decision.

Questions are shaped differently. Dialect questions are often shorter, elliptical, and rely on context from three messages ago. MSA test questions tend to be complete sentences. The demo passes; the real queue does not.

Why the handover rate is the number that matters

Handover rate — the share of conversations the assistant gives up on — is the honest measure of an Arabic deployment, and it is the one most vendors will not show you. A system that answers confidently but wrongly looks better on a dashboard and is far worse in reality.

Track it as three separate numbers, because they have three different fixes:

Failure	What it looks like	Where the fix lives
Did not understand	Assistant asks the customer to rephrase, twice	Input normalisation and dialect coverage in retrieval
Understood, could not answer	Correct topic, no information to give	Knowledge gap — content, not language
Answered, but wrongly	Confident reply, customer corrects it	Retrieval quality and confidence thresholds
Refused by policy	Payment, complaint, anything binding	Correct behaviour — do not engineer this away

That last row is worth defending. A system that escalates a complaint to a person is working, not failing. Collapsing all four into one “handover” figure is how teams end up optimising the wrong thing.

This is a measured problem, not an anecdote

It is worth knowing that the pattern is documented across the region rather than peculiar to your deployment. Regional analysis of Arabic-first AI reports that a meaningful share of Arabic chatbot conversations still end in human handover, and that general-purpose models perform far worse on Arabic financial and business documents than their English benchmarks suggest — the consultancy.me analysis of Arabic-first AI in the Gulf is a reasonable summary of where the field stands.

The underlying reasons are structural. Arabic script carries optional diacritics, letters that change shape by position, and several characters that are visually and semantically near-identical in casual writing — the Unicode Arabic block contains multiple encodings that users treat as interchangeable and machines do not. Mixed-direction text adds another layer, governed by the Unicode bidirectional algorithm, which is why an Arabic sentence containing an English product code and a price can reorder in ways that surprise everyone who has not read the specification.

None of this is exotic or unsolved. It is simply work that has to be done deliberately, and that most deployments skip because the demo passed.

The five changes that move the number

1. Normalise input before anything else touches it. Strip diacritics, unify alef and hamza variants, unify ta marbuta and ha, collapse repeated letters used for emphasis, and map Arabic-Indic digits to Western for matching. This one step is cheap, deterministic and routinely removes a large slice of “did not understand”.

2. Detect and convert Arabizi. A short transliteration layer that recognises Latin-script Arabic and converts it before retrieval is a small piece of work with a disproportionate effect on message volume you were previously dropping.

3. Write the knowledge base in the language customers use. This is the change teams resist most, because internal documents are in MSA or English. But if your policy page says the MSA word and every customer types the dialect one, retrieval fails no matter how good the model is. Keep MSA for the formal record; add the dialect phrasing customers actually use.

4. Set a confidence threshold and honour it. Below the threshold, hand over — and say so plainly rather than guessing. The same principle that governs document extraction, which we set out in reading Arabic invoices with AI: a system that admits uncertainty is worth more than one that is confidently wrong.

5. Test on real messages. Take 200 actual customer messages from your own history, not invented ones. Every team we have worked with discovers that their real queue looks nothing like their test set, and the difference is always in the direction of messier.

What this does to cost

Two of these changes cut cost while improving accuracy. Normalisation reduces retries, and better retrieval means less context attached to each request, which is the largest single driver of model spend — the mechanics of which we laid out in what AI costs after launch.

One of them adds cost: human review of the handover queue, at least for the first weeks. Budget it. It is also where your best training data comes from, because every handover is a labelled example of something your system could not do.

What we would not promise

We would not promise zero handovers, and you should be sceptical of anyone who does. Some questions should reach a person: anything binding, anything involving money moving, anything where a customer is angry. The realistic goal is that the assistant handles the repetitive majority cleanly and hands over the rest with the context attached, so the person picking it up is not starting from nothing.

And we would not promise a number without seeing your messages. The honest first step is measurement, on a narrow slice of your real messages — including the version of the answer where your volume does not justify building one at all, by the arithmetic in which process to automate first.

If you already have an assistant that hands over too often, send us a few of the conversations it gave up on. The pattern is usually visible in ten of them.

Frequently asked questions

Do the big models understand Kuwaiti dialect?

They understand it far better than they did two years ago, and that is not the binding constraint. The constraint is usually retrieval: the model can read the question perfectly well, but the passage that answers it is written in different words, so nothing relevant is found to answer from.

Should we just reply to everyone in MSA?

Replying in clear MSA is generally safe and is what we default to for anything formal — it reads as professional rather than cold. Understanding dialect on the way in is the part that is not optional. The two directions are separate decisions.

How many example messages do you need to improve this?

A few hundred real messages is enough to see the pattern and fix the biggest causes. Thousands help with the long tail. What does not help is a hand-written test set, because it will quietly reflect how your team writes rather than how your customers do.

Is a dedicated Arabic model better than a general one?

Sometimes, and it is worth testing rather than assuming. In most deployments we have measured, normalisation, retrieval content and thresholds moved the handover rate more than swapping the model did.

What handover rate is realistic?

It depends entirely on how wide the question space is, so any number quoted without seeing your queue is marketing. What is reasonable to expect is steady measured improvement against your own baseline, with the residual handovers being the ones a person should have handled anyway.