Synthetic Users Don't Speak Egyptian: Where AI Respondents Fall Short for MENA Research

Synthetic users are AI-generated respondents that answer surveys and interview questions as if they were your target users. For a startup with a small research budget, they're tempting: answers in minutes, no recruiting, no incentives. In MENA, where research budgets are often the first thing cut, I hear the question more and more: "Can't we just ask the AI?"

NN/g recently compared synthetic respondents with a survey of 604 real people. Both reached the same headline conclusion, but the synthetic answers collapsed into far fewer distinct responses. The variation was gone. The study didn't look at different languages or regions, and that's exactly where I'd expect the gap to get wider.

What the NN/g study found, and what it didn't test

The finding in short: the synthetic and human groups agreed on the main conclusion, but the synthetic group gave a narrower, more uniform set of answers. Real people disagree, misunderstand questions, and bring up things you didn't ask about. Synthetic respondents mostly don't.

What the study didn't cover is language and region. Large language models learn mostly from English text, and much of their Arabic content is Modern Standard Arabic rather than the dialects people actually speak. So when you ask a model to answer "as a 35-year-old mother in Cairo", you get a plausible average. Plausible averages are the least useful thing in research.

Why MENA users are a long-tail problem

"MENA users" isn't one group. A product that works in Riyadh can fail in Cairo for reasons that have nothing to do with the interface:

• Dialect and wording. The same feature name can sound natural in Egypt and odd in the Gulf.
• Trust and payments. Cash on delivery, card trust, wallets and installments vary a lot by market and income.
• Family decisions. Big purchases are often discussed with family, which changes how people compare and commit.
• Device and connectivity. Older Android phones, shared devices and limited data plans shape behavior.

These are long-tail differences, and long tails are exactly what synthetic respondents smooth away.

Where synthetic respondents are useful

I don't think they're useless. They're a good thinking partner before research starts:

• Drafting hypotheses. Generate a wide list of possible objections and motivations, then decide which to test with real people.
• Writing interview scripts. Dry-run your questions to find confusing or leading wording.
• Testing survey wording. Check that answer options cover the obvious cases before you send the survey.
• Preparing stakeholders. Show the client the range of assumptions the team is making, so real research has something to confirm or overturn.

Where they mislead

• Dialect and tone. They tend to produce polished, neutral Arabic, not the language your users use.
• Cultural norms. They give the expected answer, not the real one. Real users often say one thing and do another, and that gap is what you're looking for.
• Payment and trust behavior. These depend on personal experience with fraud, delivery failures and banks, which a model can't have.
• Validation. If a decision depends on "users said yes", synthetic users can't provide that evidence.

A lean hybrid plan for small budgets

For most early-stage clients, this plan costs a fraction of a full study and keeps real users in the loop:

1. Day 1: Use synthetic respondents to draft hypotheses and an interview script. Two hours.
2. Days 2–4: Recruit five to eight real users from the main market, split across the segments that matter (for example, two cities or two income groups).
3. Days 5–7: Run 30-minute remote interviews in the users' own dialect.
4. Day 8: Compare what real users said with the synthetic drafts. The differences are your findings.

[Waleed: add a real example of a small comparison you ran, or plan to run, with results.]

Explaining the trade-off to a client in one slide

When a client asks why we can't skip real users, I put it simply:

Synthetic users tell us what people would probably say on average. Real users tell us what our users actually do. We use the first to prepare and the second to decide.

Then show the cost of being wrong next to the cost of eight interviews. That comparison usually settles it.

See it in real projects

Many of the products in my portfolio were designed for users across Egypt and the Gulf, where small, focused research rounds shaped the final design. If you're planning an MVP on a tight budget, read how I design MVPs.

Sources: Synthetic Users vs. Human Respondents: Same Conclusion, Less Variation (NN/g), The State of AI in UX & Product Design: 2026 (Designlab).

FAQs