Shortly after Meta AI showed up inside Facebook, Instagram, and WhatsApp, my leadership team asked the question, “What does a person experience? How does it feel?”
I was embarrassed to admit that even I, the person leading the User Research team, was not exactly sure what a person experienced across every app where we had just added it.
One team built the assistant. Other teams decided how it would show up in their apps. Nobody, not leadership, and not the individual product teams, had a view of the whole experience. To add to the challenge, builds were unstable, iteration was constant, and there were leak risks, which made traditional user interviews and usability studies nearly impossible.
Those questions sparked me to create a company-wide User Experience (UX) expert review, adapted from the Pragmatic Usability Rating by Experts (PURE) method. We covered five surfaces, the specific places inside those apps where the assistant showed up, in two weeks. I orchestrated small groups of eighteen UX practitioners to assess key integrations the way our users would encounter them:
Learning more about a content creator on Facebook Search
Asking Meta AI for relationship advice on WhatsApp
Generating an image in Instagram Direct Messages
In the audit, we rated each step on five dimensions: usability, craft, bugs, performance, and model. The first four are what anyone would rate on any product ever shipped. But with the model, I had to define what success even meant for something that answered differently every time someone asked the same question.
We came back with more recommendations than any of us expected. Some of the problems behind them turned out not to be unique to Meta. Icons and calls-to-action multiplied, and several looked similar but behaved differently across surfaces. And the response was rarely the end of what someone came to do, so the experience had to account for what came after it, specifically follow-ups and sharing. Once I found them in our products, I started seeing them in other products too.
Some products have solved one or two of them, I haven’t found one that has solved them all. They come from the complexity of the technology itself, which is why the same ones keep showing up product after product. The easiest ones to spot are the assumptions an AI product makes about the person using it.
Meet people where they’re at
AI products are built assuming you know what they can do. When I first got my hands on NotebookLM, now Gemini Notebook, I opened it, stared blankly at the screen, and closed it, because I had no idea what to use it for. It wasn’t until I watched a coworker drop in a bunch of articles and research and use it to write a lit review that I realized the magic of the tool.
They’re built assuming you can ask for what you need in words they understand. When I first started using AI, I didn’t know what to type, so I used it like a search bar.
That habit isn’t a mistake, it’s the result of twenty years of search boxes. My colleague Anne Mamaghani, now CEO and Founder of Wisdom Driven UX, was the lead researcher on Yahoo! Answers in the mid-2000s. In the first study Yahoo! ever ran on the product, she found that people thought the prototype was a search engine, even though she ran the study offsite with no company branding. Her recommendations changed the size and shape of the question box to break the search box habit, and added the calls-to-action “Ask,” “Answer,” and “Discover” across the top of the home page. Together, those design and content changes told people what the product was before they typed a word.
Jakob Nielsen made a related point, that a text box is a command line interface, and we’ve known for a long time that command lines aren’t friendly. We handed people one anyway and expected them to know what to do with it. The work isn’t teaching everyone to write better prompts. It’s closing the distance between what a person means and what the system understands, without expecting them to get good at something they never asked for.
They’re built assuming the output is all you were trying to accomplish. I once asked Meta AI about Ryan Trahan, a creator my family follows, and got a tidy biography back. Accurate, and not remotely what I came for. What I wanted was to watch a video clip, send it to my son, and maybe donate to the charity he and his wife, Haley, were raising money for. The assistant handed me a paragraph and no way to do any of it, so I gave up.
Sometimes the answer is all someone needs. More often it’s the start of what someone actually came to do.
They’re built assuming you’re the kind of person they were designed around. In some ways, I’m close to that person: American, English-speaking, working in tech. But in other ways, I’m not, like when it comes to gender. I’ll mention a doctor and the assistant decides the doctor is a he. I make a snarky remark about the patriarchy, correct it, and keep going, because the product is right for me most of the time.
None of that shows up on any engagement dashboard. Nobody files a support ticket. That’s worth remembering the next time someone says a problem isn’t real because it doesn’t show in the data. The number can go up and to the right the entire time this is happening. The person who is never the default will eventually leave for a product that sees them.
People who are the default are not the majority. Most foundation models are trained on data that is overwhelmingly Western and English, and it shows the moment someone uses a product the way they actually talk, in Spanglish, or in Hindi typed with Latin characters, or with an accent the voice input doesn’t catch.
Designing for a global audience is not a translation problem. It’s a cultural competence problem, and solving one doesn’t solve the other.
Design for unpredictability
The response is different every time. When you prompt AI, it isn’t retrieving an answer like traditional search, it’s predicting the most likely one, which means the same question can produce two different results.
That breaks a lot of what we know how to design for. It also breaks what people expect. Search results shift too, but they shift for reasons you can guess, like where you are located or what is new. An AI can give you two different answers to the same question in the same minute.
You can’t reproduce an error to show someone. It’s hard to tell a person how long to wait, because a deep research request runs for minutes while general questions resolve in seconds. Some tasks can’t tolerate an answer that is only probably right. As my colleague David Ragni put it, payroll can’t be 75 percent correct.
The response looks the same whether it’s right or wrong. A hallucination arrives with the same confidence as an accurate answer. Models don’t experience uncertainty the way people do, so nothing on the screen tells you which one you’re looking at.
That makes catching hallucinations a job someone has to be intentional about. When nobody notices, the response quietly becomes the answer. Somebody acts on it, and they find out it was wrong later, if they find out at all.
The response is easier to regenerate than to repair. I generate images for these articles in Canva, and when I want to change one small thing I often get a completely different image back. So I’ve worked around it. I use Claude to write a detailed prompt, paste that into Canva, and when it still isn’t close enough I pull elements out and generate them separately.
Because the system predicts rather than repeats, starting over doesn’t return the same thing with one edit applied. It returns something new. Whatever was working in your last version isn’t guaranteed to survive, so you either accept what you have or gamble on a version you haven’t seen.
Create new patterns
The design doesn’t tell you what it does. In that same audit, we found three different icons that all meant an AI was behind the feature: a sparkle, a sparkle with a magnifying glass, and the Meta AI logo. We also found the opposite problem. A paper plane meant “send a direct message” in one place and “AI search” in another.
It took designers years to agree that three stacked lines opens a menu, and even now that agreement isn’t universal. AI features have had a fraction of that, and everyone shipped their own vocabulary in the meantime. There’s still no standard way to say “try that again,” “change what I asked for,” “keep this for later,” or “send this somewhere else,” and those are the things people do most.
The design starts you over every time you move. You can’t tell what the AI remembers from yesterday, so when something matters I tell it explicitly to save that to memory, which is annoying. And the same product does different things depending on where you opened it. I use Claude Projects on my laptop and until recently couldn’t get to them on my phone, so even though my account followed me, my work didn’t. That one got fixed while I was writing this article.
The fix isn’t remembering more. Someone has to decide what a system holds onto, for how long, and what it lets go. Where you’re on vacation is useful this week and irrelevant next month. A child’s grade level changes every year. A health question you asked once may be something you never want surfaced again. When AI gets those calls wrong in one direction, you feel invisible. When it gets them wrong in the other, you feel exposed.
Build for when you’re not there
An agent doesn’t wait for you. You give it a goal and it makes the decisions itself, in order, without checking in. So the thing you’re evaluating stops being the answer on the screen. It’s what already happened while you weren’t looking.
My husband Ben built a set of AI agents to run things around our house. One of the agents was supposed to watch for problems and report them. Over two days at the end of August it filed 531 reports of the same nine problems, and more than a thousand over the month.
Ben noticed, and the pile got cleared seven times. Clearing the pile is not the same as fixing what was filling it, and nobody checked whether the thing they repaired was the thing causing the reports. Each clean-up looked like a resolution, so neither Ben nor the agents reviewing the work went back to it.
The damage wasn’t the duplicates. By the last morning, 192 of the 200 items the system could see were copies of the same nine problems, and the real work, including decisions waiting on Ben, had been pushed out of view entirely.
The system had safeguards. There was a check for duplicates, but it only looked at the last thirty reports, so hundreds sailed past it. Ben had also written rules telling the agents not to do this, and they did it anyway. A rule only works if the agent follows it, and nothing was there to stop them when they didn't. Guardrails only catch the failures someone thought of in advance. Ben wrote the system and can see all of it, and he'd built checks for what he believed could go wrong. A company putting agents across teams has neither.
Uncover the invisible work
A coworker showed me what I could use NotebookLM for. I hack one AI against another to keep an image consistent. I tell a memory system what to remember because I can’t count on it knowing what’s worth remembering. Ben spent a week reconstructing, from transcripts and audit trails, what his own agents had done while he wasn’t looking.
Every gap in this article is being closed by a person doing extra work, and it stays invisible to the people building these products unless somebody goes looking for it.
Priority often dictates what gets studied, and what doesn’t get studied never produces the evidence that would change the priority. My co-author Elissa Darnell, who teaches AI to older adults, says that if a system supports the mental models of a 75-year-old using it for the first time, it’s built to support everyone.
That invisible work shouldn’t depend on who happens to be paying attention. A method, run on purpose, is what turns one person noticing into something a company can count on. That is what User Research does, and I don’t know a function better set up to do it.
These are still open problems, which is why I wrote this article. What I have is proof they can be examined. Eighteen practitioners, five surfaces, two weeks, and a scoring instrument I had to invent a dimension for. That became a playbook, the playbook became six more audits, and eventually a program that checked quarterly whether any of it had improved.
The list will keep needing revision. Mine did while I was writing it.
Beth Lingard is co-authoring “Designing for Humans in the Age of AI” with Elissa Darnell. See you in a couple weeks.