What are you missing when user research stops at launch?
Beta users are recruited to test something, and they understand why they’re doing it. They know they’re helping identify problems. When they encounter friction, they often work through it because they’re committed to the test.
Real users encounter a live service because they need it. They’re motivated by necessity, not by helping the process. When the service doesn’t work easily, they don’t report the problem — they abandon it and find workarounds. When a real user stops using the service, that moment tells us something useful and important.
These are fundamentally different contexts, and the behaviours they produce are foundationally different too. Many development teams stop researching once a service goes live, losing access to the most valuable data; how real users actually behave at scale.
What can happen next is often discovered months down the line through support data, complaints, and operational workarounds, not through active research. By then, the opportunity to iterate and improve has usually passed.
Why you should keep watching
One of our government clients faced this dynamic in practice. They had a service that went live after thorough beta testing. It worked well for users who met the standard criteria and had straightforward needs. For others – those with complex circumstances, incomplete documentation, or conflicting eligibility signals – the service broke down.
In beta, these cases were visible but sparse. In live, across thousands of users, they became common. The client team noticed the pattern in support data but had already moved on to new projects. No one was running user research in live or testing whether changes would help. Consequently, the service sat in a state where it worked, but with significant friction for a minority of users who required access the most.
When we stepped in, we continued research through live, looking at where people stopped and talked to users who had failed to complete the service. We discovered that things we thought were edge cases in beta were actually more common than expected. This understanding guided where we focused our subsequent testing. Changes to eligibility questions, clearer error messages, and earlier prompts for missing information made a tangible difference. As a result, complaints reduced and completion improved as the service became noticeably easier to navigate for people in difficult circumstances.
Observing real users
When you build time and capacity after launch to continue testing with real users, patterns emerge that recruited testers never reveal. You see where people run into obstacles or actually stop. You understand what friction is too high and you refine based on evidence, not assumptions.
When helping establishments meet the GDS Service Standard, this phase is non-negotiable. You’re testing how people access the service and whether it is easy for them to use but now with real users, at real scale, with real consequences. Where users stop or give up is as revealing as where they progress.
For government, this matters beyond usability. A service that works for most people but creates barriers for those who need access most is a compliance and equity problem. It generates failure demand, escalations, and workarounds. Continuing research in live means you catch these patterns early, when small iterations can still move the needle.
