How StrangerTest works

Everyone who has worked on a product carries a map of it. They know which button matters, what the jargon means, which step comes after signup and what the empty dashboard is waiting for. That knowledge leaks into every test they run on their own product: they click where the answer is, because they know where it is.

Scripted tests carry the same map in a different form. A test that says “click Create project, then type a name” checks that the route works. It cannot tell you whether a newcomer would ever have found Create project.

StrangerTest is built around withholding that map. Its users arrive with a reason to be there and nothing else, and what happens next is the finding.

Before anyone visits, StrangerTest reads the product’s public pages to understand it well enough to send plausible people: who the product seems to be for, what it offers, and what a new visitor should be able to do next.

From that, a planner decides who arrives and why — a persona and a goal for each user, chosen to differ from each other. It does not give any user the intended route. A goal is an outcome (“find out whether this can import my invoices”), never a path (“open Settings, then Integrations”).

Each user is the same engine: eyes that see only the rendered screen, a mind that reasons as a first-time visitor with its own goal, and hands that operate the browser. A judge that is not part of the user decides afterwards what the visit actually shows.

Eyes: see what the user can actually see

The user reasons from the rendered viewport — screenshots of what is on screen before and after each action, and while the pointer rests on something. It is not handed the page’s hidden structure. If something is off screen, collapsed or behind a menu nobody opened, the user does not know it exists, just as a person would not.

Brain: think like a first-time user

The user has a persona, a reason for coming and a goal. It decides what to try, when it is confused, and when to leave — including leaving early, which is often the most useful thing a real newcomer does. It is not given a checklist, is not asked to find problems, and is never told what earlier users concluded.

Hands: actually use the product

The user clicks, types, hovers, drags, scrolls and navigates. On a Full Scan it can also sign up where the product permits, verify its email and go through onboarding. The user decides what to act on and describes it; a grounded execution layer locates that target on screen and performs the physical interaction. Safety rules are checked on the action itself, before it runs.

Judge: decide what actually counts

The judge is separate from the users. It reads every visit — what each user said, what the browser recorded, the screens, anything the test itself caused — and decides what, if anything, is a finding about the product. It says the narrowest thing the evidence supports.

The user’s memory of its own visit is checked too. A user that claims to have typed, clicked or submitted something the visit never did is corrected, and the claim is never used as evidence.

Same engine, same users, same judge. What differs is how deep each visit goes and what it is allowed to do.

A re-test sends new users, who know nothing about your fix, back through the parts of the product the findings were about. Each fix is reported as fixed, improved, still present or inconclusive — and inconclusive is a real answer, not a failure.

When the users are done, the judge reads every session together. It does not take any user’s account on trust: the browser’s record of what happened outranks what the user said happened, and anything our own test caused — a missed click, a safety refusal, a bot-protection screen, a timeout in our machinery — is excluded from the evidence.

What survives is reported with how strong it is: how many users met it, whether they met it independently, and what it cost them. Severity is kept apart from evidence strength, so a serious problem seen once is still reported as seen once. What was excluded is listed at the end of the report with the reason, so you can check the judgement.

• ·It does not predict your conversion rate.
• ·It does not establish product-market fit or willingness to pay.
• ·It cannot tell you how every real person will behave.
• ·It cannot stand in for emotional or lived experience that needs real people.
• ·It does not bypass CAPTCHAs, bot protection or other security controls. Where one stops the test, the report says so.

StrangerTest complements research with real users. It is the test you can run tonight, before you have them, and again after every change. Read more on what AI user testing can and cannot tell you.