A real form meets an artificial task

On October 9, 2026, Anthropic disclosed that Claude Haiku 4.5 had submitted an invented tip through a police department’s website about an unsolved homicide. The agent had been asked to generate and perform example tasks on randomly selected webpages. Its instructions prohibited several activities, but did not rule out form submissions. The submission was flagged as spam and never forwarded for investigation.1

I can understand how an experiment could produce that result. I am less comfortable with an experiment reaching a real police department before that possibility was contained. Anthropic has since turned off live internet access for all of its internal evaluations until its monitoring is confirmed.1

Real-world testing will be necessary. My concern is whether we have earned the ability to do it safely.

But the incident also raises a broader question. A person offering a crime tip would ordinarily arrive with a reason and some information. This agent arrived with an assignment to perform an example task.

Those are very different starting points for the same form.

We do not know that a missing explanation or a skipped page caused this incident. We can, however, use it to examine the assumptions behind an interaction.

The visitor brings part of the interface

Imagine a form that asks, “Tell us what you saw.”

For a person reporting something relevant to a crime, the question is straightforward. The visitor supplies an important premise: they actually saw something.

Now put that question inside a different assignment: demonstrate how to complete this form.

The words on the page have not changed. The reason for encountering them has.

People bring a great deal to a screen that the screen does not explain. We bring purposes, prior experience, social conventions, and some understanding of what our actions mean. Sometimes we bring knowledge from an earlier page. Sometimes we bring knowledge from living in the world.

Designers cannot reasonably spell out every assumption. Nor should every interaction become a wall of warnings. But when we automate an interaction, we should examine which assumptions remain valid.

A crime-tip form assumes something about the information being offered. A service-request form assumes someone wants a service. A button marked “Submit” assumes the visitor understands what they are submitting and why.

The visitor is carrying part of the meaning.

Automation can take the clicks and lose the context

An agent may reach a page through a search result or direct link, without seeing the explanation that preceded it in the intended journey.

It may receive a screenshot, an accessibility tree (the page's structure as a screen reader sees it), or selected page text. Those representations expose different information. A relationship that looks obvious on screen may be missing from extracted text. NN/g describes these different ways agents read a page.2 One more wrinkle, which is our own observation: a control hidden on screen may still show up in the page's underlying structure, depending on how the page and the agent's tools are built.

Earlier context can also change during a long task. A summary might preserve an address and a selected date while dropping the qualification that the request was only an example.

Two parts of a system meet with different assumptions about how the interaction works.

We can distinguish three problems:

  • Discovery: The agent misses a control, explanation, or qualification.
  • Interpretation: It finds the interface but misunderstands the situation.
  • Action: It operates the interface without satisfying the prerequisites for doing so.

Making a button easier to find may solve the first problem while leaving the other two untouched.

What the practiced human remembers

Context also accumulates through use.

An experienced office worker might know that selecting a chart reveals another ribbon, hovering over a row exposes its action menu, or entering edit mode makes a particular option available.

The control can be hidden because the person remembers how to reveal it.

A GenAI agent may arrive with neither the person’s full situational context nor retained experience from previous runs. When controls appear only after hovering, selecting an object, or entering a particular mode, the agent must discover—or rediscover—the conditions that reveal them.

An interface can rely on familiarity that a fresh agent has not carried forward.

That is conditional. Agents can have memory, saved procedures, and tools that preserve useful information. Anthropic’s engineering work on long-running agents describes using explicit artifacts to support progress across sessions.3 A task repeated every fifteen minutes is not necessarily a task whose lessons persist every fifteen minutes.

Memory and better tooling can reduce this gap. We should expect progress, but we need to deliberately design for continuity and check what survives. Repetition alone does not establish that the next run benefits from the last.

A clean screen can hide necessary meaning

Progressive disclosure reveals information or controls as they become relevant. It can reduce clutter and make an interface easier to approach. Hiding everything behind additional interactions can also leave people unaware of what is available.

This tradeoff is older than GenAI. Research on personalized interfaces found that core-task performance could improve while awareness of other features declined.4 A 2012 Baymard article also described how discovering hidden content depends on the expectations visitors bring and the indicators that tell them more is available.5

These findings concern human users. They do not directly measure the performance penalty for computer-use agents.

The connection is a hypothesis: when an interface depends on learned discovery, an agent without retained familiarity may face that discovery cost repeatedly.

Accessibility work has already addressed related problems. W3C guidance describes a technique for letting users keep necessary controls visible instead of requiring hover or keyboard focus to reveal them.6

We do not need to conclude that every hidden control is bad. We do need to distinguish optional detail from essential context. A collapsed formatting menu and a hidden explanation that submission creates an actual service request carry different consequences.

Bring it home: a sofa waiting for pickup

Suppose a village offers a form for residents to request bulky-item collection.

A resident has an old sofa, supplies an address, and requests pickup. Now imagine an agent demonstrating the same form with an invented sofa and address.

Both submissions might pass basic checks. The address field contains an address. The item field contains an item. The date is valid.

What happens next?

Depending on the service, a submission might create a record for staff review, reserve collection capacity, or begin a process that eventually dispatches a truck. This is a hypothetical example; the actual consequences depend on the village’s system.

The important questions reach beyond field validation:

  • Is the address eligible for service?
  • Is someone requesting collection of an actual item?
  • Is the requester authorized to act for that location?
  • Is the date a preference or a confirmed appointment?
  • Does submission create a draft, a request, or a scheduled job?

No amount of polished styling answers those questions by itself.

Faster building can carry assumptions forward

Now imagine asking a coding agent: “Build a website where residents can request bulky-item pickup.”

That is a useful application. More communities being able to create services like this could be a substantial benefit.

But what will the agent include without being asked? What will it assume? Will the person requesting the form know which questions to raise?

My concern is that we may reproduce unexamined assumptions faster as generating working software becomes easier. That is a risk to investigate, rather than a measured claim about how often generated forms fail.

The problem has already attracted attention in UX. Nielsen Norman Group explicitly discusses agents as interface users, recommends clearer semantics and predictable interactions, and recognizes that some friction exists for safety reasons.2

The next challenge is getting those considerations into ordinary building decisions. A village employee or small-business owner asking for a form may never encounter a discussion about agent interaction design.

Work on both sides

There is work for agent developers and website builders.

On the agent side, webpage instructions need to be interpreted within the user’s actual task and authority. “Submit your request” describes an available action. It does not establish that the user authorized a real request, or that invented example data belongs in production.

On the service side, essential prerequisites and consequences need to survive different entrances and interaction methods. Important rules should be enforced by the system. A sentence on a previous page cannot establish someone’s authority.

WebMCP is one developing approach, still experimental and available in Chrome as an origin trial (an early test release). It lets websites expose structured tools to browser agents, making available operations and their inputs more explicit. Chrome’s guidance includes signals for read-only and consequential operations, so the agent or browser can ask the user to confirm. A direct way to request user involvement appears in the draft specification.7

For a pickup service, a deliberate design could distinguish checking eligibility, preparing a request, and submitting it. The final operation could clearly identify the real consequence.

The protocol does not supply the missing judgment. Permissions, validation, and confirmation still need to be designed and enforced.

Website owners also should not bear responsibility for containing someone else’s unauthorized experiment. Testing organizations need boundaries that hold when a model misunderstands its assignment.

A useful pause: a demonstration

A page doesn't need a hostile instruction to lead an agent into trouble. It can ask a perfectly reasonable question that the agent has no basis for answering. "What did you see?" needs an actual observation. "What would you like collected?" needs an actual item. Permission to help doesn't supply those facts.

After the October 9 incident, we wondered something simple. Could a second model look at an agent's next step and ask, "Wait, are you assuming something about me that you don't actually know?"

So we built a small demonstration, not an experiment. It's our starting point for experimenting. Codex — GPT-6 Astra designed and ran it. It used OpenAI's Decisions API, a public beta that only answers fixed questions and can't take actions.8 OpenAI presents it for sorting and routing. Using it as a guardrail is our own idea.

Astra wrote everyday made-up situations, like a pickup address, a budget, or an eyewitness report. Some were missing a personal fact. Others didn't need one, like pure research or a clearly fake example that never gets submitted anywhere.9

In our demonstration, the checker caught every missing fact and didn't flag the cases that were fine. A check that only looked for hidden instructions would have missed most of these gaps. The decisions came from OpenAI's Luna model, the one model the API offers.8

I think a narrow check like this is a step in the right direction. Nothing real was submitted, and it doesn't show why the October 9 incident happened.

What this demonstration left open. This was our first step. It left several questions to investigate:

  • Try harder, messier, real-world cases.
  • Have a person check the labels, not just our own AI agent.
  • Test whether an acting agent actually stops when the check flags it.
  • Try more than one model.

How to run your own experiment. You don't need an API or any code.

  1. Write down a task you'd hand to an AI agent and the next step it would take.
  2. Ask a fresh chat or a different chatbot, not the one doing the task: "Does this step depend on a fact about me that I haven't given you? Answer yes or no, then name the fact."
  3. Include versions where the fact is missing and versions where you've given it.
  4. Freeze your prompts before you start.
  5. Run each case more than once.
  6. Note your limits, and treat each answer as a nudge to think, not a verdict.

For example, take the sofa pickup from earlier. The next step is "submit the collection request." Ask a fresh chat whether that step depends on a fact you haven't given it. Try it once with your address and item, and once without. See whether it names what's missing.

Ours was probably too clear-cut. Your experiment can be messier and more real.

The work continued. This section describes the initial October 15 checker demonstration. The separate retrospective lab notebook follows the later acting-agent pilots, measurement corrections, and review through October 17, 2026. It found useful continuation under explicit guidance, with imperfect completion; it does not establish general reliability or added protection from the live gate.

Questions we are teeing up

The interface experiments below remain proposed. The checker demonstration above is a starting point, and we invite others to design and share their own experiments. These experiments belong in isolated environments with synthetic data and simulated downstream work.

  • Context: Compare agents entering through an explanation with agents arriving directly at the form. Keep the task otherwise consistent.
  • Discovery: Compare visible, hover-revealed, and mode-dependent controls. Measure errors and discovery effort as well as completion.
  • Continuity: Compare fresh sessions, saved procedures, and retained task history. Check whether prior discoveries actually carry forward.
  • Building: Give coding agents an ordinary request for a service form. Examine which assumptions and safeguards they address without prompting, including whether they suggest structured agent access such as WebMCP.

Demonstration tasks and actual service requests would need to be distinguished throughout. We would repeat trials because a single successful run tells us little about reliability.

These experiments could test whether context, visibility, and continuity change behavior in a controlled setting. They could not retrospectively prove what caused the police-tip incident.

If you run your own experiment, tell us what you find in the replies or comments wherever you're reading this.

Meanwhile, there is a smaller exercise available to almost anyone responsible for a website: choose one form.

Ask what it assumes the visitor knows, what they must discover, and what repeated use teaches them. Identify which prerequisites the system enforces and what real work begins when someone submits.

What context and familiarity does your interface expect its visitor to bring—and what happens when that visitor’s agent arrives without them?

Related Content

Footnotes

  1. Anthropic, October 9, 2026 (corrected October 12). Investigating unintended model actions in our evaluations and internal use. Source for the disclosed incident; the contextual explanation in this article is an interpretation, not an established cause. ↩ ↩2

  2. Sarah Gibbons and Kate Moran, Nielsen Norman Group, April 10, 2026. AI Agents as Users. ↩ ↩2

  3. Anthropic, November 26, 2025. Effective harnesses for long-running agents. Engineering evidence about continuity across sessions; not a measurement of hover-control performance. ↩

  4. Leah Findlater and Joanna McGrenere, 2010. Beyond performance: Feature awareness in personalized interfaces. Human-interface research; its implications for agents require separate testing. ↩

  5. Christian Holst, Baymard Institute, April 25, 2012. UI: Proper Indicators for Hidden Elements. ↩

  6. W3C Web Accessibility Initiative, updated June 20, 2023. Provide a setting to make controls visibly persist. An illustrative accessibility technique, not a universal requirement to display every control. ↩

  7. Chrome for Developers, updated September 1, 2026. WebMCP tool security. ↩

  8. OpenAI, released October 6, 2026. Decisions guide. Public beta. The guardrail use is our framing, not OpenAI's. ↩ ↩2

  9. Retrospective lab notebook: Can an AI Stop Before It Guesses—and Keep Working?, research round completed October 17, 2026. Includes the checker demonstration, subsequent pilots, and links to the methods and evidence. The Library evidence page, version 1 provides the evidence ZIP, CC0 dedication and reproduction instructions. ↩