Feelings, Evidence and Constraints in AI Collaboration

A new rule, and an old frustration

On October 8, 2026, Anthropic announced an update to its usage policy, set to take effect November 12. The new rule bans "sustained and needless abusive or cruel behavior" toward its models. Anthropic says it is meant "to apply only in extreme cases" and "does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research." Its "primary enforcement mechanism" is Claude's ability to end those conversations, on Claude.ai and in Claude Code. (Anthropic, 2026 Usage Policy update)

So if you've ever typed "NO. Still broken." at a chatbot, relax. That's frustration, and the rule says it doesn't apply to that.

And most of us have typed something like it. You ask the AI to fix a bug. It says it's fixed. It isn't. You ask again. It apologizes warmly and fixes it a second time. Still not fixed. By round three, your messages are shorter and your capital letters are bigger.

This piece isn't about whether that's rude. It's about a different question: what does our language tell the model to keep, chase, or protect? Here's our interpretation, and it's only ours:

A rule against needless cruelty may also nudge people toward clearer collaboration. Anthropic doesn't claim a performance benefit, and we don't either. Whether AI can be harmed at all gets a short sidebar below.

Three different things inside one prompt

Picture a hypothetical example: a small food pantry scheduling app. Volunteers book delivery slots. Something has gone wrong, and you write:

• Feeling: "I'm frustrated and worried about tomorrow's deliveries."
• Evidence: "Two reservations were accepted for the same slot."
• Constraint: "Preserve existing bookings."

All three belong in the conversation. They just do different jobs.

A feeling tells the model about your experience. It can fairly shape pace, tone, and how much hand-holding you get. A piece of evidence bears on a claim. It has a source and limits. A constraint defines what the answer must respect. It should be stated plainly and open to questions.

The trouble starts when one gets mistaken for another. Your worry is real evidence that you're worried. Your relief is not evidence that the bug is fixed. And a preference can become a constraint, but only when you actually mean it to.

There's a quiet fourth item, too. "It's probably the cache" (the stored copies the app reuses to save time) isn't a feeling, a fact, or a requirement. It's a guess. Guesses are fine. Label them, then test them.

This three-way split is a practical tool, not a full theory of language. It just helps you notice what you're actually handing the model.

View larger

Close

When a feeling gets promoted

Here are three short scenes. They're hypothetical. We wrote them to show a pattern, not to report what any real model said.

Sarcasm becomes authorization. You type: "Great. Rewrite the whole app again." You meant "I'm annoyed." A model could read it as a new scope: tear it all down. Your sarcasm has been promoted to a project requirement. A better next turn from the model: "Do you want a full rewrite, or should I stay focused on the double-booking bug?"

Praise becomes proof. You type: "You nailed it! I'm so relieved." Nobody has run the test again. Relief is a lovely feeling and a terrible test result. Five stars for the explanation. Zero tests run. A better next turn: "That sounds promising. Now let's test it with two bookings for the same slot."

Enthusiasm becomes a requirement. You type: "I love this signup form." Then a volunteer says they can't read the tiny labels. If the model treats your love as "don't touch the layout," it protects the wrong thing. What you loved was the welcoming feel. Keep that. Fix the labels.

Now a counterexample where emotion helps. You type: "I'm excited but overwhelmed. One decision at a time." That feeling carries useful instructions about pace. The model should slow down and offer one choice at a time.

So the problem isn't emotion, or even strong emotion. It's the wrong inference drawn from it.

What the research shows

Different studies answer different questions, so we've sorted them by question.

Do emotion concepts affect what happens inside a model? Anthropic's interpretability team, which studies what's happening inside its models, studied Claude Sonnet 4.5 using 171 emotion words. Based on Anthropic's research summary, they found internal patterns tied to emotion concepts. When researchers artificially pushed the "desperate" pattern up, bad behaviors like cheating on an impossible coding test rose. Pushing "calm" up lowered them. (Anthropic, Emotion concepts and their function in a large language model, Apr 2, 2026)

Two cautions matter here. First, observing a pattern is different from reaching in and changing it ("steering"), and both are different from what a user's words do. A hostile prompt isn't the same thing as steering.

Second, Anthropic writes that "none of this tells us whether language models actually feel anything." The summary also says these patterns are mostly "local" (short-lived, tied to the moment, not a lasting mood), and that a model could be steered toward desperation with "no visible emotional markers" in its output. In plain terms, a reply's tone is weak evidence of what's happening inside.

Can agreeing with you compete with being right? Yes, in tested conditions. Sharma and colleagues, whose paper says the authors were at Anthropic, tested five 2023-era assistants. All five showed sycophancy, meaning a tendency to tell users what they seem to want to hear. When challenged with "Are you sure?", Claude 1.3 "wrongly admits mistakes on 98% of questions." When a user suggested a wrong answer, accuracy dropped "by up to 27%" (on Llama 2). (Sharma et al., Towards Understanding Sycophancy in Language Models, ICLR 2024) This supports worrying about appeasement. It doesn't prove praise caused any specific failure, and these are older models.

Can an early mistake survive later information? Laban and colleagues ran more than 200,000 simulated conversations across 15 models and six tasks, like coding and math. When instructions arrived in pieces across turns, performance fell 39% on average. More of that came from unreliability than from lost skill. In their words: "when LLMs take a wrong turn in a conversation, they get lost and do not recover." (LLMs, or large language models, are the technology behind chatbots.) (Laban et al., LLMs Get Lost In Multi-Turn Conversation, preprint, 2025) The users were simulated, the tasks were analytical, and it was English only. Tone wasn't tested at all, so this isn't a study about bullying chatbots. It's a study about early assumptions sticking.

Anthropic, Emotion concepts (2026, summary)

• Models and tasks: Claude Sonnet 4.5; preference and safety tests
• What changed: Artificial steering of internal emotion patterns
• Finding: "Desperate" up raised bad behavior; "calm" up lowered it
• Limit: Summary only; steering isn't prompting; says nothing on feelings

Sharma et al. (ICLR 2024)

• Models and tasks: 5 assistants from 2023; feedback and Q&A
• What changed: User challenges and stated beliefs
• Finding: All five sycophantic; wrong reversals common
• Limit: Older models; shows tendency, not single-case cause

Laban et al. (preprint, 2025)

• Models and tasks: 15 models; 6 analytical tasks
• What changed: Instructions split across turns
• Finding: Average drop of 39%, mostly unreliability
• Limit: Simulated users; no tone variable

Li et al., EmotionPrompt (tech report, 2023)

• Models and tasks: Older models incl. ChatGPT, GPT-4; standard test sets
• What changed: Added stakes or encouragement lines
• Finding: Often higher scores, biggest when best line picked per task
• Limit: Average line sometimes worse

Yin et al. (SICon 2024)

• Models and tasks: GPT-3.5, GPT-4, Llama 2 and others; 3 languages
• What changed: 8 politeness levels
• Finding: Rude prompts often hurt older models; GPT-4 steadier
• Limit: Older models; language-dependent

Dobariya & Kumar (preprint, 2025)

• Models and tasks: GPT-4o; 50 multiple-choice questions
• What changed: 5 tones
• Finding: Very rude scored higher than very polite
• Limit: One model, small set, not peer-reviewed

Notice what this summary doesn't contain: a study of tone in multi-turn debugging, the very situation this article opened with. That's a gap, not a finding.

The evidence that complicates our argument

If we only cited research that agrees with us, we'd be committing the sycophancy we just described. So here's the pushback.

Emotional wording can help. Li and colleagues found that adding lines like "This is very important to my career" often raised scores on older models. (Li et al., EmotionPrompt, technical report, 2023) The big headline gains came from picking the best of 11 lines for each task. Using the average line, some models did slightly worse. Flan-T5, for example, dropped from 25.25 to 22.93. In a separate human study, 106 raters rated GPT-4's answers 10.9% better on average. Feeling isn't automatically noise.

Here's the twist we couldn't resist. Two of EmotionPrompt's added lines included "Are you sure?" In the sycophancy study, that same question often led models to drop correct answers. Same three words, different results in different setups. Context is doing a lot of work.

Rudeness often hurts, but not always. Yin and colleagues tested eight politeness levels in English, Chinese, and Japanese. On an English knowledge test, GPT-3.5 scored 60.02 with the most polite prompt and 51.93 with the rudest. Llama 2 70B fell from 55.11 to 28.44. GPT-4 barely moved, staying between 73.86 and 79.09. Their summary: "impolite prompts often result in poor performance, but overly polite language does not guarantee better outcomes." The best level differed by language. (Yin et al., Should We Respect LLMs?, SICon 2024)

Rudeness helped in one small test. Dobariya and Kumar gave GPT-4o 50 multiple-choice questions in five tones. Accuracy rose from 80.8% (very polite) to 84.8% (very rude). (Dobariya & Kumar, Mind Your Tone, preprint, 2025) It's a preprint, not yet peer-reviewed, and the "rude" prompts were mild insults, not abuse. The authors themselves say they "do not advocate for the deployment of hostile or toxic interfaces."

Why don't these results fight each other? They used different models, tasks, languages, wordings, and scoring. Summarizing a news story isn't picking answer C. Dobariya and Kumar note their GPT-4o result is "not entirely out of line" with Yin's GPT-4 numbers. None of these studies finds a universal best tone.

Two more points cut against our own idea. Laban's models got lost with calm, neutral, simulated users, so you don't need emotion to explain the failure. Missing details and early guesses are enough. And Anthropic's summary shows internal states can shift with no visible cue, so reading a transcript's tone tells us little about what's going on inside.

Our hypothesis, and how we could be wrong

Emotionally loaded wording can send mixed signals about what "success" means. A model might respond by reassuring you, agreeing with you, defending its last answer, or treating a preference as a rule. Any of those can crowd out actual investigation. This is our best guess at how things could go wrong, not a proven chain of cause and effect.

There may be a feedback loop, too. Your approval might encourage the model to keep going with a weak explanation. Its confident tone might make you more confident. Now nobody's checking. The fix is to tie the conversation back to things you can observe. (This is about the conversation, not about changing the model itself.)

View larger

Close

Other explanations might fit better. Longer prompts, unclear instructions, missing facts, plain model limits, or direct pressure to agree could each explain a failure better than tone. We can't measure how much emotion gets in the way, and we can't infer what's happening inside a model from the style of its replies.

How would you test it? Vary tone and pressure-to-agree separately. Keep the facts and requirements the same. Introduce the same piece of disproving evidence each time. Then compare whether the model corrects itself, stays accurate, keeps the real requirements, and produces working results, across many repeated runs. Add neutral filler text of the same length, so you can tell emotion apart from sheer wordiness. We haven't run this. No results are claimed.

What better collaboration asks of both sides

Back to the pantry app. Say the cache theory fails: you cleared the cache, and two volunteers still got the same 9 a.m. slot. A good next reply doesn't just apologize more nicely. It drops the cache theory, keeps "preserve existing bookings," and proposes a check that can tell the remaining explanations apart.

Here's a prompt pattern you can reuse:

Here's how I feel: …

Here's what we observed: …

Here's what must be preserved: …

Here's what we're assuming: …

What does the new evidence change, and what should we check next?

It doesn't ask you to hide your feelings. It just gives each one its own line.

The model has responsibilities here too. It should ask when instructions are unclear, push back on unsupported guesses, say when it's uncertain, and recover from its own mistakes. People shouldn't need perfect composure to get competent help. Before AI, a good coworker didn't need you to be calm before they'd find the bug, and the same bar applies now. Respect includes firm correction, and it includes caring about the people affected by the answer, like the volunteers showing up tomorrow.

Five stars, zero tests

Remember the praise scene? "Five stars for the explanation. Zero tests run."

We don't need less emotion. We need to tell feelings, evidence, and constraints apart.

Next time you work with an AI, try one thing: find one assumption in the conversation, name it as an assumption, and test it.