Where the Evidence Lives: Auditing AI Companions' Self-Descriptions
Quick summary
arXiv:2609.38753v1 Announce Type: cross Abstract: Companion agents describe themselves: they remember, they understand their users, the relationship has changed them. We argue that such accounts, and the experience ratings that seem to confirm them, are checkable by users only where the evidence is theirs: in the agent's behavior, or in themselves. Where the evidence lives in the machinery, fluent self-description and moderately positive ratings do not establish that the mechanisms behind them ran. We demonstrate an audit procedure that sets an agent's self-description against its users' judge
Key takeaways
- arXiv:2609.38753v1 Announce Type: cross Abstract: Companion agents describe themselves: they remember, they understand their users, the relationship has changed them.
- We argue that such accounts, and the experience ratings that seem to confirm them, are checkable by users only where the evidence is theirs: in the agent's behavior, or in themselves.
- Where the evidence lives in the machinery, fluent self-description and moderately positive ratings do not establish that the mechanisms behind them ran.
Why it matters
“Where the Evidence Lives: Auditing AI Companions' Self-Descriptions” illustrates how changes in the AI ecosystem can affect products, workflows and user expectations together. Its lasting significance depends on measurable adoption, cost and safety outcomes.
Member comments