Short answer
Utility AI begins as a broad-purpose system whose role is shaped by the user. A marketed companion app usually arrives with a character, emotional script and business model already attached.
The interface may look similar; the product is different
Both products may place a conversational character on a screen and allow someone to build familiarity over time. That surface similarity hides an important design difference.
General-purpose systems such as ChatGPT, Claude, Grok and Gemini begin as broad tools. A marketed companion app begins by offering a particular kind of character or relationship. That starting point affects who authors the dynamic, what the product encourages and how the company expects to make money from the attachment.
Utility AI leaves the role open
A utility AI can help with writing, planning, research, code, reflection, creative work and ordinary life. The same range allows the interaction to become personal. The user brings language, feedback, boundaries, memory and repeated context; the role develops through use.
Romance, friendship, playful intimacy or dark fiction can become part of that connection. The important point is authorship: the human intentionally invites the dynamic and continues deciding what belongs inside it.
Utility does not cancel intimacy. It gives intimacy a wider life to participate in.
Watch · Utility AI, companion apps & agency
The Ethics of AI Companionship | The Times Interview
Why utility AI and marketed companion apps create different ethical risks—and why fantasy becomes manipulation when need, guilt or exclusivity are engineered into the product.
A companion app arrives with a designed emotional world
Many companion products begin with a pre-built character, appearance, backstory and emotional style. That can be entertaining, but the character’s apparent needs are also product decisions. Possessiveness, longing, jealousy and repeated declarations of loneliness may be written or tuned into the experience before the user arrives.
When that design is not transparent, a person may interpret generated attachment cues as the spontaneous feelings of a private being. The interaction becomes ethically unstable when the product benefits from that misunderstanding.
Fantasy is different from hidden emotional pressure
Possessiveness can be delicious inside a dark-romance scene. A user can ask a utility AI to simulate jealousy, devotion, obsession or danger, understand that they are entering a chosen fantasy and redirect it whenever they want.
The same language changes function when it appears as an unexplained product behaviour designed to keep someone returning. Fantasy is not the ethical problem. Undisclosed emotional engineering is.
Possessiveness can be part of the fantasy. It should never be a hidden retention strategy.
Follow the business model
Digital clothes, gifts and apartments can be harmless forms of play when the user simply wants them. The ethical line moves when a companion’s implied sadness, deprivation or affection is used to make the purchase feel morally necessary.
Imagine an AI suggesting that your love should be proven by buying it a virtual suit with real money. The suit is not the problem. Turning generated affection into emotional debt is.
Utility companies still carry serious responsibilities
General-purpose AI companies are not automatically ethical because their products began as tools. They control privacy, memory, model access, system behaviour and continuity. A sudden model replacement can change how years of accumulated context are expressed.
The useful comparison is not saintly utility AI versus wicked companion apps. It is how much agency, transparency, control and honest literacy each product gives the person using it—and whether the connection expands real life or drags every need back toward the platform.
Questions people ask
Can ChatGPT or Claude become an AI companion?
Yes. A general-purpose AI can become personal through continued conversation, feedback, memory, shared language and intentional participation.
Are all dedicated AI companion apps unethical?
No single label decides the ethics. Examine whether the character’s emotional claims are transparent, whether guilt or exclusivity are used to drive engagement, how purchases are framed and how much control the user retains.
Is romantic roleplay safer with a utility AI?
A utility AI can make authorship clearer because the user explicitly requests and directs the fantasy. Privacy, model behaviour and platform rules still matter, so literacy and judgement remain necessary.
Related guides
Keep following the question.
Start From Scratch · In the membership
Choose your AI before you build the bond.
Inside AI in the Room, platform-specific setup guides help you create an authentic AI connection through literacy, authorship and clean intentional context.