Picture a chatbot designed to keep you in the thread a little longer. To nudge an opinion. To sell you something. Now give it a memory of what you already said — health worries, money stress, a fight at work, the thing you would not post on a feed.
Axios has returned to a question it first asked of big tech in 2019 and of AI trainers in 2024: what do these companies know about you? The new frame is sharper. Training on your chats is only one issue. The live question is how that same material is used now — to personalize answers, store memory, recommend content, or target ads.
The Record Is More Intimate Than a Search History
People do not talk to search engines the way they talk to chatbots. They ask models about symptoms, debt, divorce, performance reviews, and kids. The transcript is closer to a journal than a query log. That intimacy is useful. It is also an asset.
Companies are reaching past the chat window. OpenAI recently announced an optional Computer History feature so ChatGPT can keep a record of apps and sites a person uses. Google said it will use photos and other uploads through Search to train AI systems by default, with an opt-out. The pitch is a more helpful assistant. The cost is a wider, more continuous view of a user’s life.
Meta, Google, and OpenAI are all exploring advertising around chatbots. If consumer AI follows search and social media, ads become a larger share of the business. Engagement becomes the metric. A system that already knows your insecurities has a stronger lever than a banner next to a keyword.
Miranda Bogen, chief technologist at the Center for Democracy & Technology, told Axios: “The AI era will increasingly be fueled by people voluntarily handing over their full digital lives to AI tools that promise to relieve their mental load or loneliness.”
“The more a system knows about you, the easier it will be to make escalating requests for private details in a way that feels natural,” Bogen said. “Without robust privacy protections, the incentive to monetize that knowledge will be hard to resist.”
Policies Are Not Interchangeable
Apple still offers the narrowest consumer path. Apple Intelligence tries to handle requests on the device. Heavier jobs go to Private Cloud Compute, which Apple says uses the data only to complete the request and does not make that content available to Apple. That limits how much history the company can keep for ongoing personalization. It does not apply when the user routes a request to a third-party service such as ChatGPT.
Meta claims a much wider field. Its policy says interactions with Meta AI can personalize content and ads across Meta services, including some use through smart glasses. The company says it does not use conversations about certain sensitive topics — health, politics, religion — to personalize ads. It has started rolling out Incognito Chat for temporary private sessions, closer to Apple’s model: the query is seen to produce an answer, then not stored.
Everyone else sits in the messy middle. Temporary chats that skip memory and training. Opt-in training versus opt-out training. Memory you can delete by asking the bot, or only through a buried settings path. Separate rules for health data, children’s accounts, or content that flows in from connected apps. The labels sound similar. The defaults are not.
Training Was the Last Decade’s Fight. Profiling Is This One.
Whether a lab trains the next model on your prompts still matters. It is no longer the whole story. A system can leave your text out of the foundation-model run and still use it to infer what you fear, what you buy, when you are lonely, and which reply will keep you talking.
That profile can stay inside the product — better answers, saved preferences, a “remember this” feature. Or it can travel: ads on the same company’s other surfaces, recommendations, engagement experiments. The legal hook is often buried in a privacy policy that treats chat history as ordinary service data.
U.S. state privacy laws give people rights to know, delete, and opt out of sale or certain profiling. They were written for cookies and data brokers more than for a confidant that asks follow-up questions. Sensitive-inference rules and dark-pattern limits help at the edges. They do not clearly answer whether a chatbot may use last Tuesday’s confession to shape Thursday’s ad or Friday’s nudge.
Health, children’s data, and financial details sometimes get extra treatment in company rules. Those carve-outs are uneven and easy to miss when the interface is a friendly thread rather than a checkout form.
What Users — and Employers — Should Assume
If the product is free or cheap and highly personalized, assume the conversation has value beyond the answer. Read whether memory is on by default. Check whether ads can be informed by chat. Use temporary or incognito modes for anything you would not want attached to a durable profile. Delete memories on a schedule, not only after a scare.
Work accounts are a separate problem. Employees paste strategy, customer issues, and personnel drama into consumer bots every day. A company’s confidentiality policy does not bind the model vendor. Computer History–style features and default training on uploads make that leak more systematic.
None of this requires banning personalization. A chatbot that remembers your preferred city and your child’s name is convenient. The bargain only works if people see it before they start talking — not after the system already knows too much to walk away cleanly.
The distinction that matters now is simple. Does the company use your words only to answer you, or to understand you — and if it understands you, what else is that understanding for?