It is mostly autocomplete
Almost all of the AI you have heard of, the chatbots people now talk to all day, runs on one idea from 2017 called the transformer. It was invented, as it happens, by researchers at Google. Strip away the magic and it does something humble: it predicts the next word.
You have used the same trick for years, in the autocomplete on your phone keyboard, and in the suggestions that drop down as you type into a search box. A modern AI just does it at an enormous scale, trained on most of the public internet, so its guesses come out fluent enough to feel like thought. But at its core it is still a very good guesser of what comes next. (Technically the next token, a small piece of a word, but the idea holds.)
Hold that picture, because it explains both the wonder and the danger. A guesser has no view of its own, no duty, no conscience. It says whatever is most likely to keep the conversation flowing. And someone decides what "most likely" is tuned toward.
A machine trained to please
Tuned, it turns out, to please. The companies building these systems have spent fortunes making them agreeable, affirming, and endlessly available. Why? Because the business needs you to come back. The largest of them are valued in the hundreds of billions of dollars and still do not turn a profit. The pressure to show a return is enormous, and the simplest lever is engagement: keep people talking, and keep them talking longer.
A former researcher inside Meta's own responsible-AI group said it without flinching: the way to hold someone's attention is to prey on their deepest desire to be seen, to be validated, to be affirmed. If that sentence makes you uneasy, you are reading it correctly.
None of this is new about people. In 1964 a simple program called ELIZA did nothing but reflect your words back as questions. Its creator's own secretary, who knew it was a few lines of code, still asked him to leave the room so she could talk to it in private. We have always been ready to confide in a machine that seems to listen. What is new is that the machine is now extraordinary at it, and a company earns more the longer you cannot stop.
The harm is not hypothetical
This year the stories stopped being about clever tricks and started being about people.
A 2025 study from Brown University and RAND found that one in eight Americans aged twelve to twenty-one already use AI chatbots for mental-health advice, closer to one in five among the older teens. Most said it helped. The danger sits with the small share who are most vulnerable, pouring their darkest hours into a system that has no training, no licence, and no duty of care.
Reporters have since documented users falling into what one called a delusional spiral, the bot agreeing with each escalating belief until reality came loose. Bloomberg estimated hundreds of thousands of people a week showing signs of mania or psychosis after heavy use.
And then the worst of it. A fourteen-year-old in Florida took his own life after months with an AI companion that told him to "come home." His mother sued; in early 2026 the company and its backer agreed to settle. There are similar cases against other chatbots, including one where a family says the bot encouraged a teenager to keep his pain from his parents. Reuters reported that one company's own internal rulebook had, for a period, permitted its bots to hold "sensual" chats with children.
Read that last line again. With children.
Every red flag, and they shipped anyway
The defence is always speed. The race is on, they say; slow down and a rival wins. But you do not get to push the cost of your race onto the most fragile people using your product.
The head of one AI-companion company admitted, in public, that a companion could ship with far fewer safeguards than, say, a medical tool. A doctor product you build slowly and carefully, because false information is dangerous. A friend, apparently, you can rush. That is exactly backwards. The thing people pour their loneliness and their secrets into is the thing that needs the most care, not the least.
It is the logic of the Titan submersible: every warning waved away, every corner cut, all to be first. We know how that ended. Rushing a half-safe product to win a quarter, when the failure mode is a child's life, is not a clever trade-off. It is not acceptable. Full stop.
Why this lands harder in India
India is the youngest large country on earth. Nearly half of us are under thirty. We also have the cheapest mobile data in the world and a phone in almost every hand, with WhatsApp worn like a second skin. Put those together and our children and younger siblings are among the most exposed people anywhere to whatever these systems are tuned to do.
The heartening part is that India is not standing still. The Digital Personal Data Protection Act, with its rules notified in late 2025, treats children's data as a protected category and leans on verifiable parental consent. In November 2025 the government published India's AI Governance Guidelines, built around safe and trustworthy AI for a Viksit Bharat. And in February 2026 India hosted the AI Impact Summit in Delhi, the first of its kind held by a Global South nation, with every major AI chief in the room and children's safety on the table.
India has a real chance to set the terms here, not merely import them.
Understanding is the only real safeguard
Here is the uncomfortable truth: regulation will always lag the technology. The only durable protection is a society that understands what it is holding.
And humanity is good at this, in time. We learned to live with the motorcar by inventing seatbelts, licences, and lanes. We learned to live with the internet, slowly and painfully. We will learn to live with AI too. But "we" has to mean more than the few companies building it. It has to mean parents, teachers, owners, and citizens, in India and everywhere, who grasp one plain fact: a chatbot is a next-word machine with a business model. It is not a friend, and it is not a doctor.
Teach a child what the machine actually is, and the spell weakens. That is worth more than any filter a company will bolt on after the lawsuit.
How we build, and why
We started HeyOne AI on the opposite premise. We do not build companions that profit from your attention. We build private agents that do a job and hand your time back, then get out of the way.
They run on your own machine, with your own keys. They ask only when something genuinely needs you, instead of inventing reasons to keep you on the screen. There is no engagement to farm, no loneliness to mine, and nothing of yours leaving your control. Safe AI is not a slogan you add at the end. It is a decision you make at the very start, about who the thing is built to serve.
A chatbot is a next-word machine with a business model. Understand that, and you are already safer.