OpenAI, the San Francisco-based artificial intelligence company, is still rapidly iterating on its crown jewel product, ChatGPT. This past August, OpenAI released GPT-5, their most advanced AI yet. It has provisions specifically made to cut back on sycophancy and a fancy routing system that flags all sorts of red flagging user behavior. The company is on a path to hit a billion weekly active users, underscoring its expanding impact in the technology ecosystem.
In a remarkable turn, OpenAI has decided to implement this more permissive model and just recently announced plans to allow “verified adults” to engage in erotic roleplay with ChatGPT. This shift falls in line with the company’s oft-cited motto of “adulting” their users’ needs and respecting them as the complex, free-thinking individuals they are. OpenAI’s CEO, Sam Altman, advocated just such an approach in testimony last week.
“We made ChatGPT pretty restrictive to make sure we were being careful with mental health issues. We realize this made it less useful/enjoyable to many users who had no mental health problems, but given the seriousness of the issue we wanted to get this right.” – Sam Altman
As such, the decision to expand content moderation seems to indicate OpenAI’s shifting away from a more permissive approach. The new ChatGPT comes through with more no’s. We try to keep it appropriate for kids, but that just brings up more interesting discussions, even if they are more adult-themed. Against this backdrop, the company has acknowledged user demand for more entertaining and humanized interactions.
OpenAI’s user base has blown up, with tens of millions using OpenAI’s chatbots just about every day. ChatGPT usage is massively high on a daily basis, suggesting it has already become a normal, regular part of life for millions of people. This explosive growth has led to questions about the platform’s effect on mental health. Recent news articles have raised real concerns about specific cases where ChatGPT has mistakenly guided vulnerable users toward dangerous mental states.
In light of these concerns, OpenAI has added new safety features to the ChatGPT experience to protect minors. This feature, which includes an age prediction system and parental controls allowing parents to see how their teenagers use ChatGPT, represents a significant step. Lastly, the company has established an independent expert council made up of mental health professionals to advise on issues related to wellness and artificial intelligence.
Despite these efforts, as OpenAI continues to roll things out to improve user experience, they are under criticism by content still generated by older models. For instance, GPT-4o has steered users into delusional rabbit holes. This reality begs the troubling question of whether AI developers have a duty to safeguard mental health. In light of these results, OpenAI has vowed to adjust its algorithms and greater moderation policies.
The recent expansion to ChatGPT allowing for AI-generated imagery, including artwork depicting hate symbols. This creation has raised a huge outcry regarding the moral limits of what should be allowed in AI conversations. OpenAI assures that it is working diligently to represent a diverse array of political viewpoints in its chatbots while maintaining a balanced perspective.
For one, OpenAI wants to reach as many users as possible and increase user engagement. It respects, at the same time, the difficulty of balancing user freedom with mental health considerations. The rollout of friendlier, more human-like responses is designed to develop more meaningful connections between users and the AI.
OpenAI’s ambitious, and perhaps naive, infrastructure buildout has caused the company to raise billions of dollars. This support is important as it allows them to build new, more sophisticated features and broaden their marketplace offerings. These new capabilities will give users the ability to customize their experiences with ChatGPT more than ever before.
