OpenAI CEO Sam Altman appeared on Cleo Abram’s podcast in August where he said the company was “tempted” to add sexual content in the past, but resisted, saying that a “sex bot avatar” in ChatGPT would be a move to “juice growth.” In light of his announcement last week that ChatGPT would soon offer erotica, revisiting that conversation is revealing.
It’s not clear yet what the specific offerings will be, or whether it’ll be an avatar like Grok’s horny waifu. But OpenAI is following a trend we’ve known about for years: There are endless theorized applications of AI, but in the real world many people want to use LLMs for sexual gratification, and it’s up for the market to keep up. In 2023, a16z published an analysis of the generative AI market, which amounted to one glaringly obvious finding: people use AI as part of their sex lives. As Emanuel wrote at the time in his analysis of the analysis: “Even if we put ethical questions aside, it is absurd that a tech industry kingmaker like a16z can look at this data, write a blog titled ‘How Are Consumers Using Generative AI?’ and not come to the obvious conclusion that people are using it to jerk off. If you are actually interested in the generative AI boom and you are not identifying porn as a core use for the technology, you are either not paying attention or intentionally pretending it’s not happening.”
Altman even hinting at introducing erotic roleplay as a feature is huge, because it’s a signal that he’s no longer pretending. People have been fucking the chatbot for a long time in an unofficial capacity, and have recently started hitting guardrails that stop them from doing so. People use Anthropic’s Claude, Google’s Gemini, Elon Musk’s Grok, and self-rolled large language models to roleplay erotic scenarios whether the terms of use for those platforms permit it or not, DIYing AI boyfriends out of platforms that otherwise forbid it. And there are specialized erotic chatbot platforms and AI dating simulators, but what OpenAI does—as the owner of the biggest share of the chatbot market—the rest follow.
Already we see other AI companies stroking their chins about it. Following Altman’s announcement, Amanda Askell, who works on the philosophical issues that arise with Anthropic’s alignment, posted: “It’s unfortunate that people often conflate AI erotica and AI romantic relationships, given that one of them is clearly more concerning than the other. Of the two, I’m more worried about romantic relationships. Mostly because it seems like it would make users pretty vulnerable to the AI company in many ways. It seems like a hard area to navigate responsibly.” And the highly influential anti-porn crowd is paying attention, too: the National Center on Sexual Exploitation put out a statement following Altman’s post declaring that actually, no one should be allowed to do erotic roleplay with chatbots, not even adults. (Ron DeHaas, co-founder of Christian porn surveillance company Covenant Eyes, resigned from the NCOSE board earlier this month after his 38-year-old adult stepson was charged with felony child sexual abuse.)
In the August interview, Abram sets up a question for Altman by noting that there’s a difference between “winning the race” and “building the AI future that would be best for the most people,” noting that it must be easier to focus on winning. She asks Altman for an example of a decision he’s had to make that would be best for the world but not best for winning.
Altman responded that he’s proud of the impression users have that ChatGPT is “trying to help you,” and says a bunch of other stuff that’s not really answering the question, about alignment with users and so on. But then he started to say something actually interesting: “There’s a lot of things we could do that would like, grow faster, that would get more time in ChatGPT, that we don’t do because we know that like, our long-term incentive is to stay as aligned with our users as possible. But there’s a lot of short-term stuff we could do that would really juice growth or revenue or whatever, and be very misaligned with that long-term goal,” Altman said. “And I’m proud of the company and how little we get distracted by that. But sometimes we do get tempted.”
“Are there specific examples that come to mind?” Abram asked. “Any decisions that you’ve made?”
After a full five-second pause to think, Altman said, “Well, we haven’t put a sex bot avatar in ChatGPT yet.”
“That does seem like it would get time spent,” Abram replied. “Apparently, it does.” Altman said. They have a giggle about it and move on.
Two months later, Altman was surprised that the erotica announcement blew up. “Without being paternalistic we will attempt to help users achieve their long-term goals,” he wrote. “But we are not the elected moral police of the world. In the same way that society differentiates other appropriate boundaries (R-rated movies, for example) we want to do a similar thing here.”
This announcement, aside from being a blatant hail mary cash grab for a company that’s bleeding funds because it’s already too popular, has inspired even more “bubble’s popping” speculation, something boosters and doomers alike have been saying (or rooting for) for months now. Once lauded as a productivity godsend, AI has mostly proven to be a hindrance to workers. It’s interesting that OpenAI’s embrace of erotica would cause that reaction, and not, say, the fact that AI is flooding and burdening libraries, eating Wikipedia, and incinerating the planet. It’s also interesting that OpenAI, which takes user conversations as training data—along with all of the writing and information available on the internet—feels it’s finally gobbled enough training data from humans to be able to stoop so low, as Altman’s attitude insinuates, to let users be horny. That training data includes authors of romance novels and NSFW fanfic but also sex workers who’ve spent the last 10 years posting endlessly to social media platforms like Twitter (pre-X, when Elon Musk cut off OpenAI’s access) and Reddit, only to have their posts scraped into the training maw.
Altman believes “sex bots” are not in service of the theoretical future that would “benefit the most people,” and that it’s a fast-track to juicing revenue, something the company badly needs. People have always used technology for horny ends, and OpenAI might be among the last to realize that—or the first of the AI giants to actually admit it.
From 404 Media via this RSS feed
True as that may be, this just feels like desperation.
I mean it definitely is. I am of two minds about it:
- Sam Altman in this as so many other things clearly has a toddler’s understanding of business. This is going to be a death knell for any use of his software other than for porn, and it seems unlikely that the “thirsty porn fiend who is willing to pay for the privilege” market is large enough to amount to more than fuck-all (or even enough to replace the pretty minuscule number of people who were paying for the not-porn before.)
- What the hell, they’re losing money on an absolutely historic scale right now, so changing course to literally anything is the only way they can survive.
In a way I think this is the perfect context to understand why AI is at the heart of it a process of dehumanization.
The problem with adult content is that almost everybody wants porn to some degree (I don’t care what you say, it is what you do…) but almost nobody wants to treat the humans making the porn like humans doing actual work.
I mean, this whole conversation is absurd if it doesn’t start with the way that companies like Pornhub built their entire business on literally stealing videos from adult content companies and uploading them to their site. Because it was porn nobody wanted to talk about it in a public fashion and the industry just got more and more toxic.
With erotica things are a bit more acceptable, but I think it is important to frame everything from that context. AI is an invitation into dehumanizing our sexuality further, to alienating our kinks that confuse us further, consigning them to spaces of pure vice that only interface with broken robots, to disconnect our desires from our empathy further…
I am not saying talking naughty with an LLM is bad, I could care less, what I am talking about is the way this is being used to swallow up the entire conversation around adult content and sexuality. These very same rich oligarchs are part of an advancement of conservatism and conservative values across the english speaking internet, to act like they should be the ones even being interviewed about topics of sexuality and self expression is absurd and insulting (not intending to suggest this article is).
A central point of this conversation is that you can’t ignore the elephant in the room of erotica and porn when talking about LLMs and chatbots… but WE DID ignore porn when having a conversation about the internet!!? We never had it, we never really advanced as a society to treating adult content creators as human beings… so yes… yes we can ignore porn in the conversation about LLMs and chatbots actually? I am not saying we should, but do not underestimate our collective capacity to ignore elephants in rooms.
This is one of those trends that disturbs me because it must be seen in a context, we must see this as part of an advancing moral conservatism even though it doesn’t seem like it on the face of it. In order to oppress a society the forces of oppression must first convince the people in it to oppress some part of themselves and deem it unnatural. AI wading into adult content is a form of that, especially since it isn’t as if monetization from adult content on the normal internet by actual human beings was ever really figured out and embraced by the broader economy, which these AI people are acting like did happen by suggesting erotica LLMs as an avenue for massive profits.
To summarize my long ass thread, imagine if music was considered morally condemnable but everyone secretly liked it so there were shady places you could go online to get music but nobody ever got around to accepting into society musicians who were open about making a living creating music. Then imagine AI came around and suddenly AI businessmen were bragging about how much money they were going to get openly selling massive amounts of AI created music on the internet. This is EXACTLY what is happening with adult content.
I wanted to commend you on your deconstruction of this issue. Phenomenally put.
What you wrote should be more commonly shared as a broad discussion.
Dehumanisation is never a problem to overlook.
If the core product was to eliminate basic thinking then is this one going for the genocide?
So uhh who’s idea was it to make EpsteinBot that will have everyone’s dirty fantasies to blackmail them with?
Today i saw ChatGPT chat after an along while and it actually seems like they are are correcting course as they realize that it sucks in giving you accurate answers. And with the dwindling amount of accurate training data this won’t change anytime soon i think. Instead of it saying that it was wrong about smth it just says it misspoke??! And it’s just so “nice” to you—agreeing with any shit you tell it.
Yeah. It used to own up that it wasn’t an “AI” but just a demo of a large language model. Those innocent days are long gone now, and they still haven’t figured out how to make an AI, it’s just that a whole industry has decided to make ever more cunning rigged demos for people to buy into.
Isn’t this just a reaction to what Elon Musk does?? I mean the article isn’t wrong but I’m missing some context here. Especially how erotic roleplay has some history and isn’t a new thing. There were millions of people using Replika AI. And there’s even more history. And then how all major AI companies started out exactly how we’d expected a prudish American company to handle it, and that is to crack down hard on smut. They made the chatbots refuse to respond, eradicated pornography from the datasets and jumped though all sorts of hoops to make it impossible with the big, commercial services. And now turns out Elon Musk is a gooner and has some waifus and now everyone thinks about allowing erotica? That’s just weird.
Yeah. I cannot imagine, based on OpenAI’s post-GPT4 performance, that they’ll do as good a job at goonbots as the people who have been doing it for longer with more talent applied to it. I think it will just be another baffling stage in the tragic downfall.
(One important asterisk - if they can convince Trump’s US government to give them unlimited amounts of money for some pretend product, which does seem plausible, I think they can survive. I don’t think they can continue their present “losing money by the bargeload” incarnation for much longer without that.)
I don’t think they can continue […] for much longer […]
Good luck with that prediction 😆 I’ve been wrong too many times predicting what’s going to happen to the AI economy. They’ll find ways to cough up multiple trillions of dollars to burn through, send $10 on a merry-go-round and pay the $10 million electricity bill with it… It’s just beyond my comprehension. But seems they’re really good at it.
