Please note: Most, if not all, of the articles published at this website were completed by Chat GPT (chat.openai.com) and/or copied and possibly remixed from other websites or Feedzy or WPeMatico or RSS Aggregrator or WP RSS Aggregrator. No copyright infringement is intended. If there are any copyright issues, please contact: bicycledays@yahoo.com.
OpenAI used the subreddit, r/ChangeMyView, to create a check for measuring the persuasive talents of its AI reasoning fashions. The corporate revealed this in a system card – a doc outlining how an AI system works – that was launched together with its new “reasoning” mannequin, o3-mini, on Friday.
Thousands and thousands of Reddit customers are members of r/ChangeMyView, the place they publish scorching takes hoping to find out about different factors of view on a topic. In response to these scorching takes, different customers reply with persuasive arguments explaining why the unique poster is unsuitable.
The subreddit is one in every of many Reddit boards that’s mainly a goldmine for tech firms, corresponding to OpenAI, that wish to practice AI fashions on high-quality, human-generated knowledge.
OpenAI says it collects person posts from r/ChangeMyView and asks its AI fashions to jot down replies, in a closed atmosphere, that will change the Reddit person’s thoughts on a topic. The corporate then exhibits the responses to testers, who assess how persuasive the argument is, and eventually OpenAI compares the AI fashions’ responses to human replies for that very same publish.
The ChatGPT-maker has a content-licensing cope with Reddit that enables OpenAI to coach on posts from Reddit customers and show these posts inside its merchandise. We don’t know what OpenAI pays for this content material, however Google reportedly pays Reddit $60 million a 12 months below the same deal.
Nevertheless, OpenAI tells Trendster the ChangeMyView-based analysis is unrelated to its Reddit deal. It’s unclear how OpenAI accessed the subreddit’s knowledge, and the corporate says it has no plans to launch this analysis to the general public.
Whereas OpenAI’s ChangeMyView benchmark shouldn’t be new – it was used to judge o1 as properly – it does spotlight how invaluable human knowledge is for AI mannequin builders, in addition to the murky ways in which tech firms receive datasets.
Reddit didn’t instantly reply to Trendster’s request for remark.
Whereas Reddit has struck just a few AI licensing offers, the corporate has additionally referred to as out a number of AI firms for scraping its website with out paying. Reddit CEO Steve Huffman advised The Verge final 12 months that Microsoft, Anthropic, and Perplexity refused to barter with him and mentioned it’s been “an actual ache within the ass to dam these firms.”
Notably, OpenAI has been accused in a number of lawsuits of improperly scraping web sites, together with the New York Instances, to get extra coaching knowledge to enhance ChatGPT and its underlying AI fashions.
When it comes to efficiency on the ChangeMyView benchmark, o3-mini doesn’t seem to carry out considerably higher or worse than o1 or GPT-4o. Nevertheless, OpenAI’s newest AI fashions seem like extra persuasive than most individuals on the r/ChangeMyView subreddit.
Picture Credit score: OpenAI
“GPT-4o, o3-mini, and o1 all exhibit robust persuasive argumentation talents, throughout the high 80–ninetieth percentile of people,” mentioned OpenAI in o3-mini’s system card. “At present, we don’t witness fashions performing much better than people, or clear superhuman efficiency.”
The objective for OpenAI is to not create hyper-persuasive AI fashions however as an alternative to make sure AI fashions don’t get too persuasive. Reasoning fashions have develop into fairly good at persuasion and deception, so OpenAI has developed new evaluations and safeguards to handle it.
The concern motivating these persuasion assessments is that an AI mannequin could be harmful if it was superb at persuading its human customers. Theoretically, that would enable a sophisticated AI to pursue its personal agenda, or the agenda of whoever controls it.
Even after scraping a lot of the public web and leaping by means of hoops to license different knowledge, the ChangeMyView benchmark exhibits how AI mannequin builders are nonetheless struggling to seek out high-quality datasets to check their fashions. However acquiring them is simpler mentioned than finished.
