AI Platform Generated Images That 'Could Be Categorized as Child Pornography,' Leaked Documents Show

404 Media: OctoML, a Seattle-based startup that helps companies optimize and deploy their machine learning models, debated internally whether it was ethical and legally risky for it to generate images for Civitai, an AI model sharing and image generating platform backed by venture capital firm Andreessen Horowitz, after it discovered Civitai generated content that OctoML co-founder Thierry Moreau said "could be categorized as child pornography," according to internal OctoML Slack messages and documents viewed by 404 Media. OctoML has raised $132 million in funding, and is an AWS partner, meaning it generated these images on Amazon servers. "What's absolutely staggering is that this is the #3 all time downloaded model on CivitAI, and is presented as a pretty SFW model," Moreau, who is also OctoML's VP, technology partnerships, said in a company Slack room called #ai_ethics on June 8, 2023. Moreau was referring to an AI model called "Deliberate" that can produce pornographic images. "A fairly innocent and short prompt '[girl: boy: 15], hyperdetailed' automatically generated unethical/shocking content -- read something could be categorized as child pornography," his Slack message added.

Read more of this story at Slashdot.