AI-Driven Misinformation Increasing, Report Finds

In an era where digital content creation is evolving at breakneck speed, OpenAI has launched Sora 2, a platform capable of generating short-form videos directly from text prompts. The October 1 release marks a significant advancement in AI-generated media, coming just days after California Governor Gavin Newsom vetoed Assembly Bill 1064, which sought to regulate artificial intelligence systems targeting minors.

Sora 2 builds upon its predecessor released in December 2023, offering users the ability to create increasingly lifelike videos. While some view this technology as revolutionary for filmmakers, educators, and advertisers who can now visualize concepts with unprecedented speed, others express concern about a digital landscape where distinguishing between authentic and AI-generated content becomes increasingly difficult.

The rapid evolution of AI-generated content presents complex challenges for society, legal systems, and individual users. Currently, existing laws on copyright and defamation are being applied to new forms of AI misuse as policymakers debate whether entirely new regulations are necessary. Media literacy is becoming increasingly important as the ability to identify AI-generated content varies widely among consumers.

Lei Mei, an attorney and managing partner at intellectual property firm Mei & Mark LLP, notes the regulatory gap in this emerging field. “I’m not aware of any specific AI-related law that is governing this area,” Mei said. “There are some existing laws that potentially can cover a situation for AI-generated work. For example, there could be copyright issues and there could be defamation claims.”

This legal patchwork creates a complicated question of liability. Mei suggests that primary responsibility likely falls on individuals who use AI to create misinformation, though creators who fail to verify harmful AI-generated content before sharing it could also face legal consequences.

For young people and regular social media users, the concern is less about legal liability and more about navigating daily content. Carlmont student Eli Chen, who encounters AI content frequently on social media, remains confident in his ability to identify artificially generated material.

“As AI is improving currently, the content is getting really real, but there are still very obvious tells that you can see,” Chen said.

Not everyone shares this confidence. Dan Campion from the Maricopa County Sheriff’s Office has observed a troubling increase in sophisticated deepfakes over recent months. He points to a fabricated video of baseball star Aaron Judge as an example of the technology’s advancing capabilities.

“It was his face moving and talking. It looked exactly like him,” Campion explained. He expressed particular concern about the potential for such technology to undermine public trust, especially if weaponized against political figures.

“AI tools could change public trust because somebody could take a deepfake video of a politician or an elected official, and they could put things out there that are not true, and that could cause concern in the community,” Campion warned.

The question of responsibility remains central to the debate. Dennis Yang, a Carlmont student who competes in AI Olympiads, believes accountability should be shared between users and companies.

“If the video generation comes from an input where the individual puts in their input, I think the individual using the AI is misusing it,” Yang said. “However, there definitely should be restraints on the author’s side.”

Despite growing concerns about potential misuse, many still recognize the transformative potential of AI technology. “AI is a wonderful tool, and it could serve a lot of purposes and make life easier for everyone,” Campion acknowledged.

Looking toward regulatory solutions, Mei advocates for a straightforward first step that might avoid political controversy: transparent labeling of AI-generated content.

“It should be a good practice for the creators to label the product as AI-generated,” Mei said. “I don’t believe that’s politically controversial.”

As AI-generated content becomes increasingly sophisticated and widespread, society faces the challenge of balancing technological innovation with safeguards against misinformation. The coming months and years will likely see continued debate about the appropriate legal frameworks, industry standards, and individual responsibility in navigating this rapidly evolving digital landscape.