Indonesian Police Warn Grok AI Photo Manipulation May Be Criminal Offense

Indonesian Police Launch Investigation into AI-Generated Deepfakes on X Platform

Indonesia’s National Police Criminal Investigation Department (Bareskrim) has initiated investigations into multiple cases involving deepfakes and digital media manipulation created through artificial intelligence tools, officials confirmed Wednesday.

The investigation specifically targets potentially illegal content created using Grok, an AI service available on the social media platform X, formerly known as Twitter. Authorities are examining whether these manipulations constitute criminal violations under Indonesia’s electronic data laws.

“We are currently investigating in that direction. This has become a focus of our Cyber Crime Directorate,” said Brigadier General Himawan Bayu Aji, Director of Cyber Crime at Bareskrim, during a press briefing on January 7.

The police probe comes amid growing concerns about the misuse of AI tools to create deceptive or harmful content. In recent weeks, manipulated images generated using Grok have proliferated across X, with users issuing text commands that instruct the AI to alter photographs in inappropriate ways, including attempts to create non-consensual nude imagery.

“When it comes to AI, as long as it can be established that there is electronic data manipulation, it can be treated as a criminal offense,” Himawan explained. He added that several cases are currently under review, with potential criminal charges looming if investigators determine the actions violate Indonesia’s laws on electronic information and transactions.

Indonesia has one of the largest social media user bases in the world, with an estimated 170 million active users across various platforms. The country has previously shown willingness to regulate digital platforms, including temporarily blocking access to certain services that failed to comply with local regulations.

Digital rights experts note that this case represents a growing trend of nations grappling with the legal and ethical implications of generative AI technology. Indonesia’s Information and Electronic Transactions Law (UU ITE) contains provisions that could potentially apply to deepfakes, particularly those involving defamation or non-consensual intimate imagery.

The investigation into Grok highlights the challenges facing regulators worldwide as AI tools become increasingly accessible to the general public. Grok, developed by xAI, was launched by X owner Elon Musk in November 2023 as a competitor to other AI chatbots. The service was initially available only to premium subscribers of X.

This isn’t the first time Indonesia has expressed concerns about AI services on the X platform. Government officials previously warned of a possible ban on Grok if the service failed to implement adequate safeguards against misuse.

Digital rights advocates have called for a balanced approach that protects individuals from harm while avoiding overly broad restrictions that could impede legitimate technological development. They emphasize the importance of clear guidelines for AI developers and platforms regarding content moderation and safety features.

The Indonesian police investigation comes as other countries, including the United States and European Union members, are also developing frameworks to address AI-generated deepfakes and misinformation.

For social media platforms operating in Indonesia, this investigation signals increased scrutiny of AI features and potential liability for harmful content generated through their services. Companies may face pressure to implement stronger content filters and verification systems or risk regulatory consequences in one of Southeast Asia’s largest digital markets.

Authorities have not yet announced a timeline for the conclusion of their investigation or potential charges against specific individuals or entities.