The Union Government has repeatedly issued advisories warning against the growing threat of Artificial Intelligence (AI)-generated deepfakes and their potential to compromise digital systems
NEW DELHI — The Union Government has repeatedly issued advisories warning against the growing threat of Artificial Intelligence (AI)-generated deepfakes and their potential to compromise digital systems.
The Securities and Exchange Board of India (SEBI) has also warned against the “Boss Scam”, in which fraudsters impersonate chief executives and senior officials to deceive finance teams into transferring funds.
AI-generated deepfakes have been a concern for some time. However, intelligence agencies have warned that the problem could assume alarming proportions in the coming months, with elections in several states approaching and the potential for deepfakes to penetrate the political system on a much larger scale.
Seven states are scheduled to go to the polls in 2027, namely Goa, Gujarat, Punjab, Himachal Pradesh, Uttarakhand, Uttar Pradesh and Manipur. Assembly elections in these states are expected between February and December 2027.
Intelligence agencies are keeping a particularly close watch on Punjab, Uttar Pradesh and Manipur, where they anticipate a heightened risk of attempts by anti-national elements.
An Intelligence Bureau official said the atmosphere in the country is already charged.
There are ongoing anti-Special Intensive Revision (SIR) protests taking place, and with the political atmosphere heating up, many elements would look to have a field day by putting up content that is AI-generated.
“AI-generated videos of top politicians will be the most preferred choice of these elements. Elements, not just from across the border, but those within the country would look to create such deepfake videos and circulate them widely on social media,” the official said.
Another official said that apart from the generation of such deepfake videos of political leaders, the other major problem is the scale at which it is being planned.
“In normal circumstances, such elements would push five to ten videos a day of political leaders with fake messages being attributed to them. However, as the elections near, such elements would look to generate at least 100-150 deepfake videos a day and circulate them widely,” the official said.
The intention of generating a large number of videos is clearly aimed at stressing the system and the fact-checkers.
Debunking so many videos a day would be extremely challenging, and there is a good chance that a few of them may stay within the system and remain uncontested.
This is the goal of these elements, the official said.
Another official said those generating such videos are clearly targeting rural sectors of states where elections are scheduled in 2027.
In the urban areas, the awareness of such technology and content is higher among the people. At least 30 per cent of people are able to identify such content and debunk it. That is not the case in rural India. Those parts of these sectors blindly believe what they see and circulate it. By the time the video is fact-checked and debunked, the damage may already have been done.
“In many cases, it has also been found that many do not even bother to check what the fact-checkers or the government and agencies are saying about these videos,” the official said.
This helps scammers keep the content out in the public domain for a longer time and hope the fake narrative they are pushing is set in the minds of the public.
In 2024, one survey found 22 per cent of Indians encountered political deepfakes. A study found that 50 per cent of respondents watched deepfake videos within 12 months.
During the 2024 Lok Sabha elections, it was found that 75 per cent of the voters were exposed to synthetic media.
Officials say these elements are adopting different strategies for rural and urban India. In urban areas, political deepfakes would continue to be circulated on platforms such as X and YouTube.
The problem, however, is bigger in rural areas where such content is pushed directly to phones through WhatsApp forwards and massive voice-cloning broadcast campaigns.
On platforms such as YouTube, X, or Instagram, legal requests can be made to take down such content. However, the problem is when it is pushed directly on mobile phones through WhatsApp. Such content is widely forwarded, and by the time action is taken, the damage is already done.
The agencies are particularly concerned about such videos in states like Punjab, Manipur and Uttar Pradesh. In UP, the push would be heavy in the rural parts.
In Punjab and Manipur, the problem can be immense as both states are volatile in nature and anti-national elements would look to vitiate the atmosphere by pushing in massive amounts of political deepfake videos, an official said.