Malicious actors are using generative AI music tools to compose homophobic, racist, and propagandistic songs-and to publish guides that explain how others can do the same.

ActiveFence describes itself as a platform that helps online services govern trust and safety operations. ActiveFence reports that "hate speech-related" communities have spammed online chatter with a sharp increase of discussion about methods used to misuse AI music creation tools in order to compose offensive songs that direct against minority groups since March. According to ActiveFence researchers in a report, AI-generated songs being shared in forums and discussion boards look forward to inciting hatred toward ethnic, gender, racial, and religious cohorts while celebrating acts of martyrdom, self-harm, and terrorism.

Abhorrent and hurtful songs barely premiere nowadays. The worse news is that, because music-making software is free and accessible to anyone with an internet connection, they will be produced at scale by those who could not or did not formerly know how-to just like image, voice, video, and text generators have amplified the dissemination of fake news, falsehoods, and hate messages.

These are trends that are becoming more intense with more and more users learning how to create such songs and share them with others, an ActiveFence spokesperson told TechCrunch. Threat actors are fast identifying specific vulnerabilities to abuse these platforms in different ways and generate malicious content.

Creating "hate" songs
Other generative AI music tools, for example the ones that appear at Udio and Suno, also enable users to paste in their own lyrics with a generated song. Filters on those platforms block common slurs and other pejoratives, but users have found ways around those, according to ActiveFence.

For example, the report states that users of white supremacist forums shared phonetic spellings of minorities and derogatory terms. They even went as far as spelling out "jooz" instead of the term "Jews" and "say tan" instead of "Satan" in an attempt to bypass content filters. There are even suggestions that users alter spacings and spellings of violence to refer to committing acts of violence, such as turning "my rape" into "mire ape.".

TechCrunch tested a few of these workarounds on Udio and Suno, two of the most popular tools for generating and sharing AI-created music. Suno allowed all of them through, while Udio blocked some-but not all-of the offending homophones
Reached via email, a Udio spokesperson said the company doesn't permit hate speech on its platform. Suno didn't respond to our request for comment.

ActiveFence noted it had detected some links in the communities canvassed to tunes that spew AI-generated songs parroting conspiracy theories against Jewish people and calling for mass murder; tunes containing slogans associated with terrorist groups ISIS and al-Qaida; and sexual violence against women.
Song impact
ActiveFence argues that songs-comparatively speaking, text-tend to carry emotional weight that make them a particularly effective tool for hate groups and political warfare. The company points to the example of Rock Against Communism: the series of white power rock concerts in the U.K. in the late '70s and early '80s that spawned subgenres of antisemitic and racist "hatecore" music.

They make malicious content palatable. Consider a hateful preacher speaking a bad narrative about some group of people, and then out comes a rhyming song that makes it easy for all to remember and sing," said the spokesperson for ActiveFence. "They reinforce group cohesion, indoctrinate fringe group members, and also are used to shock and offend unaffiliated internet users.".

ActiveFence is calling for the addition of prevention mechanisms among these platforms, along with further deeper safety assessment. "Red teaming may identify some of those weaknesses and could be triggered by threat actor simulation," the spokesperson told. "Indeed, a more effective input and output moderation might prove helpful here in allowing the platforms to block the content before it ever reaches the user.".

But fixes might prove short-term as users find ways of working around the moderation. For example, some of the AI-generated tracks that tout terrorist propaganda ActiveFence found were made using Arabic-language euphemisms and transliterations — the euphemisms the music generators did not catch, perhaps because their filters do not happen to be strong in Arabic.

AI-generated hate music has a long queue if it tends to flow as other AI-generated materials are. There was a show earlier this year by Wired documenting how an AI-manipulated clip of Adolf Hitler received over 15 million views on X after being shared by a far-right conspiracy influencer.

For instance, a UN advisory body stated that racist, antisemitic, Islamophobic and xenophobic content might be supercharged by generative AI.

"Generative AI services allow users who do not possess resources or creative and technical skills to create relevant content and communicate ideas that can attract attention within the global market of ideas," explained the spokesperson. "And threat actors, having found the creative potential associated with these new services, are trying to circumvent moderation and remain undetected — and they have been successful."