To catch influence operations, look at how accounts behave together, not at whether any single piece of content is violative. Most content from influence operations looks like ordinary political opinion, which is exactly why it gets through, but at scale it can be harmful.
At Musubi, we help platforms find coordinated inauthentic activity. We’ve found that bad actors use a variety of tactics to meet their goals. They will push a single narrative, but they’ll also seed inflammatory content on both sides of the political spectrum. They’ll build up followers slowly before activating, but they’ll also create profiles in bulk and go straight to mass posting. Individual pieces of content often look innocuous, but when you see how they’re all connected, you can clearly see the harm at scale. This means you need a system that can constantly adapt to new patterns and that looks across accounts, behavior, and content.
Here are some examples of what we’ve found:
- Both-sides partisan bait: Templated personas posting partisan US political commentary, some anti-Trump and some patriotic, mixed with nostalgic engagement bait.
- LLM political commentary bots: Coordinated accounts posting LLM-generated commentary on US politics, including institutional decline, tariffs, primary results, and immigration spending.
- Yoga-loving anti-MAGA personas: Templated female personas who say they love yoga and travel and oppose "America First" politics.
- A fake-soldier persona farm: A bot network running varied personas, including fake Ukrainian soldiers, to farm engagement and build follower counts.
- A pro-Kremlin link network: Coordinated accounts promoting pro-Russian, Putin, and USSR-themed links.
- A harassment campaign against Dutch climate activists: Coordinated accounts sending severe abuse and death wishes, and threatening to evade bans.
Outside of political content, we also saw clusters of fake giveaway scams, SEO spam, and gambling promotion networks. AI showed up across all of it, used by bad actors to speed up disinformation, fraud, and scam campaigns.
Content moderation usually looks at one post at a time. A single post from the yoga personas or the both-sides network might not break any policy. What gives these accounts away is the coordination, and that only shows up when you look across them.
We use Content Atlas, our graph-based detection system, to surface clusters of coordinated bad actors and similar content at scale. Content Atlas connects whatever data you send us (content as well as account metadata) into a cluster map, grouped by what the content means rather than its format, label, or language. Accounts posting the same templated personas or narratives end up in the same cluster, even when the phrasing is slightly different. Because this works at the account level, it makes recommendations about accounts and the relationships between them, not individual pieces of content.
The same playbook is showing up across the internet
Each platform can only see what happens on its own service, but the reports coming out of the larger platforms and AI labs describe the same patterns we saw.
AI labs see operations being built before anything goes public. Anthropic's September threat report describes operators using AI to build persona systems, run networks of fake news sites, and invent reporters. Anthropic also found that commercial firms increasingly run influence operations for paying clients. OpenAI's October report on "false front" operations describes fake research centers and invented journalist bylines used to push content into real outlets.
Social platforms see the distribution side. Meta's latest Adversarial Threat Report describes operations working like supply chains, with freelancers and contractors handling different pieces. It also found that a Russian operation broke into smaller campaigns targeting several countries after sustained takedowns. Google's Influence Operations Bulletin lists coordinated YouTube networks from many countries. X says it suspended accounts tied to suspected Iranian, Russian, and Chinese operations ahead of the midterms.
Best practices for platforms
- Write policy around inauthentic coordination and deception, not just around content. Create concrete examples of what’s allowed as well as what crosses the line.
- Evaluate content together, not posts one at a time. Influence operations are using generative AI to make convincing and high quality content at scale. Content will be slightly different from each other so keyword and hash lists won’t work, but when you view them in aggregate you can see the similarities.
- Behavioral and network signals expose operations even when the content itself isn’t suspicious or violative. This is the tricky thing about influence operations. You will see the same exact type of post from an individual citizen talking about their own country and a mass group of fake personas controlled by another nation-state.
- Watch both sides of polarizing topics, since one operator may be seeding both.
- Look for templated personas and fake fronts. Fake but similar identities are common from these networks and are a strong signal of coordinated abuse.
- Some rings spend months building reputation before activating. Look for new (or newly activated) accounts that have similar narratives, templates, or links.
- Plan for networks to come back after you remove them. They may split into smaller campaigns or use slightly different tactics.
- Route fraud and integrity signals into one view, because spam, scam and influence operations often share infrastructure, or are even using the same third-party contractors.
- Set up monitoring before an event, not during it. Coordinated activity tends to build ahead of elections, along with genuine political opinion online. The further ahead you can set up systems, the better you’ll be able to spot anomalies. You also want to line up appeal and escalation capacity to handle the additional volume ahead of time.
We're here to help
If you want stronger protection against coordinated influence operations, we’d love to chat and show you what we can do.
Wondering if Musubi could help?
We love to talk about the basics, the gnarly edge cases, and everything in between.