Stocktwits is a social platform where retail investors and traders discuss stocks in real time. Scammers were using those conversations to lure users into off-platform "trading groups," and they adapted to every new rule within the hour. Stocktwits now uses Musubi to evaluate every post against its own fraud policy and to ban coordinated scam accounts automatically. The team detects 2.5x more scams than manual review alone, at an estimated 92% lower cost per decision.
The Challenge
With fast-moving conversations about financial decisions, Stocktwits became a target for sophisticated fraud and scams.
The core problem: Bad actors were running affinity fraud schemes by posting messages and images in chat designed to lure users into off-platform "private trading rooms" or to invest in "expert guidance".
Examples:
- if you're interested in copying my trade. DM . I provide real trading guidance.$
- A great discussion group👇
- Stay updated on trends and opportunities by connecting with a community of active traders! 🙏 ▶️
- If you like money, come with us.
Every time we would put a new rule in place, they would instantly adapt. Literally within the hour, there was a whole new regime of scamming and spamming going on.
— Noah Loffredo, Senior Product Manager, Stocktwits
What made this problem hard:
- Speed matters: Chats move quickly, so fraudulent content needed to be caught before other users saw it.
- Nuance matters: There was a fine line between sharing excitement for a stock and promoting it in a fraudulent way. The core use case of the platform was discussion about stock and giving each other advice, and it was important to not stifle legitimate conversations.
- User-level patterns matter: Sending a few promotional messages appears innocent, but multiple accounts sending similar messages with similar unusual behavioral patterns indicated coordinated fraud activity.
- Scammers adapt relentlessly: Scammers kept probing defenses until they found ways to create accounts and send messages that didn't trigger detection.
Traditional keyword filtering, content moderation, and ML models couldn't adapt. The Stocktwits team needed a system that understood context and could take action at both the content and account level.
Scammers are posting very manipulative content around a certain stock, and they're posting what seems like financial content that a normal trader would be posting. But if you dig into their user behavior, they're posting slightly different messages across hundreds of different stock pages. And it's very botlike in nature, in terms of the repetition and also how they're coordinating across different accounts.
— Noah Loffredo, Senior Product Manager, Stocktwits
The Solution: Two-Layer Detection System
Musubi implemented a layered approach that enabled Stocktwits to:
- Catch scams at the content level, in real time. Musubi's LLM-based, policy-aware content moderation sits in the message pipeline and evaluates each post and image against Stocktwits' custom fraud policy, flagging off-platform solicitations and "join my trading group" pitches while letting normal stock discussion flow.
- Escalate risky messages into account-level decisions. When Musubi flags suspicious content, it automatically reviews the full history and behavior of that account, looking for coordinated patterns across multiple profiles and channels.
- Automate action at the account level. Accounts that exhibit clear fraud patterns are automatically banned before they can post again.
- Continuously adapt as scammers change tactics. Because policies are expressed in natural language and Musubi learns from moderator feedback, the system quickly incorporates new scam and fraud variants without weeks of model retraining or manual rule tuning.
Stocktwits had tried other tools, but generic content categories couldn't handle a platform where hyping stocks is the point.
We had another vendor running in the platform for a year, but only 2% of the decisions or the categorizations that they were giving us ended up actually being valuable to us. Until we used Musubi and could actually define what we consider to be spam and create those policies ourselves, we were just lost.
— Noah Loffredo, Senior Product Manager, Stocktwits
Layer 1: Agentic LLM-Based, Policy-Aware Content Evaluation
Stocktwits writes fraud policy in plain language, and Musubi enforces it with agentic LLMs that weigh context and judgment rather than keyword matches or fixed categories. Agentic means multi-step: instead of stopping at a single classification pass, a flagged message kicks off the same investigative sequence a moderator would run, from account history review to coordinated-cluster analysis to enforcement. That's what lets the policy handle edge cases human moderators were catching manually:
- Off-platform links to chat apps (Discord, Telegram, private chatrooms) → Flag as scam
- Off-platform links to news sources (Bloomberg, Reuters, financial news sites) → Allow
- Context-aware detection: "Join my trading group" vs. "Here's an analyst report"
- All types of media: When scammers moved from text-only chat to embedding text in images, Musubi continued to ingest and evaluate those images without losing coverage.
Defining the policy in broad terms for Musubi to enforce rather than using dozens of rigid, manual rules meant it could detect novel scam approaches just like human moderators, but do it at scale.
Layer 2: Whole-Account Assessment and Action
When Musubi identifies suspicious content, it automatically triggers a holistic review of that user's whole account. This catches bad actors early:
- Reviews user's full posting history and behavioral patterns and clusters in real time
- Automatically bans accounts showing coordinated fraud behavior
- Prevents fraudsters from posting additional scam content after the first detection
Many fraud prevention systems only look at individual pieces of content. By connecting content-level detection to account-level action, Musubi prevents the next 5, 10, or 50 posts from that same bad actor, reducing exposure for users and creating a safer community.
How the policy creation works, the different trigger points, violation severity scoring, and how we handle all of that stuff was very intuitive. Once we actually had it up and running and saw it working on real data, it was very easy for us to iterate from there.
— Noah Loffredo, Senior Product Manager, Stocktwits
Iterating on policy as fast as scammers adapt
When scammers change tactics within the hour, the policy has to keep changing too. Noah now does that work with the Musubi Agent, which he uses both in the Musubi app and through its MCP extension in his coding workflow. The Agent works directly with Stocktwits' policies, datasets, and decision history, so testing a policy change no longer means clicking through each step by hand.
I actually did three iterations of the main policy very quickly, and got the accuracy up significantly in the first day that I was using it because I could rapidly iterate a lot quicker than it would have been manually.
— Noah Loffredo, Senior Product Manager, Stocktwits
The Agent has also changed how Noah maintains golden datasets, which he says "has always been a pain." The Agent samples decision history into a CSV, Noah validates the labels, and that set becomes the baseline for the next round of policy iterations. He now refreshes those datasets every few days. "To do that on a weekly or daily basis is definitely something I would have never done before."
The Results
Stocktwits' moderation is no longer limited to taking spam down after it appears. The team now detects bad actors at their first flagged message and removes them before they can post again, which reduces user exposure to harmful content and makes the community safer.
Now we hear from users that have been here for a long time and they say, 'I don't know what you guys did, but it's gotten so much better' — and we get messages like that every single week now, which is a great feeling.
— Noah Loffredo, Senior Product Manager, Stocktwits
Fraud Detection & Prevention
- 2.5x more violations detected vs. manual review alone, catching sophisticated schemes human moderators were missing
- Faster decisions: each piece of content is evaluated in less than a second, and a full account review takes a few seconds, much faster than human review
- Thousands of fraud accounts automatically banned. Stocktwits estimates that those accounts would otherwise have made around 22,000 fraudulent posts every day.
In terms of legitimate appeals, the numbers actually stayed very consistent despite the number of bans going up 10x from where we were in the past. And that has been a huge surprise to me, to be honest. Like I said before, when we were using other vendors the amount of false positives was astronomical. We eventually just had to pull it all back and say, 'We'll accept more spam on the platform than dealing with all the false positives.' But this has been the complete opposite of that.
— Noah Loffredo, Senior Product Manager, Stocktwits
Operational Efficiency
- Estimated 92% reduction in cost per content decision
- Moderators redeployed to handle complex edge cases and community building instead of battling back coordinated attackers
Looking Ahead
The next area the Stocktwits team wants to tackle is trend detection: using Musubi's content insights and analytics to spot new scam variants and shifts in how the community communicates as they emerge, so policies and support guidance can adapt sooner.
Beyond that, Stocktwits is shifting toward algorithmic feeds, and moderation signals are becoming part of how content gets ranked. Musubi decisions and severity scores already feed into the algorithm the team is building. The team is also exploring the flip side: using Musubi's policy evaluation to identify good content worth promoting, not just violations worth removing.
Moderation is an interesting duality — you can see what is all bad content, but you also can see what is good content. And Musubi is interesting in that sense too, because you can compare every single message that gets posted against this policy of spam, but you can also compare it against the policy of 'this is really good content.'
— Noah Loffredo, Senior Product Manager, Stocktwits
About Musubi
The same challenge shows up on any platform where legitimate activity looks a lot like the abuse: generic categories can't tell the two apart, but a policy written in the team's own words can.
Musubi is the Trust & Safety suite that covers content, accounts, and AI in one place: content moderation and insights, fraud and fake-account detection, AI guardrails and agent oversight, and a moderation console, all trained on your team's decisions. It runs through one unified API, and Musubi is accessible to AI agents through MCP and a CLI.
If your team is dealing with something similar, reach out and bring your hardest policy. We'd love to chat.