You've decided to use LLMs for content moderation (or you’re thinking about experimenting with them). What skills does your team actually need? Can you upskill your existing policy experts, or do you need to hire entirely new roles? How do you develop these capabilities when the technology is evolving faster than anyone can keep up?

Most T&S leaders can build AI-ready teams by upskilling existing members rather than starting from scratch. But it requires understanding which skills matter, who can learn them, and how to create the conditions for success.

The Core Skills Your Team Needs

LLM-based moderation requires a different skill set than traditional ML classifiers or purely human review. 

Policy Engineering

What it is: Translating human-readable content policies into instructions that LLMs can follow consistently and accurately. This is more than just "writing prompts"—it's a discipline that sits at the intersection of policy expertise and understanding how language models interpret instructions.

Why it's different from policy writing: Traditional policies are written for human moderators who bring context, judgment, and common sense. LLM-based policies need to be explicit about edge cases, structured in ways models can parse, and tested systematically for consistency.

As we explain in our article on policy engineering for LLM-based content moderation, effective prompts require three key elements:

• Concise, precise language - LLMs understand natural language, but clarity matters, and descriptors like “egregious” that people intuitively understand need to be defined exactly.
• Clear structure - Use markdown, bolding, bullets, and logical sections.
• Examples of both violations and non-violations - LLMs learn from examples.
• Background- LLMs are surprisingly responsive to creative prompts that give background color to their task. Giving an LLM a persona, a scenario, or a mission with detail can help.

Can policy experts learn this? Usually, yes. Policy engineering is fundamentally about clear communication and structured thinking, which are skills most T&S professionals already have. 

But assess early. Some people find prompt engineering intuitive and energizing. Others struggle no matter how much training they receive. Give your team a low-stakes experiment early. For example, have them transform one simple policy into a prompt and test it. See who gets excited and who gets frustrated.

1. Pick one simple, clear policy (e.g., hate speech)
2. Have them transform it into a structured prompt using the framework from our policy engineering guide
3. Test it on 10-20 examples. You can do this through any genAI chat client - just paste the policy and the examples and ask it to label the examples.
4. Debrief: How did they approach it? Did they get energized or frustrated? Did they iterate when it didn't work?

Not everyone needs to be good at this, and there are tools which help you to optimize policies and make the process easier (Musubi’s PolicyAI is one). But you do need 1-2 people on your team who can do this manually in a pinch and who understand how to “speak LLM”. 

Some resources:

• Learn Prompting - Best free resource, practical and accessible
• Google’s prompting essentials course
• MIT basics on prompting
• Anthropic's prompt engineering interactive tutorial

AI Literacy

Your team doesn't need to build models or write code. But they do need to understand:

Model outputs and confidence scores: ML models and LLMs work very differently, and different LLMs have different structured outputs. Many struggle with outputting a confidence score (LLMs are famously overconfident). Your team needs to know what outputs to trust and why.

Model limitations and blind spots: LLMs struggle with certain tasks, such as intent, built-in bias, or subtle sarcasm. Your team needs to recognize these patterns so they can design systems and accommodate shortcomings accordingly. (More info in some sections of our guide here). 

Prompt iteration: Understanding that the first prompt rarely works. Getting good results requires testing, measuring, and refining. This is similar to how you'd train a new human moderator, but with different feedback mechanisms.

Practical application: This means your team can look at model decisions and understand whether the issue is the policy, the prompt, the training data, or the model's fundamental limitations. That diagnostic skill is what separates effective teams from struggling ones.

Systems Thinking

Traditional moderation often focuses on individual decisions: Is this post okay or not? LLM-based moderation requires thinking in systems: How do all the pieces work together?

Designing escalation workflows: When should AI make autonomous decisions? When should it flag for human review? What triggers escalation? Trust & Safety best practices emphasize that for complex abuse types or lower confidence assessments, AI should route content for human review rather than make autonomous decisions.

Identifying feedback loops: How do human decisions improve the AI? How do edge cases discovered during review feed back into prompt refinement or golden dataset updates?

Understanding interconnections: If you change one policy's prompt, does it affect how the model handles related policies? If you adjust confidence thresholds, what happens to your escalation queue?

Quality Assurance & Analysis

Building and maintaining golden datasets: These are your ground truth. Golden Datasets are carefully labeled examples used to test and evaluate model performance. Someone needs to own this work: selecting representative cases, ensuring labeling quality, updating as policies evolve.

These datasets should include edge cases, clear violations, clear non-violations, and cases where reasonable people disagree.

Monitoring for drift and degradation: Models don't stay static. Your team needs people who can spot when performance is declining, identify patterns in where the model struggles, and diagnose whether the issue is prompt quality, model limitations, or changing abuse tactics.

False positive/negative analysis: When the model gets it wrong, someone needs to investigate why. Was the policy ambiguous? Did the prompt have gaps? Is this a systematic issue or an outlier?

The "Translation Superpower"

If there's one highest-value skill, it's what the All Tech Is Human research calls "bilingualism"—the ability to translate between different institutional logics.

In practice, this means:

• Explaining technical constraints to policy teams ("Here's why the model struggles with that edge case")
• Translating policy nuances to engineers ("We can't just flag keywords; context matters this way")
• Communicating risk to business stakeholders ("Here's what happens if we get this wrong")
• Presenting findings to legal/compliance ("Here's how we're meeting regulatory requirements")

The people who can bridge these gaps become indispensable.

Team Structure

Here's how roles typically break down in LLM-based moderation:

Policy Experts define and refine your moderation rules. These are your T&S veterans who understand abuse vectors, cultural context, and where edge cases hide. Their core work doesn't change, but how they express policies does.

Prompt Engineers translate policies into effective LLM instructions. As we explain in our policy engineering guide, this role focuses on writing, testing, and iterating on prompts to achieve high accuracy. In many teams, policy experts and prompt engineers are the same people.

Analysts monitor performance metrics, identify patterns in where the model struggles, investigate false positives/negatives, and manage golden datasets. They're your early warning system for problems.

Engineers handle infrastructure, API integration, and technical implementation. Depending on your organization, this might be a separate engineering team rather than embedded in T&S.

In smaller teams, these roles overlap significantly. You might have:

• 2-3 policy experts who also do prompt engineering
• 1 analyst who also helps maintain golden datasets
• Engineering support from a shared tech team

You might not need four separate people, but need to ensure all four functions are covered.

Name a Point Person

LLMs, just like human moderation teams, require continuous attention.

Monitoring metrics, updating prompts, evaluating new models, refreshing golden datasets, investigating edge cases– this work never stops. Industry best practices outline that even the most diligent AI systems require ongoing human oversight and adjustment.

You need dedicated ownership of this work, which could be either a specialized role (even if it's just 50% of someone's time initially) or a clear allocation of how much time for this work is protected each week.

Without this, your system will drift, degrade, and eventually fail in ways you won't notice until it's too late.

How to Develop These Skills

For Leaders

• Create psychological safety. Team members need to feel safe admitting when they don't understand something, when they made a mistake, or when the AI got something wrong. Celebrate learning, not just results.
• Budget actual time for learning. "Learn this in your spare time" doesn't work. Allocate specific hours: "Everyone gets 4 hours per week for learning and experimentation." What your team learns today may be obsolete in six months. Continuous learning isn't optional.
• Make feedback loops visible. Show your team how their work improves the system. When someone's edge case discovery leads to a prompt update that improves accuracy by 5%, celebrate it. Make the connection explicit between human expertise and system performance.
• Protect time for strategic work. If your most experienced analyst is spending 90% of their time in the moderation queue, you're wasting their highest-value skills. Create protected time for pattern analysis, prompt refinement, and system improvement.

For Teams

• Document why you override AI. Every time you disagree with a model decision, write down why in a structured way. These overrides are teaching moments. Over time, they become data that improves the system.
• Share edge case discoveries. Found a new evasion technique? A policy ambiguity? A model blind spot? Document it and share it with the team. Build institutional knowledge, don't hoard it.
• Think like a model trainer. Your decisions aren't just individual judgments, they're data that will train future iterations. What patterns are you teaching the AI?
• Experiment and be curious. Try different prompt phrasings. Test the model on corner cases. Ask "Can the AI handle this?" as a constant practice. Use AI to help you learn—generate summaries, identify patterns, draft communications.

Skills Matter

The technology will keep changing and new models will emerge with different capabilities. The tools your team uses six months from now might not exist today (and at Musubi, we're constantly working to innovate in this space!). But the skills in this guide (policy engineering, AI literacy, systems thinking, quality assurance, and translation) remain valuable regardless of which specific model you're using, or what technology is available.

The teams that succeed are the ones that figure out how to blend human expertise with AI capabilities, and build the capacity to keep learning and experimenting as everything evolves.

----

Need hands-on help? Musubi's PolicyAI makes it easy for T&S teams to write, test, optimize, compare, and deploy prompts so your policy experts can focus on what they do best.