I was responsible for moderating a GroupMe with hundreds of people talking in it. Keeping up meant reading through a huge amount of conversation just to find the occasional message that needed action.
So I built a moderation bot to do the first pass and send me updates in a separate chat when something needed my attention.
The system has several layers. Every message goes through OpenAI's free moderation service. Then it goes through a broad word list covering the terms I wanted the bot to watch. Different terms lead to different decisions.
For terms I'd set up for automatic bans, the bot can take that action directly. Potential warnings take another path. The bot queues a low-cost AI model to review the message in the context of the conversation. That review happens before it issues a warning or sends me a notice about one.
The order matters. A word showing up is enough to take a closer look, but a possible warning still needs context. The model helps decide whether the word actually warrants action.
The notifications matter just as much. Whenever the bot acts, it sends a message to me in another chat. I can see when someone was warned or banned without keeping the whole group open and reading every message.
So the system has two jobs: decide which messages need action, and decide which events need my attention. Ordinary conversation goes on without creating a notification for me. The separate moderator chat becomes a record of when the system acted.
The free moderation pass checks every message, the word list routes the known cases, and the contextual model handles the warning decisions that need interpretation. Each part has a specific job.
I built it for a practical reason. I was already responsible for the group, and I wanted a manageable way to do that job. The bot let me focus on the events it surfaced instead of following hundreds of people's conversations one message at a time.
Keeping the contextual decision narrow
The AI review has one job: look at the message and the recent conversation, then return an action. That action can be no action, a warning, or removal. A term can be the reason to ask for a review, but on its own it isn't enough to justify a warning.
That matters in a group chat because the same words can describe a situation, criticize an idea, or go after a person. The review instructions tell those uses apart, and the model is allowed to do nothing. The word list decides which messages get that extra interpretation, so ordinary exchanges don't all go through another AI call.
The code also separates signs that someone is struggling from behavior that breaks the rules. A possible wellbeing concern can be raised privately without removing anyone from the chat. That's a different path with a different purpose.
The work around the AI calls
A bot running in a real chat has to deal with how the chat service behaves. The code deduplicates messages, so when the same message is delivered twice, it doesn't trigger two actions. Warning cooldowns keep the chat from filling up with notices when a short burst of messages comes in together.
Group setup is part of the design too. The bot works only with the chats I configure, and the admin chat is excluded from moderation. That keeps the place where I get reports from turning into another chat the bot moderates.
Automatic removal is a setting. The version I described can remove a member. The code also supports reporting a suggested removal when that setting is off. Those are two different ways to run it, and I don't want to imply every deployment used the same settings.
When the contextual review fails, the code defaults to taking no AI-directed action. A failed API request shouldn't turn into an accusation against someone. Choices like that shape what people in the group actually experience, so they belong in any explanation of the project.
What I can say is this: the bot routes messages, takes the actions I configured, and tells me about them in another chat. I haven't measured an accuracy rate or counted the hours it saved. Its purpose was to let me carry the moderation job without reading every message as it came in.