How AI decision models could change content moderation

Musubi announced on Tuesday a lightweight decision model for real-time content moderation called PolicyLM-1.7B, released with open weights. The model is designed to take a content policy written in plain English and apply it to messages in under 50 milliseconds.

According to Musubi, the model is intended to be similar in cost and speed to the AI classifier systems that power moderation on most social platforms, but because it has the flexibility of a modern LLM, it can apply complex policies without special training. Musubi says the model will not need new training when a policy changes, allowing human policy-setters to iterate.

Musubi co-founder and chief AI officer Filip Jankovic said the approach gives platform managers a way to label content proactively. "Product teams just want a better understanding of what's happening on their platform, especially as the amount of content is exponentially increasing," Jankovic said. "Being able to label all of that in a very scalable, customizable way is extremely useful."

Decision models have drawn attention since the release of Typesafe AI's Jev in September, followed by competing decision models from OpenAI and Amazon. Rather than outputting text, a decision model outputs outcome probabilities; in this case, it outputs a binary judgement of whether content falls in a category or not. Limiting output to predetermined choices lets decision models run faster and cheaper than large language models while retaining the flexibility of the transformer architecture, per the report.

Jankovic said his interest in decision models predates Jev, tracing it to a 2024 project called GLiNER (Generalist Model for Named Entity Recognition) that deployed many of the same techniques. Musubi's product announcement reads: "If Jev caught your eye, PolicyLM-1.7B is the same kind of model, trained specifically for content moderation, that you can run yourself."