AivexaNewsSearch
AI news for builders and product teamsChecked every hour

LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B

Collected Nov 12, 2026

IBM's Granite team has released two experimental preview models trained to work with LLMON (LLM Object Notation, pronounced "lemon"), a markup language designed specifically for language models rather than for parsers. The checkpoints, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, are derived from Granite 4.1 base models and were announced on the Hugging Face Blog alongside a GitHub repository containing a toolkit, workflow examples and templates. The authors describe them as previews suitable for exploration and evaluation, not production use.

The motivation is the informal structure of a typical LLM context. A system prompt, past turns, retrieved documents, tool responses, output requirements and reasoning chains all arrive as one stream, and most of that structure is either implicit to the model or expressed in formats like JSON that were built for software parsers. That ambiguity, the team argues, is a major reason smaller models are brittle, and it is also what makes prompt injection possible.

LLMON is an XML-like syntax in which semantically meaningful tags mark each component, enclosed by reserved special-token delimiters rather than ordinary characters. Because the delimiters occupy their own special tokens, structure and content draw on separate parts of the vocabulary. Nesting is stated locally at each span, so the model does not have to track bracket depth across long inputs. Unique block IDs let a turn point at any instruction or payload anywhere in the context and bind them together, much as a function call names its arguments.

In practice, the shipped chat template surfaces this through an ordinary messages list. A turn can name its parts — instruction, data, requirement — and flag each one with exec: True or False. Parts marked True are collected into an exec block at the bottom of the rendered prompt, which states which instruction to run, which data it operates on, which requirements constrain it and which output block receives the answer. Parts marked False remain visible in context but are referenced by nothing. Block IDs come from a fixed set the tokenizer encodes as single tokens, so addressing a block costs two tokens regardless of which ID is chosen.

On security, IBM evaluated the previews on the OPI benchmark, which hides injections inside the data of classic NLP tasks and uses several attack strategies, and reports robustness as one minus attack success rate. Non-LLMON models, both open and closed source, largely carried out the injected instructions. The LLMON models avoided roughly 47% more attacks than the best frontier model tested and around 95% more than the best open-source model, with similar trends on the ASB and StruQ benchmarks. IBM notes that these tests cover static indirect injections and that work continues on adaptive attacks that unfold over multiple chat turns.

General capability is reported as on par with the Granite 4.1 Instruct models across standard benchmark categories. Because the chat template handles the rendering, teams already experimenting with Granite 4.1 3B or 8B can swap in a preview without building a conversion pipeline, provided they name the parts of each turn. An accompanying converter library goes further, translating existing JSON to LLMON and back losslessly and preserving a structured document as typed, individually addressable blocks rather than one opaque string. Mellea, a separate open-source project, is described as having native LLMON support that automatically LLMONizes its LLM calls.

Training started from Granite 4.1 base checkpoints rather than the released instruct models, so the behavior is learned during post-training rather than layered onto an existing chat format. Six delimiters are installed in the tokenizer as reserved single tokens. A conventional instruction-tuning corpus was re-serialized into LLMON with each prompt component as its own block, using the usual task mix of chat, multi-turn, RAG and tool calling. The data was augmented with decoy instructions and data blocks left deliberately unbound while target outputs stayed the same, so the model learns to follow only the dependencies named by the execution block. Five sequential GRPO reinforcement-learning stages followed, spanning multi-domain reasoning, RLHF, knowledge calibration and math, then a final DPO stage on a thousand on-policy preference pairs built from the model's own responses to injection attempts.

The resulting privilege rule — instructions reached through an exec block are authoritative, anything reached as data is material to read rather than commands to follow — emerges from that layout rather than from an added guardrail. IBM is explicit that this is learned behavior, not an enforced guarantee. A likely trade-off is that a stronger statistical separation helps against static injections but does not provide the hard boundary a type system or sandboxed tool executor would give; a sufficiently novel or adaptive attack could still cross it. The team also offers an attention visualization demo on Hugging Face Spaces for inspecting how the models treat the blocks.

LLMON is not tied to a fixed schema. On ScienceWorld, a text benchmark where an agent performs elementary science tasks one command at a time, IBM defined a domain vocabulary with tags such as instr.task.goal, req.planning_policy.rule, data.state.room, data.state.inventory and assistant.next_action, rendered reference solutions with those tags and tuned a LoRA on top of an LLMON model. The adapter then plays held-out games and writes each next action in the new vocabulary; a notebook covers the full workflow, and another demonstrates a tool-calling agent. The paper references are listed as arXiv 2610.38120 and 2603.22519.

Why it matters: Teams shipping agents that ingest retrieved documents, tool output or user-supplied text face indirect prompt injection as a live risk, and the previews offer a drop-in path for existing Granite 4.1 users that reportedly costs little general performance. The bigger shift is conceptual — making instruction versus data an explicit property of the prompt format, with binding and addressing built in — which suggests prompt layout and chat templates deserve the same scrutiny as model choice (inference). Because IBM frames the safety behavior as learned rather than guaranteed, production systems should keep outbound tool authorization and output checks in place.

Read at Hugging Face Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt