The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models

Apple Machine Learning Research published a NeurIPS paper, dated September 2026, titled "The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models." Authors are Xavier Suau, Alex Ferrando de las Morenas, Luca Zappella, and Samy Bengio. The work is listed under Methods and Algorithms and Speech and Natural Language Processing.

The paper studies how much tree-structured compositional content survives when language models serialize structured information into natural language during chain-of-thought reasoning or free-text exchange. The proposed round-trip protocol has a generator convert a procedurally generated arithmetic expression into a word problem, a separate extractor recover the expression from the word problem alone, and symbolic equivalence serve as an exact oracle. Evaluating all pairwise combinations of sixteen models produces a communication matrix whose marginals separate generation quality from extraction quality.

The authors report three findings. First, the channel is lossy and asymmetric: swapping which model generates and which extracts shifts accuracy by up to 60.4 points, and the best pair reaches 92.9% by combining different models on each end rather than the same model on both. Second, at least 73.6% of round-trip failures originate at generation, and difficulty is driven by tree structure, specifically operator count, depth, and right-branching, rather than model family. Third, the channel is trainable: approximately 3600 fine-tuning examples sharing the evaluation's operators and tree shapes lift every open-weight model above untrained Gemini-3.1-Pro, described as an upper bound under matched semantics. A disjoint-domain regime with new operators and vocabulary also raises every open-weight model, confirming the gain is not an artifact of matched semantics, though a gap to the frontier remains.

The authors conclude that tree-structured expression serialization is a primary limiting factor when models communicate hierarchical structure through natural language.