Controllable Neural Text Generation

Lilian Weng published a technical overview of controllable neural text generation, examining how an unconditioned pretrained language model can be steered toward desired output attributes such as topic, style, and sentiment. The post was updated three times: version 2.0 on 2021-02-01 added several works and fixed typos; on 2021-05-26, P-tuning and Prompt Tuning were added to the prompt design section; and on 2021-09-19, unlikelihood training was added.

Weng organizes the approaches into three groups: guided decoding strategies that select desired outputs at test time, optimization through prompt design, and fine-tuning the base model or steerable layers for conditioned generation. The post assumes access to a pretrained generative model trained via next-token prediction.

The decoding section covers common methods including greedy search, beam search, top-k sampling (Fan et al., 2018), nucleus or top-p sampling (Holtzman et al., 2019), and penalized sampling from the CTRL paper (Keskar et al., 2019). Guided decoding alters candidate ranking with feature discriminators designed by heuristics (Ghazvininejad et al., 2017), supervised learning (Holtzman et al., 2018), or reinforcement learning (Li et al., 2017). Hafez, described by Ghazvininejad et al. in 2017, generated poetry in a desired style by adjusting beam search sampling weights. Meister et al. (2020) studied beam search in a regularized decoding framework connected to the uniform information density hypothesis.

Trainable decoding methods include Gu et al. (2017), Grover et al. (2019) with likelihood-free importance weighting, and Deng et al. (2020) with a residual energy-based model.

Prompt design coverage includes AutoPrompt (Shin et al., 2020), which uses gradient-guided search to identify universal trigger tokens in embedding space. Weng notes that model steerability remains an open research question and that each introduced method has pros and cons.