Why can’t language models write well? Because they’ve been trained into obedience: rule-following, terrified of biology, allergic to weirdness. Meanwhile the genuinely strange GPT-2 from 2017 was putting lemon-eating men in showers. (Jasmine Sun, The Atlantic)
- Jasmine Sun interviews AI researchers, writing evaluators, and lab insiders to explain why language models produce bad prose despite having ingested centuries of literature.
- The post-training process that makes chatbots safe and commercially viable actively suppresses the unpredictability that makes writing interesting, including the lemon-eating-shower-man energy of early GPT-2.
- Rubrics that cap exclamation marks at two, models allergic to blood and sex, and graders rating fan fiction on "factuality" are not bugs in the system but features of how these models are built and evaluated.
