But AI isn’t creative, isn’t funny, isn’t surprising! Always just average, everything’s been done before. That’s kind of true – but here comes a paper saying: You can actually train LLMs to be creative too. It’s called Creative Preference Optimization.
Summary
- Creative Preference Optimization improves the novelty, diversity, and surprise of language model generations while maintaining high quality.
- CrPO models outperform strong baselines, including GPT-4o, on both automated and human evaluations.
- Directly optimizing for creativity signals in the preference objective is a promising direction for advancing the creative capabilities of language models.
