Inducing Sustained Creativity and Diversity in Large Language Models

We address the early stages of research, where scholars conduct “search quests,” hunting for research topics, qualitative cases, game tree primitives, survey questions, compelling arguments, etc. Instead of the “correct” answer, the goal is to understand the search space by evaluating many diverse and creative possibilities before making a personal choice. Large language models (LLMs) offer more ideas for these open-ended queries than anyone can on their own. However, the advantage is temporary and Faustian: because LLMs optimize for questions with correct answers, they return orthodox answers to search quests, repeat ideas when asked for more, give away ideas a researcher chooses to competitors asking the same question, and risk homogenizing the discipline. We develop an easy-to-use algorithm that induces 2–3 times more sustained diversity and creativity than standard LLMs, and more with newer LLMs, across 16 separate analyses, 12 evaluation types, and hundreds of thousands of comparisons.