We wrongly assume that the best ideas come to us quickly – a cognitive illusion that limits our creativity.

Whether you are a scriptwriter trying to find the right plot twist for the next episode of your sitcom, an entrepreneur searching for the name of a new product or a programmer trying to fix a software bug, you’ll know the feeling. After an initially productive brainstorm, you find new ideas increasingly hard to come by. Expending any further energy feels like a wasted effort, so you decide to call it a day, and quickly choose one of your first suggestions as the best possible solution.

Many of us assume that creativity declines over time, and that our first ideas originate from our best and freshest thinking. Yet according to cutting-edge new research by Brian Lucas, a professor of organisational behaviour at Cornell University, this assumption could not be more wrong – and we’re falling victim to a phenomenon known as the “creative-cliff illusion”.

The best ideas, he says, come to those who wait. And by learning to exercise a bit of patience and perseverance, we can all find more innovative solutions to our problems.

The accessibility principle

Lucas’s theory originates from a logical consideration of the way ideas are formed. For any problem, the more obvious solutions are going to be within easy reach – perhaps resembling something that you have previously encountered, with a few tweaks for the current situation. That initial idea generation is incredibly satisfying – Lucas compares it to the hit you might get after eating a sweet.

Once you’ve exhausted those possibilities, any further solutions will come more slowly, and will require much more mental effort to draw out and develop. Lucas proposes that the frustration and hard work, give the impression that you are facing rapidly diminishing returns, and the sense that your creativity is about to drop off a cliff.

In reality, the number of ideas may indeed have fallen, but the quality of each individual idea could be much higher – with the extra thinking contributing to greater originality and insight. If you work through the frustration, you may therefore find a truly innovative concept that stands out from all the other ideas.

“It’s the basic principle of accessibility: the first thing you consider is probably what everyone else is considering, which means that they're not novel,” says Lucas. “So, you have to push past those to get to the things that are a little more unique.”

False beliefs

Lucas put his theory to the test with a series of experiments, the first of which tasked participants with creating strategies to increase donations to a local charity. To assess their beliefs in the creative-cliff illusion, the participants first had to predict how creative they would be (on a scale of -50 to +50) at various intervals, before actually beginning the assignment.

Sure enough, Lucas found a remarkable mismatch between people’s assumptions and their actual abilities. Most participants believed their creativity would decline rapidly when they were around halfway through the allotted time. Yet, independent judges of their work found that the quality of the ideas – as defined by their novelty and usefulness – stayed stable, or actually rose throughout the task.

Subsequent experiments asking participants to consider a creative challenge from their own personal lives – with regular progress reports over a five-day period – replicated these findings. Lucas also examined whether people’s backgrounds affected how susceptible they were to the creative-cliff illusion. As you might hope, regular experience of creative problem solving did make people a bit less susceptible, he found.

The power of persistence

In his final experiment, Lucas investigated the consequences of the creative-cliff illusion.

He looked into whether the erroneous assumption that participants had exhausted their most creative work led people to abandon brainstorming before they reached their best ideas. (That’s because it was possible that even if people assumed their ideas were declining, they would continue persevering anyway - in which case the creative-cliff illusion would be less relevant to actual workplace behaviours.)

To test this possibility, Lucas asked the participants to rate a series of statements on a scale of 1 (strongly disagree) to 7 (strongly agree):

• People tend to generate their best ideas first
• A person’s best idea is usually among the first few ideas generated
• Ideas generated earlier are often better than ideas generated later

They then took part in a short brainstorming session – in this case, a cartoon-caption competition, with a maximum $150 (£109) prize – which they were free to finish whenever they wanted. As expected, the higher they scored the statements, the less time they spent on the task, and the worse the quality of their ideas. Indeed, each additional point on the scale corresponded to 18% fewer solutions that were deemed to be “highly creative” by the independent judges.

“Your best ideas will typically take more iterations and more resources than you might initially expect,” concludes Lucas.

The 3:20 rule

While Lucas’s tests of the creative-cliff illusion examined relatively short brainstorming sessions, he suspects the effects may accumulate in more ambitious projects.

“People typically approach long-term projects by breaking them into sub-goals and tasks,” he says. “A product-launch team may spend the afternoon brainstorming ideas about suppliers, or a writer might spend the weekend developing a character’s backstory. Our data suggest that within each of these tasks, people could fall prey to the creative-cliff illusion.”

His research has immediate implications for managers and leaders. At the very least, they might try to gauge their team’s general beliefs about brainstorming, and correct the misperception that creativity drops after a certain amount of time – so that team members know to persevere past the ‘cliff’.

We should also try to incorporate the findings into the structures of our working practices – such as budgeting much more time and resources for the idea generation and development process. In general, Lucas suggests a simple rule of thumb: “If you're trying to put three good ideas on the table, you should probably be asking your team to generate 20.”

Wilma Koutstaal, a professor of psychology at the University of Minnesota, Minneapolis, agrees that recognizing the creative-cliff illusion could be useful for anyone trying to innovate. “One of the things I try to teach my students is that we need to be patient with our own thought processes and give ourselves the time to have ideas emerge,” she says.

Lucas’s findings also chime with some recent observations by Zorana Ivcevic Pringle, who is director of the Creativity and Emotions lab at the Yale Center for Emotional Intelligence. She recently surveyed various ‘creatives’ – artists, designers and writers – and asked them to describe the emotions they feel during their work.

As Lucas’s work shows, many laypeople assume that innovation comes easily and feels “fun”, she says. “But the number one thing these creatives mentioned was frustration,” she says, as they worked through their problems.

It’s only by learning to live with those uncomfortable feelings, as we traverse the creative cliff, that we can come up with work that is truly novel.

More about: