"Blogging innovation and marketing insights for the greater good"
Business Strategy Innovation Consultants

Blogging Innovation

Blogging Innovation Sponsor - Brightidea
Home Services Case Studies News Book List About Us Videos Contact Us Blog

A leading innovation and marketing blog from Braden Kelley of Business Strategy Innovation

Sunday, March 07, 2010

Distributed Idea Generation Outperforms Team Brainstorming

by Hutch Carpenter

"This has significant managerial implications: if the interactive build-up [of team brainstorming] is not leading to better ideas, an organization might be better off relying on asynchronous idea generation by individuals using, for example, web-based idea management systems."

Distributed Idea Generation Outperforms Team BrainstormingThat quote is from a report by three researchers from the INSEAD and Wharton business schools. They published a study, "Idea Generation and the Quality of the Best Idea", that analyzes a mainstay of corporate life: the brainstorming session.

Is it effective in generating quality ideas?

To find out, the researchers conducted a field experiment in which they compared two models of generating ideas:
  • Team structure: Group works together at the same time together in a room to generate ideas.

  • Hybrid structure: Individuals generate their ideas independently, then meet together in a group.

Their objective was to determine which of those two structures generated more ideas, ideas of higher quality and is better able to discern the quality of ideas. They found in all cases that the hybrid structure outperformed the team structure.

Extreme Value Theory

The success of idea generation in innovation usually depends on the quality of the best opportunity identified. For most innovation challenges, an organization would prefer 99 bad ideas and 1 outstanding idea to 100 merely good ideas. In the world of innovation, the extremes are what matter, not the average or the norm.

The researchers - Karan Girotra, Christian Terwiesch, Karl T. Ulrich - were interested in determining what methods generate the best ideas. They distinguish their approach from previous research which analyzed the quantity or average quality of ideas generated.

They use extreme value theory to understand the factors impacting the quality of ideas. Extreme value theory shows that the maximum value of an idea from a set of ideas is based on:
  1. The sheer volume of ideas generated
  2. Average quality of all ideas generated
  3. The level of variance in the quality of generated ideas

These concepts are put together nicely in this graphic:

Extreme Value Theory
Once you understand this framework for innovation, it becomes a matter of maximizing the values for each component. Watching, of course, for correlative impacts between them.

Field Research Experiment

The three researchers conducted an exhaustive experiment to determine which of the two methods - team structure or hybrid structure - generated the highest quality ideas at the top end of the scale. Here is the summary of their experiment.

Subjects: 44 juniors, seniors and grad students at the University of Pennsylvania

Challenges: They generated 443 ideas around two challenges.
  • You have been retained by a manufacturer of sports and fitness products to identify new product concepts for the student market. The manufacturer is interested in any product that might be sold to students in a sporting goods retailer.
  • You have been retained by a manufacturer of dorm and apartment products to identify new product concepts for the student market. The manufacturer is interested in any product that might be sold to students in a home-products retailer.

Idea generation formats: Subjects were split into four clusters. Half the clusters did the team structure first, half did the hybrid structure first. The clusters then switched structures for the different ideation challenges.

Idea quality: The quality of the ideas was assessed in two ways.
  1. Business value: Panel of 41 Wharton MBA students each assessed the business value of the ideas on a 1 - 10 scale
  2. Purchase intent: Panel of 88 college students (the target market for the ideas) each assessed their own likelihood of buying a given product proposal on a 1-10 scale

Experiment format: Subjects conducted idea generation exercises as follows.
  • Team structure: 30 minutes together in a room to generate ideas together. Then 5 minutes of assessing and selecting the best 5 ideas.
  • Hybrid structure: 10 minutes of generating ideas on their own. Then 20 minutes of discussing these and new ideas. Finally, 5 minutes of assessing and selecting the best 5 ideas.

Results: Hybrid Structure Tops Team Brainstorming

The results of the experiment are eye-opening. The researchers analyzed the two approaches on the three components of extreme value theory. They find hybrid is better on the individual components of the theory, and in the ultimate test: quality of the top ideas produced.

Number of ideas generated. Hybrid structure generates three times more ideas than does the team structure. Researchers attribute this result to three dynamics:
  1. Free riding: it's easy enough to ride the idea coattails of the group
  2. Evaluation apprehension: the fear of negative reaction when proposing an idea in front of a group
  3. Production blocking: participants have to wait while one person is speaking, limiting idea generation throughput

Idea quality: The average quality of the hybrid structure ideas was higher than that of the team structure. Specifically, 0.25 points better in business value, 0.35 points better in purchase intent. To put this in perspective, these differences translate into roughly a 30 point differential in percentile rankings. In other words, the difference between the 1st and 30th idea in a pool of 100 ideas.

Researchers attribute the decrease in idea quality for team structures to the same free riding dynamic that reduces the quantity of ideas.

Idea quality variance: The researchers found no discernible difference in idea quality variance between the hybrid and team structures.

What this means is that from extreme value theory, the quantity and average quality of ideas are the key drivers of generating the highest-ranked ideas.

Best ideas: Here's where the rubber meets the road. Which approach had the highest ranked ideas? Hybrid structure, by a landslide.

The researchers looked at the top 5 ideas, by quality scores, that emerged from the two approaches. The hybrid structure ideas were of much higher quality than those generated from the team structure. This finding held for looking at the top 3, 4 and 6 ideas as well.

To recap:

The hybrid structure produced:
  • More ideas
  • Ideas of better quality on average
  • Highest rated ideas

Ability to Select Best Ideas

Perhaps the one down note from the study is the ability of the group to select the best ideas. Remember that in both the team and hybrid structures, the group did a consensus selection of the top ideas. Participants weren't asked to select the top ideas individually.

The researchers found a small advantage in the hybrid structure group's ability to select the top 5 ideas resulting from their ideation exercises. But it wasn't material. Indeed, they note:

"The hybrid process may generate better ideas, but that due to the noisy selection process, its relative advantage is much diminished, to the point of becoming statistically insignificant for one of our quality metrics."

"Noisy selection process", indeed. Ever been in a brainstorming session where you're supposed to rank the ideas at the end? Imagine the dynamics of resolving differences of opinion, time constraints and the extraordinary influence of certain individuals that drowns out other opinions. This is not an optimal way to determine the ideas that define innovation for your organization.

What This Means for Companies Seeking Innovation

As we described previously in "Crowdsourcing Is the New Collaboration", there are many benefits to taking a new approach to idea generation, peer collaboration and integrating innovation more deeply into an organization's culture. Advanced innovation management platforms are ideal for this approach.

As this study confirms, distributing the idea generation process, as well as the idea selection process, results in higher quality ideas for organizations. This study dovetails well with another study by Professor Ron Burt, that found that employees with access to a wider range of viewpoints and feedback generate higher quality ideas.

Brainstorming does have its benefits in terms of face-to-face interactions. Perhaps the nature of what is brainstormed needs to change. Brainstorming can be valuable for project-oriented tasks and problem-solving. But don't consider it your go-to activity for the best ideas.

Don't miss an article - Subscribe to our RSS feed and join our Continuous Innovation group!

Hutch CarpenterHutch Carpenter is the Vice President of Product at Spigit. Spigit integrates social collaboration tools into a SaaS enterprise idea management platform used by global Fortune 2000 firms to drive innovation.

Labels: ,

AddThis Feed Button Subscribe to me on FriendFeed


Blogger Walter Bilgram said...

It’s my experience that brainstorming has become a mainstay for any team leaders or facilitator that wants to generate a list they can count, prioritize, and publish. It demonstrates results. The activity itself of developing a list, consolidating, voting on the top responses is reassuring and comfortable. The activity feeds the team just enough to proceed down a path of mediocrity.

We have all completed assessments like the DiSC profile that highlight individual behaviors of people within their environment. Each personality dimension, by definition, reacts differently in a similar environment. Brainstorming in an open environment can be effective with a segment of the population; maybe half. What about the other half?

The hybrid structure allows for both introverted and the extroverted participants to engage, provide input and build on the team’s synergy.

6:41 PM  
Anonymous calvin said...

I'm really glad you share these results with us. It's clear from this the role individuals and groups have in the innovation process.

I would like to add though, that there are many structural forms of brainstorming. The hybrid approach is inherent in some of them. So maybe those who want to use brainstorming to get the best ideas should simply chose the best brainstorming method first.

8:59 PM  
Anonymous Graham Horton said...

Comparing the effectiveness of different arrangements of classical brainstorming is popular among scientists.

unfortunately, doing this is rather like comparing methods for straightening deckchairs on the titanic - you may well find that one method is more efficient than another, but the whole exercise is ultimately kind of pointless.

the reason is simple: classical brainstorming is the weakest of all idea generation techniques. any more advanced technique is so superior to brainstorming in terms of quality of results, that worrying about the style of brainstorming to use is essentially irrelevant.

5:16 AM  

Post a Comment

<< Home

Site Map Contact us to find out how we can help you.