I worked with a colleague who strongly believed that successful innovation was a numbers game and companies needed to put a lot of ideas out there to ‘see what sticks’ with consumers.
Over a 6-month period informal predictions of 4 new products' success (over/in line/under expectations) proved to be more accurate than research form the focus groups. However his idea was never able to be utilized as an in-house ‘market’ for new concepts, as products could not be served if they had not met the legal and product safety standards.
Management at the time was steadfast in their policy that no product would be put through testing and certification if it had not been through focus groups (who could work around legal implications by requiring them to sign waivers). Not to be stopped, he started using the same methodology to test packaging changes on products that were already being produced for the public. He had set up a display of various products, including those with the new packaging. As a ‘reward’ for various things (i.e., working late to complete a project) he would let us pick a bottle from the display. What we did not know is that he was carefully paying attention to our reaction to the display and our selections. This information was used to narrow in on which innovations would proceed further.
Following a wholesale change in executive management over the last 24 months there has been a definite shift towards trialing products not only at our in-house bar, but also with major bar/restaurant partners. The focus has not yet been on idea generation/selection as concept testing, but it has improved the company's new product success rate (even if they are more step changes than true innovations).
Explaining basis of decisions to superiors. Superiors were very focused on either historical data (Neilsen and other trend based data) or formal product testing (focus group results). These are the methods that are/had been entrenched in consumer goods as they have worked in the past. They were quick to dismiss his information on the grounds that it did not tailor the audience to target demographics.
I will be honest and say that I am not sure he ever really got past this, and did leave the company prior to the new executive team joining. However the solution he came up with, which was to introduce the internal marketplace principle in a smaller more controlled way was an effective way to show how effective the principle could be.
More product research could be conducted at a fraction of the cost.
Feedback was more genuine, people were expressing actual opinions to friend, not a tester.
Most companies have access to phenomenal amounts of untapped data. One area this prompted me to think about pushing into was analyzing travel and expense card receipts. We already had to compile this data for processing and audit purposes, we could get a good insight into purchasing habits and preferences of clients and consumers.
People unaware that their actions are being monitored act differently to those who are aware that their actions are being observed and will have outcomes (his bar test proved a more accurate predictor of launch success than focus group testing).
Can drive two benefits from single cost. The testing he was doing was positioned as a reward, and as a result drove feelings of goodwill.
That even if there appear to be road blocks the applicability of good ideas (in this case testing on employees) can be used on smaller scale to establish validity. Must be willing to persevere though.