A complication was that he had previously worked in software/technology, which has a lot less regulatory hurdles to launching products than alcoholic beverages. Further the company’s performance not meeting shareholder expectations and expansions in marketing/R&D budgets were out of the question.

The needed to generate growth and test ideas in an environment where resources were very limited.

At first my colleague wanted to use the company bar, where our products were served to employees after work as an in-house focus group. The rationale to senior managers was that it could be an extension or precursor to formal focus group testing that the company typically pursued. His thoughts were that it mirrored more accurately the actual marketplace, as participants were unaware that their behavior was being observed. The idea largely came from wandering through the bar and soliciting feedback during campaign launches. He found responses were more likely to be honest, especially when negative than in the focus groups.

Over a 6-month period informal predictions of 4 new products' success (over/in line/under expectations) proved to be more accurate than research form the focus groups. However his idea was never able to be utilized as an in-house ‘market’ for new concepts, as products could not be served if they had not met the legal and product safety standards.

Management at the time was steadfast in their policy that no product would be put through testing and certification if it had not been through focus groups (who could work around legal implications by requiring them to sign waivers). Not to be stopped, he started using the same methodology to test packaging changes on products that were already being produced for the public. He had set up a display of various products, including those with the new packaging. As a ‘reward’ for various things (i.e., working late to complete a project) he would let us pick a bottle from the display. What we did not know is that he was carefully paying attention to our reaction to the display and our selections. This information was used to narrow in on which innovations would proceed further.

Following a wholesale change in executive management over the last 24 months there has been a definite shift towards trialing products not only at our in-house bar, but also with major bar/restaurant partners. The focus has not yet been on idea generation/selection as concept testing, but it has improved the company's new product success rate (even if they are more step changes than true innovations).

Explaining basis of decisions to superiors. Superiors were very focused on either historical data (Neilsen and other trend based data) or formal product testing (focus group results). These are the methods that are/had been entrenched in consumer goods as they have worked in the past. They were quick to dismiss his information on the grounds that it did not tailor the audience to target demographics.

I will be honest and say that I am not sure he ever really got past this, and did leave the company prior to the new executive team joining. However the solution he came up with, which was to introduce the internal marketplace principle in a smaller more controlled way was an effective way to show how effective the principle could be.  

Testing could be conducted more frequently and at early stages in the product development lifecycle, allowing for more ideas to be considered. 

Most companies have access to phenomenal amounts of untapped data. One area this prompted me to think about pushing into was analyzing travel and expense card receipts. We already had to compile this data for processing and audit purposes, we could get a good insight into purchasing habits and preferences of clients and consumers.

People unaware that their actions are being monitored act differently to those who are aware that their actions are being observed and will have outcomes (his bar test proved a more accurate predictor of launch success than focus group testing).

Can drive two benefits from single cost. The testing he was doing was positioned as a reward, and as a result drove feelings of goodwill.

That even if there appear to be road blocks the applicability of good ideas (in this case testing on employees) can be used on smaller scale to establish validity. Must be willing to persevere though.