People are different. What works for employees in the Facebook generation may not work for other stakeholders. The critical management challenge becomes to identify and take those actions that at any point in time best favor the entire organization. New, reliable survey analytics are crucial for success in such endeavors.

Problem

When Tom Peters and Bob Waterman in the early 80’s wrote their famous book “In Search of Excellence – Lessons from America’s Best-Run Companies”, they based their findings on companies that had achieved 20 years of uninterrupted success. I was enthused by their findings and a few years later I was on the Swedish management team of one of the excellent companies, Digital Equipment Corporation (DEC).

DEC was a great place to work with much of the freedom and the open communication that MIX is looking for, and our customers loved our products.

Yet DEC collapsed, just like many of the other “excellent” companies. What went wrong?

In DEC’s case, we developed the wrong products. Some cost fortunes to develop and were decades ahead of demand, some were just not in line with what the customers needed at that point in time. For example, we had three lines of personal computers but they could neither communicate with each other nor with DEC’s state-of-the-art servers.

When CEO Vineet Nayar now describes how he has transformed HCL Technologies, I recognize many of the principles that guided DEC. Is there a risk that Vineet describes a success formula that is unique for HCL or can it with equal success be applied to any other organization in any situation?

Today I have no generic answer to that question, but in my company ValueMetrix AB – a small R&D company in the management field based in Stockholm, Sweden – I have the methodology, the experience, and the software that dynamically can help organizations determine what their key success factors presently are and what to do to be better off in the future.

The methodology builds on stakeholder surveys extended with financial facts and analyzed with factor analysis and structural equations.

Properly understanding the drivers of human behavior is an absolute must for any successful manager, but inadequate survey analytics have given many managers a negative view on the value of surveys. Running a profitable business just isn’t as easy as raising survey scores by giving the stakeholders whatever they want.

Yet many companies today base their strategies on the assumption that high survey scores are shortcuts to better financial performance.

Such a message appeals to managers. It is positive, simple to communicate, and intuitively a good way to run a sound business.

If these companies would check their survey scores against financial facts on respondent level, they would be worried. Based upon actual data from many different countries, industries, companies and stakeholder categories, they would most likely find that

High-scoring customers are NOT more profitable than other customers
High-scoring employees are NOT extra willing to support actions that improve the employer’s competitiveness
CSR efforts are NOT increasing revenue enough to motivate the added costs

Check the attachments to my MIX contribution 'Listen to Your Customers, Attract and Retain Professionals' for supporting evidence.

The list of similar findings could be made much longer and further increase the worries: Are present strategies really viable?

To make matters worse, such findings would be unpopular to convey and may take long to get accepted. The positive, simple interpretation of high survey scores has long been an accepted and unchallenged “truth”.

From school we all know that high scores are good scores – but were all scores equally important for you later in your professional career? And in retrospect, were all the scores worth the effort?

Several widely dispersed academic papers claim positive linkages between satisfaction scores, CSR programs etc. and profitability – but aren’t they just referring to simultaneous occurrences without eliminating all other possible explanations, such as the business cycle in general? For example, wouldn’t it be possible that Customer Satisfaction goes up when customers have more money to spend, can buy nicer products, and the suppliers can afford to quickly correct delivery mistakes? That Employee Satisfaction is high in companies that do well so the employees have more fun and get more challenging tasks and added benefits? That CSR programs attract attention and are initiated during good times based on unknown future benefits rather than proven short term bottom line effects?

There are also many consulting firms that gladly sell surveys, present the results as cross-tabulated scores in colorful graphs and complex tables, and praise the importance of high scores – but did they objectively prove that it is their services that make the difference? Some try to make their points using regression analyses – but did they mention that in surveys, most scores correlate so regression analyses can prove any sub optimization that the client wants to hear or the consultant wants to send?

Other consulting firms sell scorecard services – but do they take any kind of responsibility for picking the scores to put on the scorecards, or don’t they care if they help their clients implement unviable strategies?

So it is easy to understand why companies have ended up allocating scarce financial and human resources to actions that actually may destroy value. Most companies are not even aware of this serious threat to their strategies.

Thus, without verified financial linkages, survey analytics are seriously misleading the companies that buy them and need to be reformed to match the needs of management for the 21^st century.

Solution

The reformed survey analytics need to identify and quantify those actions that really create financial value.

This approach differs from conventional survey analytics in three main ways:

The holistic approach
The role of the investor
The value calculations

The end product is reliable links between survey scores and financial performance for later use in connection with strategy development and resource allocation.

1. THE HOLISTIC APPROACH

The holistic approach is a necessity as major actions may affect several stakeholder categories. If an action is positive for the customers but negative for the employees, what is the net effect for the investors?

The need for the holistic approach is particularly obvious in connection with CSR efforts where many stakeholder categories may be affected, the costs may be endless, and the claims for benefits may be many but the supporting evidence scarce.

Note that the holistic view is equally important for the stakeholders themselves. Their behavior is not decided by individual actions of the company but on their overall impression of that company and what it stands for.

2. THE ROLE OF THE INVESTOR

The role of the investor is central when estimating the financial value of actions.

“Create value” is an expression often used in companies without clear definitions. It may mean anything from an increase in the market value of the company or just added revenue, or a better image, or a feeling that certain resources have been well spent.

Few people seem to realize that it is the investors who ultimately determine whether an action creates value or not. Even fewer seem to realize that the investors balance future profitability and cash flow against the level of risk in the business. And as many investors are risk-averse, particularly during the last couple of years, also actions that reduce profitability may create value provided they sufficiently reduce the risk.

The profitability/risk balance is extra relevant when assessing the financial value of CSR actions, at least if the company is big and exposed to environmental hazards or relies heavily on purchases of from low-cost countries.

However, the profitability/risk balance can also explain less well-known stock valuation differences. As an example, customer satisfaction drives loyalty so a bank that excels at attracting and keeping low-risk, high-profit customers is likely to be valued higher than a bank with more risk-prone customers although the two banks may show similar profitability over a complete business cycle.

3. THE VALUE CALCULATIONS

The value calculations are divided into two main parts.

The first part consists of estimates of how strongly survey score changes would be likely to impact on stakeholder behavior.

In the second part, the value calculations are finalized by converting the behavioral changes into profitability and risk assessments.

Estimating the Impact of Survey Score Changes

The likely impact of survey score changes can be estimated using factor analysis and structural equations (see also the attached document called 'The ValueMetrix Approach in Summary' or www.valuemetrix.net for graphics that further describe how to execute and apply the analyses; more detailed versions can be obtained directly by e-mail from me, see below).

The analysis is a simulation of the human decision process. The human brain hates chaos so we take the bits and pieces we have learnt about an interesting offer and group them into broader factors. Having weighed them both against each other and against available options, we decide what to buy, where to work or invest, what to praise or criticize.

For example, when buying a product or a service, we first systemize details into 3-5 broad Driving Factors. Then we summarize the driving factors into an Overall Evaluation of the alternatives at hand. At some point we feel that we have enough information considering the importance of the decision so we pick the alternative that suits us best. This is the Desired Behavior as seen by the selected supplier.

Often the driving factors are not as distinct as one might think when developing the questions for the survey. This problem must be removed before proceeding to the structural equations, otherwise the results will be unreliable and the magnitude of the actions may be wrongly estimated. The tool to use for this purpose is statistical factor analysis.

Once the factors are distinct, they can be organized into a mathematical structure that copies the decision process and the steps from detailed observations to changes in various types of behavior.

The linkages are estimated in the structural equations and presented in the format

“If the score for Driving Factor A goes up by 5 units (on a scale from 0 to 100), the Overall Evaluation is likely to increase by X units”
“If the Overall Evaluation increases by 5 units, the Desired Behavior is likely to change by Y units”

These estimates can be delivered with various reliability statistics, such as explanatory power, confidence intervals, and collinearity measures that describe how distinct the factors are.

There are several issues to observe both when defining the surveys and during the quantitative analyses:

The choice of overall evaluation must be made dynamically to reflect changes in the environment. It is a very important choice as it may have a heavy influence on the conclusions. Customer Satisfaction has other driving factors than Supplier Attractiveness, Trust, or Corporate Responsibility. Employee Satisfaction has other driving factors than Employee Motivation or Joy for Work. The list of similar, alternative Overall Evaluations can be made much longer, and there is no generic proof that a certain Overall Evaluation is the best in all situations. With this in mind, the choice of overall evaluation should be based on the strongest links to the Desired Behavior of the stakeholders covered in the survey, and the choice should be reconsidered regularly as the strength of these links is likely to change over time
There are often major differences between segments. Relatively high customer scores are generally given by women, elderly people, persons who live in the countryside and those with little formal education. Conversely relatively low customer scores are generally given by men, young people, persons who live in the main cities and by those who are well-educated. For employees, the scoring pattern tends to be the other way around, with young employees giving the highest scores. See the attached document called 'Observations of Demographics-Driven Response Patterns' for such examples. In addition the various segments may also have different driving factors. For example, the Facebook generation is certainly having other preferences than elderly people. Basing actions on what the average of these segments prefers may be a severe mistake, perhaps resulting in higher customer satisfaction but also in a less profitable customer mix. The attached document called 'Observations of Differences in Customer Preferences' illustrates this.
Pay special attention to “non-stakeholders”, i.e. the ones who actively have decided to select another alternative that is available on the market. Take customers as an example. By definition customers have bought a certain offering at a certain price. A customer survey can identify glitches in the delivery, but not what it would take to make non-customers change their minds, such as a modified offering or different prices or pricing schemes
Consider the meaning of the response rates. When using internet surveys, the negative stakeholders tend to reply first. Pushing for higher response rates, the scores will go up and the message from the stakeholders will be diluted by those who dutifully respond after reminders. The response rates indirectly also indicate the respondents’ interest in the company in the survey and thereby also how likely they are to observe and care about the messages that the company may send out about actions after the survey

By combining scores and impacts, it is now possible to start evaluating different types of actions. Are there cost reduction opportunities to capture where scores are high but the impact on stakeholder behavior is low? Are there glitches to fix in regard to driving factors with high impact on the overall evaluation and desired behavior? Are there attractive segments where the fit with the company’s offering is good but present position is weak?

To determine which actions to take, the findings from the first part of the value calculations regarding the impact of score changes on stakeholder behavior are now the input into the second part where the financial value of score changes is calculated.

Converting the Impact of Score Changes into Financial Performance

The complexity of the second part of the value calculations depends on the type of action to evaluate, from simple cost/savings comparisons to investor surveys with focus on risk assessments.

First define an action to evaluate. What would it cost, and how long would it take to implement it?

If the action reduces costs without expected impact on stakeholder behavior, the analysis is as simple as a direct comparison between costs and savings.

For actions intended to improve stakeholder experiences, you need to begin by estimating by much the actions are likely to change the scores for the driving factors concerned.

This is best treated as realistic targets at certain points in time, remembering that score changes often take longer and are harder to achieve than what companies expect. After all, score changes are a matter of both taking the actions and ensuring that the stakeholders take notice of them and appreciate them sufficiently to allow them to influence their behavior.

Guidance for realistic end score levels can be obtained from studying what other parts of the organization have achieved.

As the impact of score changes on the behavior of individual stakeholders has been estimated in the first part of the value calculations, the next step is to compute what the estimated score change would mean in terms of gross value per stakeholder based on the changed behavior.

Then scale the value to reflect the size of the stakeholder base. Consider reductions if the response rates in the survey were low; they indicate how susceptible the stakeholders are to notice and care about the actions that the company implements.After this adjustment you get the likely volume change that the score change would be expected to cause. Now you can deduct variable and action costs to arrive at the profit contribution that the action is expected to generate.

For major investments it may also be appropriate to include the cost of capital as a deduction from the profit contribution.

Repeat the analysis for all the actions that are being considered and all the stakeholders that the company may want to influence giving particular attention to the preferences of the various segments with each stakeholder category.

When a profitability estimate has been calculated, it can be converted into shareholder value by using the company’s or the industry’s usual profit/earnings ratio.

If the risk profile is changed as a result of the action, not only the profitability but also the price/earnings ratio changes. Also this aspect can be included in the calculation of shareholder value, possibly after conducting an investor survey to be sure to understand the origins of the investors’ valuation of the company’s market value.

Practical Impact

The practical impact of the solution described above can be dramatic and lead to major changes both in the substance and in the manner companies are managed:

Strategic redirections may be necessary if the strategy previously was based on misleading assumptions
If the strategy changes, many other changes may also follow such as revised organization, new management roles, reallocation of investment resources for example updated marketing and communication activities, new IT systems, modified production processes, etc.
The solution gives added empowerment possibilities as the solution calls for greater involvement in the action discussions, it contains new performance measures linked to future financial performance, and it has a built-in predictive capacity because score changes typically precede behavioral changes by several months
The empowerment can be supported by rewards for good performance as budgeted score improvements may be detected also when the overall financial performance is weak as a result of aspects outside the control of specific managers

The net effect should be better financial performance represented by viable strategies and effective resource allocation.

First Steps

The suggested solution may not be suited for “quick & dirty” efforts but companies can take a few steps to remove the mental blocks as a preparation for taking additional steps later:

Get access to raw data
Add facts, preferably per respondent
Display the variables in scatterplots

These steps require little special competency and could be performed by most business controllers.

Begin by getting access to the raw data from previously conducted surveys.

The data collection firms are often hesitant about supplying the raw data but generally and formally they are the property of the company that ordered the survey. The hesitance may just reflect that the data collection firms are afraid to reveal the poor state of their files and filing systems.

Personal integrity may be a valid reason for not sharing the files, particularly when consumers and employees are the respondents. If so, all personal references must be removed by the data collection firm before sharing the files.

If there are no financial facts per respondent in those files, you can try to have the data collection firm add them afterwards but before removing the personal references and sharing the files.

In either case, it may be valuable for the company to know the demographics of the person who gave a certain open answer, but never the name of that person.

Sometimes it is necessary to rely on higher level of financial data, such as branch office profitability or market share in various segments, when comparing scores and financial performance. This is particularly true for employee surveys where results on respondent level are rare.

Once data sets from surveys are available, display two variables at the same time in a scatterplot, for example in Excel. Use standard features to draw a regression line and include the explanatory power.

If the line has a significant inclination, you have an indication that there might be an interesting positive correlation between the two. Before drawing any conclusions, look at the explanatory power to see how strongly they seem to be related. If one of the variables is a measure of financial performance, expect the explanatory power to be low, probably below 10%.

Repeat the analysis for other variables, and you are likely to find that most of them correlate, except with the measure of financial performance. If the financial measure is drawn on the X axis, expect the line to be more or less horizontal.

Credits

The methodology described above builds on the methodology developed by Prof. Claes Fornell at the University of Michigan for the American Customer Satisfaction Index (ACSI) and similar national satisfaction indices.

I am also thankful for the input by Dr. Marc Orlitzky, ass. prof. at Penn State and a prominent researcher in the field of linkages between CSR and financial performance.

Documents