The best way to analyse your employee survey data

Running a survey is pointless unless you can make sense of the data. This post provides a practical perspective on how we tend to analyse employee survey data. After each survey we run for our clients we meet to review the survey data using our survey platform so this is an overview of how we do that.

As such, analysing your survey data needs to be seen as just one step in improving employee engagement in your organisation.

Key survey statistics

We tend to start by reviewing some of the key survey statistics.

This will include a review of the survey completion rate and a brief review of any demographic data captured.

Completion rate provides a sense of how many people took part in the survey (we actually calculate participation rate as not everyone will complete the survey fully). Most organisations want to feel that they have had a reasonable participation rate for the data to be seen as useful. We often get asked what the response rate needs to be to be ‘valid’, but there is actually no definitive answer to this. Employee surveys are based on asking people’s opinions of how they feel about working for the organisation so we tend to take the view that all views are valid no matter how many there are. Response rates will also vary depending on the type of business. For example, retail and hospitality businesses will usually be pleased with a 50% response rate, in the NHS it can be as low as 20%, whilst we have a couple of clients who are disappointed if they don’t achieve 80%.

Reviewing demographic data can provide an overview of how survey responses were distributed such as which part of the organisation respondents came from, what the gender or age range profile was etc. Demographic data becomes more useful when we get into slicing and the dicing the data to see if there are any differences between different groups of employees.

Survey scores by theme

Most of your survey questions are likely to be quantitative rating scale questions. These are the detailed questions where everyone rates how they feel and they are likely to be grouped into themes. We start by taking a high level view of the aggregated scores for each theme.

Rating scale question data can be analysed in two ways:

1. We can calculate the mean average for each theme. In this survey we used a rating scale with 6 points on it (strongly agree – strongly disagree) so the maximum possible score is 6 and the minimum possible is 1. We tend to recommend with a 6-point scale that you aim for a minimum score of 4 as this is where most people are tending to agree with the survey question statements.

2. We can calculate the proportion of people who use each part of the rating scale.

Each type of chart uses the same data, it is just displaying it in two different ways. The stats can be explained by looking at the distribution curve of the data. The mean average chart shows the peak of the curve, the percentage chart shows the size of each rating scale ‘chunk’ of the curve.

The data for the survey themes shown in the charts above is giving us a clue as to where people are more or less satisfied.

Individual question scores

Now we have an overview of which themes in our survey are scoring high or low we can begin to drill into the detail so the next thing we do is start to explore the scores for each individual question in the survey.

Again we can view the data using either mean averages or the percentage distribution, but this chart shows the mean average scores for each question.

How do different groups of employees feel?

Reviewing the data at the question level allows us to really home in on the specific issues and we usually find at this point that clients want to start exploring to see if there are any differences between different groups of employees.

This is where the demographic data in the survey really helps as we can use it to slice the rating scale question data using the demographic questions in the survey. For example we can look at how people in different parts of the business feel, we can see how people with differing lengths of service feel etc. This is why it is important to use demographic questions in your employee survey.

In our platform we can either create multiple datasets by applying filters or we can use heatmaps to do the same thing.

So where do we focus our efforts?

The analysis we have outlined so far will provide some very strong pointers around where the issues might be. From a high level we know which themes we need to focus on, we have identified some specific questions where we can make improvements and we have explored to see if there are any variations based on demographic data.

There is a temptation to home in on the questions that have the lowest score, but this does not necessarily mean that it will have an impact on overall levels of engagement. Questions with low scores indicate that people are dissatisfied with them, but they might not actually have a big impact on overall engagement. For example, people might be dissatisfied with aspects of their job, but they might still be engaged with the organisation as a whole. In our experience, pay tends to fall into this category. Pay has undoubtedly become more of an issue for people in the last few months, but our analysis shows that although people would like their pay to be higher, other aspects of the employee experience tend to have a bigger impact on how engaged they feel.

We need to do some more analysis to understand which specific survey questions are having the biggest impact on overall levels of engagement. These are your engagement drivers.

We use a bit of simple stats to do this by correlating how every employee rates each survey question compared with how they rate one ‘engagement outcome’ question. The most common engagement outcome question is “I would recommend the organisation as a great place to work”. We call it an engagement outcome question because if you score highly on all the other survey questions people will tend to also recommend the company as a good place to work.

We combine the results of this correlation calculation and use some colour coding to identify which specific questions are having an impact on overall engagement and how people feel about those questions. In the table below the ‘r Value’ column (nothing to do with Covid!) shows the strength of the relationship between the survey question and how much people would recommend the company. The higher up the list, the stronger the impact. The colour coding is based on how employees feel about each survey question (shown as average scores).

• Green indicates a question that has an impact on engagement and is scoring well.
• Amber indicates a question that has an impact on engagement, but could be improved.
• Red indicates a question that has an impact on engagement, but needs significant improvement.

Your red questions are the ones where you should focus – improving satisfaction levels with these questions will drive improvements in overall levels of engagement.

Establishing overall levels of engagement using eNPS

We started referring to Net Engagement Score a few years ago after taking learning from experts in customer experience management and seeing how we could apply it to employee experience management. Since then, eNPS (Employee Net Promoter Score) has become a more established term.

In customer experience management Net Promoter Score measures how much people would be likely to recommend a company’s products or services to others. Transferring this thinking to employee experience management, the Net Employee Engagement Score is based on how much employees would advocate your organisation as a good place to work.

How is net engagement score or eNPS calculated?

When it comes to employee surveys, we use one of four approaches to measuring net engagement score.

1. The simplest method, and arguably the purest, is to include the question “I would recommend the company as a great place to work” in your employe survey. We then use how people respond to this question to calculate your eNPS.
2. The second approach is to use other ‘outcome’ questions in the survey as part of the measure. Examples include “I am proud to work here”, “I can see a future for myself working here” etc.
3. The third approach is to take a more research-based approach by looking at which survey questions appear to be driving employee engagement. To do this we run a correlation analysis to identify your engagement drivers. We still use the “I would recommend…” question as the basis for the correlation calculation, but we explore how the question correlates to all the other questions in the survey. For example, we might find that questions such as “I feel valued” have a strong correlation to how much people would recommend the company. Correlation does not imply a causal relationship, but it indicates that whether or not people feel valued has an impact on whether or not they would recommend the company to others as a good place to work. Using this approach means that we can define specific organisational behaviours to include in the eNPS calculation. The thinking is that if these questions all have a strong statistical linkage to engagement then it will increase the focus of the business on improving these specific engagement drivers.
4. The fourth approach is kind of similar to the previous one, but it simply uses all of the survey questions to arrive at an overall net engagement score. It is the least scientific method, but it is easy to apply.

Once we know which approach you prefer it is just a case of calculating the net engagement score. We can do this in two ways depending on whether or not your prefer to work with average scores or percentage score.

Net Promoter Score is actually based on percentages. The calculation takes the percentage of people who would advocate the product/service/company and subtracts the percentage of people who would not advocate the product/service/company. The end result is a net percentage, hence Net Promoter Score. When it comes to eNPS we do exactly the same thing by taking the percentage of people who use the positive parts of the survey rating scale (usually Strongly Agree, Agree etc) for the questions in our calculation, minus the percentage of people who use the negative parts of the scale for the questions in our calculation.

However, some companies prefer to use average scores when analysing their survey data rather than percentages so we also often use mean averages to calculate Net Engagement Score. This is a little different as it does not subtract the negative percentage from the positive percentage, it just calculates the overall average score for the questions in the calculation.

In conclusion

Analysing your survey data takes time and effort, but the value is not in the analysis, it is in the story that the analysis tells. There is no point spending weeks analysing data and producing overly complex information if your stakeholders can’t easily make sense of it or quickly understand where they need to focus their efforts. Our survey platform was built to significantly reduce that effort and help you make sense of your survey data as quickly and intuitively as possible.