Questionnaires

Trainers Tips

Questionnaire Specific Best Practices and Discussion Topics

Overview:

Before creating a questionnaire, there are some things to consider:

• What scoring method should be chosen? Percent or Points?
• How should the weight be distributed?
• Which answer options should be selected besides yes/no?
• Should there be options for "N/A" and how should this calculation be done?
• Etc.

In this Trainers Tips section, you will find tools to support your work as well as best practices for creating an effective questionnaire within Eleveo Quality Management.

Scoring Considerations

It is recommended that you choose percentages as the scoring system. Therefore, each question and answer section is given a percentage by default. Percentage allows for cleaner reporting and more effective calculation of N/A scores. With percentages, it is possible to choose a N/A calculation method that effectively factors out the question. This is not possible with points, as you either give the full score or no score at all, which can negatively impact the overall score.

The option to calculate questionnaires by points is available, but not recommended. To make changes to the default settings, see Setting Questionnaire Properties.

Note: When creating a questionnaire, there are 3 requirements that must be met in order for it to be considered valid and usable:

1. The sum of the values of all sections of the questionnaire must equal 100%.
2. The sum of the values of all questions within each section must be 100%.
3. Each question must contain at least 1 answer

A complete list of all status messages can be found on the page Understanding Error Messages

Weighting Considerations

When deciding what weight to apply to question sections or the questions assigned to those sections, consider what kind of message the choice in weight sends to agents. Questions assigned a higher weight clearly indicate how important they are to your organization.

It's important to note that weighting can be adjusted for each section as well as for each question within the section.

Notice in the example below how Customer Service has been given more weight. An agent will quickly understand how important this category is to the organization versus the Closing, which is assigned weight. While both sections impact the final score, a difference in weighting clearly indicates what areas are of importance in your organization.

Weighting is not only done at the section level, but also for the questions within a section. In this example, we see that verification of customer information is more important than the opening script, so the questions within the section are weighted accordingly.

Visit the page dedicated to Adding and Editing Question Sections, Questions, and Answers for step-by-step instructions on adding sections and questions.

Creative Answer Options – Beyond Yes/No

Have you thought about answer options beyond Yes, No, or Not Applicable?

It is possible to create answers that receive partial credit, or even fail all options!

When considering possible answers to a particular question, it is important to explore the pros and cons of partial credit.

• Is it fair to offer only full credit or no credit answers? (Yes/No)
• If partial credit answers are introduced, will this create confusion for employees?
• Is the business looking at clear-cut processes or performance-based behavior?

Why not get creative in naming the answer choices? Instead of 'Yes', perhaps you could offer the option 'Meets Expectations' or even 'Exceptional'. Creative answer options that go beyond yes/no answers can be beneficial in several ways.

To add an answer to a question, see Adding and Editing Question Sections, Questions, and Answers.

Keeping It Simple

Full-credit and no-credit answers, refer only to yes/no answer options. In this case, the agent would receive full credit or 100% for a Yes answer and 0% for a No answer.

This option is the preferred response method when there is no ambiguity for both the agent and the analyst. For example, "Did the agent perform action X?" If there is a possibility of deviating from or only partially following the default,  full-credit and no-credit answers could have a negative impact on the agent's score.

Not Applicable N/A

Sometimes there are situations that do not apply. Therefore, be sure to include a not-applicable option. This option can be used in situations where a question should not be used to evaluate an agent for the specific contact. Consider when a Not Applicable option should be provided. It should not be created for questions that are required for all contacts.

There are several ways to calculate the N/A response. To determine the calculation method and for more information on the available settings please refer to the page named Preferences.

Partial Credit Choice

When an agent is required to follow a well-defined action during a conversation, the ability to use partial credit options allows the evaluator to select a specific answer based on the defined processes. Interpretation of outcome is then minimized.

Example Question: Did the agent open the call properly?

Requirement: Agent said Hello, states the entire company name, and states the agents full name.

• Covered All Requirements – 100% Credit
• Missed One Requirement – 67% Credit
• Missed Two Requirements – 33%
• Missed Three or More Requirements – 0% Credit

This approach is preferred in businesses where standards are clear, reproducible, and defensible when challenged. Scores obtained using this approach are usually reproducible when different evaluators evaluate the same call.

Fail All/Reduce All/Success All

• Fail All – Assigns a final score of 0 for the entire scorecard.
• Fail Section – Assigns a final score of 0 for the entire section.
• Reduce All – Reduces the overall max score by or to a certain amount.
  • The reduction amount can be customized. See the section Adding Answers on the page Adding and Editing Question Sections, Questions, and Answers for additional configuration details.

These three options can have a significant impact on the agent's overall score. Since Fail All/Section can have a significant impact on the agent's overall score, this option can be an effective way to stress business critical functions. Examples of this include functions that could lead to litigation. i.e., If an agent does not verify a caller's information or HIPAA, there could be legal consequences and therefore it may be appropriate to assign an overall score of 0 for this action.

Reduce All, on the other hand, is often used for more subjective questions. For example, "Was the agent rude?" or "Did the agent provide exceptional customer service?". If the answer is "No," the overall score may be reduced by a certain percentage.

• Success All – Assigns the highest possible score for the entire scorecard. (i.e. 100%)
• Success Section – Assigns the highest possible score for the entire section.

These options can have an overall positive effect on an agent's overall score and are rarely used. They are typically used in situations where the business expects all answers to be "yes" and is looking for a quick way to pre-set answers to that question. For the first question, the answer option "Success All/Section" can be selected. If you select this option, all questions will be automatically answered with "Yes" and the agent will get 100%, although modifications to the questions are possible.

All of the above options can be paired with the "0%" sections option. In this scenario, a "Yes" answer would result in no credit, but a "No" answer would result in Fail Section/All or "Reduce All". This would have the advantage of enforcing business critical questions. If "0%" Sections are used, please note that all questions within each section must still equal 100%, even if they do not grant the agent any points.

Consider all possible options when creating questions and answers to ensure that the evaluations meet your business needs.

Scoring Tiers

Have you considered hiding the result of the questionnaire from your agents?

How about showing a color or phrase instead of the actual score?

For companies that want to take the focus off of actual numerical scores, Scoring Tiers may be an effective way to do this. Scoring Tiers use predefined tiers and language in questionnaires. In most cases, the language and tiers are customized to match yearly or quarterly reviews. With this method, instead of exact scores or no scores at all, a range of answers would display a phrase with an associated color.

• Note – Scoring Tiers are supported in combination with a percentage-based Scoring System.

Why might this be better?

As an example, let us take an agent who failed their assessment by 2%. It matters to the business that the agent failed, and training and/or education might be required, even if she is only off by a small or significant percentage. The agent, on the other hand, might think, "It's only 2%. What does it matter?" While this response may not apply to all agents, in cases where it does, some organizations may want to shift the focus away from the numbers and instead focus on the intent of the evaluation itself. For this example, a scoring tier might be developed that simply states "Meets Expectations" or "Does Not Meet Expectations." The agent would then only see these statements and not the assessment that contributed to achieving this outcome. Perhaps a text label is not appropriate and you would rather display a color palette that shows the agent if they have met the goals set for them. For example, purple could represent a neutral or acceptable score, orange could represent needs improvement, and finally black could represent unacceptable scores. The rating levels can be customized to show either the color or the phrase or both. The rating levels can also be tailored to the questionnaire, with as many or as few levels as required. Further information and instructions on how to configure scoring tiers are available on the page named Scoring Tiers.

The scoring system can be set individually in each scorecard. Detailed instructions are available on the page Setting Questionnaire Properties.