Best Practices for Questionnaire Creation
Overview
Before creating a questionnaire, there are some things to consider:
What scoring method should be chosen? Percent or Points?
How should the weight be distributed?
Which answer options should be selected besides yes/no?
Should there be options for N/A, and how should this calculation be done?
This section contains questionnaire-specific best practices and tools for finding the most suitable configuration. It is based on the hands-on experience and expertise of people who work with this feature daily.
Scoring Considerations
It is recommended that you choose percentages as the scoring system. Therefore, each question and answer section is given a percentage by default. Percentage allows for cleaner reporting and more effective calculation of N/A scores. With percentages, it is possible to choose an N/A calculation method that effectively factors out the question. This is not possible with points, as you either give the full score or no score at all, which can negatively impact the overall score.
The option to calculate questionnaires by points is available but not recommended. To change the default settings, see Setting Questionnaire Properties.
When creating a questionnaire, three requirements must be met for it to be considered valid and usable:
The sum of the values of all sections of the questionnaire must equal 100%.
The sum of the values of all questions within each section must be 100%.
Each question must contain at least one answer.
A complete list of all status messages can be found on the Understanding Error Messages page.
Weighting Considerations
When deciding what weight to apply to question sections or the questions assigned to those sections, consider what kind of message the choice in weight sends to agents. Questions assigned a higher weight clearly indicate their importance to your organization.
It's important to note that weighting can be adjusted for each section as well as for each question within the section.
Notice in the example below how Customer Service has been given more weight. An agent will quickly understand how important this category is to the organization versus the Closing, which is assigned weight. While both sections impact the final score, a difference in weighting clearly indicates what areas are of importance in your organization.
Weighting is done not only at the section level but also for the questions within a section. In this example, we see that customer information verification is more important than the opening script, so the questions within the section are weighted accordingly.
Visit the page dedicated to Adding and Editing Question Sections, Questions, and Answers for step-by-step instructions on adding sections and questions.
Creative Answer Options – Beyond Yes/No
Have you thought about answer options beyond Yes, No, or Not Applicable? It is possible to create answers that receive partial credit or even fail all options!
When considering possible answers to a particular question, it is important to explore the pros and cons of partial credit.
Is it fair to offer only full credit or no credit answers? (Yes/No)
If partial credit answers are introduced, will this create confusion for employees?
Is the business looking at clear-cut processes or performance-based behavior?
Why not get creative in naming the answer choices? Instead of 'Yes', perhaps you could offer the option 'Meets Expectations' or even 'Exceptional'. Creative answer options beyond Yes/No answers can be beneficial in several ways.
To add an answer to a question, see Adding and Editing Question Sections, Questions, and Answers.
Keeping It Simple
Full-credit and no-credit answers refer only to the Yes/No answer options. In this case, the agent would receive full credit or 100% for a Yes answer and 0% for a No answer.
This option is the preferred response method when there is no ambiguity for both the agent and the analyst. For example, "Did the agent perform action X?" If there is a possibility of deviating from or only partially following the default, full-credit, and no-credit answers could have a negative impact on the agent's score.
Not Applicable (N/A)
Sometimes, situations do not apply. Therefore, be sure to include a not-applicable option. This option can be used when a question should not be used to evaluate an agent for the specific contact. Consider when a Not-Applicable option should be provided. It should not be created for questions that are required for all contacts.
There are several ways to calculate the N/A response. To determine the calculation method and for more information on the available settings, please refer to the Preferences page.
Partial Credit Choice
When an agent is required to follow a well-defined action during a conversation, the ability to use partial credit options allows the evaluator to select a specific answer based on the defined processes. Interpretation of outcome is then minimized.
Example Question: Did the agent open the call properly?
Requirement: The agent must say “Hello,” state the entire company name, and state the agent's full name.
Covered All Requirements – 100% Credit
Missed One Requirement – 67% Credit
Missed Two Requirements – 33% Credit
Missed Three or More Requirements – 0% Credit
This approach is preferred in businesses where standards are clear, reproducible, and defensible when challenged. Scores obtained using this approach are usually reproducible when different evaluators evaluate the same call.
Fail All/Reduce All/Success All
Fail All – Assigns a final score of 0 for the entire scorecard.
Fail Section – Assigns a final score of 0 for the entire section.
Reduce All – Reduces the overall max score by or to a certain amount.
The reduction amount can be customized. For additional configuration details, see the section Adding Answers on the page Adding and Editing Question Sections, Questions, and Answers.
These three options can significantly impact the agent's overall score. Since Fail All/Section can significantly affect the agent's overall score, this option can be an effective way to stress business-critical functions. Examples of this include functions that could lead to litigation. For example, if an agent does not verify a caller's information or HIPAA, there could be legal consequences. Therefore, assigning an overall score of 0 for this action may be appropriate.
Reduce All, on the other hand, is often used for more subjective questions, such as "Was the agent rude?" or "Did the agent provide exceptional customer service?". If the answer is "No", a certain percentage may reduce the overall score.
Success All – Assigns the highest possible score for the entire scorecard (for example, 100%)
Success Section – Assigns the highest possible score for the entire section.
These options can have an overall positive effect on an agent's overall score and are rarely used. They are typically used when the business expects all answers to be "Yes" and is looking for a quick way to pre-set answers to that question. The answer option "Success All/Section" can be selected for the first question. If you select this option, all questions will be automatically answered with "Yes" and the agent will get 100%, although modifications to the questions are possible.
The above options can be paired with the 0% sections option. In this scenario, a "Yes" answer would result in no credit, but a "No" answer would result in Fail Section/All or "Reduce All". This would have the advantage of enforcing business-critical questions. If 0% Sections are used, please note that all questions within each section must still equal 100%, even if they do not grant the agent any points.
Consider all possible options when creating questions and answers to ensure that the evaluations meet your business needs.