QM Questionnaire Scoring Considerations

Overview

Eleveo’s Quality Management module offers two primary methods of scoring questionnaires, surveys, and quizzes: percentages and points. This document will detail how each method works (with a focus on questionnaires) along with the downstream impact on reporting.

Audience

This document is intended for those who have purchased the Quality Management module and are responsible for transitioning their current evaluation forms into the Eleveo Workforce Optimization solution.

Percentage Based Example (Preferred)

Benefits of Percentages vs. Points. Scoring Method

Cleaner Reporting 

• A 100% Scale is intuitive, whereas a points scale has an arbitrary maximum score 

• Questionnaire sections (groups) are easily compared, i.e. "apples to apples" 

*See below for points example if needed 

Weighting and Additional Considerations

Partial Credit Answers

When considering the potential answers for a given question, it is important to investigate the pros and cons of offering partial credit.  The following considerations must all be considered: 

• Is it fair to only offer full-credit and no-credit answers? 

• Is the business looking at performance on a question level (partial credit not advised), or looking at performance based on specific requirements/behaviors that may fall within questions (partial credit advised) 

• If partial credit answers are introduced, will this create confusion for employees? 

• If partial credit answers are introduced, will this introduce subjectivity on the part of evaluators? 

Keeping it Simple - One Partial Credit Choice 

• 100% credit for a correct answer 

  • 50% credit for a partially correct answer 

  • 0% credit for a wrong answer 

This approach's strength is also its weakness.  The conditions under which 50% would be scored is vague.  This can work both to your business' advantage and against it. 

This method is preferable to offering many gradations of partial credit with no mapping to specific requirements. 

Scores generated by this approach may not be reproducible when different evaluators evaluate the same call due to subjectivity. 

Tie it Together - Partial Credit Choices that map to specific requirements 

Example Question: Did the agent open the call properly? 

Example Description: Did the agent say hello, use the full company name and his/her full name? 

• 100% credit for covering three requirements 

• 67% credit for covering two requirements 

• 33% credit for covering one requirement 

• 0% credit for covering no requirements 

This approach is preferable for businesses where standards must be crystal-clear, reproducible, and defensible when challenged. 

This method can be difficult to maintain if the number of specific requirements within a question change; the form in Eleveo must be retired and a new form with updated scoring created. This approach also assumes all the requirements are equal, when in fact, some may be more important than others. 

Scores generated by this approach are easily reproducible by when different evaluators evaluate the same call. 

Maintaining Flexibility

Partial Credit Choices that allow for changing requirements 

Example Question: Did the agent open the call properly? 

Example Description: Did the agent say hello, use the full company name and his/her full name? 

• 100% credit for covering all requirements 

• 50% credit for missing one requirement 

• 0% credit for missing two or more requirements 

This approach is preferable for scenarios where the number of requirements differs from question to question, but an overall partial credit standard must be crystal-clear, reproducible, and defensible when challenged. 

This method is good choice if the number of specific requirements changes from question-to-question and/or over time; the form will undergo minor updates, but can continue to be used in Eleveo.  This approach can be punitive when there are more than two requirements per question, and it becomes more punitive as the number of requirements increases. 

Scores generated by this approach are easily reproducible by when different evaluators evaluate the same call. 

Points Based Examples (Alternative) 

*N/A for Points must be total credit or no credit  

Agent Skills Report using Percentage Scoring  

Note that each of the following must be true:

• Question sections must total 100%

• Each question section must have questions that total 100%

• Each question must have at least one answer and is considered a 0% to 100% potential.

With all of these in mind, it becomes easy to compare one group to another group, regardless of whether they have differing weights, since they must equal 100% based on the questions assigned to them (e.g. Verification may weigh 20% and Product Knowledge may weigh 60%, but one may still compare them against one another since each have questions that total 100%).

Agent Skills Report using Points Scoring 

Note that each question section may total a different amount based on the maximum potential of each question. For example, in the example provided on pg. 5, Verification can total a maximum of 8 points and Product Knowledge can total a maximum of 20 points. Thus, comparing the two of these is less useful. Trending reports are more for looking at how a given agent or team has done overall, or at the specific question section, or question, over time.

Not Applicable Calculation Considerations 

 Percentages  

Can effectively factor out the question so it has no impact on the Questionnaire. 

 Points 

Typically, a choice must be made as to whether a N/A answer gives the agent all points for the question or no points. 

Scoring Tiers

Color Scores 

 One method of scoring Evaluations involves utilizing pre-defined tiers and language most often seen in yearly or quarterly reviews.  In this method, rather than no scores or exact, a range of answers would include examples ranging from simple to complex.  Multiple color scoring tiers may be used for viewing review results, but only one is assigned as primary on a given questionnaire. This primary scoring method is the scoring methods agents see in the user interface. 

 For the below responses, the expectation is that the agent remains within the range that has been designated as 'Meets.’  Examples of a Percentage weighting in this case might be: 

Simple Example: 

Within these responses, the goal set for the agent would be set at the 'Meets', with expectations that the agent should strive for 'Exceeds'.  Examples of a Percentage weighting in this case might be: 

Complex example: