Curves and Speedbumps in Training: Normative-based Testing Versus Criterion-based Testing

The training industry has many approaches to measure the effectiveness of training on learners. However, to obtain the most accurate measurement of skills, HR and L&D professionals must pay attention to the normative method for testing and assessments. The two best approaches to accomplish this are norm-referenced testing and criterion-referenced testing.

What is the problem with normative-based assessment?

The normative model compares test results to a corpus of other completers of the same testing instrument. It is similar to grading on a curve. However, critically the curve for corporate learners is an individual’s livelihood and career. Given the importance of such grading, is it useful or fair to compare learners to others with completely different backgrounds, experiences, capabilities, knowledge, and skills? For example, what about a situation where your performance scores are judged against a cohort with a high preponderance of MIT grads taking that test? Or what if you are judged when an offer to win a free Xbox is circulating the dorms at MIT and all of a sudden your scores are mixed with future tech-titans? Or imagine that there are a bunch of 10-year-old boys wanting to win that same Xbox and the ad for the Xbox makes its way through a junior high-school. The likelihood is that this will produce a false measurement as the results are lower than those generated in a corporate scenario. Granted, these are two extremes, but the point is you never know the data pool in which your results are mixed. Would you want your career based on this sort of measurement?

What is a criterion-based assessment?

In a criterion-based scenario, if you answer a question right or complete a practice lab component correctly, you get a positive score. If you get enough positive scores, you pass without any artificial comparison to 10-year-olds or future tech-titans. The chart below depicts the difference between measuring people against others versus comparing an individual’s knowledge and skills against a pre-determined standard.

Source: Renaissance

You can see how the average of the taller people will bring the smaller people up to or above the line rather than just measuring each against their ability to answer questions correctly or to complete a practice assignment. Which would you want your career? For corporate learners, the only accurate measure of proficiency is to demonstrate both the knowledge obtained and the ability to put that knowledge into practice with a lab, case study, or project that measures true applicability.

Why criterion-referenced testing is better for the workplace

Let's explore normative reference tests a bit. An individual who measures themselves against an IQ for a role or skill may find themselves in a high percentile, but not pass in a proficiency test. As a learner or an employer, do you care that someone ranks high on a scale or percentile but is unable to proficiently do the job or role they are expected to perform? With normative testing, learners can perform poorly and pass as long as they are not the worst-performer. Sounds like a solid plan to build your tech future. Not.

Source: Renaissance

Our educational systems typically use criterion-referenced tests because the goal is to determine if schools are successfully teaching students what they are expected to learn. Why would you use any other measure to evaluate the knowledge and skills of your most valuable corporate assets -- your employees?

When you are navigating all the options in corporate learning and training, make sure you know what goes into a quotient, IQ solution, or other normative measurements. Likewise, make sure you know if your learners can pass the criterion-referenced testing at a proficient score. Responsible training providers provide assessments and testing that looks at the objectives for each course and then tests to ensure those objectives have been met or the knowledge has been acquired and retained.

Mike Hendrickson is the VP, Technology & Developer Products at Skillsoft.