Search
Close this search box.
Search
Close this search box.

How Effective Is That Test Item? Item Discrimination Made Simple

The credentialing community relies on exams to find out which candidates are sufficiently competent, and which are not. Exam scores help determine whether or not they earn the desired credential. 

Well, with a little statistical magic – and we emphasize little – it’s possible to measure how well each test item (aka question and answers) contributed to the purpose of such exams. This is “item discrimination.” 

The statistical magic involved can be accomplished with various statistical tools. (Look for our simple example below.) Regardless of the method used, the result is often referred to as the discrimination index (DI). The theoretical range of the result is -1.0 to +1.0.

For example, an item that top performers always get correct and low performing candidates always get wrong would yield a DI of 1.0. 

Consider a different item that is correctly answered by the same proportion of high and low performers. Its DI would be 0.0, because it doesn’t discriminate.

What about an item that is always answered correctly by low performers while high performers always get it wrong? Its DI would be -1.0. It discriminates in reverse. 

Psychometricians will be quick to point out that negative DIs indicate invalid items. Perhaps the key – the answer that is supposed to be correct – isn’t. Or maybe the item is outdated. In any case, a negative DI tells us that the item is unfair to candidates.

Items almost never correlate perfectly to exam scores in either a positive or negative direction. Table 1 captures the range of DI values that Kryterion’s psychometricians typically observe.

Table 1
Discrimination Index (DI)Efficiency
< 0 (i.e., negative value)Invalid measure
0 – 0.09Poor discrimination
0.10 – 0.14Low discrimination
0.15 – 0.19OK discrimination
0.20 – 0.34Moderate discrimination
0.35 and aboveHigh discrimination
Source: Psychometric Services, Kryterion, Inc.

Calculating the Discrimination Index (DI)

There are multiple approaches to calculating a DI. However, the following – much simplified – example should clarify the general concept.

A hypothetical exam was given to 10 candidates. Table 2 lists candidates’ total score as a percentage in descending order. It also shows whether they got Question One (Q1) right (R) or wrong (W). The DI for Q1 can be estimated in three steps.

Table 2
CandidateTotal Percentage CorrectQ1 Responses
Candidate 1100R
Candidate 290R
Candidate 380R
Candidate 480R
Candidate 570R
Candidate 660W
Candidate 760W
Candidate 850R
Candidate 950R
Candidate 1040W

Step one is to find the number of correct Q1 answers for the top and bottom 30% of candidates, though the percentages that define the “top” and “bottom” can range from 25% to 50% in this method.

In this example, the top three performers (30%) responded correctly three (3) times. The three low scoring performers got Q1 right twice (2).

Step two subtracts the low performers’ result from the top performers’ result: 3 – 2 = 1.

In step three, divide that result by the number of performers in either group: 1/3 = 0.33.

The DI of Q1 is 0.33. Meaning, this item – see Table 1, above – is a moderate discriminator between high and low performers.

It’s that straightforward to get a rough estimate of DI. However, Kryterion recommends larger candidate samples and more robust methods to determine item discrimination indices for items being used on credentialing exams.

If you found this moment of psychometric clarity refreshing, we invite you to review our equally accessible discussion of P-value.

Have a question about the psychometric status of your test development project or credentialing initiative? Reach out to Kryterion’s Psychometrics Team here.

Our experts specialize in the delivery of clear and helpful insights. You might enjoy it.

Subscribe Today

Stay informed! Sign up to receive periodic business and
assessment industry news and updates. Unsubscribe at any time.

Take Control of Your Certification Program!

Want more information on how to start or expand a credentialing program? Fill out the form below and we’ll get back to you right away!

Read Reviews