Human Resources
What's Your Favorite Band(ing) Method?


What's Your Favorite Band(ing) Method?


Loading

Dec 14, 2016 | by Jennifer Cavanaugh


No one is perfect, right? Well, the same can be said about selection tests. Can we really be upset with imperfect tests that are designed to help us hire the best people? Maybe a little, because these tests can be costly to an organization when qualified applicants are rejected or unqualified applicants are hired.

We also run into problems when sub-group differences on selection tests put minority applicants at a disadvantage and organizations at risk of legal challenge because of adverse impact (AI).

Fortunately, one popular option—“banding”—helps us deal with prediction errors, reduce adverse impact, and boost diversity. When we use banding, we categorize applicants’ test scores within a given range and treat all scores within that range as equivalent.

Banding Controversy

Although it has been used over the past 20 years as an alternative to the strict top-down approach, banding is a highly debated topic in personnel selection. The basic principle behind statistical banding is the generation of scores that are equivalent; however, it gets tricky to treat all scores within a band as equivalent when the lowest scores in the band are not significantly different from the scores in the band below. This is often referred to as the “fatal logical contradiction” and is one of the major sources of banding controversy.

The legality of banding within the context of AI is another controversial issue. While over 30 court cases have mentioned banding in a variety of contexts , no consensus has been reached regarding its legality. The U.S. courts generally do not support the use of banding where minorities are given preference. But some experts note that banding without minority preference may do little to reduce AI.

On the other hand, proponents of banding argue that it’s a legal alternative to race norming (the practice of adjusting cut-offs or otherwise altering the scores of candidates based on race), which is prohibited under the Civil Rights Act of 1991. Moreover, research suggests that banding may be particularly appropriate when selecting for professions that require content-specific testing (e.g., firefighters), and courts have ruled that banding is a way to simplify scores by eliminating meaningless differences between test takers.

Banding Calculations—The Technical Stuff

Okay, this part can get a bit heavy, but it’s important and we’ll try to keep it brief. Statistical banding uses the standard error of measurement (SEM) to compute the standard error of the difference (SED), both of which are used to determine the range of scores that fall in a given band (and are therefore considered “equivalent”). However, very little attention has been given to the commonly used formula for calculating SEM for statistical banding.

There are two primary methods for calculating SEM. First is the traditional SEM calculation from Cascio et al. (1991), which essentially provides a range of observed scores around the hypothetical true score of a test taker. Some researchers argue that the concept behind SEM-based banding violates the basic assumption of banding, which is to develop a range of true scores around an observed score (did you catch the difference there?). These same researchers argue that the formulation of the SEM derived by Lord and Novick (1968), referred to as the standard error of estimation (SEE), accurately assesses the range of true scores around an observed score.

What’s this all mean? Basically, it comes down to this… using the SEE-based banding formula may result in smaller bands than the SEM-based banding , which could mean differences in subsequent selection decisions. And that could impact important outcomes like adverse impact and return on investment (ROI).

Interesting argument, right? We thought so, too, so we took a look at how differences in banding calculations might impact an organization’s selection decisions by examining average test scores, AI ratios, and ROI across the two formulas.

So, what’d we learn? We learned that the picture is still a bit foggy.

  • SEE-based banding resulted in higher average test scores with a smaller range of scores falling in each band.
  • Both SEE- and SEM-based banding resulted in marginal AI for protected groups, but there was no clear pattern where one method consistently resulted in higher AI than the other.
  • The ROI analyses showed us that SEM-based banding resulted in higher dollar gains than SEE-based banding—which isn’t all that surprising for those of you who remember that SEE-based banding resulted in smaller bands, consequently causing fewer individuals to be selected using the pre-employment test.

 

If you’d like to learn more about our research and the implications of our findings, please contact Jennifer Cavanaugh to get a copy of her poster presented at SIOP’s 2016 annual conference.

Want to learn more about our Leadership & Assessment solutions?

About the Author