Human Resources
The Fairness Factor: Decoding Adverse Impact


The Fairness Factor: Decoding Adverse Impact


Loading

Jan 10, 2017 | by Katherine MacKay & Eleni Lobene


Adverse Impact (also known as Disparate Impact) occurs when members of a particular race, ethnicity, color, religion, sex, age, or national origin are systematically disadvantaged by an employment practice. Investigating Adverse Impact is how employers ensure that hiring practices, such as assessment usage and promotion systems, are fair. Critically important, right?

The tricky thing is, analysis of Adverse Impact can be approached in a variety of ways. The three most common are based on the 4/5ths Rule, the 2 Standard Deviation (SD) Rule, and Cohen’s d. Confused yet? No worries – we’ve created a cheat sheet below.

Adverse Impact Metrics

The 4/5ths Rule

The 2 SD Rule

Cohen’s d


   The magnitude of the
   difference between
   the pass rates of two
   different subgroups
   (i.e., impact ratios)

   The standard
   deviation (SD)
   difference between
   expected pass rates
   and observed pass
   rates

   Effect size estimate
   used to indicate the
   standardized
   difference between
   the means (on a test
   or assessment) of
   two subgroups
   (Cohen suggested
   that d=+/-0.20 be
   considered a ‘small’
   effect, +/-0.50
   represents a
   ‘moderate’ effect size
   and a +/-0.80 a ‘large’
   effect)


   Ratios of less than
   0.80 or above 1.20 
   are viewed as a
   substantial difference
   between pass rates

   A result of
   +/2.00SD or greater
   is a statistically
   significant difference
   between expected
   and observed
   selection rates for
   different subgroups

   An ideal result would
   be a small to
   negligible Cohen’s d

   If an effect size is
   small, it means there
   is relatively small
   difference between
   the means of the two
   subgroups on the test


   Simple to calculate
   and interpret

   Most commonly
   reported estimator of
   Adverse Impact


   Provides a statistical
   test of significance
   for a difference
   between pass rates

   Best when size of
   subgroups are small. 
   Overpowered when
   sample size exceeds
   100


   Provides a description
   of the magnitude of
   the difference
   between 
   the means and is not
   influenced by the size
   of each subgroup

The 4/5ths Rule is commonly used both by organizations and in litigation. Thus, Aon Hewitt suggests a review of impact ratios on a continuing basis. However, the Equal Employment Opportunity Commission does not require any one specific metric to determine Adverse Impact. In most cases, a combination of two or all three is strongly suggested as they each provide different types of interpretations regarding the magnitude of the difference between subgroup selection rates. We recommend considering all available results in making decisions!

Pros and Cons of Adverse Impact Metrics

The 4/5ths Rule

The 2 SD Rule

Cohen’s d


   Easy to calculate and
   interpret

   Requires relatively
   small samples per
   group (N of about 50)

   Provides a statistical
   test of the difference
   between pass rates

   Provides the practical
   magnitude of the
   difference between
   the averages of each
   group

   Not as susceptible to
   sample size but still
   use caution in
   interpreting results for
   small samples


   Can incorrectly
   indicate Adverse
   Impact exists when
   samples are lower
   than 50 per group

   Useful with small
   samples

   A  large disparity
   between subgroup
   samples may
   artificially inflate the
   effect size


   Only takes into
   account the subgroup
   means and standard
   deviations, but does
   not take into account
   specific cut scores or
   pass rates

 

Ensuring fairness is an absolutely critical part of any selection process. Mitigating Adverse Impact is a top priority for the scientists at Aon Hewitt who develop and implement assessments. We are curious to know–what are your experiences with various approaches to evaluating Adverse Impact in pre-hire testing?

Want to learn more about our Leadership & Assessment solutions?

About the Authors