Human Resources
Testing When No One is Watching: The Good, The Bad, The Validity


Testing When No One is Watching: The Good, The Bad, The Validity


Loading

Nov 24, 2016 | by Jennifer Cavanaugh and Sarah Strahan


When searching for new talent, are you or your clients still using paper forms and tests?   While this was once the norm, the growing popularity and accessibility of laptops, tablets and smart phones has led many organizations to embrace online recruiting and selection tools. According to a recent Global Assessment Trends Report, 81% of companies currently use online assessments and 23% of companies indicated that candidates are requesting to complete assessments using mobile devices (Kantrowitz, 2014). In response to the growing technological capability and candidate needs, more and more organizations are moving to unproctored internet testing (UIT).

Despite the growth in popularity of online and unproctored testing, it is important that I/O psychologists are reminded that there are many organizations that may have concerns about this approach. Some, for example, may struggle to implement online and unproctored tests due to practical limitations and challenges. Before deciding whether UIT is right for your organization, it’s important to consider some of the following information.

The Good

  • Allows for a streamlined process for applicants, in which they can go to an employer’s website, review job postings, fill out an application, and take a pre-employment test in one sitting.
  • Organizations are able to reach a larger applicant pool more quickly and reduce time to hire.
  • Reduces the costs associated with pre-employment testing, such as paying for computers or tablets to administer online tests and paying test administrators to schedule, administer and proctor testing sessions.
  • Can easily administer new forms of tests, adjust cutoff scores, and replace or correct items.
  • Makes the application and assessment process much more convenient for applicants who are not be located near the organization or are currently employed and have limited time during the day to complete applications or employment tests.
  • Allows applicants to engage in the process from the comfort of their own home, without taking time off work or alerting their employer to the fact that they are applying for other jobs.
  • Allows for standardized accommodations for applicants with disabilities.

The Bad

... or Not So Bad

Cheating: How do you know the person taking the test is actually the applicant?
  • Cheating is more of a concern for cognitive than non-cognitive tests and research suggests the occurrence of cheating is low.1
  • Verification testing can be used to detect cheating after the fact.
  • Cheating can be deterred by using honesty clauses with strong language around the consequences of cheating and the use of verification testing.
Test Security: How do you know the test is secure? Don’t these tests get out on the internet?
  • Test security is an issue for paper-and-pencil tests as well since copies inevitably go missing and can end up getting passed around or published online.
  • Test security can be increased by using multiple/alternate forms of assessments with large item banks; online testing makes randomizing test forms/item banks easier.
  • Computer adaptive tests increase test security because each applicant sees a different test.
Accommodations: How can you provide accommodations online? How do you know a candidate needs them?
  • As with any testing scenario (online or paper/pencil) candidates need to request accommodations; companies should include very clear language around this in communications to candidates.
  • Many online testing platforms, including our Global Assessment Talent Engine test platform, are 508 Compliant, and can easily handle accommodation needs by providing additional time or untimed testing options or allowing for changes in screen size Companies should have a proctored option available for those candidates who require assistance as a part of the accommodation.

The Validity

In addition to advantages and disadvantages of UIT, a large body of research has been devoted to discovering whether, and under what conditions, tests delivered in an unproctored environment are valid, due to concerns with cheating and unstandardized test environments. Experts suggest that cheating is more of a concern for cognitive tests than non-cognitive tests, as there is no “right” answer on non-cognitive tests (Tippins, et. al., 2006).

It is also suggested that candidates have equal opportunity to inflate scores on non-cognitive tests, regardless of whether the test is proctored or unproctored, therefore the administration (i.e., proctored vs. unproctored) method does not differentially affect the validity of non-cognitive tests. Finally, although it is impossible to ensure candidates are taking UITs in ideal settings (i.e., quiet, distraction-free environments), research suggests that very few candidates report being bothered by noise or distractions while testing in unproctored settings (Tippins et. al., 2006).

Overall, findings suggest that, despite these concerns, online, unproctored tests are able to predict job performance as well as proctored tests (Kaminski & Hemingway, 2009; Tippins, et. al., 2006; Weiner & Morrison, 2009).

How to know if UIT is right for you:

Unproctored internet testing is growing in popularity, but each organization needs to determine whether it is the right choice for them. If you’re thinking about “making the switch” to unproctored internet testing, you may want to consider:

  • Are you using cognitive tests? If so, are there multiple versions/large item banks or are they adaptive?
  • Will it work for your candidate pool? Do they all have access to the necessary technology and would it deter or encourage them to apply?
  • Have you considered using a hybrid approach, with some assessments online/unproctored as an initial hurdle, followed by proctored testing later on?
  • What is your organization’s culture/attitude toward testing, in general, and online/unproctored testing, specifically? Will you receive the support you need from your stakeholders to implement a new testing solution?

 

For more information on proctored or unproctored tests, including available computer adaptive tests, visit us here.

Want to learn more about our Leadership & Assessment solutions?

About the Authors