February 2010

By Jeff Shen, Head of Human Capital Practice, China 

"The ultimate throttle on growth for any great company is … above all others: the ability to get and keep enough of the right people".- Jim Collins

In this article

The need for talent assessment  Reliability and validity  Face validity Choosing an assessment tool  Measures of success  Conclusion  Contact

The recession has forced many companies into a defensive stance and triggered great survival instincts. They have had to deeply cut their resources and assets to a manageable level in order to simply survive. However, the real test begins in the aftermath of the recession. The companies that come out of the recession reinvigorated are those who have invested time and effort focused on re-thinking and improving their businesses. The companies that will continue to struggle are the ones that simply return to the ways things used to be.

With the recovery of the economy, more companies in Asia Pacific are forecasting positive plans for hiring, with China showing the most confidence in terms of their recruitment strategies.¹ To capture the best talent before your competition does, you will need a strong and effective recruitment practice. Recommended steps by experts include: anticipating your future talent needs based on your strategic business plan; identifying the specific competencies required in each position you need to fill; and developing a sufficiently large candidate pool to increase the likelihood you will find the right person for each job.²

With last year's redundancies and the current high unemployment rates, the pool of potential applicants will be larger than what most organizations have experienced in the past. Companies will need good assessment tools and apply best selection practices to identify the most qualified applicants. 

A number of companies, such as Hewlett-Packard and Yum!, are now using pre-employment assessments in new ways to enhance their competitive advantages in this recovery. Apart from putting together a best-in-class profile of the qualities of top performers, the companies attempt to identify those individuals with the highest probability for success by matching their scores with the profile during the external hiring process. Current employees are given these same assessments to see how they match up with the profile and to identify those internally who are in need of development and remedial training.³ In this way, the degree of error in making hiring and selection decisions is minimized.

Labor is typically one of the largest components of total operating expense, about 20 to 80%, across various industries. Consequences of selection errors include customer service complaints, increases in work-related accidents and injuries, high absenteeism, poor work quality, increased turnover, and damage to the reputation of the organization.⁴ Accurate assessments benefit employees who experience greater job satisfaction because they are matched to roles and organizations for which they are well suited.⁵

Why do many companies in Asia shy away from using assessment tools even though they can greatly enhance the quality of selection decisions? In our work with clients, we have encountered three reasons for this reluctance:

• Selection testing can be inherently technical and what it can and cannot achieve is often misunderstood. Companies frequently have unrealistically high expectations and subsequently become disillusioned with the return on investment (ROI) from using assessment tools. This has led to an under-utilization of formal assessments in the selection process. 
• Decision makers are frequently confused with the multitude of consulting firms selling different selection products and making contradictory claims about assessments.
• The field of industrial and organizational psychology in Asia is not as developed as in the US, UK or Australia. As a result, hiring managers in Asia frequently do not have the experience or knowledge to evaluate the effectiveness of alternative assessment tools.

  Back to top 

Reliability and validity

Two features of an assessment are critical in determining its quality and usefulness - its reliability and validity. Reliability will tell you how trustworthy a score on a test will be, while validity will tell you how good a test is in assessing a particular situation.

Reliability means that the score an applicant receives on the assessment is consistent. For instance, if the applicant were to take the test more than once, with no performance intervention, the test score should be similar each time. Reliable assessment tools produce dependable, repeatable, and consistent information about people. Test developers have the responsibility of reporting the reliability estimates, how reliability studies were conducted and the characteristics of the sample group.⁶

Validity is the relationship between the results of an assessment and current or future performance on the job (or some other work-related outcome, such as training success, productivity, absenteeism, or turnover). Validity is the most important issue to consider when deciding whether to use a particular assessment tool. If a particular assessment does not provide useful information about how an individual will perform on the job, it is of little value to the organization.

Specifically, validity demonstrates how well an assessment measures a job-related characteristic. All else being equal, a person scoring high in an assessment should perform better than a person scoring low, if an assessment has been demonstrated to be a valid predictor of performance on a specific job.

You cannot draw valid conclusions from a test score unless you are sure that the test is reliable. Even when a test is reliable, it may not be valid. You should be careful that any test you select is both reliable and valid for your situation. Therefore, use only assessment procedures and instruments that have been demonstrated to be valid for the specific purpose for which they are being used. For example, the test you use to make valid predictions about someone's technical proficiency on the job may not be valid for predicting his or her leadership skills.

Back to top

Face validity

When candidates participate in an assessment process, they are not the only ones being evaluated; the organization is being evaluated as well. Candidates who complete an assessment process leave with impressions about the face validity and overall fairness of the assessment procedure. Whether they believe they had sufficient opportunity to display their job-related competencies is likely to color impressions.

Organizations using grueling assessment procedures may end up alienating applicants. It is important to recognize that candidates use the assessment process as a means to gather information about the prospective company. Failure to recognize this in selecting job-related assessments can be very costly to companies who aim to attract and retain the best talent in this highly competitive market.

Back to top

Choosing an assessment tool

When selecting an assessment tool, be sure to consider the reference groups used to develop the test and the jobs for which the test has proven to be valid. For example, an assessment designed to predict the performance of a general manager requiring problem solving may not allow you to make valid or meaningful predictions about the performance of frontline sales supervisors. Therefore, it is important to know whether the kind of problem-solving ability required for the two positions is similar or different.

In determining the appropriateness of an assessment for your target group/s, consider factors such as occupations, job families, reading level, cultural differences and language barriers. We recommend that you only use assessments that are valid for what you intended, are job-related, and adhere to legal and employment guidelines in your country.

Back to top

Measures of success (Return on Investment)

Finally, to measure the ROI of assessments in your organization, we recommend that you take these steps:

We encourage you to use the following data for your measurement of assessment ROI:

• Input measure
  • Estimate the cost per hire of the assessment currently in use and the costs of purchasing and designing the assessment as well as the labor involved in administering these assessments.
  • Estimate the reductions in costs as a result of using these assessments when compared on a year-on-year basis. Such cost reduction might, for example, take into consideration staff turnover, recruitment cost, and training cost.
• Output measure
  • Consider conducting criterion-related validation studies that reveal the relationship between the assessment results and the actual job performance or a desired outcome, such as the new hire retention rate, after utilizing an assessment tool.
  • Consider conducting a stakeholder survey in order to measure the achievement of intended outcomes, such as the quality of hire, staffing efficiency, or the time taken to achieve the desired level of productivity.

Back to top

Conclusion

As my colleague Richard Payne recently described in Competition for Asia's talents: The risks that lie ahead (Nov 2009), it is critical that companies speed up their process in creating the best tactical and strategic system to identify and select talents. Incorporating a reliable and valid set of assessments into your selection process will help your organization to create a competitive advantage by hiring the right talents. 

Back to top

Contact

For more information about Aon Consulting's capabilities and experience in talent assessment, please contact Jeff Shen, Head of Human Capital Practice in China, at jeff_shen@aon-asia.com. 

Back to top

Notes: