1. What can standardized employee selection testing do for our company?
2. How can RA’s testing be any better than my current selection process?
   
3. How can selection testing take the place of a good interviewer?
   
4. How can I justify the costs of selection testing?
   
5. How good are RA’s validated tests as predictors of job performance?
   
6. How do you measure whether a test is a good predictor of job performance?
   
7. What is a good cutoff-score for applicants?
   
8. Can RA’s tests support our workforce diversity efforts?
   
9. Is testing legal?
   
10. What is "Adverse Impact"?
    
11. Are Resource Associates’ tests fair?
    
12. How does testing compare to other selection methods?
    
13. How should test scores be used to make hiring decisions?
    
14. When should test scores not be used as a Pass/ Fail criterion?
    
15. What are the first steps in starting a testing program?
    
16. What is the best way to integrate the testing process into our other selection activities?
    
17. Don’t people exaggerate their “good” traits on personality tests?
    
18. Shouldn’t an aptitude test measure the same thing that the person would have to do in the job?
    
19. I took the personality test for practice and got mostly low scores -- what's wrong?
    
20. People complain about how long the tests take.  How can I justify this requirement?

Questions about Selection Test Validation Studies

1. What is a Validation Study?
   
2. Does our company need a validation study before starting selection testing?
   
3. Is it possible to do some preliminary work to ensure that tests are valid before we go through a full validation study?
   
4. What if our company doesn't want to do a validation study?
   
5. What are the various types of selection test validation studies?
   
6. How do we know what type validation study is right for our company?
   

What can standardized employee selection testing do for our company?

Overall, RA’s standardized selection assessments are designed to enable you to identify and retain high performing employees – a critical component of profitability.

In order to identify the best employees, RA’s tests can measure mental aptitudes, which are one of the most powerful predictors of success on the job. RA’s personality scores can help you understand how the person is likely to act at work over a long period of time—after they have shed their “best behavior” for the interview!    

It doesn’t do much good to identify and hire the best employees if they are going to quit after working for a short period of time. Job and Culture Fit are important elements of employee retention, as well as each individual applicant’s Resiliency. RA’s test can measure these so that you can reduce employee turnover.

>Back To Top

How can RA’s testing be any better than my current selection process?

Statistical evidence shows without doubt that standardized selection testing improves upon what companies are already doing in their selection. Not only does standardized employment selection testing provide objective data from which to base informed hiring decisions, but it also creates efficiency by “weeding out” poor candidates before they are interviewed by HR.

>Back To Top

How can selection testing take the place of a good interview?

The short answer is that it can’t.

Most companies use RA’s selection tests in two ways. First, as a pre-screen for candidates who appear to meet the minimum qualifications, yet their Job and Culture fit are unknown. In this case, the company can see the Job and Culture fit results from a number of candidates, and then select the “best fits” for the interview stage.

The second way the test results are used is during the interview itself. RA’s test results provide “red flags” to be discussed during the interview along with suggested interview questions to probe those “red flags”.

>Back To Top

How can I justify the costs of selection testing?

Testing can dramatically increase the percent of superior employees hired, while at the same time reducing the number of “bad apples” hired.

Superior employees are an excellent investment of company resources because they:
  

• Learn more quickly so the investment in training dollars is spent wisely.  
• Use their job knowledge to make wise decisions to increase your company's profitability.
• Make fewer errors so they are more efficient in their everyday work.
• Are observant and thoughtful about their work so they help you improve the processes and save even more money in the long run. 
• Are dependable and reliable.
• Make better team members.
• Are a good fit for the job and the company culture, so they have higher morale and are less likely to quit.

Published research shows that the value of an average worker is approximately 2X their annual salary, so if you could minimize the chance of hiring one “bad apple” per year, that could more than justify the cost of the selection testing program for one year.

Here are just 3 examples of how selection testing pays off:

• In a retail-clothing store, RA’s tests were able to identify superior salespeople produce on average 500% more sales than the average worker. 
• RA’s proprietary STAY-Score generated evidence that our client could cut turnover by more than half.
• A large convenience store chain realized over 3000% ROI on testing of their store managers.

To read about more examples of how RA testing has produced tangible savings, CLICK HERE to view RA Case Studies.

How good are RA’s validated tests as predictors of job performance?
The graphic below provides a graphic representation of the relationship of typical scores from a valid test and job performance.  The higher a person’s score on a valid test, the more likely he or she will be a Superior Employee. 

Successful Hires are those people who passed the test and turn out to be good employees.  These are your "Superior Employees."
Rejects were people who failed the test and who would, indeed, have been undesirable employees. These are the "Bad Apples" you want to avoid.
False Negatives are people who would failed the test and would have been good performers if they had been hired – they are "Missed Opportunities."
False Positives are people who passed the tests but should not have been hired because they turned out to be bad performers -- these are "Bad Hires."
RA’s Validated tests are not perfect predictors, but they almost always give you an added measure of predictability over less valid hiring practices.

>Back To Top

How do you measure whether a test is a good predictor of job performance?

Typically, a correlation coefficient is used to assess whether there is a statistically significant relationship between test scores and job performance. It is represented as a small r, and ranges from -1 to +1 with no relationship being defined as r=0.00.  The correlation in the example above is about r=.45. In addition to looking at the absolute value of a correlation coefficient, you also need to know if it was significant, i.e., if the probability of random error is less than 5%.  Typically, if a correlation coefficient is significant at the 5% level then one asterisk will be used, e.g., r=.25*, or if it is significant at the 1% (which is an even more rigorous test of significance) then two asterisks will be used, e.g., r=.31**.

What is a good “cut-off score” to use in testing?

In the example above, a 50th percentile cut-off score is used. This is just one example of where a cutoff score could be set -- where you set the cutoff score for your testing program depends on your situation.  The graph also shows that:

• If you set a low cutoff (test is “easy”), you will increase the number of “Bad Apples” that must be interviewed, but you will have fewer “Missed Opportunities”.

• If you set a higher cutoff (test is “hard”), then there is a strong likelihood that almost all of the people you interview will be “Superior Employees”, but you increase the number of “Missed Opportunities”.

To see how different “Cut-Off Scores” can be used, click here.

>Back To Top

Can RA’s tests support our workforce diversity efforts?

RA’s tests can be an integral part of your workforce diversity efforts. Using a test that has been validated and checked for adverse impact reduces the chance for human bias to be introduced into the hiring process (this of course depends on how each company uses the test results in their overall decision making process).

Some people do better on selection tests than do others.  Better test performers usually do better work on the job.  However, sometimes different groups of people perform better or worse than other groups of people.  If that is the case in your selection process, then it is especially important to use tests that have been validated either specifically for your company, or validated in similar settings. RA’s experienced staff of Industrial Psychologists can work with you to provide either type of validation data, upon request.

>Back To Top

Is testing legal?

Yes, testing is legal.  Title VII of the Civil Rights Act of 1964 and the Uniform Guidelines on Employee Selection of 1978 both include pertinent guidelines and instruction on ways in which testing is appropriately used to make hiring decisions.  If you follow these guidelines, you can feel safe that your testing program is “legal”.  You may find it interesting that in over 25 years of doing business in this area, Resource Associates has never been sued.  Should any legal questions or challenges arise, our Industrial Psychologists are ready to help you generate a rapid response.

>Back To Top

What is "Adverse Impact"?

The concept of Adverse Impact is legally defined Uniform Guidelines on Employee Selection, 1978, and is overseen by the Equal Employment Opportunity Commission (EEOC). Several formulas exist to define adverse impact (e.g., 80% rule and Two Standard Deviation rule). In our society, we are concerned about whether minority groups have as good a chance to be selected into a job as majority candidates. 

If a testing process is judged to have adverse impact, then a minority group typically has a statistically significant lower chance of passing a selection test than someone in the majority group.  If the EEOC determines that adverse impact is evident for a particular selection test, then the company would need to demonstrate evidence that the test is a valid predictor of job performance. Presence of adverse impact should not, by itself, eliminate a valid test from your hiring process.

>Back To Top

Are Resource Associates’ tests fair?

Yes, Resource Associates’ tests have demonstrated their fairness time and time again.  Fair tests are ones where the research shows them to be good predictors of future job performance.  High scores on fair tests mean that the person has a better chance of being a Superior Employee in the company, whereas unfair tests either don’t predict well or they over-predict or under-predict certain groups more than others.  The researchers at Resource Associates help you create a testing program that can demonstrate good predictability for your particular job.

>Back To Top

How does testing compare to other selection methods?

Studies published in leading scholarly journals support what we see repeatedly with our own clients: standardized testing of aptitude is more reliable, accurate, and cost effective than other selection tools.  It is a well-established scientific fact that cognitive ability is the best predictor of long term success at work.  And, using valid, reliable measures of personality builds on that – it doubles your ability to predict who will be an outstanding performer on the job!  Resource Associates selection testing can add value to whatever you are already doing in your selection process.

The graphic above shows that standardized testing for cognitive ability is superior to all other selection practices. (See Hunter and Hunter, 1984 and Ones, Viswesvaran & Schmidt, 1993)

>Back To Top

How should test scores be used to make hiring decisions?

We recommend that scores on a validated test be incorporated into your hiring process as only one component of the overall decision. That being said, sometimes it is appropriate for you to establish a Pass / Fail Criterion on the test (based upon one or more Cut-Off Scores). How should that score be set?  This is a judgment made by the company (we will certainly provide useful advice) based on several factors:

• What are your hiring goals?  Do you seek to staff the position with a person of superior talent or are you more interested in weeding out people who are likely to be poor performers? Different hiring goals lead to different test cutoff scores.   A company with a lot of high tech equipment that employees are supposed to manage using a  great deal of independent judgment will probably find that setting a high cutoff score would reap many rewards.  On the other hand, where most people can succeed in the position, your goal may be simply to weed out those who are incapable of learning the tasks or those who have poor retention potential--here a low cutoff score would be appropriate.
• How healthy is your applicant flow?  If you have an adequate pool of job applicants, you can afford to weed out a lot and still have a reasonable number of people from which to choose (but it may create negative attitudes in the community if people get the idea that no one can pass your test). If you do not have many applicants, then you may have to set the cutoff score lower and possibly select more “marginal” candidates.
• How much time and money are you willing to spend on recruiting?  If you desire to hire "cream of the crop" individuals, then you will need a large pool to draw from (advertising cost) because a high cutoff score on the test will create a high rate of test failures.  Setting a “too low” cutoff score can increase the amount of “wasted” interviews (labor cost) – and put your workforce at risk for hiring “Bad Apples”.
• How crucial is it for the job to be staffed with Superior Employees? 

Balanced Approach:   We have found that many of our clients find it useful to establish a target percentage of the applicants to weed out of the hiring process.  This is accomplished by establishing a baseline test score distribution for your applicants (usually 100 or more applicants), then setting test score cutoffs based on what percent you wish to fall into three categories:  (a) unacceptable candidates, (b) marginal recommendation, and (c) full recommendation.  
Note: The Industrial Psychologists at Resource Associates can work with you to create a customized approach to setting cutoff-scores for an Overall Recommendation on test reports. 

When should test scores not be used as a Pass/ Fail criterion?

In some hiring situations, test score information will simply be used as a guide in the interview process, providing insights about potential problem areas that need to be probed during the interview. In small work teams, where a major criterion for hiring is whether the candidate will get along well with teammates, a pass-fail designation would be inappropriate.

>Back To Top

What are the first steps in starting a testing program?

The Resource Associates Industrial Psychologist who works with you to set up a selection testing program will guide you through this process. Step one is to thoroughly understand the job requirements. Since Resource Associates has a large number of validated tests, we can help you choose which tests are the best ones to provide the information you need to make wise hiring decisions. If a test validation study or a benchmarking study is called for in your situation, our experience and knowledge in this area will be invaluable.

>Back To Top
 

What is the best way to integrate the testing process into our other selection activities?

Most companies find that selection tests are most cost and labor effective if it is used early in the selection process. Information on cognitive ability and personality can help you identify those people on whom you should focus most of your time with reference checking, interviewing, providing a tour, etc.  Likewise, this information gives you valuable clues to help you probe into potential problem areas during other contacts with the candidates.

>Back To Top

Don’t people exaggerate their “good” traits on personality tests?

To some degree, almost everyone wants to think of themselves in a positive light and most of us want others to think well of us.  The scoring formulas we use to measure personality are designed to take this “self-inflating” phenomenon into account.  Also, the items (questions) are designed so that not everybody answers the same way--those differences between people give us the information on which to distinguish between people and make predictions about future behavior on the job.

>Back To Top

Shouldn’t an aptitude test measure the same thing that the person would have to do in the job?
Sometimes YES.  If the candidate must know specific things prior to being hired, the test should present questions about actual job content. For example, a bus driver should be a competent driver before being hired into the job.
Usually NO.  In most jobs, people are given the opportunity to learn most aspects of the job during on-the-job training. In this case, it would be unfair (and possibly illegal) for a job candidate to be expected to know exactly the same things as a current employee.  Therefore, aptitude tests for selection purposes are good predictors of whether the person would be quick to learn new job information, capable of using that knowledge to reason through problems in the job, and efficient in managing that type of information.

I took the personality test for practice and got mostly low scores, what's wrong? 

First, our personality tests are designed for people who in the “job applicant” frame of mind. A “job applicant” often will try to figure out what the "best" answer is and then pick it in order to present themselves in glowing terms. For example: “never late for meetings, work is always perfect, never get annoyed with people, always work hard”, etc. So, if you are responding to the inventory in a very "honest" frame of mind, your scores will be low.

Second, sometimes the better answer is on the left and sometimes it is on the right.  When people are taking a personality just to get a feel for how it works and how the scores will turn out, sometimes they are not as careful as a real job applicant would be.  Averaging in scores for a "wrong" answer here and there in the set of numbers will bring the overall score for that dimension way down.

Last, we are the first to admit that personality inventories are not perfect, but they are valid in most cases. Being a valid predictor means that they offer some value above and beyond the pre-existing hiring process.   And, bear in mind that no predictor is perfect -- even face-to-face interviews.  The best example of that I give is a person's choice of marriage partner: we spend a tremendous amount of time getting to know a person before deciding to marry; but for about half the population, that choice / prediction turns out to be wrong! Hopefully our clients can use the information presented in a personality report as a general indicator of the type of person the candidate will turn out to be months and years down the pike.

>Back To Top

If people complain about how long the tests take.  How can I justify this requirement?

The amount of time that needed to take a test depends greatly on the type or types of tests that you the company decide to use. Some of our clients use an exhaustive series of tests that can take up to two and a half hours – some rely solely on a 15 minute test!

Adequate testing provides benefits to both the company and the prospective employees. The company benefits from reduced interviewing costs, higher likelihood of hiring “Superior Employees”, decreased likelihood of hiring “Bad Apples”, and reduced Turnover. The prospective employee benefits from ensuring a good fit with the company and the specific job, so asking a candidate to invest some of his or her time to help make sure a hiring decision is a good one is a small thing to ask.

>Back To Top

Questions and Answers About Validation Studies

What is a validation study?

A validation study generates evidence that the pre-employment tests you are using are good predictors of future job performance.  When a testing program has been “validated” you have documented assurance that the information used to make hiring decisions is predictive of future job performance and is thus fair to job candidates.  

Typically, a validation study includes:

• A job analysis.
• A rationale for tests used.
• Information on how employees or job candidates perform on the tests.
• A demonstrated relationship between test scores and some measure(s) of success on the job --usually supervisor ratings or other objective measures.
  
  

>Back To Top

Does our company need a validation study?

Every company that uses standardized testing for selection purposes needs some documentation to support their compliance with EEOC and / or ADA Guidelines, so any and all employment testing  must be “validated.”

Resource Associates’ tests have already been validated in many companies for many jobs, and as a Resource Associates client, you have access to this accumulated validation evidence.  For most companies and most types of jobs, no additional validation studies are necessary.  

If a client company feels that they have a “unique” type of job or that challenges are likely to arise, a full-scale validation study can be conducted (provided that they have a large enough workforce to allow a suitable research study). Resource Associates can, upon request, provide the expertise and assistance to complete the full validation study.

The Industrial Psychologists at Resource Associates can help you decide what type of validation is best for your specific situation.

>Back To Top

Is it possible to do some preliminary work to ensure that tests are valid before we go through a full validation study?

First, be aware that Resource Associates has accumulated validation evidence from many companies and for many jobs. It is likely that a Resources Associates professional can “match” and existing validated test to one of the jobs at your company. 

“Benchmarking / Profiling”. Is another tool that can be used prior to (but not in place of) a validation study. Rather than going through the process of testing a large number of current employees, many companies elect to establish a profile of their best performing people.  With this approach, you would identify about 15-20 of your best performers, give them the test, look at their pattern of scores, then try to establish a target profile against which to compare job candidates’ profiles.

Please note that if your testing program was challenged in court, a benchmark study does not meet the EEOC requirements for a validation study.

>Back To Top



What if our company doesn't want to do a validation study?


>Back To Top

What are the various types of validation studies?
Content Validation Studies:  Sometimes a validation study involves simply documenting that the content of the job is similar to jobs for which the tests have been previously developed and validated.  This approach is often used with a new company where no employees have been hired, or a situation where there are only a small number of employees in one job category.
Concurrent Validation Studies:   Sometimes, a validation study involves using the tests on current employees and correlating their test scores with measures of their job performance.  When the company can provide a large enough group of employees in one job category, we can do a full-scale correlational study.  The Concurrent Validation Study requires large sample sizes (150 or more people in one job category).
Predictive Validation Studies:  Other times, we can start with job candidates and follow them through the job orientation and initial training process to evaluate whether the tests could predict which people were more successful than others.  This type of study also requires a large sample size (150 or more in one job category.)
Criterion Related Validation Studies:   This is another name for what was described above.  Anytime you take test scores from a group of people, then correlate them with an outcome measure of some type (e.g., supervisor ratings, error rates, annual sales, etc.) then you are doing a criterion-related validation study.

>Back To Top

How do we know what type of validation study is right for our company?

The Industrial Psychologists at Resource Associates are experienced in helping you make good decisions about appropriate kinds of test validation.  The final choice of which strategy is best depends on:

• How many employees are currently working in this job. A research project of this type requires at least 150 people in the job family for which the testing program is being implemented. 
• Whether there are any reasonable ways to reliably measure job performance.  If yours is a new company, you may not have employees yet.  Or, your employees may not have been on the job long enough for you to judge how successful they are. 
• The rate at which new employees are brought into the company.   If new employees are not hired very often, it would not be feasible to do a predictive validation study.
• The length of time before you can reasonably judge the value of a new employee. If it takes only a month or two to evaluate the worth of an employee, then a predictive study is a reasonable choice, vs. if it would take several years, a predictive study becomes more difficult and time consuming to complete.

Your representative from Resource Associates can help clarify the right strategy to take to ensure legal defensibility of your testing program.

>Back To Top

Do you have more questions?  The Industrial Psychologists at Resource Associates would be happy to talk with you about your particular situation. Please call us. Toll free:  1-866-840-4749