What happens when an employer uses a flawed test to select job candidates? There are many ways that selection tests can be flawed. For example, an 'invalid test' fails to measure the factor that it is intended to measure. A 'biased test' favours some groups – as defined, for example, by age, gender or race – over others. Writing in Personnel Psychology, Herman Aguinis and Marlene Smith have now devised an on-line computer programme that allows employers to predict the consequences of using an invalid or biased test, in terms of selection errors and adverse impact.

Consider an organisation that used a biased test as if it were unbiased. In this case, the organisation would select recruits based on a single cut-off point (e.g. they must score x number of points above the population test average). But if the test is for some reason biased against women, then men and women should be judged against two separate test averages – women against a lower average and men against a higher average. Not doing so will mean that a proportion of women who ought to have been selected will be missed, while a proportion of male applicants will have been selected when they shouldn't have been.

The computer programme developed by Aguinis and Smith allows employers and researchers to input the relevant group averages for a given test, and to calculate the impact on candidate selection of using the test as if it were unbiased. For example, given the example above, the programme would calculate what percentage of suitable women would have been missed and the percentage of unsuitable men who would have been picked, depending on the chosen test score cut-off point.

Aguinis and Smith argue that researchers usually tackle issues of test bias, test validity, selection errors and adverse impact individually or in pairs, whereas their programme integrates these factors for the first time. "We have not been able to locate any published source that investigated the interrelationship among all four of these concepts explicitly", they said.

The authors conclude that their programme will be useful to both policy makers concerned with societal issues and test developers and employers concerned with accuracy. "Our framework is sufficiently broad to allow each of these stakeholders to answer key questions about human resource selection tests", they said.

Link to the programme: http://carbon.cudenver.edu/~haguinis/selection/

Aguinis, H. & Smith, M.A. (2007). Understanding the impact of test validity and bias on selection errors and adverse impact in human resource selection. Personnel Psychology, 60, 165-199. http://dx.doi.org/10.1111/j.1744-6570.2007.00069.x