When inferences about a person are made based on their performance on a psychological test, this is usually after they have completed that test only once. But now Professor Timothy Salthouse at the University of Virginia has demonstrated just how much variation there is in a single person's repeated performance on the same neuropsychological test, a fact he says could dramatically undermine evaluations based on single assessments. Writing in the journal Neuropsychology he has called on more attention to be given to this 'within-person' variability.

Salthouse gave 16 psychological tests, including measures of memory, vocabulary and spatial ability, to 1600 people aged between 18 and 97 years. The participants took three versions of each test within a period of two weeks (an earlier experiment had established the equivalence of the different versions). The fluctuation in performance by each participant on a given test was striking, being equivalent to the variation in performance that would be expected across an age range of between 10 and 29 years. In other words, it was as though the same person performed like a 19-year-old on Monday, a 46-year-old on Thursday and then like a 30-year-old, the following week.

'I don't think many people would have expected that the variability would be this large, and apparent in a wide variety of cognitive tests – not simply tests of speed or alertness,' Salthouse said. He argues it may be helpful to modify the way we think of people's cognitive ability. Rather than seeing it as a 'single discrete level that is highly stable over short intervals,' we should perhaps see cognitive ability as 'consisting of a distribution of many potential levels of performance,' like a one-person bell-curve.

To accommodate the large amount of within-person variability on neuropsychological tests, Salthouse's proposal is for a 'measured burst design', in which a person is assessed multiple times at each test occasion. That would not only provide a more accurate, stable evaluation, but would also allow a person's own variability across tests on one day to be compared with their variability across tests on another day.

Salthouse characterises the challenge as being akin to a balancing act: the more repeats of the same test, the better, in the sense that more within-person variability will therefore be taken into account, yet too many repeated tests will obviously place a strain on the examinee, and will also require too many parallel versions of the same tests.

Salthouse didn't find an association between within-person variability and age, but nonetheless he believes changes in a person's own performance variability across time could prove to be a useful diagnostic marker, perhaps as an indicator of mental decline. He plans to test this with colleagues in the future.

References

Salthouse, T.A. (2007). Implications of within-person variability in cognitive and neuropsychological functioning for the interpretation of change. Neuropsychology, 21, 401-411.

http://dx.doi.org/10.1037/0894-4105.21.4.401