Considering technical precision of measurement

The technical precision of measurement is of high importance in particular for tests that assess attention functions, reaction behavior and/or psychomotor functions. Even errors of measurement of only a few milliseconds can cause a significant shift of the normed test score and thus result in incorrect interpretation of the test results.

Measuring reaction times to the nearest millisecond is not straightforward. Many test programs or neuropsychological experiment generators quote reaction times in milliseconds in the test results but may nevertheless be affected by measurement errors of several times this amount, depending on the hardware and software used (see Häusler, Sommer & Chroust, 2007; Plant, Hammond & Turner, 2004). For this reason, special hardware components are developed for the time-critical tests of the Vienna Test System that can guarantee a measurement that is precise down to the millisecond and therefore down to the percentile ranks by way of controlled input and output of stimuli.

Stimulus presentation

Visual stimulus presentation

For tests with visual stimuli, the Vienna Test System outputs a signal to the monitor. A calibration module is available to compensate for the internal delay of the screen and the preparation and output of data. This module makes it possible to measure the exact screen delay. When performing a time-critical test for the first time, a query is posed automatically if calibration should be performed. The calibration can then be called up at any time via the Hardware Test function. The screen delay and possible delays in USB transfer from the panel determined in such a way will be used as the correction value for all successive time-critical tests. Calibrated test systems are guaranteed to yield measurements that can be converted accurately into milliseconds or percentile ranks regarding the stimulus output on CRT and LCD monitors.

If the tests are administered on an uncalibrated system using a panel, minor technical measurement errors of an average of between -2 and 8 milliseconds may occur (depending on the hardware and software used).

Acoustic stimulus output

In order to ensure the highest level of precision for auditory stimuli, it is recommended to use a standard audio output device. These are USB audio output devices and the audio output of the panel. If a non-standard headset or external loudspeakers are used for audio output, there is a risk that the driver software of these devices will produce measurement errors of up to 100 ms. In addition, these devices may have a different sound curve, so that - for example - low sounds may be reproduced more softly in comparison to other tones than was the case in the standardization of the respective time-critical tests.

If the audio output device used does not conform to the standard, this will be pointed out before the test session starts. A comment will also be included in the test results to the effect that the results were obtained under non-standard conditions.

Reaction input

An ergonomic SCHUHFRIED panel is available for input entry for time-critical tests. The panel also ensures fair test results for test takers with no computer experience and guarantees to yield measurements that can be converted accurately into percentile ranks or milliseconds in combination with a calibrated screen or a standard audio output device.

For some time-critical tests, a PC keyboard can also be used for entering the response. However, when using a PC keyboard, greater errors of measurement and percentile rank shifts can be assumed. Internal measurement series with calibrated screens that included six different PC keyboard models resulted in an average deviation of around 25 milliseconds for an average of a minimum of 5 and a maximum of 50 milliseconds. Since PC keyboards always show a positive delay of reaction times, it can only be assumed that when achieving a certain test score that it would not have fallen short for an exact measurement. For example, assuming a percentile rank of over 25 with a PC keyboard, it can be assumed that the result would not have been below average for an exact measurement.

Practical effects on the normed test result

The effects of measurement errors on the norm-referenced scores depend on the type of test and the position of the raw score. For very simple stimulus reaction tests, the distribution of raw scores in the norm sample is usually insignificant, so measurement errors can result in greater distortion in the normed test results. In addition, deviations are caused by measurement errors due to the distribution of the norm sample and are usually greater than for average performances than for above or below-average performances.

 The test RT can be taken as an example to illustrate this:

This indicates that inaccuracy in technical measurement in the medium range would result in a loss of 17 percentile ranks while the result in the upper and lower range would only be skewed by 4 percentile ranks.