Applied Measurement by Unknown

Author:Unknown
Language: eng
Format: epub
ISBN: 978-1-135-59662-0
Publisher: Taylor & Francis (CAM)

Empirical Keying

Three methods frequently used for empirical keying of background data measures are the correlational method (Lecznar & Dailey, 1950), the differential regression method (Malone, 1978), and the weighted application blank (England, 1971). The first two methods are most appropriate when continuous response options (e.g., 5-point Likert scales, where 5 = Strongly Agree and 1 = Strongly Disagree) are used.

The correlational method involves using the magnitude and direction of the correlation between the item and the criterion to determine the item’s weight. Item weights may be the actual correlation coefficients or unit weights, depending on the statistical significance of the item correlation (e.g., plus or minus 1 for p < .05; Stokes & Searcy, 1999). Techniques can also include phi coefficient and point-biserial correlation when using correlations between each item response option and a binomial criterion or continuous criterion as response option weights (Dean et al., 1999; Lefkowitz, Gebbia, Balsam, & Dunn, 1999). As a general rule of thumb, a minimum correlation of .10 to .15 has been suggested for inclusion of items on a scale (Mumford & Owens, 1987). The differential regression method uses least squares regression analysis to develop a model that maximizes explained criterion variance. Items are selected based on the increment in criterion variance accounted for over and above that which is explained by items already in the equation. It is important to note that variance maximizing procedures are likely to capitalize heavily on chance, making cross-validation (using weights from one sample to predict performance for a different sample, and vice versa) very important. The weighted application blank method involves weighting alternative response options on the basis of differences in option selection for different criterion groups. Criterion groups that are well differentiated are needed to produce differential weights and these groups need to be large enough to produce stable weights. There are several studies that suggest that unit weights produce scales with validities that are nearly equal to those produced with differential weights (Dawes, 1971). This approach has been shown to be useful for professional entry-level selection where the goal is to reduce a large number of applicants to a smaller, more manageable set (Harvey-Cook & Taffler, 2000).

In summary, the empirical keying approach requires that items be retained for the final version of the questionnaire based solely on their ability to predict performance on a criterion (Mumford & Owens, 1987). To develop empirical keys, scores of a criterion group (people who are good performers on the criterion) are compared to the scores of a reference group (population of job applicants). Mean differences (when items are scored on a continuum) or percent response differences (when response options are scored) are obtained reflecting the differences between these groups (Dean et al., 1999). Items are retained if they discriminate between criterion group and reference group members; items that fail to distinguish between these groups are eliminated. Retained items are weighted based on the magnitude of the differences observed between the groups (Hogan, 1994).

An empirical keying strategy maximizes the ability of background data items to predict performance on the criterion of interest.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.