Many companies use scores from personality tests when evaluating job applicants. Meta-analytic evidence suggests that scores from well-developed personality tests are predictive of job performance and other organizationally relevant outcomes, with the strongest findings observed across settings for the trait of conscientiousness. A potentially important issue with many personality tests, however, is the accuracy of responses provided by applicants. The assumption is that applicants will respond to items in a manner consistent with their behavior, producing scores that indicate their trait standing. However, for many personality items it is possible for applicants to respond in a way that enhances their attractiveness to the hiring organization.
For illustrative purposes, consider the personality test item “I am rarely late for appointments” and an associated five-point Likert response scale that ranges from strongly disagree to strongly agree. Some individuals tend to respond to this item more toward the strongly agree end of the response scale in an applicant setting than when responding to the same item in a research setting or when taking the test for self-exploration. This phenomenon of changes in responses to such items across motivated (e.g., applicant) and nonmotivated settings has been referred to as impression management, socially desirable responding, intentional distortion, and faking.
Interest in the subject of intentional distortion is probably as old as interest in measuring personality through self-report methods. Although reviews of research on intentional distortion date back to 1946and much is now known, more knowledge has led to more questions. Indeed, interest and research in this issue are as keen and perhaps as controversial as ever. Through the years, research has focused on four broad questions: (a) Can people distort self-descriptions of their personality? (b) Do applicants distort their self-descriptions? (c) What effect does intentional distortion have? (d) Are there remedies for intentional distortion?
Can People Distort (Fake) Self-descriptions Of Their Personality?
Research clearly demonstrates that when asked to do so, people can distort their responses in the desired direction. Laboratory studies with transparent personality items—that is, items for which the supposedly good or bad response options are readily apparent— show that the mean scores for respondents asked to describe themselves in as good a way as possible are significantly higher than the mean scores for respondents asked to describe themselves as they really are (honest responses). The differences are even greater between respondents asked to portray themselves in an undesirable way and those asked to portray themselves honestly.
Do Applicants Distort (Fake) Their Self-Descriptions?
Although people readily recognize answers to transparent items that enhance their self-description, whether or not real-life job applicants do enhance their self-descriptions is a separate question. Research suggests that at least some applicants do enhance their self-descriptions. Much of this research involves comparisons of job applicant and job incumbent test scores, and it is based on the assumption that job applicants are likely motivated to enhance their self-descriptions to increase their chances of being hired. Results of this research indicate that mean applicant test scores are notably higher than mean incumbent scores. Nonetheless, the size of the difference between incumbent and applicant mean scores varies depending on the particular personality characteristic, the type of job, and, undoubtedly, other factors, as well.
This research does not suggest that all applicants fake. Perhaps only 25% of job applicants enhance their self-descriptions on personality inventories consisting of transparent items. Moreover, job applicants who enhance their self-descriptions do so to different degrees. One applicant might raise her score by a couple of points on a scale, whereas another applicant might raise his score by several points, perhaps dramatically. Very little is known about the individual differences and situational factors that influence an individual’s decision to distort or the amount of distortion. It is likely that people are differentially able to distort their scores. Certainly, one’s true score on a characteristic influences the amount of distortion in a positive (or negative) direction. For example, an individual with an already high true score on a characteristic is simply unable to raise his or her score as much as someone whose true score is lower. Factors such as these seriously complicate the tasks of identifying those who have distorted their self-descriptions and determining the amount of the distortion.
What effect does intentional distortion have?
Faking might have a number of possible effects on personality test scores and the usefulness of those scores. Some research has explored the extent to which faking affects the construct validity of personality measures. Generally, these studies have focused on factorial validity, comparing the factor structures of a personality measure across samples of honest responders (typically incumbents) and people who have completed the measure in a motivated situation (typically applicants). It has been hypothesized that faking will increase the correlations between the scales and that a general faking factor will be observed in samples in which respondents were motivated to provide a positive self-impression. Research suggests this hypothesis is inaccurate. Evidence to date suggests that faking does not substantially change the factorial validity of personality measures.
Researchers have also investigated the effect of faking on the relationship between personality test scores and job performance (i.e., criterion-related validity). The expectation is that if scores on the test change (i.e., faked responses) but performance on the criterion (e.g., job performance) does not change, criterion-related validity will decrease. In other words, faked responses will lead to predictions of performance that are higher than would be predicted based on honest responses, and thus greater errors of prediction. Investigations into the effect of faking on criterion related validity have provided mixed results, with some studies showing that faking has no effect on criterion-related validity and others showing a notable decrement in the correlation when faking is present. A review of these studies led Leaetta M. Hough to argue that the mixed results may arise from differences in setting or research context. That is, she showed that studies conducted in the laboratory tend to evidence a stronger impact of faking on the criterion-related validity of personality measures than those studies conducted in field settings (with actual applicants), where findings indicate a much smaller impact. Importantly, and consistent with Hough’s conclusion, meta-analytic research indicates that criterion-related validities obtained in settings with real applicants for real jobs are similar to criterion-related validities obtained with real incumbents in real jobs. Thus, it would appear that intentional distortion has a minimal impact on the criterion-related validity of personality test scores.
Even though the criterion-related validity of a personality test might not be strongly influenced by faking in applicant settings, it remains possible that the quality of hiring decisions made using such tests will be adversely affected by faking. That is, for small selection ratios, the quality of selection decisions might be negatively affected to the extent that individuals who appear to have a high level of the trait (but who would be expected to perform in accordance with the actual trait standing) displace individuals whose test scores are slightly lower but who actually have a high standing on the trait (and who would be expected to perform in a manner consistent with that trait standing). Some findings to date are consistent with this logic, suggesting that intentional distortion may detrimentally affect the quality of selection decisions, especially when the selection ratio is rather small. It should be noted, however, that much of this research has been conducted in laboratory settings or evaluated through simulation methods. Consistent with Hough’s observation regarding the effects of faking on criterion-related validity, it is quite possible that lab and simulation studies overestimate the impact of faking on the quality of selection decisions. Moreover, not all research has reached the conclusion that faking harms the utility of personality tests. Findings from a recent simulation study by Neal Schmitt and Frederick L. Oswald suggest that removing fakers from the applicant pool changes mean levels of performance very little.
Are there remedies for intentional distortion?
Even though questions remain regarding the consequences of intentional distortion on the usefulness of applicant scores, many people assume that such distortion does degrade the validity of self-report responses for predicting later work performance. As applicant and client perceptions of fairness and validity are important, a search for and evaluation of remedies for intentional distortion is warranted.
Attempts to address issues of intentional distortion can be divided into two broad approaches. The first has been referred to as the detection and correction approach. The detection aspect of this approach involves the inclusion of a set of items in the personality test specifically designed to identify those individuals who may be attempting to respond in an overly desirable manner. These sets of items have been referred to as faking scales, lie scales, unlikely virtues scales, and impression management scales. The correction aspect of this approach involves adjusting scores on the various scales, often through a statistical procedure, based on scores on the impression management scale. In general, research suggests that the detection and correction approach makes little difference in terms of increasing the accuracy of individual scores or the criterion-related validity of the scores.
A second approach to dealing with faking is the prevention approach. In this approach, steps are taken to dissuade applicants from faking or to make it difficult to intentionally raise one’s scores. One prevention strategy is that of warning statements. In this strategy, a statement is included in the instructions of the test informing test takers that there are items in the test designed to identify individuals who are responding in an overly desirable manner. Meta-analytic research has demonstrated that personality test scores are notably lower when a warning statement is present in the instructions of the test. However, that benefit appears to occur only when test takers are informed that there are consequences, such as being dropped from the applicant pool, for being identified as having responded in an overly desirable manner. In other words, warnings appear to be an effective deter-rent of faking only when the warning “has teeth.”
A second prevention strategy involves the use of multidimensional forced-choice (MFC) item formats. An MFC item comprises two or more statements, each reflecting a different trait. The respondent must choose, rank order, or otherwise indicate preference among the statements presented in the item. Because the individual cannot make himself or herself look good on all of the statements within the item, it is thought that a respondent’s capability to fake is reduced. Consistent with this line of thinking, research has shown less score inflation between motivated and nonmotivated testing situations on MFC scales than on scales with Likert-type formats. However, research also suggests that when completed under motivated testing situations, scores from MFC measures provide representations of trait standings that are no more accurate than scores from Likert-type measures. Thus, MFC formats, at least of the type that have most recently been discussed and explored, do not appear to be a remedy for faking.
Summary
Personality tests are often used in personnel selection systems, and many of these tests are susceptible to socially desirable responding. Research shows that applicants can and will fake personality tests in applicant settings, although precise estimates of the percentage of applicants who fake is not well documented. The impact of faking on the criterion-related validity of personality test scores appears to be rather small, with larger effects observed in lab studies than in field studies. Likewise, studies with real applicants in real selection settings indicate the effects of intentional distortion are minimal. Lab and simulation studies suggest that faking may adversely affect the quality of selection decisions when top-down selection is used and the percentage of the applicant pool hired is very low, although these findings need to be confirmed in field settings. Finally, with regard to remedies for faking, research suggests there are no methods that fully prevent faking or that effectively deal with the effects of faking after it has occurred. At present, the best recommendation is to include an impression management scale and warn applicants that high scores on the scale might result in their removal from the applicant pool.
References:
- Dwight, S. A., & Donovan, J. J. (2003). Do warnings not to fake reduce faking? Human Performance, 16, 1-23.
- Ellingson, J. E., Sackett, P. R., & Hough, L. M. (1999). Social desirability corrections in personality measurement: Issues of applicant comparison and construct validity. Journal of Applied Psychology, 84, 155-166.
- Hough, L. M. (1998). Effects of intentional distortion in personality measurement and evaluation of suggested palliatives. Human Performance, 11, 209-244.
- Hough, L. M., & Furnham, A. (2003). Importance and use of personality variables in work settings. In I. B. Weiner (Ed.) & W. Borman, D. Ilgen, & R. Klimoski (Vol. Eds.), Handbook of psychology: Vol. 12. Industrial and organizational psychology (pp. 131-169). New York: Wiley.
- Hough, L. M., & Ones, D. S. (2001). The structure, measurement, validity, and use of personality variables in industrial, work, and organizational psychology. In N. R. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), International handbook of work psychology (pp. 233-377). London: Sage.
- Mueller-Hanson, R., Heggestad, E. D., & Thornton, G. C., III. (2003). Faking and selection: Considering the use of personality from select-in and select-out perspectives. Journal of Applied Psychology, 88, 348-355.
- Schmitt, N., & Oswald, F. L. (in press). The impact of corrections for faking on noncognitive measures in selection settings. Journal of Applied Psychology.