Selection and Assessment Consulting
Selection strategies differ from organization to organization in any number of ways. Some rely mostly on tests, others on interviews. Some are computer or Web based, others paper-and-pencil tests. Some automatically select candidates out, whereas others inform decisions that select candidates in—and so on. Selection strategies are the result of many design decisions, and it is safe to say that no two strategies are the same.
The purpose of this entry is to describe different selection strategies and evaluate the effectiveness of those strategies in different employment situations. These descriptions and evaluations will be limited to the manner in which information from the selection procedures is used to make selection decisions. This entry will not address strategies relating to delivery methods or types of assessment procedures. Rather, the focus will be on describing and evaluating the different strategies that organizations use to make selection decisions.
Framework
This entry is organized into two major sections. The first section will identify and describe the most common types of selection decision-making strategies. This section is primarily descriptive and highlights key differences among the strategies. The second section will evaluate these strategies based on their fit with each of several types of employment context. The employment context is defined based on three considerations: (a) employment volume (high/low), (b) mode of employment processing (continuous/ episodic), and (c) the organization’s culture for accountability (systems accountability/manager accountability). The decision-making strategies will be evaluated based on their fit with each of the eight combinations of volume, mode, and accountability. Although this approach is somewhat artificial—actual employment contexts are not likely to be exact invariant combinations of volume, mode, and accountability—it has the advantage of being systematic and reveals key principles that can be applied to any employment context.
This essay will not evaluate strategies based on considerations of legal risk. Certainly, legal risk is an important consideration in the design of any selection strategy. The primary means of controlling legal risk is the documentation of validation evidence. However, legal risk can also be influenced by the extent to which selection decisions are not precisely prescribed and uniformly applied to all candidates. Although the latter considerations certainly relate to decision-making strategies, legal risk often depends on subtle and nuanced aspects of the situation and the particular charge, making it difficult to offer general guidelines beyond the principles of validity, adverse impact, and consistency of application, which may be applied to any decision-making strategy.
Selection Decision-Making Strategies
Some of the strategies described here are typically mutually exclusive, such as multiple hurdles and compensatory scoring, whereas others are not, such as compensatory scoring and profile matching. The descriptions offered here summarize the relationships among the most commonly used strategies. The first two strategies, multiple hurdles and compensatory scoring, are typically viewed as mutually exclusive, although hybrid strategies are beginning to emerge. The third and fourth strategies, cut scores and non-cut-score-based judgment methods, are deliberately defined as mutually exclusive to clarify their most important differences. The fifth strategy, banding, is a class of methods introduced primarily to manage legal risk in certain types of situations but not equally applicable in all employment contexts.
Multiple Hurdles
A major consideration in designing a selection strategy is the cost-effective management of candidate flow. One common approach to this issue is the strategy of multiple hurdles. According to this strategy, selection procedures are administered in sequential steps. After each selection procedure is administered, it is then scored. At each step, candidates whose scores fall below an established threshold are eliminated. Candidates who are not eliminated proceed to the next step, where the next selection procedure is administered and scored and additional candidates are eliminated.
Two methods of multiple hurdles can be considered. The first and most common method, the independent method, eliminates candidates based only on scores from the selection procedure administered at the current step in the series. The second method, the accumulative method, eliminates candidates at each step based on scores from all of the previously administered procedures and the current procedure. For example, consider a multiple hurdles strategy in which Step 1 is a problem-solving test, Step 2 is a work sample exercise, and Step 3 is an employment office interview. At Step 2, the independent method would eliminate candidates based solely on scores from the work sample exercise, whereas the accumulative method would eliminate candidates based on some combination of both the problem-solving score and the work sample score.
The primary advantage of the multiple hurdles strategy is that it minimizes administration cost and time by not administering later selection procedures to candidates who performed poorly on earlier procedures. Typically, this advantage is maximized by administering the selection procedures in order of cost from lowest to highest. Its primary disadvantage is that it sacrifices some predictive accuracy at each decision step by considering information from only some (usually one) of the selection procedures that the organization views as relevant to the job. The accumulative method mitigates this disadvantage somewhat.
Compensatory Scoring
When organizations prefer to maximize the predictive power of each selection decision, all candidates are administered a common set of selection procedures. A composite score is then created for each candidate that combines information from the entire set of selection procedures. This composite score is then used in some decision method to select among the candidates.
In virtually all applications of compensatory scoring, only one composite of selection procedures is used as the basis for making selection decisions. This common practice reflects an assumption that there is only one model or type of successful employee. All selection decisions seek to choose employees who fit that particular model of success. In contrast, some organizations recognize that different employees are successful and valued in different ways. Some employees may be successful because they are accurate and fast producers; others may be successful because they are able to engage others in achieving objectives. Still others may be successful because they are reliable employees who show up every day, perform well enough, and are loyal to the organization. An organization that values different models of success may prefer to use more than one composite of the same predictors and choose candidates who are predicted to be successful by at least one of the valued models. Little research has examined this possibility of simultaneous, separate composites to determine how much gain could be realized compared with the standard practice of a single, presumably optimal composite.
Compared with multiple hurdles, compensatory scoring is more expensive but has somewhat more predictive accuracy. There is no general conclusion about the actual trade-off between cost and accuracy. This trade-off depends on the actual costs and predictive validities of each of the selection procedures. However, it is common for cost differences between types of selection procedures to be very large, sometimes 500% to 1,000%, though the incremental accuracy that the second procedure adds to the first is often slight, say 10%. Of course, a 10% increase in predictive accuracy may actually have more dollar value than the cost of administering the second procedure to all candidates.
Cut Scores
The use of cut scores can be combined with any of the other strategies described here. In fact, some strategies, such as multiple hurdles and banding, invariably rely on cut scores as part of the decision-making process. However, it is useful to describe cut scores as a distinct strategy, perhaps a substrategy, and to provide information about specific methods for determining and using them.
A cut score is a particular score on a selection procedure or composite of selection procedures that serves as a threshold value for determining which candidates are excluded or included. There is no law or professional standard that requires cut scores to be either low or high. Generally, it is acknowledged that the hiring organization may consider a variety of factors, including expected work proficiency, cost of employment, labor market conditions, and efforts to avoid discriminatory employment practices, in determining a cut score that optimizes the organization’s value for its new hires.
Because many factors may influence cut scores, there are many methods for setting cut scores. Some methods rely primarily on the judgment of experts, whereas other methods rely on quantitative analyses of desired outcomes. Some methods focus on the probability that hired candidates will be successful employees, whereas other methods focus on the probability that people who will be successful employees will be hired. In any case, no cut score can be set without some form of value judgment being made on behalf of the organization. All cut scores rely on some judgment about the outcomes the organization values and desires from its selection strategy.
More than one cut score may be used to enable selection decisions. In the simplest decision process, a single cut score is used to determine that candidates who score at or above the cut score will be hired. More complex decision processes may use multiple cut scores to create ranges of scores that distinguish the most qualified candidates from the next most qualified, and so on. In all cases, a cut score defines a boundary line between candidates who are treated in different ways.
The primary advantage of cut scores is that they simplify the selection decision process. The primary disadvantage is that the use of cut scores ignores potentially useful information both above and below the cut score. For example, by relying on a single cut score set at the 75th percentile of candidate scores, the organization treats the highest-scoring candidates the same as candidates scoring at the 75th percentile. Both are hired. This process can be disadvantageous to the organization if it only needs to hire a small percentage of the candidates. For example, suppose an organization only needs to hire 5% of the available candidates but has adopted a cut score that 25% of the candidates satisfy. In this situation, the organization is losing value from its selection strategy because the cut score does not allow it to choose just the top 5% of all candidates. Of course, in such a case, the organization may change the cut score to be closer to the 95th percentile so that it does not lose useful information.
Judgment Methods
For convenience, the term judgment methods is used to refer to methods of making selection decisions that are not based on cut scores to determine automatic decisions. Judgment methods include all methods for using quantitative information from the selection procedures to inform the judgment of the person making the hiring decision. Certainly, there are innumerable such methods. The two most common types of judgment methods will be described here: expectancy methods and profile-matching methods.
Expectancy methods convert the score results from the selection procedures into quantitative predictions or expectations about outcomes of interest to the organization. This conversion requires quantitative analysis that relates the scores on the selection procedures to scores on outcome measures. One method, probability of success, converts each selection procedure score into a probability of success. This, of course, requires the organization to define what level of outcome result corresponds to success and to provide a measure of this outcome in some study sample of employees that can be used to define the conversion. For example, suppose the organization is interested in making selection decisions to minimize the number of new hires who leave during the first 12 months. Success is defined as staying on the job for 12 months or longer. To convert scores on the selection procedure into a probability of staying for 12 months or longer, retention would need to be tracked for candidates who had taken part in the selection procedures in question. Based on an analysis of the results of such a study, for each candidate, the person making the hiring decision would be provided with a measure of the probability that the candidate will stay for 12 months or longer, as predicted by the selection procedure. The decision maker could then make an informed hiring decision based on this information and whatever other information might be available about the candidate.
Similarly, another expectancy method, predicted performance, provides predicted levels of performance rather than probability of success. The same types of data and analyses are required to produce this type of information, although it is no longer necessary for the organization to define success.
A second judgment method, profile matching, provides a different type of quantitative information to the person making the hiring decision. According to this method, the information describes the extent to which the candidate’s selection procedure scores are similar to the selection procedure scores of people who have demonstrated success on the job.
Consider a sales organization, for example. Suppose this organization is implementing new selection procedures to assess achievement orientation, independence, intelligence, sociability, and integrity. As part of the study of this new process, the organization administers selection procedures to its current salespeople. Suppose that the top sellers are different from average sellers on achievement orientation, independence, and intelligence but no different on sociability and integrity. In addition, top sellers score very high on achievement orientation, moderately on independence, and moderately high on intelligence. This profile of a top seller becomes the ideal pattern of scores on these three selection procedures and distinguishes top sellers from average sellers. Each new candidate’s profile of achievement orientation, independence, and intelligence scores is compared with this ideal profile. In some fashion, usually graphically or quantitatively, the decision maker is shown the extent to which each candidate’s score profile is similar to the ideal profile. Like expectancy methods, the profile method provides quantitative information to decision makers as the basis for making a hiring decision but generally does not trigger any automatic decision.
Banding Methods
During the early 1990s, banding methods were introduced as a technique for selecting candidates so as to equalize the hiring rates among different groups of candidates. Although these methods were designed to minimize group selection rate differences, they also have broader applications. Banding methods all separate the full range of scores on the selection procedure of interest into several bands. The essential feature of banding is that each band of scores is defined as a range within which the organization is indifferent to the highest- and lowest-scoring candidates. Some banding methods define the indifference bands based on the reliability of differences between scores on the selection procedure; other methods define the indifference bands based on differences between predicted outcomes. Regardless of the method used to define the bands, the selection decisions are made by first considering the candidates in the top band, choosing among them, and, if any vacancies remain, moving to the next highest band and repeating the process until all vacancies are filled.
The primary advantage of banding methods is that they provide an explicit definition of the organization’s indifference to the highest- and lowest-scoring candidates within a band. This enables organizations to base selection decisions on other considerations because they can be confident that candidates within bands are similarly qualified with respect to the attributes assessed by the selection procedures. The primary disadvantage of banding is that it is most effectively applied only when the whole set of candidates is known before any selection decisions are made.
Evaluation of Strategies
Eight employment contexts are listed in Table 1. For each context, the selection strategies most likely to fit with the demands of that context are shown, as well as the most significant considerations in choosing among the selection strategies. This evaluation is presented in a table format for ease of comparison and use.
The distinction between high and low volume is not based on absolute numbers of vacancies or candidates. Rather, high volume exists when the number of candidates or vacancies to be filled stretches the capacity of the regular employment process.
Mode of employment refers to the continuity of recruiting and screening processes. Continuous-mode operations manage ongoing recruiting to maintain an available pool of candidates to be ready when vacancies occur or to support continuous vacancies. Continuous employment processes include processes that are modulated from time to time but maintain ongoing recruiting and screening processes. A common example of continuous employment is the process used by retail sales organizations that need to maintain fully staffed sales clerk positions in the face of typically high turnover rates. In contrast, episodic employment processes stop and start over time. Typically, they start when a batch of new vacancies is to be filled or a batch of new candidates is to be recruited. Once the batch of vacancies or candidates is completed, the process stops. A common example of episodic employment is public-sector employment for police and firefighter jobs, which are frequently managed as periodic episodes of employment.
The culture for accountability is perhaps the least well-prescribed context feature. It refers to the organization’s tendency to place high accountability on either its systems or its managers, especially with regard to human resource processes and systems. For example, some organizations manage annual performance management and compensation processes largely as a matter of inputting performance results into a system and then following that system’s rubrics for performance assessment and compensation decisions. In contrast, other organizations rely entirely on managers to make performance management decisions and compensation decisions with little, if any, systematic structure or guidelines.
References:
- Aguinis, H. (Ed.). (2004). Test score banding in human resource selection. Westport, CT: Praeger.
- Kehoe, J. F. (Ed.). (2000). Managing selection in changing organizations: Human resource strategies. San Francisco: Jossey-Bass.
- Kehoe, J. F., Dickter, D. N., Russell, D. P., & Sacco, J. M. (2005). E-selection. In H. G. Gueutal & D. L. Stone (Eds.), The brave new world of eHR: Human resources management in the digital age. San Francisco: Jossey-Bass.
- Kehoe, J. F., & Olson, A. (2005). Cut scores and employment discrimination litigation. In F. J. Landy (Ed.), Employment discrimination litigation: Behavioral, quantitative, and legal perspectives. San Francisco: Jossey-Bass.
- Schmitt, N., & Borman, W. C. (Eds.). (1993). Personnel selection in organizations. San Francisco: Jossey-Bass.
- Tippins, N. (2002). Issues in implementing large-scale selection programs. In J. W. Hedge & E. D. Pulakos (Eds.), Implementing organizational interventions: Steps, processes, and best practices. San Francisco: Jossey-Bass.