Tests and other forms of assessment give important insight into key human characteristics that drive performance. These tools are used widely in both business and educational settings, with the goals of their use ranging from low stakes (such as personal insight and development) to extremely high stakes (such as selection or promotion into coveted job positions). The fairness and accuracy of tests are critical to enable valid inferences about human behavior and to protect examinees from the misuse of test results. For this reason, psychologists have a professional and ethical obligation to ensure the security of these processes and materials at all times. Test security in this context refers to the continuous maintenance and control of all test material within a testing program by only those individuals who are qualified and/or designated to have access to them. Test material is used broadly throughout this discussion to include any document, device, or process used to assess human characteristics (psychometric tests, behavioral simulations, structured interview protocols, etc., are all included under this definition).
Psychologists are concerned with test security because of the strong impact that a breach of security can have on the standardization of the test, the integrity of the results, and the long-term value of the test itself. If the data collected from a test do not accurately reflect the examinee characteristics that the test was designed to measure, then the inferences that can be made from it may be meaningless. Proper interpretation and use of test results thus relies on the assumption that test security was maintained throughout the assessment process—that, simply stated, the test questions, answers, and/or scoring protocols were not known to any of the examinees prior to the point of assessment.
Common Forms of Security Breach
Individuals or organizations willing to breach test security may do so for any number of reasons, ranging from a single test taker’s efforts to be hired for a job to an organized attempt to profit from the unauthorized release of high-stakes test material. Breaches of test security come in many forms; some common examples include the following:
- Unauthorized removal of test materials from a test site
- Posting of test answers over the Internet
- A test taker asking another person to take the test in his or her place
- Unauthorized copying of copyrighted test materials
- Hacking into data storage locations to gain access to test results
Steps should be taken to identify the potential threats to test security for any operational testing program.
Consequences of Security Compromise
Breaches in test security not only affect measurement integrity and reliability; they also diminish the potential financial and social benefits associated with testing. Several significant risks associated with a poorly secured testing program are elaborated as follows.
- Decreased validity and reliability. Security breaches in test content and scoring affect validity and reliability. Test scores affected by a breach of security may not reflect the target construct of the assessment; this impinges on the ability to draw inferences between other constructs and outcomes and restricts the ability to measure consistently over time.
- Ineffective selection procedures. Reduced validity in testing processes can have a significant impact on the effective selection of examinees into organizational settings; as validity declines, the number of selection errors increases.
- Decreased organizational performance. Ineffective selection procedures may further affect organizational performance outcomes and eliminate any advantage provided by the use of well-constructed assessments.
- Lowered confidence in psychological assessment. High-profile cases of cheating and pirating of copyrighted content may also have an impact on social perceptions of the overall fairness of specific testing applications or even of the testing industry in general.
Although these risks may differ in the level of importance for different test applications and user groups, they each represent significant concerns that can arise when test security is not maintained.
Ensuring the Security of Test Materials
Several practical challenges in securing test content, processes, and data exist. Threats to test security may be thought of as a series of factors that interact to influence test responses, test reliability, and validity. These factors include the following:
- The physical and electronic security of testing sites, content, and data
- Appropriate qualification levels of test users and administrators
- Differences in delivery technology (e.g., paper-and-pencil, interactive voice response, and computer-based)
- Differences in stakes (e.g., educational admissions vs. self-development)
- Cultural values within which the tests are administered
Although this list is not exhaustive, each item can have some influence over the need to monitor and protect test materials.
To maintain security, test publishers and users follow several sets of guidelines and recommendations that help both to directly protect testing-related property and information and to guide safer practices. The Standaizis for Educational and Psychological Testing includes suggestions for maintaining safe and secure testing protocol and content. The Society for Industrial and Organizational Psychology provides guidance for maintaining the security of tests and assessments used within personnel selection systems in the Principles for the Validation and Use of Personnel Selection Procedures. Similarly, the International Testing Commission (ITC) has developed standards for test security through their International Guidelines for Test Use. The ITC has also issued guidelines that are specific to computer-based tests (the International Guidelines on Computer-Based and Internet Delivered Testing); these guidelines make several suggestions regarding test administration, data storage, and ensuring test-taker authenticity.
Methods for Securing Testing Programs
Psychologists and other test users have devised several methods for protecting testing material and procedures, including developing alternate/parallel test forms; proctoring testing sites; physically and electronically securing locations of test materials and results; monitoring and analyzing test response data for abnormal response patterns; regularly searching the Internet for test content and test preparation materials; and copyrighting all testing materials. Advances in testing procedures and electronic security software work toward enhancing our ability to protect tests, test materials, and information. For example, the expanded use of computer adaptive testing (in which examinees with different ability levels are likely to receive different test questions) will help to reduce opportunities for cheating by limiting the number of times a test question is presented to examinees. Although these methods can be effective at controlling for some loss of materials and content, ways to circumvent test protection are often found when the stakes associated with the assessment are high. In these cases, psychologists and other testing professionals may take legal action to protect the security of a testing process (e.g., by asserting and defending the copyright on testing materials).
One of the most challenging threats to test security arises when a party to a legal or regulatory action requests test materials. Psychologists may be asked to provide, to lawyers and other individuals, confidential results, test content, and scoring keys as evidence or support for a case. However, the individuals who request the information may not be ethically or legally required to maintain test security. Test materials may then become part of a public record, thereby putting the test materials and examinee information at risk. In these situations, psychologists may ask that the requested material be delivered only to other qualified psychologists who are ethically and legally obligated to maintain the confidentiality of the material. If this is not an option, psychologists may ask that the materials be covered under a protective order and that the materials be returned directly to the psychologist following litigation.
Social awareness of test security issues will also help to enhance test security and discourage individuals or groups from obtaining testing materials for the purpose of compromising the usefulness of an assessment process. One sign of the growing importance of test security is the emergence of specialists who can assist those who are responsible for testing programs with the development of policies and practices that facilitate test security.
Over time, test security may be enhanced by the continued development of the knowledge base surrounding security issues. Research in this area can help build an understanding of the factors that contribute to the likelihood of a security compromise, as well as the factors that help maintain the quality of a test under conditions in which security compromise may be likely, such as when a test is provided over the Internet. Advances in test security are also directly related to innovations in the techniques used to monitor and analyze test data for aberrant or unlikely patterns in test data, so that security breaches can be identified quickly.
Summary
The security of testing material has both a direct and an indirect impact on the validity of the inferences that can be made from a test result. However, test security is a concern not only because of the impact a security breach may have on the psychometric qualities of a test and its proper interpretation, but also because of the financial and social implications that compromised assessment processes can have within organizations and the public. As the use of testing increases in organizational settings, across international borders, and through an increasingly technology-based delivery framework, careful attention to the enhancement of security has become more critical than ever before.
References:
- American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: Author.
- American Psychological Association. (1999). Test security: Protecting the integrity of tests. American Psychologist, 54, 1078.
- American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060-1073.
- International Testing Commission. (2000). International guidelines for test use. Stockholm, Sweden: Author.
- Naglieri, J., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A., et al. (2004). Psychological testing on the Internet: New problems, old issues. American Psychologist, 59, 150-162.
- Schroeder, L. (1996). Examination security. In A. H. Browning, A. C. Bugbee, & M. A. Mullins (Eds.), Certification: A NOCA handbook (pp. 125-147). Washington, DC: The National Organization for Competency Assurance (NOCA).
- Society for Industrial and Organizational Psychology. (2003). Principles for the validation and use of personnel selection procedures. Bowling Green, OH: Author.