Give due regard to technological issues in Computer-based (CBT) and Internet Testing
Give consideration to hardware and software requirements
Ensure that you have sufficient understanding of the technical and operational requirements of the test (i.e. hardware and software), as well as the necessary hardware, software and human resources to obtain, use, and maintain the CBT on an on-going basis. Confirm that the system the test-taker is using is documented as being suitable. Ensure there is a good justification for the use of complex software, graphics, and technical IT features in the CBT/Internet test. Monitor supplier for information on future changes to the hardware requirements, test system, or software. Ensure understanding of the implications of changes and their impact on the testing process.
Take account of the robustness of the CBT/Internet test
Before beginning a test, verify that its robustness has been adequately tested (e.g. documentation provides supporting evidence). Ensure processes are in place to log and resolve problems that may arise during testing. Check availability of the information necessary for contacting the provider of technical support and use technical support services as necessary Inform test publishers/developers where problems occur with the responsiveness of the computer to the test-taker input. For Internet testing, know the recommended procedures for dealing with hang-ups, lost connections and slow downloads, and advise test-takers accordingly. Provide the test-taker with the technical support specified in the test documentation if any routine problems occur.
Consider human factors issues in the presentation of material via computer or the Internet
Be familiar with the screen design requirements of the test and ensure that such features are compatible with the systems being used. Ensure that test-takers are informed of screen design conventions, including where instructional text and prompts are placed, and how instructions can be accessed once testing begins. Be familiar with how items are presented and how the test-taker is required to respond. Verify that error messages are non-alarming and inform how to proceed.
Consider reasonable adjustments to the technical features of the test for candidates with disabilities
Check that the hardware/software features facilitate the participation of test-takers with disabilities and those with special needs. Follow best practice as in other modes of testing [see ITC Guidelines on Test Use]. Ensure that any necessary test modifications specifically address the test-taker’s special needs and are within acceptable limits so as to not adversely affect score validity. Be aware of the impact these modifications may have on the test-taker’s score. Consider the use of alternative assessment procedures, rather than modifications to CBT/Internet tests, (e.g., paper and pencil test or alternative structured forms of assessment).
Provide help , information, and practice items within the CBT/Internet test
Understand the technical support documentation provided with the test and how to access additional technical support when needed. Know how to set up, load and log onto the system. Ensure the test-taker has access to information on the test and the testing process before beginning the test and is able to access on-screen help while completing the test. For Internet testing, provide clear information to the test-taker on how to log-in to and off from the system (e.g., the use of passwords). Provide sufficient opportunity for the test-taker to become familiar with the testing software and the required hardware. Where appropriate, direct test-takers to appropriate Internet testing practice sites. Where appropriate, i nform the test-taker about available practice tests. Make it clear that it is the test-taker’s responsibility to practice any embedded tutorials and responses to test items (e.g., use of the input device). Where appropriate, collect data on test-taker reactions towards Internet-delivered testing and provide feedback to test developers to help them ensure a more positive experience for test-takers.
Attend to quality issues in CBT and Internet testing
Ensure knowledge, competence and appropriate use of CBT/Internet testing
Assess the appropriateness of the content and technical adequacy of CBT/Internet testing relative to alternative testing methods for each client. Inform test-taker s of the purpose of the testing so they are able to make an informed decision on the appropriateness of the test for their situation. Have adequate knowledge of the CBT/Internet test and its modes of operation. When required, attend appropriate training events and read and have knowledge of relevant training materials. Follow best practices in the use of CBT/Internet testing and, where appropriate, create ‘best practice’ testing policies. Verify that test-takers know how to interact with an Internet testing system (e.g., basic browser operation, use of access passwords). Maintain and regularly update knowledge about CBT/Internet testing, including pertinent changes in legislation and policy. Adhere to legal, professional, and ethical mandates related to CBT/Internet testing. Inform test-taker s of the limitations of the Internet test in terms of the professional relationship expected from this medium. For Internet testing, provide a contact point (e.g., email or phone) for those who do not understand the purpose of the test.
Consider the psychometric qualities of the CBT/Internet test
Ensure that documentation of the appropriate psychometric evidence is supplied with the CBT/Internet test. Ensure that current psychometric standards (test reliability, validity etc.) have been met even though the way in which the tests are developed and delivered may differ. Be able to distinguish between tests with and without documented psychometric properties. Those with documented evidence ensure that the evidence is appropriate for the intended use of the test. For Internet testing, use only those websites supported by publishers who offer validated psychometric tests. Check that the CBT/Internet test does not require knowledge, skills or abilities that are irrelevant to the construct being assessed. Where appropriate, review and understand the documentation that describes how the CBT/Internet test uses algorithms for item generation, selection, or test construction, for controlling the order of testing, and the model underlying the development of the test. When necessary, access appropriate training to ensure continuing professional development. Document information provided about changes to test items or parameters and their impact on the test properties.
Where the CBT/Internet test has been developed from a paper and pencil version, ensure that there is evidence of equivalence
Confirm that the evidence regarding the equivalence of the CBT/Internet test to the manual version is sufficient. If norms are based on manual versions of the test, confirm that evidence has been obtained to show equivalence of test means and SDs across versions and for appropriate subpopulations. Verify that the technical features of the CBT/Internet test (e.g., test-taker control and item presentation) allow the results from the CBT/Internet test to be equivalent to the manual version. Only use the test in those modes of administration for which it has been designed (e.g., do not use a test in an unproctored mode when it is specified for use only in proctored modes).
Score and analyse CBT/Internet testing results accurately
Review and understand the rules underlying the scoring of the CBT/Internet test. Inform test-takers, when appropriate, about how scores are generated. Know how the statements in the CBTI are derived and be aware of the limitations such methods may have. Ensure the accuracy of test data that are hand-entered into the computer.
Interpret results appropriately and provide appropriate feedback
When interpreting the CBTI results, be aware of potential limitations, general and specific, to the reports being used . For example: - Score interpretations are based on administration in proctored, standardised conditions and the test has been administered under open or controlled modes and there is no evidence provided to support the validity of the report under such conditions.
- T ests are completed in an administration mode that makes it impossible to guarantee the true identity of the test-taker.
- T ests alone, however administered, may not provide a complete assessment of an individual , as other confirmatory or ancillary information is not considered .
Select and use the most appropriate CBTI template for the client or intended audience. Ensure that the language and information given in the CBTI fit the needs of the intended stakeholder (e.g., test-taker, organisation, and client). Confirm that there is a sound basis for the CBTI and that its rationale is well-documented. Where possible, edit CBTI reports to include information obtained from other sources to ensure a comprehensive treatment of the test-taker’s background, behaviour, ability, aptitude, and personality. Ensure appropriate, relevant, and timely feedback is provided to the test-taker and other relevant stakeholders. Ensure that Internet testing presents test interpretations in a comprehensible and meaningful form. Provide client test interpretations that are appropriate for the context and intended use of the test (e.g., high or low stakes testing, corporate versus individual applications). Take account of ethical issues surrounding the provision of feedback using the Internet (e.g. the difficulty of knowing the effect of providing negative feedback to a test-taker, the lack of knowledge of the emotional state of the test-taker, or the difficulty of providing immediate support to a test-taker when feedback has a negative impact). Where appropriate, feedback should include directions on how to access support and other information.
Consider equality of access for all groups
To monitor for possible adverse impact, collect data on the number of individuals accessing the CBT/Internet test from protected/minority groups . - For most countries such groups may be legally defined in terms of one or more of the following: ethnicity, gender, age, disability, religion, and sexual orientation.
Where there is evidence of possible inequality of access, offer the use of alternative methods of testing. Where possible, collect data to monitor group differences in test scores. Consider the appropriateness and feasibility of Internet testing if testing in locations with limited access to computer technology or the Internet. If testing internationally, use the country-specific adapted versions of the test, if available. Provide appropriate levels of control over CBT and Internet testing
Detail the level of control over the test conditions
When administering the test, adhere to the standard hardware, software, and procedural requirements specified in the test manual. Before testing, ensure that software and hardware are working properly. When testing at a specific test centre, ensure that the test-taker is comfortable with the workstation and work surface (e.g., the ergonomics are suitable). For example, test-takers should: - be encouraged to maintain proper seating posture,
- be able to easily reach and manipulate all keys and controls,
- have sufficient leg room, and
- not be required to sit in one position for too long.
When testing via the Internet, provide instructions to test-takers that specify the best methods of taking the test. Ensure that the facilities, conditions, and requirements of the testing conform to national health and safety, and union rules. For example, there may be rules governing the length of time a person should work at a monitor before having a break, or rules as to adequate lighting, heating, and ventilation. When testing over the Internet, inform test-takers of such rules and regulations.
Detail the appropriate control over the supervision of the testing
Identify the level of supervision required to administer the CBT/Internet test. U se the CBT/Internet test only in the appropriate testing scenarios for which it was designed.
Give due consideration to controlling prior practice and item exposure
Document for test-takers the equivalence of parallel or multiple forms of a test. Protect the CBT/Internet test from previous item exposure by not coaching test-takers with actual test content. Where appropriate, provide test-takers with practice without compromising the security of the actual test items themselves.
Give consideration to control over test-taker’s authenticity and cheating
Ensure test-takers provide the appropriate level of authentication before testing begins. Remind test-takers (in the Controlled mode) of the need to obtain a password and username to access the test. In supervised and managed testing conditions, test-takers should be required to provide authentic, government approved picture identification. For moderate or high stakes testing confirm that procedures are in place to reduce the opportunity for cheating. Technological features may be used where appropriate and feasible (e.g., Closed Circuit Television, CCTV) but it is likely that such testing will require the presence of a test administrator, a follow-up supervised assessment, or a face to face feedback session (e.g., for post-sift assessment in job selection situations). For moderate and high stakes assessment (e.g., job recruitment and selection ) , where individuals are permitted to take a test in controlled mode (i.e. at their convenience in non-secure locations ) , those obtaining qualifying scores should be required to take a supervised test to confirm their scores . - P rocedures should be used to check whether the test-taker ’s original responses are consistent with the responses from the confirmation test.
- T est - take r s should be informed in advance of th ese procedures and asked to confirm that they will complete the tests according to instructions given ( e.g. not seek assistance, not collude with others etc).
- T his agreement may be represented in the form of an explicit h onesty p olicy which the test-taker is required to accept.
Provide test-takers with a list of expectations and consequences for fraudulent test taking practices, and require test-takers to accept or sign the agreement form indicating their commitment. Make appropriate provision for security and safeguarding privacy in CBT and Internet testing
Take account of the security of test materials
Know the features that have been developed to ensure the security of test materials, and develop procedures that reduce unauthorised access to such materials. Respect the sensitive nature of test materials and intellectual property rights of test publishers/developers. Protect test materials from being copied, printed, or otherwise reproduced without the prior written permission of the holder of the copyright. Protect passwords and usernames from becoming known to others who are not authorised or qualified to have them. Inform the service provider/publisher of any breach in security.
Consider the security of test-taker’s data transferred over the Internet
Prior to test administration, have knowledge of and inform test-takers of the security procedures used to safeguard data transmitted over the internet. Confirm with the service provider that they frequently back up data. Verify that the service provider is able to allow test users and authorised others to discharge their responsibilities as data controllers under local data protection and privacy legislation (e.g. the European Union’s Directive on Data Protection).
Maintain the confidentiality of test-taker results
Know how confidentiality will be maintained when data are stored electronically. Adhere to country-specific data protection laws/regulations governing the collection, use, storage and security of personal data. Protect all material via the use of encryption or passwords when storing sensitive personal data electronically on test centre facilities. A pply the same levels of security and confidentiality to backup data as to the data on the live system when backups are used to store personal data.
