• Background
• Test Developers
• Technological Issues
• Quality Issues
• Control Issues
• Security/Privacy Issues
• Test Publishers
• Test Users
• Official Translations
• Acknowledgements
• References
• Appendix
• Glossary

International Test Commission

International Guidelines on Computer-Based and Internet Delivered Testing

Attend to quality issues in CBT and Internet testing

1. Document the constructs that are intended to be measured and investigate whether CBT/Internet mode of delivery is appropriate in terms of content and technical adequacy to access the relevant constructs.
2. Ensure all those involved in test design and development (item writers, psychometricians, software developers etc.) have sufficient knowledge and competence to develop CBT/Internet tests.
3. Remain current on recent advances in CBT/Internet testing, including advances in computer hardware and software technologies and capabilities.
4. Adhere to legal, professional, and ethical mandates and guidelines related to CBT/Internet testing.
5. It is important that during the development of items and tests, the content is protected, through the use of agreements as well as sound security procedures.

Consider the psychometric qualities of the CBT/Internet test

1. Document and disseminate information on the validity, reliability, and fairness of the CBT/Internet testing process.
2. Ensure that current psychometric standards (test reliability, validity, etc) apply even though the way in which the tests are developed and delivered may differ.
3. Take care that the CBT/Internet test does not require knowledge, skills, or abilities (e.g., computer skills) that are irrelevant to or might impede the test-taker's ability to perform the test.
4. Describe the theoretical and practical applications of algorithms used in test-item selection and/or controlling item or test order (as in adaptive testing).
5. Where test -item content changes, retest and evaluate the changes.

1. Provide clear documented evidence of the equivalence between the CBT/Internet test and non-computer versions (if the CBT/Internet version is a parallel form). Specifically, to show that the two versions:
   • have comparable reliabilities,
   • correlate with each other at the expected level from the reliability estimates,
   • correlate comparably with other tests and external criteria, and
   • produce comparable means and standard deviations or have been appropriately calibrated to render comparable scores.
2. When designing a CBT/Internet version of a non-computerised test, ensure that:
   • there is equivalent test-taker control (such as the ability to skip or review items) as on the manual version,
   • the method of item presentation ensures that the results from the CBT/Internet test are equivalent to the manual version, and
   • the format for responding is equivalent.
3. For Internet-based tests, studies of test equivalence and norming should be conducted over the Internet with participants completing the test under conditions that represent those that the intended target population will experience (e.g., unproctored or unstandardised testing conditions).

1. Ensure the accuracy of rules/algorithms underlying the scoring of the CBT/Internet test.
2. Provide appropriate documentation of the use and validity of scoring rules.
3. Where reports classify test respondents into categories, such as ‘Introverted type’ or ‘High sales potential’, provide information in the test manual that specifies the accuracy of the classification system used to generate computer-based test interpretations (CBTI).
4. Describe the rationale for CBTI statements and how statements are derived from particular scores or score patterns.
5. When test data are hand-entered into a computer, devise procedures to allow for data to be checked for accuracy.

1. Illustrate potential limitations of the computer-based test interpretations (CBTI) specific to the current CBT/Internet test.
2. Design and embed individual CBTI report templates for all stakeholders in the testing process.
3. Illustrate how to obtain these various reports and what is contained within each report. In particular consider the:
   • media (e.g., text, graphics, etc.),
   • complexity of the report,
   • report structure,
   • purposes of testing,
   • degree of modifiability,
   • style and tone of report, and
   • intended recipients.
4. Provide appropriate guidance on giving feedback, including necessary training requirements for interpreting the CBTI.

1. Document the methods used to enhance psychometric fairness and equality of access.
2. Assess Differential Item Functioning (DIF) and, where DIF might be a problem for one or more groups, identify where this problem occurs and attempt to modify the test to overcome such problems.
3. When developing CBT/Internet tests that may be used internationally, take into account the fact that countries differ in their access to computer technology or the Internet.
4. For tests that are to be used internationally :
   • a void the use of language, drawings, content, graphics (etc.,) that are country or culture specific.
   • w here culture specific tests may be more suitable than culturally-neutral ones, ensure that there is construct equivalence across the different forms.
5. If developing adapted versions of an Internet test for use in specific countries ensure the equivalence of the adapted version and that the adaptation conforms to the ITC Guidelines on Test Adaptation.