International Test Commission

International Guidelines on Computer-Based and Internet Delivered Testing

Ensure knowledge, competence and appropriate use of CBT/Internet testing

Ensure that the CBT/Internet test is suitable in terms of content and technical adequacy for its stated purpose and intended test-taker groups.
Provide test users with sufficient information about the CBT/Internet test, its modes of operation, and basic computer functions. If appropriate, provide training materials that are specific to CBT/Internet tests and testing.
Provide test users with ‘best practice’ testing policies.
Provide test users with clear instructions on how to correctly access and administer Internet tests, including how to log test-takers onto the system.
Maintain and regularly update documentation relating to CBT/Internet testing, including pertinent changes in legislation and policy.
Adhere to legal, professional, and ethical mandates related to CBT/Internet testing.
For Internet testing, document the limitations of the test in terms of the professional context in which it operates:

provide a statement indicating the limitations of the relationships between test user and test-taker that can be achieved through this mode (e.g. the Internet is a impersonal medium and a test user may provide only limited advice)
provide a statement stating that there are limitations to the conclusions that can be reached just using the Internet test scores.

Consider the psychometric qualities of the CBT/Internet test

Provide appropriate documentation for the psychometric properties of the CBT/Internet test.
Ensure that current psychometric standards (test reliability, validity etc.) have been met even though the way in which the tests are developed and delivered may differ.
Publish and offer online only those tests that have appropriate psychometric evidence to support their use.
When offering assessments online, give advice to test users as to what to look for in order to help them distinguish between tests with and without documented psychometric properties.
Verify that the CBT/Internet test does not require knowledge, skills or abilities that are irrelevant to the construct being assessed.
Provide documentation that describes the algorithms and measurement models used and present evidence showing that the test has been validated using these algorithms or models.
For tests based on models that may be unfamiliar to test users, provide explanations of the relevant concepts for the user.
Verify that psychometric model fit has been re-evaluated when changes are made to the test content.

Where the CBT/Internet test has been developed from a paper and pencil version, ensure that there is evidence of equivalence

Evaluate the documented evidence of the equivalence of the CBT/Internet test, especially if norms from manual versions are to be used by test users to interpret scores on a computerised version of the test.
If the developer does not provide evidence of equivalence (e.g., comparable reliabilities, etc.), conduct appropriate equivalence studies.
If the developer does not provide evidence relating to the use of the test under conditions that represent those that the intended target population will experience (e.g., unproctored, unstandardised testing), additional studies of test equivalence and norming should be conducted.
Verify that the technical features of the CBT/Internet test (e.g., test-taker control and item presentation) allow the results from the CBT/Internet test to be equivalent to the manual version.

Score and analyse CBT/Internet testing results accurately

Confirm that the accuracy of scoring rules has been adequately evaluated prior to test use.
Inform test users about the scoring rules employed within the CBT/Internet test (e.g., use of non-scored items, penalties for guessing).
Inform test users how CBTI statements are derived and the validity of that methodology.
Stress to test users the importance of carefully checking data input by hand into a computer for scoring.

Interpret results appropriately and provide appropriate feedback

Inform test users of the potential limitations of interpreting results using CBTI. Specifically:

statements in a report may be general and not directed towards the specific purpose of the assessment (or specific individuals);
interpretation is based only on scores of those tests whose data were used as input; therefore, other ancillary data which may be important cannot be taken into account (e.g., scores on other, non-computerised, forms of assessment);
for open or controlled modes of Internet testing, test-takers may have been tested in non-standardised, unproctored, or variable conditions, whereas score interpretations are based on administration in proctored, standardised conditions;
some tests are completed in an administration mode that makes it impossible to guarantee the true identity of the test-taker.

Assess the suitability of the CBTI provided within the CBT/Internet test system. In particular, take note of

Advise test users on how best to share CBTI with test-takers and other relevant stakeholders.
Inform test users of ethical and other accepted practice issues related to providing CBTI feedback to test-takers.

Consider equality of access for all groups

Where possible, encourage test users to collect biographical data on test-takers in order to monitor the number of people from protected/minority groups who take any CBT/Internet test.
Where unequal access to CBT/Internet tests may occur, recommend that test users make alternative forms of assessment available.
Inform test users of any evidence regarding DIF for different test-taker groups.
When tests are published internationally, provide test users with advice on how to ensure equivalent access to computer technology or the Internet for geographically-diverse groups of test-takers.
Where an adapted version of a test is available, provide documentation specifying the equivalence of the adaptation to the original assessment.