International Test Commission
International Guidelines on Computer-Based and Internet Delivered Testing
Development of the Guidelines
As with previous ITC guidelines, the present guidelines can be seen as a benchmark against which existing local standards can be compared or as a basis for the development of locally applicable standards or codes of practice. The advantage of these guidelines is that local standards can be compared to these set guidelines for coverage and international consistency in order to promote consistency across national boundaries and for benchmarking purposes.
The project commenced with an initial literature search and review of existing references and guidelines on computer-based testing and Internet testing from a number of different countries (see Appendix). A number of these sources were particularly influential in the development of the guidelines:
- Bartram, D. (2001). The impact of the Internet on testing: Issues that need to be addressed by a Code of Good Practice. Internal report for SHL Group plc.
- British Psychological Society Psychological Testing Centre (2002) Guidelines for the Development and Use of Computer-based Assessments.
- European Federation of Psychologists’ Associations (EFPA). Review model for the description and evaluation of psychological tests (Bartram, 2002).
- British Standards’ Institute (BSI). BS 7988 (2001). A code of practice for the use of information technology for the delivery of assessments.
- Association of Test Publishers (ATP). Guidelines for Computer Based Testing.
The next stage involved a small scale survey of United Kingdom test publishers, examining good practice issues in Internet-delivered personality tests in the UK. Further examples of good practice were highlighted from this survey.
As a third method of obtaining relevant information, the ITC organised a conference in Winchester, England in June 2002 on Computer-based Testing and the Internet. The goal of this conference was to bring together people working in the field of computer/Internet testing (e.g., practitioners, scholars, industry leaders and others) from around the world and to extract common issues and themes that would inform the guidelines. In total 254 delegates from 21 countries attended the conference. The conference was composed of workshops, keynote presentations and themed papers, posters and symposia on a number of topics concerning computer/Internet testing. A review of the material from this conference coupled with the small survey data and literature review provided the basis for the development of the draft guidelines for initial consultation (version 0.3).
Four general issues emerged from the information gathering process and these formed the basis of the development of an initial draft version. The four issues were:
- Technology – ensuring that the technical aspects of CBT/Internet testing are considered, especially in relation to the hardware and software required to run the testing.
- Quality – ensuring and assuring the quality of testing and test materials and ensuring good practice throughout the testing process.
- Control – controlling the delivery of tests, test-taker authentication and prior practice.
- Security – security of the testing materials, privacy, data protection and confidentiality.
These four issues were considered high level issues and were further broken down into second-level specific guidelines. A third-level set of accompanying examples is provided to the relevant stakeholder. The guidelines are primarily written to provide advice to test developers, test publishers and test users; however, these guidelines also provide a useful source of reference for test-takers. Given these intended applications, the guidelines are structured in a three (main stakeholders) by three (level of guideline) matrix.
- Following development of the initial draft by the two authors, a consultation process was undertaken. This involved circulation of the draft to all those who attended the ITC Conference in Winchester and all those on the ITC circulation list for Testing International. A copy also was placed on the ITC web site. Comments on the draft guidelines were received and version 0.4 produced . In addition the report of the APA Internet Task force was published (Naglieri et al, 2004) . This was reviewed in detail and elements from the report were included in version 0.5 of the draft g uidelines.