International Conference on Computer-Based Testing and the Internet

Conference mission

The conference mission was to generate outcomes to feed into the development of international good practice guidelines.The conference addressed issues relating to the use and misuse of electronically transmitted tests and other assessment devices, and methodological, technical, professional and ethical issues arising from the application of new technology to testing. These issues included the impact of technology on those professions and professional services involved in testing. Legal and ethical issues associated with use and ownership of data, provision of feedback and related issues were also addressed.

Areas and issues addressed by the Conference

• The use of item generation, item banking, Structural Equation Model analysis and Item-Response Theory applications.Technology provides the capability for making practical use on a large scale of developments in psychometrics. We need to consider how these techniques might be used to advance the state of the art as well as areas where they may be misused (for example, by those who do not understand these technologies). This area raises new issues for test use guidelines, as well as for educating test users and the public.
• The opportunities afforded by new item formats, multimedia and new types of test. As technology advances, we will be able to make use of higher bandwidth delivery methods. Streaming video, voice and other media will be available and will add to the range of assessment tasks we can deliver. Artificial intelligence tools will increase our capability for scoring open-ended items and essays. The use of online video and audio in language testing and assessing comprehension will enhance our ability in areas such as English as a Foreign Language testing. At the same time, these tools will enable others to rapidly develop 'whizzy' applications that may have high face validity as assessments. The users need to be able to discriminate between effective and ineffective tests. In addition to looking at the impact of current technologies we will also consider emerging technologies and how they may change the current situation.
• Operational issues: managing and controlling item spotting; refreshing item banks; spotting exposed items; managing differential familiarity with technology; dealing with the provision of feedback to test takers and other users. The Internet poses a host of potential dangers as well as benefits in relation to both control over test security and access. For some time to come, there will be marked differences between countries (especially outside North America, Europe and parts of Asia) and within countries with the access to the medium and familiarity with it.
• Security, legal, ethical and technology issues: data protection, copyright issues, confidentiality, Internet bandwidth etc. The Internet raises a host of issues relating to local and international law. Countries have yet to harmonise laws on privacy and data protection. There are both technical and best practice issues to be taken into account when considering these issues. How safe and secure is this as a medium? To what extent can people and organisations trust systems with their personal data? What can and should be done to enhance both actual security and perceptions of safety? How much will emerging technologies solve these problems and how much might they exacerbate them?
• Applications and issues in employment (selection development and recruitment), psychological educational diagnostic assessment and health/clinical testing. This section focuses on the use of testing for assessing potential and for diagnosis. In the organisational field, we see an increasing use of formalised assessment in the early stages of recruitment over the Internet. This shift from proctored assessment on short-listed candidates to large-scale pre-short-list un-proctored assessment raises many questions, both of a technical nature and in relation to good practice.
• Applications and issues in licensing, credentialing and job performance assessment. This section considers issues relating to the assessment of mastery or competence. Increasing use is being made of the Internet for assessing people for job competence and for the award of vocational qualifications, licenses and certification.

Four general issues, which run across these 6 areas, were considered: