ITC Guidelines on Test Use

Download (Word Document) ITC Guidelines on Test Use (English version)

Key Purpose and Scope of the Guidelines

Key purpose

A competent test user will use tests appropriately, professionally, and in an ethical manner, paying due regard to the needs and rights of those involved in the testing process, the reasons for testing, and the broader context in which the testing takes place.

This outcome will be achieved by ensuring that the test user has the necessary competencies to carry out the testing process, and the knowledge and understanding of tests and test use that inform and underpin this process

Scope of the Guidelines

Any attempt to provide a precise definition of a ‘test’ or of ‘testing’ as a process, is likely to fail, as it will tend to exclude some procedures that should be included and include others that should be excluded. For the purpose of these Guidelines, the terms ‘test’ and ‘testing’ should be interpreted broadly. Whether an assessment procedure is labelled a ‘test’ or not is immaterial. These Guidelines will be relevant for many assessment procedures that are not called ‘tests’ or that seek to avoid the designation ‘test’. Rather than provide a single definition, the following statements attempt to map out the domain covered by the Guidelines.

Testing includes a wide range of procedures for use in psychological, occupational and educational assessment.
Testing may include procedures for the measurement of both normal and abnormal or dysfunctional behaviours.
Testing procedures are normally designed to be administered under carefully controlled or standardised conditions that embody systematic scoring protocols.
These procedures provide measures of performance and involve the drawing of inferences from samples of behaviour.
They also include procedures that may result in the qualitative classification or ordering of people (e.g., in terms of type).

Any procedure used for ‘testing’, in the above sense, should be regarded as a ‘test’, regardless of its mode of administration; regardless of whether it was developed by a professional test developer; and regardless of whether it involves sets of questions, or requires the performance of tasks or operations (e.g., work samples, psycho-motor tracking tests).

Tests should be supported by evidence of reliability and validity for their intended purpose. Evidence should be provided to support the inferences that may be drawn from the scores on the test. This evidence should be accessible to the test user and available for independent scrutiny and evaluation. Where important evidence is contained in technical reports that are difficult to access, fully referenced synopses should be provided by the test distributor.

The test use Guidelines presented here should be considered as applying to all such procedures, whether or not they are labelled as ‘psychological tests’ or ‘educational tests’ and whether or not they are adequately supported by accessible technical evidence.

Many of these Guidelines will apply also to other assessment procedures that lie outside the domain of ‘tests’. They may be relevant for any assessment procedure that is used in situations where the assessment of people has a serious and meaningful intent and which, if misused, may result in personal loss or psychological distress (for example, job selection interviews, job performance appraisals, diagnostic assessment of learning support needs).

The Guidelines do not apply to the use of materials that may have a superficial resemblance to tests, but which all participants recognise are intended to be used only for purposes of amusement or entertainment (e.g., life-style inventories in magazines or newspapers).

Who the Guidelines are for

The Guidelines apply to the use of tests in professional practice. As such they are directed primarily towards:

The purchasers and holders of test materials;
Those responsible for selecting tests and determining the use to which tests will be put;
Those who administer, score or interpret tests;
Those who provide advice to others on the basis of test results (e.g., recruitment consultants, educational and career counsellors, trainers, succession planners);
Those concerned with the process of reporting test results and providing feedback to people who have been tested.

The Guidelines will be of relevance to others involved in the use of tests as defined above. These include:

the developers of tests,
the suppliers of tests,
those involved in the training of test users,
those who take tests and their relevant others (e.g., parents, spouses, partners),
professional bodies and other associations with an interest in the use of psychological and educational testing, and
policy makers and legislators.

While aimed primarily at professional practice, most aspects of the good practice embodied in the Guidelines will also be of relevance to those who use tests solely for research purposes.

The Guidelines are not intended to cover every type of assessment technique (e.g., unstructured or semi-structured interviews, assessed group activities) or every situation in which assessment occurs (e.g., employment assessment centers). Yet many of the Guidelines are likely to be applicable in assessment situations and for purposes more general than those concerned primarily with psychological and educational testing (for example, the use of assessment centers, for employee placement or selection, semi-structured and structured interviews, or assessment for selection, career guidance and counseling).

Contextual factors

The Guidelines are applicable internationally. They may be used to develop specific local standards through a process of contextualisation. It is recognised that there are many factors which affect how standards may be managed and realised in practice. These contextual factors have to be considered at the local level when interpreting the Guidelines and defining what they would mean in practice within any particular setting.

The factors that need to be considered in turning Guidelines into specific standards include:

social, political, institutional, linguistic, and cultural differences between assessment settings;
the laws of the country in which testing is taking place;
existing national guidelines and performance standards set by professional psychological societies and associations;
differences relating to individual versus group assessment;
differences related to the test setting (educational, clinical, work-related and other assessment);
who the primary recipients of the test results are (e.g., the test-takers, their parents or guardian, the test-developer, an employer or other third party);
differences relating to the use of test results (e.g., for decision-making, as in selection screening, or for providing information to support guidance or counselling); and
variations in the degree to which the situation provides opportunity for the accuracy of interpretations to be checked in the light of subsequent information and amended if needed.

Knowledge, Understanding, and Skill

Knowledge, understanding and skill underpin all the test user competencies. The nature of their content and level of detail are likely to vary between countries, areas of application and as a function of the level of competence required to use a test.

The Guidelines do not contain detailed descriptions of these. However, when applying the Guidelines for use in specific situations the relevant knowledge, skills, abilities and other personal characteristics will need to be specified. This specification is part of the process of contextualisation, through which generic guidelines are developed into specific standards. The main areas descriptions of knowledge, understanding and skills need to cover include the following.

Relevant declarative knowledge

This includes:

knowledge of basic psychometric principles and procedures, and the technical requirements of tests (e.g., reliability, validity, standardisation);
knowledge of tests and measurement sufficient to enable the proper understanding of test results;
knowledge and understanding of relevant theories and models of ability, of personality or other psychological constructs, or of psychopathology, as necessary to properly inform the choice of tests and the interpretation of test results; and
knowledge of the tests and the test suppliers relevant to one’s area of practice.

Instrumental knowledge and skills

These include:

knowledge and skills relating to specific assessment procedures or instruments, including the use of computer-based assessment procedures;
specialised knowledge of and practitioner skills associated with using those tests that are within one’s repertoire of assessment tools; and
knowledge and understanding of the construct or constructs underlying test scores, where this is important if valid inferences are to be drawn from the test results.

General personal task-related skills

This includes:

the performance of relevant activities such as test administration, reporting, and the provision of feedback to test takers and other clients;
oral and written communication skills sufficient for the proper preparation of test takers, test administration, the reporting of test results, and for interaction with relevant others (e.g., parents, or organisational policy makers); and
interpersonal skills sufficient for the proper preparation of test takers, the administration of tests, and the provision of feedback of test results.

Contextual knowledge and skills

This includes:

knowing when and when not to use tests;
knowing how to integrate testing with other less formal components of the assessment situation (e.g., biographical data, unstructured interview and references etc.); and
knowledge of current professional, legal, and ethical issues relating to the use of tests, and of their practical implications for test use.

Task management skills

This includes:

knowledge of codes of conduct and good practice relating to the use of tests, test data, the provision of feedback, the production and storage of reports, the storage of and responsibility for test materials and test data; and
knowledge of the social, cultural, and political context in which the test is being used, and the ways in which such factors might affect the results, their interpretation and the use to which they are put.

Contingency management skills

This includes:

knowing how to deal with problems, difficulties, and breakdowns in routine;
knowing how to deal with a test taker's questions during test administration etc.; and
knowing how to deal with situations in which there is the potential for test misuse or for misunderstanding the interpretation of test scores.

Focus and background of this project

The focus of this ITC project is on good test use and on encouraging best practice in psychological and educational testing. The work carried out by the ITC to promote good practice in test adaptations was an important step towards assuring uniformity in the quality of tests adapted for use across different cultures and languages. However, there are two key issues in psychological test practice. First, one has to ensure that the tests available meet the required minimum technical quality standards. Second, one needs to know that the people using them are competent to do so.

The Test Use guidelines project was started following a proposal from the present author to the ITC Council in 1995. The aim was to provide a common international framework from which specific local standards, codes of practice, qualifications, user registration criteria, etc could be developed to meet local needs. Specific aims were:

To produce a set of guidelines relating to the competencies required of test users.
To cover all areas of test use.
To consult widely and include all stakeholders.
To formulate the Guidelines in the form of assessable performance (outcome) statements.

The intention was not to ‘invent’ new guidelines, but to draw together the common threads that run through existing guidelines, codes of practice, standards and other relevant documents, and to create a coherent structure within which they can be understood and used.

The competencies defined by the guidelines were to be specified in terms of assessable performance criteria, with general outline specifications of the evidence that people would need for documentation of competence as test users. These competences needed to cover such issues as:

professional and ethical standards in testing,
rights of the test candidate and other parties involved in the testing process,
choice and evaluation of alternative tests,
test administration, scoring and interpretation,
report writing and feedback.

The strategy was to deal with issues of test use quality and assessment context by focusing first on the competence of test users. This does not remove the need to also address directly the issue of test quality (see below). However, an approach that stresses user competence is more likely to ensure that users:

do not use bad tests,
know when to test and when not to,
know which tests are relevant and which are not,
know how to apply tests,
know how to treat people who are being tested.

In the process of development, emphasis was placed on the need to consider and, where possible, consult a number of different stakeholders. These fall in to three broad categories:

Those concerned with the production and supply of tests (e.g. test authors, publishers and distributors);
The consumers of tests (e.g. Test users, test takers, employers and other third parties such as parents, guardians etc);
Those involved in the regulation of testing (e.g. professional bodies, both psychological associations and others, and legislators).

The completed Guidelines represent the work of psychologists and specialists in educational testing drawn from a large number of different countries. They have been designed to help address issues of control and regulation of access to test materials and place an emphasis on the fair and ethical use of tests.

The ITC Guidelines in Test Use project received backing from the BPS, APA, NCME, EAPA, EFPPA, and from a large number of European and US test publishers. Following formal approval by the ITC Council in Graz, 1999 and by the EFPPA Standing Committee on Tests and Testing in Rome, 1999, the Guidelines were launched in Stockholm, at the ICP Congress in July 2000.

A copy of the full Guidelines (in English) can be obtained by clicking on the link at the top of this page; they were also printed in the first edition of the ITC’s International Journal of Testing.

Translations of the Test Use Guidelines

Official translations into the following languages are completed. These are available as Word or PDF files by clicking the appropriate link.

Polish

South American Spanish (Argentina)

Spanish

Swedish

The Danish translation of the Guidelines is contained within a document which can be obtained from this site:

Danish

Official translations are currently underway in:

Chinese

The following conditions apply to official versions of the International Test Commission (ITC) Guidelines. The conditions delegate authority for checking the quality and accuracy of translation to the local national Psychological Association.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1985). Standards for Educational and Psychological Testing. Washington DC: American Psychological Association.

Bartram, D. (1995). The Development of Standards for the Use of Psychological Tests in Occupational Settings: The Competence Approach. The Psychologist, May, 219-223.

Bartram, D. (1996). Test Qualifications and Test Use in the UK: The Competence Approach. European Journal of Psychological Assessment, 12, 62-71.

Canadian Psychological Association. (1987). Guidelines for Educational and Psychological Testing. Ottawa: Canadian Psychological Association.

Eyde, L. D., Moreland, K. L. & Robertson, G. J. (1988). Test User Qualifications: A Data-based Approach to Promoting Good Test Use