Home | Business News | Browse by Publication | A | Australian Journal of Education

Standards-referenced assessment for vocational education and training in schools.

Article Excerpt
This study examined a model of assessment that could be applied nationally for Year Twelve Vocational Education and Training (VET) subjects and which could yield both a differentiating score and recognition of competence. More than fifty colleges across all states and territories of Australia field-tested the approach over one school year. Results showed that the model allowed for a standards-referenced model to be used: that the approach was compatible with the diverse range of senior secondary assessment systems in use throughout Australia and that there were considerable cost benefits to be had in adopting the logic of item response modelling for the development of rubrics for scoring performances on units of competence from National Training Packages. A change in the logic of competency assessment was proposed, in that the performance indicators were not rated using a dichotomy but with a series of quality ordered criteria to indicate how well students performed specified tasks in the workplace or its simulation. The study validated the method of assessment development, demonstrated the method's consistency, and showed how the method could address the issue of consistency across states. The study also proposed a set of principles for a joint assessment of both quality and competence.

**********

The introduction of competency-based education to Australia in 1992 was intended to increase the skill level of the Australian workforce. Its introduction was accompanied by a debate about how the competencies should be assessed and other issues (such as grades) that were more related to reporting than assessment. The discussions were hampered by a lack of clarity of the terminology, shifting definitions of basic concepts and inconsistent use of language to describe the process (Stanley, 1993).

The debate surrounding the nature of competency-based assessment centred on two issues: first, whether grading was a suitable approach to use with competency assessment; and second, whether criteria for assessment can be generic or specific.

Discussion of both issues tends to have been confused by imprecise use of terminology and appears to have ignored the considerable body of literature available on each topic. The lack of rigorous research and theoretical models has been detrimental to the development of the field.

One of the few comprehensive studies into this field (McCurry, 2003) has shown that generic criteria cannot be applied effectively to competence assessments, and therefore that debate will not be extended here. Rather, this article focuses on the application of specific criteria in the development of assessment procedures to provide a differentiating score for students who chose to study vocational subjects in their last year of secondary school and who, under the regime of the dichotomy between the existence or absence of competency, were at a disadvantage to those students studying subjects graded on levels of quality when university selection procedures were implemented.

Griffin, Gillis, Keating and Fennessy (2001) examined the use of standards--referenced assessments in vocational education and Griffin and Gillis (2001; 2002) outlined possible procedures that could be used to examine the efficacy of a differentiated scoring system for VET in schools. Their research has shown how current approaches to competency-based assessment could be used to yield a differentiated score in addition to the recognition of competence, without altering the fundamentals of the entrenched competency-based approach but instead focusing on a customisation of record-keeping and reporting frameworks. This article reports on how those approaches were tested and the model of assessment for competency-based education that emerged.

Competency assessment and standards-referenced assessment

A major purpose of competency assessment has been to provide for the recognition of competence. This article argues that it can also provide evidence of differentiation for selection purposes. Observations used to provide for recognition or differentiation can all be based on the procedures already used by assessors in the workplace, endorsed by the Australian National Training Authority and national industry training advisory boards and documented in the relevant assessor-training package (National Assessors and Workplace Trainers Body, 1998).

When competency-based assessment was introduced into vocational education in Australia in the early 1990s it was presented as an example of criterion-referenced or criterion-based assessment, but this was a limited and misleading view of criterion referencing. Over the best part of a decade, the discussion was based on the use of a dichotomy that recognised only the existence or absence of competence and excluded the idea of the quality of performance. In 2001, Griffin et al. argued that the definition of criterion-referenced assessment could itself be used to justify the approach of the dichotomy of competent or not-yet-competent and yet still yield a differentiating score. They argued that competence was itself a decision point within a continuum of increasing quality of performance and this was entirely consistent with the original concept of criterion referencing. Glaser (1981), who conceived criterion-referenced interpretation, agreed that the concept incorporated a competence decision because it was about the ability to progress along a continuum of increasing competence as defined by performance quality. Standards referencing is recent terminology for a form of criterion-referenced interpretation. It is important to note also that criterion referencing is not an assessment method. It is not a testing procedure; rather, it is the use of a defined interpretation framework. The correct use of the expression should refer to criterion-referenced interpretation of assessment or standards-referenced interpretation of assessment, not criterion-referenced or standards-referenced assessment or testing in and of itself.

In the early 1980s, the development of subject profiles within the school systems was an extensive and practical approach to the development of criterion-referenced interpretation frameworks. This led to the implementation of national profiles and curriculum statements and within a few years they became the basis of 'outcomes-based education'. Outcomes were defined in terms of increasing levels of competence within discipline areas in the school curriculum. Outcomes were described using 'progress maps', but progress maps could be generated by a small number of people capable of conducting item response modelling analyses. Standards referencing was first proposed in Queensland in the late 1980s and early 1990s (Sadler, 1987) and gained credibility with the McGaw (1997) and later Masters (2002) reports regarding the New South Wales Higher School Certificate. Largely through these reports, standards referencing became synonymous with outcomes-based education. Outcomes and competence-based education merged through this terminology and standards referencing began to subsume the ideas of competency in school education.

Education began to report in terms of developmental frameworks using descriptions of skill levels and competencies; hence, if a similar shift occurred and a criterion or standards-referenced framework were to be adopted for competency-based education in VET, recording methods would need to be adjusted so that the records of achievement by students in 'VET in schools' programs could record their level of performance in addition to the competence dichotomy.

Universities make decisions about which students to select into their courses and most use a ranking system based on Year Twelve examinations. The system is usually based on a percentile rank reported at an assumed accuracy of two decimal places. Errors of measurement must be minimised and the quality and reliability of the assessment data need to be high in order to allow fine discrimination between candidates seeking entrance into university.

The current understanding of recognition requires that a person be described as having achieved...

View this article FREE - Now for a Limited Time, try Goliath Business News
Free for 3 Days!