Criterion referenced assessment

The unit ITN338 Information Resource Provision had evolved over a period of three years, not only to respond to the shifting dynamics in the management of and provision of access to information resources, but also to fit more closely with other units in the GDLIS. For a number of years, the principal pieces of assessment were an individual discussion paper, an oral presentation and a group project which required students to evaluate a real library collection. In 2004, the group learning activities moved to another unit in which there was strong focus on the development of teamwork skills. ITN338 was then seen as a unit which could provide students with the opportunity to focus more on their individual skills, yet with some collaborative activities.

Huba and Freed (2000) underscore the importance of clearly articulated learning outcomes as the initial step in the development of learner-centred assessment: "The first element of the assessment process is that, as faculty, we develop a set of intended learning outcomes, statements describing what students should know, understand, and be able to do with their knowledge..." (p.9-10). The learning outcomes for units at QUT are presented as objectives to cover Theory, Practice and Generic Capabilities. On completion of the unit, students should be able to understand and discuss:

• the value of information as a community resource

• the significance of diverse community information needs

• the issues associated with collection development and collection evaluation

• the issues associated with publishing, selection, purchasing and licensing of information resources

• the process of scholarly communication

• the changing environment of acquisitions work

At a practical level, students should be able to:

• research and discuss issues relevant to the unit

• prepare a written document for submission as a journal article

• undertake the process of peer review

• contribute to the collaborative development of an online journal

• deliver an oral presentation on a professional topic

The generic capabilities to be developed during the study program included skills in:

• information literacy

• critical, reflective and creative thinking and evaluation

• team work

• oral and written communication

The unit coordinator felt that an online journal would offer an innovative approach to structure the learning and assessment activities in the unit, with the goal of achieving the desired learning objectives. The university's online learning and teaching environment (OLT) provided the context where the journal could be created and published. Taking a holistic view of the learning activities, each assessment task was developed as an individual component in a cohesive model of teaching and learning.

The fact that new assessment tasks were to be designed meant that the relevant assessment criteria could be developed as part of the process. Griffin and Nix (1991) refer to criterion referenced assessment (CRA) as "a cohesive set of skills or standards" (p.4). It offers the opportunity for interpreting student performance in relation to given criteria with a set of standards, enabling performance to be described in terms of the tasks undertaken. Each criterion addresses a specific domain of content or behavior and should be clearly expressed for all students prior to assessment. "The notions of proficiency, achievement and competence need to be clarified in terms of the tasks set, and the associated standards or criteria for mastery within each domain to be tested need to be defined" (p.77). Performance is viewed as a sliding scale between low levels and high levels of demonstrated ability, with mastery at the higher end of the continuum.

The development of the assessment criteria and associated standards of achievement were developed over a period of several months. Throughout Semester 1, 2004, as Teaching Fellow, the unit coordinator was able to develop and deepen her understanding of assessment in general, and CRA in particular, through independent academic study, discussion with colleagues at Teaching Fellowship workshops and participation in focus groups on the issues associated with CRA. The theoretical knowledge was therefore tested and applied in the development of models of criteria and standards of assessment for the unit. The diverse assessment activities in ITN338 provided considerable variety in terms of the range of criteria to be developed, covering research skills, critical analysis and the structural elements of preparing a journal article, planning and delivering an oral presentation, critical reflection etc.

The unit was run in semester 2, 2004, with 36 students enrolled. As the Teaching Fellowship arrangements provided funding for a sessional staff member to be appointed to teach the unit and to mark the assessment, there was a further opportunity for testing the CRA process more objectively than if the unit coordinator was directly responsible for teaching and marking.

The CRA matrix encompassed the range of different criteria, with five different standards of achievement. QUT uses a seven-point scale of grading:

7: 85%-100%

6: 75%-84%

5: 65%-74%

4: 50%-64%

Grades 1-3 represent different levels of poor achievement. There was considerable discussion in the Teaching Fellow community about the challenges of developing more than five standards of achievement, particularly at the lower end of the scale. The motivation and engagement of learners is often found to be far higher amongst postgraduate, fee-paying coursework students than at the undergraduate level. There is anecdotal evidence to indicate that GDLIS students tend to drop out of the course rather than submit inferior work. Accordingly, the CRA matrix was developed with the 4 grades to reflect the marks of 50%-100%, and one grade of Refer/Fail to capture students who did not produce work of the required standard of < 50% in ITN338.

The assessment criteria themselves were developed to be closely aligned with the desired learning outcomes for the unit, with emphasis placed on the level of understanding of the topic and the information context, the quality of the research, analysis and critical reasoning. As each student could select their own specific discussion topic, the assessment criteria were written at a high level to have general applicability across a variety of situations. The weighting of each criterion varied, to indicate to students the value of the different areas of the work they were asked to produce.

For the first piece of assessment, the journal article, criteria such as i) understanding of the topic; ii) understanding of the information context; iii) analysis and critical reasoning were each weighted with 20 marks, totaling 60% of the assignment. Use of the literature/ evidence of reading was weighted at 15 marks, and then 5 marks were given to each of the criteria of the conclusions drawn, referencing, and written expression and presentation. The final 10 marks were awarded for a reflective discussion on the student's personal learning outcomes and the critical review of his/her own work.

It goes beyond the scope to discuss in detail the formulation of the text of each of the standards of achievement, but two examples are presented to highlight the concepts, with the standards for the grades 7, 6 and 5. Indeed only one student was awarded a 4 as the final grade for the unit.

The criterion 'understanding the topic' (Table 1) was broken down into two key attributes, the ability to:

• identify the audience

• identify and discuss the central issues of the topic.

The second criterion 'understanding the context' (Table 2) was expressed as the ability to relate theoretical aspects of the topic to practical application in specific contexts.

It should be noted that in class and in the study guide, students were given clear guidance about the need to write for a given professional audience and also to ensure that in developing their article, they should consider both the theoretical and practical aspects of the topic and should discuss case studies to highlight current practice in libraries and information centres today.

The oral presentation had a fairly even spread of weightings across the criteria of planning and preparation; understanding and analysis of the topic, quality of delivery and oral communication skills, visual aids and the reflective discussion.

In developing the assessment criteria for ITN338, semantics and vocabulary were central issues. The ability to use words and phrases that are specific and unilaterally understood in theory and in practice presents the greatest challenge. Sadler (1987) has argued that the lack of clarity or 'fuzziness' in the verbal descriptions of criteria inevitably results in differing interpretations on the part of students. O'Donovan, Price and Rust (2000) also indicate the confusion that can occur if the criteria are vague and/or non-explicit. Sadler (1998) and Ramsden (1992) underscore the importance of ensuring strategies are in place to ensure a shared understanding of the criteria and standards by students and academic staff. Carlson et al. (2000) state that "Effective shared understanding needs to go beyond simple definitions of the terms used to encompass the practical application of the criteria and standards. For example, students may understand that they have to demonstrate critical thinking but may not be able to translate this theoretical understanding or assignment" (p.111). Sadler (1987) presents the view that shared understanding is like the two sides of a coin: the first side is 'disclosure' or the public articulation of the criteria against which performance will be judged, and the other side is 'visibility' or how the student interprets these criteria. Carlson et al. (2000) quote one academic staff member who indicates that it is only when students "actually internalize [the criteria] with their own work" (p.111) that any true sense is made of the criteria or expectations.

اخر الاخبار

اخبار العتبة العباسية المقدسة

قسم الشعائر يواصل تحضيراته لتنظيم عمل المواكب الحسينية في زيارة الأربعين

قسم شؤون المعارف يناقش تحضيرات المؤتمر العلمي الأول لمركز تراث البصرة

قسم الشؤون الفكرية يقيم دورة تطويرية لملاكات هيأة المساءلة والعدالة في الأرشفة الإلكترونية

المجمع العلمي ينظم مجلس عزاء في قضاء الهندية

المزيد

الآخبار الصحية

نصائح عملية للحد من التعرق المفرط في الصيف

أطباء يحذرون: مكمل غذائي شائع قد يدمر الجهاز العصبي

تحذير طبي من تأثير جانبي "غير متوقع" لحقن التخسيس

حمامات الثلج قد تكون خطيرة وأحيانا مميتة.. ما السبب؟

المزيد

الآخبار التكنلوجية

أكبر جدارية شمسية في العالم تدخل موسوعة غينيس.. مشروع ضخم

تسارع دوران الأرض يثير القلق.. وتحذيرات من "كوارث محتملة" ؟

"سوذبيز" تطرح أكبر قطعة من المريخ على الأرض في مزاد علني

أوبن أيه آي تعتزم إطلاق متصفح إنترنت يعمل بالذكاء الاصطناعي

المزيد

مواضيع ذات صلة

Blending Process with Product: Using Assessment to Drive Learning Through the Creation of an Online Journal Conclusions

Evaluation of the pilot project

Assessment as the driver for learning in ITN338

Methodology

The learning context

Blending Process with Product: Using Assessment to Drive Learning Through the Creation of an Online Journal In September 2003, Queensland University of Technology (QUT) endorsed a new assessment policy to introduce criterion referenced assessment (CRA) in

Innovative Assessment and Learning in a Problem Based Environment Conclusions

Assessment of Accomplishments

Enhancing Teaching

Discussion; Enhancing Learning

Instructors

Professional Behavior