Mapping Self-Assessment to Achievement

The University of Portsmouth is introducing Personal Development Planning (PDP) within a three-year, staged program. The program commenced with the introduction of Level 1 activities in October 2003 and will culminate in 2006 by offering activities for postgraduate students, thus ensuring that PDP opportunities are in place for all awards. This strategy is in line with the UK Government agenda disseminated via the Tomlinson Report (2004), which stressed the importance of PDP in higher education (HE) and the Burgess Report (2004) recommending the continued introduction of PDP for all HE awards.

The strategy adopted at Portsmouth reflects the recommendations of a government-commissioned report and the Quality Assurance Agency (QAA). The 1997 National Committee of Inquiry into Higher Education, chaired by Sir Ron Dearing, recommended that all higher education institutions should introduce Progress Files for all awards comprising two elements:

• a transcript recording student achievement that should follow a common format devised by institutions collectively through their representative bodies;

• a means by which students can monitor, build and reflect upon their personal development. (NCIHE 1997)

Following the publication of the Dearing report, the CVCP¹, Standing Conference of Principals (SCoP) and Quality Assurance Agency (QAA) produced Guidelines for HE Progress Files. These guidelines were designed to support universities and colleges of Higher Education in the development of progress files for all HE awards by 2005-2006. The guidelines defined PDP as:

..a structured and supported process undertaken by an individual to reflect upon their own learning, performance and / or achievement and to plan for their personal, educational, and career development. (QAA, 2001)

The PDP element of a progress file aims to help students reflect on their own learning, set learning goals and plan how they are going to achieve them.

The team responsible for implementing PDP at Portsmouth considered that the most practical way forward would be to introduce PDP over a three year period, commencing with activities for Level 1 (usually first year) students and then adding activities for Levels 2 and 3 in subsequent years of the implementation. Level 1 activities would commence with an on-entry exercise to help student assess their skill levels and provide a base line from which they could develop and gauge their progress in the future.

In October 2002, the University piloted a paper-based on-entry assessment exercise and over 1,000 Level 1 students across all faculties took part in the pilots. After favorable feedback from academic staff and students involved in the pilots, the exercise was renamed the Individual Learning Profile (ILP) and introduced for all new undergraduate entrants in October 2003. The ILP prompted students to assess their skills in a number of key areas including communication, time management, researching, etc. After students completed the questionnaire it was processed by optical mark reader and returned to personal tutors within 1 to 4 days. Subsequently, tutors and students discussed (usually in personal tutorial session) the areas that required development and how students could take advantage of the options available to them to maximize their learning potential. In 2003, and again in 2004, over 3,000 students completed the questionnaire, each representing approximately 66% of first year student registrations for that year.

The utility of the ILP questionnaire was rapidly established and received positive formal and informal feedback from staff and students. Tutors commented that they found the ILP questionnaire provided valuable individual feedback and enabled them to work with students to address any potential problem areas at the earliest stage. Course leaders stated that the group level reports, which were returned with processed forms, provided useful information that could be used in curriculum development and to enhance program reviews. Students were generally positive about the questionnaire; many stated that it helped them reflect on their current skill levels and think about the actions that they needed to take to support their learning at an HE level.

PDP is operationally sound at Portsmouth, but key questions regarding its value as an educational tool remain unanswered. A key research issue for current PDP practice, and all institutions involved in the implementation of PDP, is its impact upon learning. Jackson and Ward (2004) articulate this concern and propose three areas for investigation: synthesis of relevant scientific knowledge, synthesis of institutional evaluations and practitioner action research, and examining PDP against theoretical models of learning.

There has been some investigation of relevant scientific knowledge, but further work is required. In 2003, Gough et al. (2003) compiled a report evaluating and synthesizing current research that focussed on the links between PDP and learning. A large number of relevant studies were considered for review; a number of key criteria were used to determine the quality and relevance of each study. A systematic map was developed which examined the various approaches to PDP, the context of the studies, methodology and the outcome measures. The majority of the studies originated in the USA with a smaller number in the UK; many examined PDP-like activity that focussed on course-specific outcomes and the use of learning logs. Study outcome measures generally related to learning approaches/learning styles, knowledge gain/attainment and, to a lesser extent, career outcomes. A subset of the studies, researcher-manipulated rather than descriptive cases, were subject to a more in-depth review. Findings indicated that where outcome measures related to learning approaches or styles, 9 of the 13 studies reported positive effects; where the outcome measure was student attainment all 10 studies reviewed reported positive effects. Only 3 of the studies used personal outcomes to measure the impact of PDP and the findings were respectively, positive effect, negative effect and no evidence of effect.

The second area for investigation proposed by Jackson et al. (2004) focuses on the experience of HE practitioners and institutions. Gough et al. (2003). have gathered data from a number of institutions and individuals. Higher Education Academy, in partnership with the Centre for Recording achievement, are compiling a catalogue of practitioner experience, current practice and proposals for future direction which will be made available to interested parties through various mediums including published papers and national newsletters.

The third area for investigation centres on the relationship between PDP and theoretical models of learning. One such model is Bandura's (1977; 1997) concept of self-efficacy, which is adopted here. Bandura (1977; 1997) argues that self-efficacy is a key motivating factor in goal achievement. Outcome expectations i.e. the expectation that certain behavior will lead to the goal (outcome) are a part of motivation. However, it is argued that outcome expectations alone are not sufficient to explain the behavior of an individual. An individual may expect that certain behaviors will result in a desired outcome, but it does not necessarily follow that they will complete the necessary behavior. Bandura (1977; 1997) argued that this could be understood in terms of efficacy expectations; an individual must be aware of the required behavior and be confident that they have the ability to execute that behavior. A person who has low efficacy regarding their ability to successfully complete a task and reach a goal may avoid the situation or expend little effort, whereas an individual with high efficacy may feel capable of tackling the situation and subsequently may use more effort. The ILP aims to measure students' confidence in key skill areas. It is possible that high confidence levels in an individual indicate higher levels of self-efficacy and the propensity to expend more effort in developing skills and subsequently may attain a higher level of achievement.

It is students' perceptions regarding their efficacy, rather than actual level of efficacy, that is important. Decisions regarding efficacy, according to Bandura (1977; 1997) can be made even in the absence of experience. Bandura (1977; 1997) argues that reference is made to performance accomplishments and vicarious experience. Performance accomplishments involve reference to the person's own experience in similar situations and are the most dependable source of efficacy expectations. More success results in an increased efficacy expectation. The effect of failure is temporal in that it is dependent upon the timing of the failure within a success/failure sequence. For example, an early failure is liable to result in a larger decrease in efficacy than a failure that follows a string of successes (particularly if then followed by more success). In the absence of personal experience, efficacy expectations may be formed through vicarious experience i.e. observing other individuals' experience of a similar task. There is a larger increase in efficacy when i) the other individual displays characteristics similar to those of oneself and ii) they have experienced difficulty with the task but persisted and succeeded. This effect is heightened when more than one other person has been successful in achieving the goal. Discussion and reflection may influence efficacy judgements since verbal persuasion is another reference point for such judgements (Bandura, 1977; 1997).

Evidence would suggest that the type of goal-setting behavior typified by PDP-style processes has an effect on achievement where goals are proximal (consequences are likely to be experienced in the near future) as opposed to distal (the consequences will not be experienced for some time). Bandura and Schunk (1981) argued that proximal goals motivate more than distal goals as they offer immediate incentives, allowing an individual the opportunity to assess their capabilities at an early stage, which combined success is achieving goals, will lead to an increase in self-efficacy. Bandura and Schunk (1981) tested this hypothesis and found that children who were set proximal goals had significantly higher self-efficacy than those who had been set distal goals. The research would suggest that there is only a nominal disparity between self-efficacy levels of those who set distal goals and those who were not set goals at all.

As self-efficacy arguably affects the amount of effort expended on a task and an individual's persistence, a significant body of research focuses on the use of self-efficacy for predicting achievement. In a review of the history of, and debates within, self-concept research, Pajares and Schunk (2001) describe the hierarchy of self-efficacy for predicting achievement where the various types of self-efficacy rank in the following descending order: subject-specific self-efficacy; academic self-efficacy, and global self-efficacy. They assert, "It is clear that self-concept becomes more empirically sensitive to, and more predictive of, achievement outcomes the more specifically that it is conceived and assessed." (p.244). They also address the issue of causality: is self-belief determined by academic achievement or is the converse true, that self-belief determines achievement? The research indicates that the relationship is considered to be a reciprocal one.

Bong (2001) longitudinally examined various constructs of self-efficacy at two different time points. The research included self-efficacy for self-regulated learning, academic achievement, course-specific, content-specific and problem-specific self-efficacy and the perceived value of the course in female Korean undergraduates. Bong also examined the future course enrolment intentions and the performance of the students (at midterm and final exams). The various types of self-efficacy were not equally predictive. Time 1 course-specific self-efficacy failed to predict achievement (as measured by midterm exams) but did predict time 1 course enrolment intentions, as did the perceived value of the course at time 1. The perceived value of the course at time 1 was predictive of achievement (midterm exams). Time 1 course-enrolment intentions were predictive of time 2 self-efficacy, perceived value of the course and time 2 course-enrolment intentions. Time 2 self-efficacy was predictive of final exam performance and course enrolment intentions at time 2. This provides evidence that self-efficacy beliefs occur at varying levels of specificity.

Andrews (1998) researched the influence of previous experience of science i.e. the study of science during the final year of high school, upon self-efficacy for science and the predictive validity of this measure for achievement. Those who had studied science had a higher self-efficacy score than did those who had not, although this difference was not significant. Scores on self-efficacy for science were significantly correlated with academic performance.

Research has indicated varying factors that relate to the previous experience of the task influence self-efficacy. Mitchell, Hopper, Daniels, George-Falvy and James (1994), using a sample of Air Traffic Controllers, explored the factors used to establish self-efficacy beliefs during the acquisition of the skills. It was found that the more experienced an individual, the less they drew on task and contextual factors as a basis for self-efficacy beliefs and instead referred to past performance

and their feelings at that time. In the first trial, self-efficacy was shown to be a more effective predictor of performance than the scores that individuals expected to gain. In subsequent trials, expected and aspired scores were better predictors of performance than self-efficacy.

Vrugt, Oort and Zeeberg (2002) examined the differences between beginners and advanced students in terms of their self-efficacy and task orientation. It was found that for both advanced students and beginners a high level of self-efficacy led to the pro-active pursuit of personal goals, which subsequently had an impact on their levels of achievement. However, these relationships were more marked for advanced students than for beginners. Vrugt et al. (2002) hypothesise that this is due to beginners being less familiar with the demands of the task. Task orientation did not contribute to self-efficacy for beginners in the way that it did for advanced students and in fact had a negative effect upon achievement for beginners. It was argued that beginners had yet to understand the necessary skills for success.

Chacko and Huba (1991) tested a causal model of cognitive and affective variables upon self-efficacy, achievement, use of study strategies/self-monitoring and concentration/preparation for class-based group of first-year nursing students. They found a direct relationship between self-efficacy and achievement. Language and math ability, motivation and the individual's concentration/preparation for class impacted on self-efficacy. Concentration and preparation were directly influenced by self-monitoring and the use of study strategies. It was argued that there was a direct relationship between time dedicated to study strategies and effective self-monitoring, concentration, self-efficacy and subsequently, achievement.

If self-efficacy is argued to affect persistence in a given task, there is a noticeable lack of research about the predictive effects of self-efficacy and the relationship with withdrawal rates in higher education. It could be argued the effect is accumulative: low self-efficacy leads to poor achievement rates, poor achievement further reduces self-efficacy levels, which can lead to high withdrawal rates. However, in many cases withdrawals occur prior to the first semester exam and therefore it is important to investigate the relationship between self-efficacy on-entry and withdrawal rates.

A 3-year longitudinal study of nursing students' attrition rates included the development and testing of psychometric tools to measure students' self-efficacy relating to their academic and clinical skills (Harvey & McMurray, 1994). The study compared the self-efficacy of those students who withdrew, those who completed and those who were continuing with the intention of completing at a later date; the three groups were similar in terms of age or gender. Two aspects of self-efficacy were investigated: academic self-efficacy and subject specific (clinical) self-efficacy. Research suggested that there was minimum disparity between the levels of clinical self-efficacy for completers, continuers and those who withdrew. However, students who completed had the highest levels of academic self-efficacy, those who intended to continue exhibited a lower level of academic self-efficacy and those who withdrew displayed the lowest level of academic self-efficacy. Harvey and McMurray (1997) argue that withdrawal rates could not be attributed to achievement alone and that there is a case for the predictive validity of self-efficacy for retention. It is important not to overstate this case. Although there were significant differences between the groups as identified by ANOVAs, the lack of multiple regression or similar techniques means that the predictive validity of self-efficacy scores to predict retention cannot be determined.

While some existing research aligns strongly to current priorities in HE, further evidenced-based research is required to establish the theoretical implications of engaging in PDP activities (Burgess, 2004).

The extent of information generated by the ILP questionnaire (individual learning profile completed by students during induction week at the University of Portsmouth), offers a valuable data source for such evaluation. The project team have identified a number of interrelated and linked research topics with both internal and external and, applied and theoretical significance, to be investigated in four stages:

• Stage 1: evaluate the reliability, stability and validity of the ILP questionnaire.

• Stage 2: determine causal relationships between previous academic qualifications, age, gender, motivation and ILP scores.

• Stage 3: examine the development of ILP data as students develop.

• Stage 4: determine the predictive validity of the ILP questionnaire, with retention progression, academic outcome and first destinations as dependent variables.

The current topic presents the findings from Stage 1 and preliminary analysis from Stages 2 and 4.

¹Now Universities UK

اخر الاخبار

اخبار العتبة العباسية المقدسة

جولة تفقدية لمتابعة خطة تنظيم مداخل المعزّين المشاركين في عزاء ركضة طويريج

لتعزيز التعاون الأكاديمي والبحثي.. جامعة الكفيل توقع اتفاقيتي توأمة مع جامعتين عراقيتين

المجمَع العلميّ يواصل إقامة مجلسه العزائي بذكرى شهادة الإمام الحسين (عليه السلام) في النجف الأشرف

قسم حفظ النظام يواصل تنفيذ خطته لتنظيم سير المواكب والزائرين في شهر المحرم

المزيد

الآخبار الصحية

لهذا السبب.. إغلاق 25 مركزًا لعلاج الإدمان في مصر ؟

خلافا للاعتقاد الشائع.. علماء يكشفون أسرار بذور البطيخ ؟!

هذا ما يحدث لجسمك عند تناولك الطماطم بانتظام !

تعرف على أطعمة تسرّع "تعافي الجسم" من حروق الشمس

المزيد

الآخبار التكنلوجية

الثلوج تغطي الصحراء الأكثر جفافا في العالم!

بتلسكوب "ألما".. التقاط صورا غير مسبوقة لبدايات الكون !

هل هاتفك مراقب؟.. 5 علامات خطيرة تكشف التجسس عليك

دبي تطلق أول رحلة تجريبية للتاكسي الجوي

المزيد

مواضيع ذات صلة

Online Collaborative Learning and Assessment (OCLA)

Online Collaborative Learning and Assessment Rationale, aim and method of study

Mapping Self-Assessment to Achievement Conclusion

Results, Analysis and Discussion

Stage 4: Determine the predictive validity of the ILP questionnaire, with retention progression, academic outcome and 1st destinations as dependent variables Method

Results, Analysis and Discussion

Stage 2: Determining causal relationships between previous academic qualifications, age, gender, motivation and ILP scores Preliminary analysis Method

Development of Revised Tool

Internal Consistency and Split-half Reliability of Existing Factors

Results, Analysis and Discussion Evaluation of the Existing Tool

Stage 1: Evaluating the reliability, stability and validity of the ILP questionnaire Method

Introducing Computer-Aided Peer Assessment (CAPA) in Engineering Conclusions