Hitomi
'Hi all,
I read this article with interest (suggested by Shane).
http://www.cshe.unimelb.edu.au/assessinglearning/06/normvcrit6.html
'There is a strong culture of norm-referencing in higher education.'
'The goal of criterion-referencing is to report student achievement against objective reference points that are independent of the cohort being assessed.'
'Norm-referencing, on its own — and if strictly and narrowly implemented — is undoubtedly unfair. With norm-referencing, a student’s grade depends – to some extent at least – not only on his or her level of achievement, but also on the achievement of other students. For example, a student who fails in one year may well have passed in other years! '
'Criterion-referencing requires giving thought to expected learning outcomes: it is transparent for students, and the grades derived should be defensible in reasonably objective terms – students should be able to trace their grades to the specifics of their performance on set tasks.'
Im my case, I apply 'criterion-referencing' methods as a) my student cohort is very small (around 10) and 2) my discipline (urban and regional planning) requires cross-cutting knowledge and skills. Some students might be very good with particular skills but may not with others. With criteria-referencing, students can identify which particular areas they are weak and need improvement. This may guide their future learning.
It is sometimes difficult for me to well explain the results at the faculty assessment board meeting. At the meeting, we discuss based on 'norm referencing distribution'. I understand that there are some cases that 'norm-referencing' works well but I don't feel comfortable in applying norm-referencing with the reasons above.
Which is appropriate in your case?
Bestt wishes,
Hitomi'
Dalma
'Hi Hitomi!
I am familiar with both systems, as at my former university there was a bell curve used for norm referencing, while in my current discipline we all work with criterion-referencing. We include (more-or-less) clearly defined assessment criteria in the unit outlines, and define them in further detail for marking. (The criteria listed in the unit outline do not contain the expected answers to an assessment question, of course.) The critera are aligned with the learning outcomes, inlcuding both knowledge and skills, but weight differently - e.g. getting an answer 'right' is obviously more important, even if presented in an unedited form, than having a perfectly edited submission with an absolutely wrong content.
In spite of the expectation to use criterion-referencing, the reporting to the Assessment board has to contain the percentage of each grade, and I've heard about colleagues having difficulties justifying grades that were not spread across the entire range. I never had this problem, as all my units where results were close to each other had small cohorts, either consisting in a small elite, or making very effective teaching possible, both these being accepted as justifictaion for grades generally above the discipline average.
Finally, I also perform a cross-reference as a final review of the marking I've completed, not to compare the results, but to make sure that the criterion-based evaluations are consistent across the entire cohort, and not influenced by external factors, like me getting tired in marking or frustrated by bad submissions.
Dalma'
Carlos
'Hi Hitomi, Dalma, very interesting discussion. Personally I have only become aware of the characteristics of norm referencing and criterion referencing methods through this module and the readings.
As Dalma, I come from having taught in a very strong Norm_refencing system. The use of Bell curves was very strict in its percentages, accross all disciplines in a big,state funded University. Furthermore, within an Asian context, study was highly valued culturally, and the university constantly promoted its ranking and the high quality of their "selected students", which were considered an "elite". The whole society in my previous teaching context values the idea of "meritocracy".
Although I quite like criterion-referencing methods, I think there is no black and white. For the sake of discussion and critical thinking, at this stage I would like to highlight a few aspects of norm-referencing. Although in this context it appears as an "unfair way of assessing, mainly by comparing and ranking of the students" I would like to highlight that competition and comparison are a reality in our world. People compete for jobs, in sports and for pure entertainment. Industries offer their products in the market, competing with similiar products, and we as buyers and consumers make daily choices based on comparison of products. Competition motivates improvement. Without going into details of the exact definitions of the words or semantics, the origins of the words "competency" (based assesssment) and being "competent" seem to me to be related to "competition". Furthermore, we live in a society with norms, codes of conduct, and professional protocols. Completely avoiding norms, or avoiding comparison and competition in assessment, could be, form this point of view, disconnected to reality.
Possibly a balance between criterion and norm referencing might give good results?'
Shane
'Hi Everyone,
This is a very lively and interesting discussion.
I would like to build on the thoughts around competition, in particualr how criterion referenced assessment approaches don't exclude the possibility of incorporating competition or competitive behaviour.
My contribution stems from a seminar I attended at UTS, whereby a software system had been created that allow academic staff to load their assignment criterion, and then grant access to students. Students would use the system to self assess their own assignments (a useful learning experience in itself when compared against the teachers judgements) and then the teacher would assess the work. The competitive element is contained in point 3 below:
The demonstrated impact of what the system allowed students and staff to do was multifaceted:
students could self-assess using the system and then compare against the teachers assessment - this resulted in students (over the course multiple assessments) developing a better sense of the quality of their own performances.
- teachers could save time by focusing detailed feedback on areas of disparity betweenthe self judgement of the student and the judgement of the teacher.
- once marking and moderation had been completed the students could see their criterion scores on a scale which contained the average of the cohort.
Hitomi
' Hi Dalma and Carlos
Thank you for sharing your experiences. It is interesting to know that some discipline has 'strong norm-referencing system'.
I agree, in Asian culture the idea of 'meritocracy' is really strong. I was one of the students in this system and as a student, didn't like it
I agree that balancing criterion and norm referencing would be good way but we may need to consider the effective 'balance' and rational for it. Wondering if there is any reserach on this.
Best wishes,
Hitomi'
Gemma
'At CQUniversity, there is a strong trend towards professional degrees, such as Nursing, Accounting, Engineering, Education etc where there is a higher need for more of these in the workforce. I think this is why we have quite a strong focus on criteria referenced assessment, because so long as someone is capable of doing the job, meet all the criteria, then they should pass and be able to go and do that job. It is not a particularly competetive marketplace. That may, of course, change at any time.'
0 件のコメント:
コメントを投稿