Moderation Strategy

In the forum discussion, some colleagues referred to ALTC's moderation strategy and assessment moderation toolkit.

 http://resource.unisa.edu.au/course/view.php?id=285

The toolkit is based on key principles identified by research under an ALTC project on moderation for fair assessment in transnational learning and teaching.

I don't need to consider moderation so much now as my unit is small and I am only the maker. But I think this is a useful toolkit that I will utilise in future when I need.

Feedback approach

I enjoyed reading ‘The mythology of feedback’ (Adcroft 2011). This paper questions: do academics and students share the same feedback mythology? If there is only a limited sharing of feedback mythology, How does the dissonance this creates manifest itself?
For me, the most interest argument in this paper is, ‘effort put into feedback that is not focused on assessment, despite the investment of time and resources, is simply not seen as feedback by students’. ‘Students cannot learn from feedback if they do not recognise that they are receiving feedback or if they are only interested in the marks they receive on assessed work’.

I agree with 'accurate measurement of feedback effectiveness is difficult and perhaps impossible'(Price et al. 2010).

My feedback approach is:
- Make effort to assist student learning, assist them to achieve their learning goals.
- To do this, understanding the expectation of student is important (I found one minute paper is useful).
-  Check if the intended learning outcome and assessment criteria (and rubric) are really addressing what it claimed to be for the unit

Rubric example - urban development project

In Urban and Regional Planning Course (both UG and PG), common assignment is 'essay on urban issues and development'. Students identify the problem/urban issues based on evidence (statistics, report from national/state/local government) and develop ideas on how urban planning could cope with the issues such as society(demographic change), environment, and economy. The example below is used in Ohio Department of Education 'Simulated Urban Planning Course Urban Planning Project Assignment'. I think this assignment is especially relevant to my unit 'Infrastructure Planning and Delivery'. In 'Infrastructure Planning Delivery', students analyse the UC campus and develop the ideas on Campus development as a planner. I refer to this example from Ohio Department of Education. Sometimes students are not sure what is expected for the assignment.  I think this example describe clearly on what should be included and considered in the assignment. Especially including 'culture' is a good point as this is very important aspect of city which is related to social sustainability but developers often focus on economic aspect only.The scoring guideline (rubric) seems broad but still a good guide for students to understand what is assessed in the assignment.

(Source: Ohio Depertment of Education)

Rubric example - Planning Report

In Master of Urban and Regional Planning, University of Canberra, the last unit that students take to finish the course is 'Planning Report' which is a mini research project. We don't have class for this unit. Students discuss with their supervisors and plan, develop, and finish the mini research project. Usually course convener (myself) supervise each student on this unit.
This unit is 'corpus' of studies that students undertake throughout the course. To develop planning report, students refer to planinng theory, analytical method, some tools such as statistics software, geotraphical information system. The evaluation of the report is thus comprehensive. I refer to the assessment criteria and rubric that is used in San Jose State University, Department of Urban and Regional Planning (below). The assessement criteria below is systematic and transparent. I think, for comprehensive report, this criteria works very well to evaluate how the assignment is addressing the criteria (learning outcome). This can be used as a check list on how assignment should be structured and what shold be included. Students can understand the strong and weak points of their assignment by using this sheet and utilise for their future learning.

(Source: San José State University, The Faculty of the Department of Urban and Regional Planning, URBP 298B: Special Study: Planning Report Completion Spring 2010)

Rubric example - written assignment grading rubric

I've found some example of rubric used in planning schools.
The below is the example of written assignment grading rubric, from Cornel University College of Art, Architecture, and Planning, Department of City and Regional Planning.

In my units, students write essay and report on planning theory and practice. I think this rubric is useful for assessing students writing from five angles, although this is still much dependent on assessor's subjective evaluation on each point. This can be also used for cheking the research paper manuscript. Especially 'argument' is important. There is still need for checking the consistency of marking by cross-checking the marked assignments.

(Source: Cornel University College of Art, Architecture, and Planning, Department of City and Regional Planning Promise and Pitfalls of Contemporary Urban Planning Syllavbus for Fall Semester 2012)

Studio-Specific Learning Outcomes Assessment Model (Nemeth and Grant Long)

Nemeth and Grant Long recently publised the article 'Assessing learning outcomes in U.S. Planning Studio Courses'. (Jeremy Németh and Judith Grant Long, Assessing Learning Outcomes in U.S. Planning Studio Courses Journal of Planning Education and Research December 2012 32: 476-490)

This paper proposes a model for assessing learning outcomes specific to planning studio courses. I don't have 'studio' style teaching but The units I teach includes 'project-based' assignment (e.g. develop infrastructure desing for University Campus). Therefore the proposed model by Nemeth and Grant Long is very useful for my units.

Examples of individual outcomes

Nemeth and Grant Long recently publised the article 'Assessing learning outcomes in U.S. Planning Studio Courses'. (Jeremy Németh and Judith Grant Long, Assessing Learning Outcomes in U.S. Planning Studio Courses Journal of Planning Education and Research December 2012 32: 476-490)

Nemeth and Grant group the dozens of individual outcomes into six categories derived from the planning literature on the subject: communication, professional experience, learning-by-doing, problem-solving, teamwork, and ethics/values as below. I think this is very usuful in desinging the learnig outcomes and associated assessment. These outcomes are applicable to any planning units if you customise to fit into the unit context and objectives.

Professional experience
Examples of individual outcome
- Provide “real world” work environment

- Gain project management skills (budgeting, workflow)

- Understanding quality standards expected in practice

- Learn the planning and plan-making process

- Understand various roles of planner

Communication
Examples of individual outcome
- Graphical/visual skills

- Written skills

- Oral presentation

- Understanding relationship between plans and physical reality
Learning by doing
Examples of individual outcome
- Understanding of how theory informs practice

- Understanding of how practice informs theory

- Application of general planning concepts to specific context

- Learning how to synthesize skills, knowledge, values

- Acknowledge uncertainty/complexity in planning practice

- Recognition of planning as iterative, long-term process

Ethics/values
Examples of individual outcome

- Recognition of broader “public interest”

- Assess planning outcomes on set of values (e.g., justice, sustainability)

- Sublimation of personal opinion

- Creation of ethical foundation for future practice

- Recognize accountability/responsibility to served group

- Acknowledge and challenge systemic power imbalances

Problem solving
Examples of individual outcome

- Ability to formulate logical, defensible planning decisions

- Learn how to evaluate several possible scenarios

- Negotiate oppositional viewpoints

- Recognize importance of flexibility in decision-making process

- Seek appropriate assistance and expertise

- Being creative designing solutions and processes

- Develop critical thinking ability
Teamwork
Examples of individual outcome

- Role recognition in collaborative work

- Understanding basic group dynamics

- Development of leadership qualities

- Gain vital listening abilities

- Development of interpersonal cooperation skills

Good learning outcome

In the forum of 'Assessment and Evaluation in Tertiary Teaching' unit, we discussed that assessment is learning fundamental and thus assessment should be well connected with learning outcomes.

'Using eVALUate to improve student learning' (Curtin University of Technology, Oliver et al.) is showing some examples of how good unit learning outcomes might be expressed. I think this is very useuful.

Successful students in this unit can:

•

Construct and justify logical arguments about [a significant issue in the field].

•

Critically analyse [a significant issue in the field].

•

Conduct a critical review of [a significant issue in the field].

•

Produce a [significant document] at a level acceptable to [stakeholders in the field].

•

Collaborate with [stakeholders or peers] in the successful production of a [significant

document]

•

Plan and carry out a [significant event] for [stakeholders in the field].

•

Use [discipline resources] to effectively [manage a discipline-related problem].

•

Evaluate [a scenario in the field] and produce [a significant report]• Work effectively as a team member to solve [a significant problem in the field].

- Use

[a discipline-related] theory to develop a solution to [a significant problem] within thecontext of discipline.

Assessment Criteria Grid

Margaret Price and Chris Rust created a very useful criteria grid for assessment. I selected some examples that are relevant to urban and regional planning field.

I think the original grid by Price and Rust is also useful for checking the quality of research paper.

	CRITERION	A	B+	B	C	REFER/FAIL
	Presentation & style
	Communication and presentation (appropriate to discipline)	Can engage effectively in debate in a professional manner and produce detailed and coherent project reports	Can communicate effectively in a format appropriate to the discipline and report practical procedures in a clear and concise manner with all relevant information in a variety of formats	Can communicate effectively in a format appropriate to the discipline and report procedures in a clear and concise manner with all relevant information	Some communication is effective and in a format appropriate to the discipline.  Can report practical procedures in a structured way	Communication is unstructured and unfocused and/or in a format inappropriate to the discipline
	Presentation (oral)	Imaginative presentation of material resulting in clarity of message and information	Well structured and signposted presentation.  Audible and pace appropriate to audience.  Visual aids used to support the presentation	Clearly structured and addressed to audience.  Pace and audibility satisfactory.  Visual aids used	Shows some attempt to structure material for presentation, pace and audibility are satisfactory most of the time	Material is difficult to understand due to poor structure and/or pace and audibility
	Conforming to instructions/clarity of objectives.
	Attention to purpose	Has addressed the purpose of the assignment comprehensively and imaginatively	Has addressed the purpose of the assignment coherently and with some attempt to demonstrate imagination	Has addressed the main purpose of the assignment	Some of the work is focused on the aims and themes of the assignment	Fails to address the task set
	Referencing	Referencing is consistently accurate	Referencing is mainly accurate		Some attempt at referencing	Referencing is absent/ unsystematic
	Clarity of objectives and focus of work	Has defined objectives in detail and addressed them comprehensively and imaginatively.	Has defined objectives and addressed them through the work	Has outlined objectives and addressed them at the end of the work	Has provided generalised objectives and focused the work on the topic area	NO INFO PROVIDED
	Content and knowledge
	Use of literature/ evidence of reading	Has developed and justified using own ideas based on a wide range of sources which have been thoroughly analysed, applied and discussed	Able to critically appraise the literature and theory gained from  variety of sources, developing own ideas in the process	Clear evidence and application of readings relevant to the subject; uses indicative texts identified	Literature is presented uncritically, in a purely descriptive way and indicates limitations of understanding	Either no evidence of literature being consulted or irrelevant to the assignment set
	Context in which subject is used	Takes account of complex context and selects appropriate technique	Takes some account of context and selects some appropriate techniques	Recognises defined context and uses standard techniques for that context	Context acknowledged but not really taken into account	Context not recognised as relevant
	Thinking/ analysis/conclusions
	Conclusions	Analytical and clear conclusions well grounded in theory and literature showing development of new concepts	Good development shown in summary of arguments based in theory/ literature	Evidence of findings and conclusions grounded in theory/literature	Limited evidence of findings and conclusions supported by theory/literature	Unsubstantiated/invalid conclusions based on anecdote and generalisation only, or no conclusions at all
	Analysis	Can analyse new and/or abstract data and situations without guidance using a wide range of techniques appropriate to the topic	Can analyse a range of information with minimum guidance, can apply major theories and compare alternative methods/techniques for obtaining data	Can analyse with guidance using given classification / principles	Can analyse a limited range of information with guidance using classification / principles	Fails to analyse information
	Critical reasoning	Consistently demonstrates application of critical analysis well integrated in the text	Clear application of theory through critical analysis/critical thought of the topic area	Demonstrates application of theory through critical analysis of the topic area	Some evidence of critical thought/critical analysis and rationale for work	Lacks critical thought /analysis / reference to theory
	Flexibility	Independently takes and understands multiple perspectives and through these can develop/adjust personal point of view	Recognises multiple perspectives which may affect personal view point	Can recognise alternative perspectives	Limited ability to see alternative perspectives	Fails to recognise alternative perspectives
	Practical/Interpersonal/Interpersonal Skills
	Performance Skills	Can perform complex skills consistently with confidence.  Able to choose an appropriate response from a repertoire of actions, and can evaluate own and others’ performance.	When given a complex task can choose and perform an appropriate set of actions to complete it adequately.  Can evaluate own performance.	Able to perform basic skills with awareness of the necessary techniques and their potential uses and hazards.  Needs external evaluation.	Able to perform basic skills with guidance on the necessary technique.  Needs external evaluation.	Fails to perform even basic skills.
	Data/information gathering/processing	Selects and processes data with confidence and imagination.	Selects appropriate data and processes using relevant tools.	Makes a selection from data and applies processing tools.	Collects some information and makes some use of processing tools.	Random information gathering.  Inappropriate use of processing tools.
	Self-criticism (include. reflection on practice)	Is confident in application of own criteria of judgement and in challenge of received opinion in action and can reflect on action.	Is able to evaluate own strengths and weaknesses; can challenge received opinion and begins to develop own criteria and judgement.	Is largely dependent on criteria set by others but begins to recognise own strengths and weaknesses.	Dependent on criteria set by others. Begins to recognise own strengths and weakness.	Fails to meaningfully undertake the process of self criticism.
	Self Presentation	Adopts a style of self presentation and selects from a range appropriate interpersonal skills consistent with the individual’s aims and the needs of the situation.	Can be flexible in the style of presentation adopted and interpersonal skills used.	Can adopt both a formal and informal style, and uses basic interpersonal skills appropriately.	Can adopt both a formal and informal style, and uses basic interpersonal skills but not always matching the needs of the situation.	No obvious sense of self and/ or interpersonal skills and/or skills used inappropriately.
	Communication and Presentation (appropriate to discipline)	Can engage effectively in debate in a professional manner and produce detailed and coherent project reports.	Can communicate effectively a format appropriate to the discipline and report practical procedures in a clear and concise manner with all relevant information in a variety of formats.	Can communicate effectively in a format appropriate to the discipline and report practical procedures in a clear and concise manner with all relevant information.	Some communication is effective and in a format appropriate to the discipline.  can report practical procedures in a structured way.	Communication is unstructured and unfocussed and/or in a format inappropriate to the discipline.
	Rationale	Uses all available data to evaluate the options.  Clear criteria are applied to demonstrate reasons for final decision/choice/outcome.	Uses data to evaluate options and selections of final outcome clearly follows from evaluation.	Uses data to evaluate some options and selection of final outcome is linked to the evaluation	Presents benefits and disadvantages of some potential outcomes but without providing clarity on reason for final outcome/choice.	Little explanation of how the final outcome/choice was made OR no indication of final outcome/choice.