Peer assessment using a digital tool

Peer assessments are a valuable tool to use with student groupwork to encourage students to reflect on and critically evaluate not just their own learning and skill development, but those of their team members as well in preparation for their professional careers.

3. Peer questions

Peer question types are either:

scale questions where each individual’s performance is rated by other group members against a variety of criteria on Likert- style scales
points division questions where a team member’s contribution to the group is recognised by each team member splitting, for example, 100 marks between each individual to reflect the impact they have made to each of the selected criteria
open-ended questions where group members provide a written response to questions team member’s contributions and performance.

What are you assessing?

Knowing what questions to ask is critical. What is it you wish to evaluate in the group work process? You need to establish what group working skills, attitudes and behaviours are important and you expect to be displayed to know what questions you need to ask. Make sure these criteria and the standards are observable and measurable to the group members.

While many of these criteria are common to all group work situations, there may be some questions you wish to include that will help your students learn what is most valued to your discipline and what that looks like. Race (2001), described this as What constitutes a ‘“good” attempt and what makes it a “good” attempt. These qualities will be reinforced from the time each student spends reflecting on:

•   their own performance and contribution
•   their teammates’ performance and contribution
•   the feedback they receive from other group members for their own collaborative impact in the group.

Division of points or Scale questions?

Scale questions are recommended as a default. They focus on group member achievements against standards rather than scores between students.

Division of points is a strategy used to have each team member make a judgement about the performance or contribution of each team member compared to other team members. It could be set up as just one question for a single category, or a number of different categories could be assessed, each with their own question or questions.

These types of questions are easy to implement and produce a result that is suitable for grades, however they do pose the risk of group members colluding and causing disharmony within the group. Some users add rules such as no two people in each peer assessment can have the same score to help eliminate the risk of collusion.Anonymous responses the best approach to prevent disharmony around the peer assessment.

If your goal is to also provide your students with improved feedback about their team-working skills, the Scale questions are a better choice and recommended in particular, for formative peer assessment. The Points division method of assessing student performance and contribution does not provide the individual with as much information about which behaviours are important in group work, and therefore are less informative for teaching the essential skills. Nor do they provide the student with specific feedback for how they can improve (Ohland et al., 2012).

Peer assessment questions

Peer assessment tools usually have a library from which you can select questions or you can import questions from other assessments you have created previously. These can usually be readily adapted to suit your current purpose.

Alternatively, you can develop your own questions to suit your specific circumstance.

Ideas to develop your Peer questions

Points division questions

The Proactively Ensuring Team Success (PETS) manual has identified four criteria (and indicative actions) which have been used for PETS Points division style of peer assessment, but they could also easily be adapted to suit Likert-style questions. These criteria have been developed over several years of consideration and help from students.

Teamwork and leadership	Contribution to overall project	Timeliness	Performance
attendance and participation at meetings email / SMS / discussion board response ability to work with and share findings with team	scoping research design testing analysis reporting editing final submission production up-front work (e.g. idea generation and literature research)	ability to meet agreed contribution times	completeness of work standard of work standard of work

Scale questions

Parratt et al.’s 2016 modification to Hastie et al.’s TeamUP Rubric provides criteria and indicative activities suitable to select from or adapt to develop your own peer assessment Scale questions. The rubric is known to have a sound theoretical framework underpinning it, so using its criteria (domain) and selecting some of its indicative activity could help you to develop your Scale (and also Points division) questions for peer assessment.

Domain	Activity
Planning	electing and supporting a project manager defining and agreeing on team goals and objectives defining and agreeing on quality standards for each part of the plan contributing to the development of the plan setting and agreeing realistic timeframes for each part of the plan participating in role allocations based on individual skills and learning needs willingly taking on a team role that can be completed on time to a quality standard.
Environment	exhibiting an open, gentle, polite and friendly manner demonstrating self-awareness and emotional regulation demonstrating sensitive awareness of the feelings of others (including interpreting body language) actively contributing to team discussions cooperating with others to achieve project goals following up with others when there is concern about their feelings or contribution showing respect for the contributions of others (even if in disagreement) expressing genuine gratitude and praise generously.
Facilitating contributions of others	leading and/or participating in team building processes establishing and honouring team ground rules ensuring that decisions are made in a timely manner participating in consensus-building decision-making.
Managing conflict	expressing concerns with team/team members in a constructive manne minimising unnecessary conflict by project planning and management participating in the team conflict transformation processes assisting the team to stay focused on the overall team goal approaching conflict with the aim to de-escalate being open to receiving and reflecting upon criticism of self challenging team processes not conducive to the achievement of team goals
Contributing to team project	demonstrating sufficient technological skills demonstrating relevant content knowledge adhering to academic standards for writing submitting assigned work at the agreed quality standard submitting assigned work within the agreed timeframe appropriately critiquing the work of others evaluating the quality of the whole project and making needed changes

(Table from Vaughan, B., Yoxall, J. & Grace, S. 2019)

Behaviourally Anchored Rating Scale (BARS)

Scale questions can be improved by introducing a Behaviourally Anchored Rating Scale (BARS). These questions would describe well-defined behavioural patterns expected in a collaborative activity as a measure to assesses the performance of group members.

Ohland et al.( 2012) adapted the Comprehensive Assessment of Team Member Effectiveness instrument (CATME) with Likert-style questions to include a Behaviourally Anchored Rating Scale (BARS) to create the CATME-B. BARS questions are designed to overcome the issue of a simple Likert-style question where students may have different ideas about what the expectation is, and therefore the score their teammates deserve. By adding a description of the desirable behaviours to the response options, the BARS lets students see what behaviour is needed to achieve a particular rating, and have a clearer insight of what rating the other group members deserve.

The CATME-B is based on CATME’s five broad categories for group member contribution:

contributing to team’s work
interacting with teammates
keeping the team on track
expected quality
having related knowledge, skills, and abilities.

Adding comments to justify peer scores

A feature of many peer assessment tools is the opportunity to require text input from each group member to justify how they determined the scores they allocated to other members of the group.

Benefits of justifying peer scores

Requiring your students to provide a written justification for the scores they give will:

encourage more informed and thoughtful assessment because your students are prompted to consider the criteria and standards used for their peers' work
increase accountability and fairness as group members are less likely to assign arbitrary or biased assessments, and more likely to ensure the scores they allocate are valid when they are required to articulate their reasons behind the score.

Preparation for justifying peer scores

To support your students in this process, it is good practice to work through the scoring rubric so students have a clear understanding of the criteria being used to assess their peers, and how each of the standards would be reflected in a group member’s activity. You may find it useful to have a scenario-based practice run in class so that your students become familiar with the process of justifying of scores, and you can provide the class with feedback about the quality of the responses received.

You may find including written justifications to be a better strategy than adding open-ended questions at the end of your assessment instrument.

Written feedback in peer assessment

Many self-assessment tools will facilitate written feedback shared directly from one group member to other group members as part of the peer assessment process. You should be able to select anonymity for the messages between students, but you should be able to see each message and know who wrote it.

Peer messages have their advantage in encouraging students to engage in deeper reflection for the feedback they develop for their peers; research shows that when students are required to explain the scores they have given, better peer assessment results.

Prepare your students

Your students may not be familiar with how to provide written feedback constructively, so good practice requires that you set expectations and share exemplars of constructive feedback to prevent unprofessional comments and to raise the quality of the feedback your students prepare. It may be prudent to have your students practice writing constructive feedback based on some scenarios before commencing the group work so they are familiar with what is required of them and how they should participate.

Your aim should be to foster a culture where group members know their responsibility is to convey to their peers their strengths, suggest areas for improvement and criticise constructively so each assessed group member is empowered to improve their capability. They can do this if the feedback they prepare is:

constructive – written in a positive tone, recognises strengths, identifies areas for improvement and suggests examples
specific – justified, easily translated into specific improvements
thorough – detailed and focused on contribution and performance within the group
balanced – avoids giving too much praise or criticism, highlights strengths and weaknesses
respectful – directed at the performance and contribution and not at the group member, considerate of the reviewee’s feelings.

(Adapted from Peer-assessment & self-evaluation, UOW, 2023)

Process and product Previous

Group questions Next