This resource has been drawn from discussions in the General Forum of the Australian Flexible Learning Community during the period July 2002 to December 2004. Many thanks to all who contributed.
The following comments from members of the Australian Flexible Learning Community were initiated by this question.
Can you recommend any readings or provide guidance for the assessment of student participation in online discussion? We are about to develop an online discussion group as part of an in-house training course. The curriculum documents state that participation is part of the assessment, so at this time, the decision on whether to assess is not an issue. What we would like is some ideas on how to best make it work.
Almost immediately a list of useful resources emerged.
Effective online facilitation
http://www.flexiblelearning.net.au/guides/facilitation.html
Designing effective computer based learning environments
/learnscope/golearn.asp?Category=11&DocumentId=3736
How to keep online students motivated
/learnscope/golearn.asp?Category=12&DocumentId=3340
How to help online students manage their workload
/learnscope/golearn.asp?Category=12&DocumentId=3482
Netiquette and chatiquette
/learnscope/golearn.asp?Category=12&DocumentId=4183
Some lessons I've learned about online courses
/learnscope/golearn.asp?Category=12&DocumentId=4761
Underlying all the responses from the Community was the understanding that student participation in online discussions should only be assessed if it’s a legitimate outcome for the course/subject. If not, why would their participation be assessed? This was reflected in the following posting.
I don't teach online, so I haven't faced this issue and probably don't have the right to comment here. But as a learner, I would only respond to strategies/requirements that made sense to me and could be justified as representing part of the learning or assessment. For this reason, if I were teaching the course, I would "forget" that I was using forum software and focus on building a set of learning and assessment strategies in a generic way and I would then worry about how to best implement these processes using the available tools. From there I would make sure both teachers and students understood how the tools worked and how the learning/assessment strategies would be deployed through the use of these tools.
A common theme was the importance of designing the tasks/activities so that the participants know exactly what is expected and how their participation will be measured.
One Community member referred to Gilly Salmon’s book, E-Moderating, then provided a quick listing of how learners can be assessed in online forums. It was pointed out that as you go down the list, the tasks are more difficult/higher level of skills so the weighting in terms of assessment would correlate to difficulty.
- Do they post...how often?
- Depth of ideas - explaining ideas to others
- Ability to pose new questions...identify problems, issues
- Ability to see complexities - not over simplifying, seeing only one perspective or aspect of a situation/scenario/problem
- Do they mention the ideas of others in their postings ie. show they have read others’ ideas and participated in a conversation.
- Also do they demonstrate that they can respond to other’s ideas in formulating their own. Should be a cross-referencing of ideas if a discussion forum is to be really successful and not just isolated postings one after the other…the learners need to demonstrate interaction between themselves.
- Some moderators summarise a discussion (post it up), other moderators ask individual learners to do it: rotated so everyone must summarise the ideas of everyone in a thread and post it up for all to read.
There was a strong reaction when one contributor suggested that a criteria could be to count the number of posts by each student. The following is a typical response to that idea.
I know that none of you are suggesting that counting posts alone represents anything meaningful, but thought I'd add these comments because it's easy to get dazzled/frazzled by the tools and confuse the medium with the message.
As a student I would find the idea of counting posts really annoying unless it was justified (ie "contribute three ideas..." or "provide three examples..." rather than "post three times") - to me counting posts alone would be a bit like a music composition lecturer counting the number of notes in a song to determine whether it's musical. In face-to-face teaching, do we count how many times learners speak? Does that mean that the loud ones (like me) get the highest marks? :-) Or during group work do we listen in on discussions/tasks for evidence that each learner in that group has said something?
Overall it seemed that everyone found it challenging to include online discussions as an assessable task. There were many who strongly believed in the benefits of online discussions for developing deeper level thinking but struggled to find a fair and valid way of assessing the contributions that students make.
One Community member offered the following rubrics.
|
Criteria |
Excellent |
Average |
Poor |
Total |
|
Number of responses |
More than 20 responses. Evenly distributed throughout the course |
Between 10 – 20 responses. Uneven distribution throughout the course |
Under 10 responses. Uneven distribution
|
|
|
Responsiveness to the discussion/ building of a learning community
|
Often presents reflections that become central to the group’s discussion; interacts freely and encourages others
Consistently presents creative reflections on topic; aware of needs of community; frequently prompts further discussion of topic |
Occasionally makes meaningful reflection on group's efforts; marginal effort to become involved with group |
Not actively involved in the online discussion. Needed constant encouragement from the teacher. |
|
|
Integration of subject content/readings etc |
Clear that readings were accessed and understood. Issues and knowledge gained was incorporated well into responses |
|
From the evidence in the postings it was not clear that readings were understood or used in the learner’s own knowledge construction |
|
|
Problem solving |
The postings indicated a willingness to be involved in online issues and problems. The learner was able to utilize problem solving strategies to address these issues and to offer options to the group for discussion. |
|
|
|
|
Synthesis and evaluation of own learning |
Shows excellent reflection of their own learning
|
|
|
|
|
Regularity of responses |
Constant engagement with the discussion. Responds promptly to postings; demonstrates good self-initiative |
Makes some contributions to the online discussion but not always present in an ongoing way |
Infrequent participation. Only participates after prompting by the teacher |
|
|
Expression/language |
The learner used clear and appropriate language for the context. The message was clear and unambiguous. |
The learner usually expressed themselves clearly but there were times when the language impeded the meaning of their message. |
Poor expression and grammar. Inappropriate language for the context and intended audience.
|
|
|
Online protocols
(set by the teacher or negotiated by the group) |
Obviously aware of online protocols and rules and addressed themselves appropriately |
Occasionally slipped in observing online protocols |
Serious misuse of the medium and failure to meet protocols |
|
And this
Category |
1. Drifting |
2. Moving in the Right Direction |
3. Valuable Performance |
4. Our Goal
(with due allowance for finitude) |
POINTS |
|
Contribution to the Learning Community |
Does not make effort to participate in learning community as it develops; seems indifferent |
Occasionally makes meaningful reflection on group's efforts; marginal effort to become involved with group |
Often presents reflections that become central to the group’s discussion; interacts freely and encourages others |
Consistently presents creative reflections on topic; aware of needs of community; frequently prompts further discussion of topic |
|
|
Relevance of Post |
Posts topics which do not relate to the discussion content; makes irrelevant remarks |
Occasionally posts off topic; most posts offer no further insight into the topic |
Posts are related to discussion topic; makes some connections with readings |
Posts consistently are related to discussion topic; brings readings into discussion; cites additional references related to topic; |
|
|
Expression Within the Post |
Does not express opinions or ideas clearly; no connection to topic |
Unclear connection to topic evidenced in minimal expression of opinions or ideas |
Opinions and ideas are stately clearly with occasional lack of connection to topic |
Expresses opinions and ideas in a clear and concise manner with obvious connection to topic |
|
|
Delivery of Post |
Utilizes poor spelling and grammar in most posts; posts appear "hasty" |
Errors in spelling and grammar evidenced in several posts |
Few grammatical or spelling errors are noted in posts |
Consistently uses grammatically correct posts with rare misspellings |
|
|
Promptness and Initiative |
Does not respond to most postings; rarely participates freely |
Responds to most postings several days after initial discussion; limited initiative |
Responds promptly to most postings; requires occasional prompting to post |
Responds promptly to postings; demonstrates good self-initiative |
|
