Over 2000 and 2001 the Flexible Learning Advisory Group (FLAG) funded a program of 9 research projects related to the implementation and use of online learning and delivery. This research has been managed by the National Centre for Vocational Education Research , the principal research and evaluation organisation for the VET sector in Australia.

Summaries of this research are being published in the LearnScope Virtual Learning Community once a week between 4 September 2002 and 6 November beginning with the article What the FLAG research says about online learning  published on September 4.

This week we look at Summary 7 -Assessing Online. This research investigates issues in quality online assessment in VET both in Australia and internationally.

So far, the research has found extensive use of online assessment for formative assessment purposes particularly multiple choice and short answer tests.  Summative assessment is still in its infancy largely due to concerns about plagiarism, security, access and stability of technologies. The main challenges for effective online assessment, according to literature review are:

• Institutional policy issues affecting online assessment
  
• Maintaining security/integrity of online assessment events
  
• Costs associated with developing online assessment systems
  
• Providing timely and meaningful feedback to students
  
• Identifying the right mix of technical skills/knowledge and assessment design/educational pedagogy skills in developing online assessment
  
• Measuring quality of student contributions in the online environment

Main benefits of online assessment for learners, designers and deliverers are cited as flexibility of time and place of assessment and the immediacy of feedback.  The potential to improve the quality of documentation associated with the assessment process is also beneficial for those designing and delivering online assessment.

Project title and authors: Creative, quality online assessment (R Booth, B Clayton, P Hyde, R Hartcher & S Hungar)

Project outline:Robin Booth and her colleagues are investigating issues in quality online assessment in VET by drawing on the experience of Australian and international online designers and deliverers from across the range of education sectors.

The study is using a qualitative iterative approach over two stages involving, in the first stage, telephone and email interviews with forty practitioners, and a second stage in which 20 identified expert users and leaders in the field comment on issues and contribute to ‘solutions’ developed for the VET sector. The project has a website for discussion and data collection athttp://www.veac.org.au

There is a high degree of support for the potential of online technologies to deliver assessment online and support learning, if well designed. The use of online technologies for summative assessment purposes is still in its early stages and there is a degree of caution about its appropriateness for a range of reasons/factors including plagiarism, security, access and stability of technologies. Respondents generally considered that the assessment principles of validity, reliability, flexibility and fairness had to be applied with equal rigour in the online context to assure quality assessment practice, regardless of the assessment delivery context. Some of the main challenges for implementing online assessment were identified as the initial investments of time and financial resources to develop and then maintain online assessment and subsequent workloads associated with feedback processes, staff skills and bandwidth capacity for more innovative graphical/audio/video interfaces for assessment purposes.

There is extensive use of online assessment for formative assessment purposes. The potential of activities such as online forums and simulation activities are being explored by a small sector of the respondents, and the interest in these areas is likely to increase. The views of Australian and international online designers and deliverers surveyed is summarised as follows:

• The major benefits to learners is cited as flexibility of the time and place of assessment can be undertaken and the immediacy of feedback to the learner.

• The major benefits cited for designers/deliverers are the potential to improve the quality, consistency and documentation of the assessment process, the flexibility of time and place to conduct the assessment, the opportunity to provide regular and immediate feedback, and that once established it is time saving.

Factors influencing the choice of an online assessment method mainly focused on flexibility and the capacity to provide feedback to students. The potential for group work and to be interactive were raised by some respondents as further factors.

• Respondents are currently using the online experience mostly for emailing essays and portfolios as well as multiple choice and short answer tests. The use of multiple choice questions for formative assessments was considered the type of assessment best done online.

• However in regard to both the current usage and the types of assessment considered best done online there is little development of the concept of interactive approaches such as discussion/bulletin boards or chat.

• Self and group assessment also has not been considered widely for development for online assessment despite the interest in interactivity.

All assessment methods and tools can work effectively in the online environment, and there are many instructive and varied examples of assessment methods being used for online learning. The assessment method(s) and tool(s) should be determined through careful consideration of the assessment purpose; audience/learner characteristics; and the level and type of learning content, process and outcomes.

Challenges still remaining for effective online assessment from the review of the literature:

• Institutional policy issues affecting online assessment

• Maintaining security/integrity of online assessment events

• Costs associated with the development of online assessment systems

• Time management in terms of providing timely and meaningful feedback to students

• Obtaining the right mix of technical skills/knowledge and assessment design/educational pedagogy skills/knowledge in developing online assessment

• Measuring quality of student contributions in the online environment

There is a contradiction that the online environment is seen by some as having the potential to improve learner centredness, and by others as restricting it. In order to enhance the online learning environment it is important to focus on the factors that can enhance learner centredness. These include how interactivity between assessors and learners can be better developed, how assessors can understand more about their learners and how assessment can be meaningful to the learner. To achieve this, further advice and examples are currently being sought from experienced practitioners in the following aspects of assessment:

• Providing online support for online assessors

• Measuring the quality of online interactions/contributions between students and between students and teachers

• Using the online environment to engage learners in the assessment process

• Self-assessment

• Peer assessment

• Group/collaborative assessment

• Authenticity of student performance

• Time and resource management for online assessors

• Assessing learner readiness for online learning

• Principles of quality online assessment

• Creating authentic online assessment

Further information and, as they are published, a full report on this national strategy project is available at:http://www.ncver.edu.au/online.htm

This topic page also allows you to:

• Access other work published by NCVER in relation to on-line learning
  
• Conduct a search on published material in the field using VOCED, the UNESCO/NCVER international database for technical and vocational education and training research
  
• Link to other sites related to on-line learning

NCVER will be running a series of face-to-face forums on this research in all states in November and December 2002. Watch their site for details at http://www.ncver.edu.au