Cultural Competencies

This document summarises the discussions that took place during the LinkUp Key Competencies event held in the Australian Flexible Learning Community from 17 - 28 May 2004. Please note that surnames have been removed and the discussion should be read from bottom to top (ie reverse chronological)

Forum Summary: Get Reflective … LINKup Key Competencies

Cultural Competencies, posted by: Daniel R at Wed, 10 Nov 2004, 07:16
Thank you all for your comments re KCs and Packages, posted by: Mike S at Fri, 28 May 2004, 15:15
An area for ANTA to revisit, I suggest. My best interpretation of KCs in packages is to read them and be prepared to modify the levels in accordance with the workplace, and then document the argument for the change for audit purposes. I find them inaccurate in many instances, although believe them to have benefit in identifying qualification level for units. Thank you Lilian - what packages have included the 8th KC? Particularly thanks for extensive comments from Glen Rob and David. I would be interested in participating in further debate regarding KCs and Packages should the opportunity arise.
Thank You ALL !, posted by: Rob Denton at Fri, 28 May 2004, 13:31
Today is the official conclusion of this discussion forum – which has been a huge success thanks to YOU, the participants from diverse backgrounds and perspectives.

Total ‘Word Count’ for this forum at time of this message was over 15,400 words !

Personally, I gained a great deal from the rich and diverse dialog that has taken place.
THANK YOU to all who have participated! I trust that YOU too have benefited.

I would like to publicly express my sincere appreciation to Glen Rogers for his willingness and dedication to participate in this discussion forum for the benefit of all involved. His experience, expertise and reputation certainly injected some high calibre contributions to this discussion topic.

Notably, Glen’s contribution has been on top of his busy workload as a Senior Research Associate of Alverno College with much of his participation happening around midnight (local Milwaukee time) !!

I trust the success of this forum is a sign of a great future for this important topic and a supportive and collaborative approach to its practical implementation.
Thank you!

Rob
Thanks Rob, posted by: David C at Fri, 28 May 2004, 13:05
Rob,
Setting up this forum was a great idea.
I have enjoyed the exchanges with other researchers, teachers and students. I hear that Glen has been pounding the keyboard into the wee hours over there. That's commitment Glen.
I took the opportunity to look again at the LinkUp web site. Also a great initiative.
As usual, you've done a great job.
So, THANKS Rob!!
David
Key Competencies as they apply to Training Packages, posted by: Mike S at Thu, 27 May 2004, 16:14
Would the panel comment on the following please:

1. Why is Cultural Understanding not a Key Competency?

2. How and why are Key Competencies integrated into Training Packages?

From: mike
- The VLESC generic skills , posted by: Lilian A at Fri, 28 May 2004, 13:13
  Hi Mike
  In Victoria we are using a list of generic skills which has been endorsed by VLESC ( Victorian Learning and Employment Skills Commission). Cultural understanding is included in this framework.
  
  Yet another variation on the theme!
- Re: Key Competencies as they apply to Training Packages, posted by: Rob Denton at Fri, 28 May 2004, 12:51
  Hello Mike
  1. 'Cultural Understanding' was considered many years ago as a potential 'eighth' Key Competency. Ultimately, it was concluded that while it was deemed important it was not suited to be a 'Key Competency'. Here is one resource that you could access for further info...
  http://www.curriculumsupport.nsw.edu.au/hsie/ftp/Folder3/pcc/gore.pdf
  An extract from this document states "This key competency (Cultural Understanding) is the capacity to apply understanding of the diversity and commonality within and between groups, organisations and societies toward the achievement of common goals. It involves the abilities to:
  • demonstrate knowledge of cultural cohesion and diversity
  • respect the rights and responsibilities of others
  • work with others to achieve common goals which reflect equity and social justice.
  
  The key competencies project in Australia found that, although most participants supported the concept of making cultural understanding more explicit in curriculum documents and in education and training practices, there were difficulties in using it as a key competency.
  
  The Australian key competency project found that there were difficulties with identifying the application and assessment of the key competency (using) cultural understandings. Discussions centred on whether cultural understanding might be best addressed as an integral component of all key competencies. The project team decided not to proceed with any further work on cultural understanding as a key competency. However, there was agreement that there should be strong consideration given to the fundamental question regarding the place of cultural understanding in the curriculum."
  
  2. Key Competencies are integrated into Training Packages in a variety of ways depending on the specific Training Package. This is currently being reviewed as part of the current high level Training Package review. It is very likely that Key Competencies (or perhaps 'Employability Skills') will become far more explicit in Training Packages and the training process itself in the near future. In general the Key Competencies are associated with Units of Competence and are specified according to Performance Levels required.
  This defines what Key Competencies Performance Levels must be achieved by students in each Unit of Competence. However, I believe there is currently no directive to explicitly assess or report on the achievement of these Key Comps - thereby making it impossible to verify whether these skills have been successfully performed or not!
  I firmly believe there is great benefit in the explicit assessment and development of these skills which are so highly respected and demanded by employers - and that is why we have implemented the LINKup Key Comps assessment strategy.
  For more info I would suggest looking up 'Training Packages' on the ANTA website...
  http://www.anta.gov.au/
  
  Rob
- Re: Key Competencies as they apply to Training Packages, posted by: David C at Fri, 28 May 2004, 12:45
  Mike,
  1. Cultural Understanding: A very good question.
  This issue has a long history. It was not one of the possible KCs given to the Mayer Committee, but it did arise in their extensive consultation process. The Committee could not come to a final decision when their report was submitted to the Education Council. There was a further period of deliberation, and eventually it was excluded.
  The reasons for this lie in the criteria that the Committee established early in its deliberations. In setting these criteria, I think the C'tee acted responsibly: rather than make decisions on the run, and note that they were receiving advice from many interested groups, they decided to establish a set of criteria for judging whether a proposed ability was a key competency or not.
  Their criteria were rather rigid, but probably with good reason. They included a requirements that any proposed skill had to be capable of being taught, learned and assessed. Recall also that the C'tee was working at a time when Competency Based Training was being introduced into the VET sector and being pushed in the others, so if something had to be assessed, that assessment was rather restrictive. On balance, the judgment was made that Cultural Understanding was not clearly enough defined (they had taken quite disparate advice on what it was) and could not be operationalised with sufficient clarity and objectivity that its assessment would not be as meaningful as the constructs that they did include.
  Many at the time, and many more since, think they made the wrong call. If a broader view of assessment is taken and some serious work done on fully describing these constructs, they are amenable to credible assessment.
  
  2. KCs in TPs.
  Are they "integrated"?
  It varies. In some TPs they are; in most they are not. Cathy Down has addressed this issue. (See Down, C. (2000, November). Key competencies in training packages [Electronic document]. AVETRA. Retrieved 1 May, 2001, from the World Wide Web: http://www.avetra.org.au/papers%202000/down1.PDF)
  How? It also varies. In most cases KCs are listed in an appendix and under each unit of competence, there is a table that lists the seven KCs and the three levels and shows what level should be reached on each KC in that unit. This is a minimalist approach to 'integration' and it achieves the expected outcome.
  Why? In theory, because KCs are valued attributes and required if Australia's industries are to improve their international competitiveness. Note that the rationale in the 'Why' is not matched by the practice of the 'How'.
  
  David
- Re: Key Competencies as they apply to Training Packages, posted by: Glen Rogers at Fri, 28 May 2004, 04:53
  Let me comment on Cultural Understanding as a potential key competence by describing a similar competence in the Alverno ability-based curriculum. It is entitled Developing a Global Perspective.
  
  Level 1. The student assess and articulates what she (this is a college for women) already knows about the world around her and identifies useful strategies for developing her awareness of others.
  
  Level 2. The student uses concepts from a variety of courses to describe some of the world's diversity and interconnections.
  
  Level 3. The student applies concepts and frameworks from disciplines in order to understand the influences and implications of diversity and/or global interconnectedness within a particular context.
  
  Level 4. The student draws on her learning, the perspectives of others, and personal reflection to articulate her own understanding of global interconnectedness and international relationships and critically examines the implications for herself and others.
  
  ----
  Note that the notion that the student is "developing a global perspective" rather than "taking global perspectives" (plural), which was the central idea behind a previous version of the ability in the curriculum. We found that entering students generally needed to develop their knowledge about the world and their skills in engaging the interconnected complexity to such a degree that our emphasis was put on their developing their own global perspective, which was an active accomplishment.
LINKup Tour – ‘REFLECT’, posted by: Rob Denton at Wed, 26 May 2004, 16:37
Having completed the TRY section you are now fully ‘Registered’ and have access to the full package of LINKup resources in the Downloads section. You can return to the site at any time and access Downloads from the homepage by entering your email address as your ID.

The fourth and final stage of the Tour is ‘REFLECT’.
This simply offers some prompts and tips on how LINKup may be used to benefit different people in different contexts.

REFLECT ON STUDENT PERSPECTIVES…
Finally, this forum offers a great opportunity to reflect on the perspectives of some real live students using LINKup Key Competencies assessment. These can be found as replies to my posting called “Student perspectives on LINKup Key Competencies Assessment”
Take the opportunity to dialog with these students about their experiences.

Thank you!
Rob
LINKup Tour - 'TRY', posted by: Rob Denton at Mon, 24 May 2004, 11:23
The highlight of this section is the online simulation of your very first Validated Self Assessment - designed to let you quickly and conveniently ‘experience’ the process first-hand and receive lots of informative feedback on your assessment immediately.

The LINKup Tour so far…
Go to the LINKup website (www.tvtafe.com.au/linkup) and click on ‘Tour’ travelling through Stage 1: 'DISCOVER'
Stage 2: ‘INVESTIGATE’ and then to
Stage 3: ‘TRY’

‘TRY’ involves 3 steps …

(1) Basic registration details (to create your ID in order to gain access to all LINKup resources at the completion of the TRY section)
(2) Self Check Quiz (short multiple-choice quiz to check your basic understanding of the nine ‘Objectives’ in the INVESTIGATE section. HINT: Make sure you have seen the powerpoint presentation in INVESTIGATE section called 'LINKup - how to do it!')
(3) Validated Self Assessment (online simulation)
You will reflect on and self assess your performance through this LINKup Tour in the Key Competency 'Collecting, Analysing & Organising Information'.

Completing these will automatically register you for FREE access to the complete package for LINKup assessments.

Please return to the forum with your comments and questions.
Thank you!
Rob
Are generic skills up to the task?, posted by: Mark L at Thu, 20 May 2004, 12:57
I would be interested to get the views of others regarding as to whether the generic skills, as identified in Australia; the Mayer competencies, have the capacity to provide the kind of 'mental schema' that is needed by all in an economy and society that is becoming increasingly knowledge intensive. The Mayer mix of skills, dispositions and attitudes is bit of a mish-mash for me- valuable up to a point- but rather at the lower end of things. If VET and other sectors are to really concentrate on generic skills, I would prefer to see a focus in our teaching and learning on the development in our students of the 'fair- dinkum' higher order thinking and learning skills. These kinds of skills seem to me to be of a kind we in education can best focus on, including the traditional technical/vocational skills, and the 'employability 'type skills, such as work readiness, attitude etc can be better addressed in other aspects of a learner's work life and experiences. Undoubtedly VET has a role here as well, but the emphasis in this case is different.
- Re: Are generic skills up to the task?, posted by: Lilian A at Thu, 20 May 2004, 15:21
  Hi Mark
  I agree that it is about time that the VET sector relooked at some of the higher order skills. However it seems to me that some of the generic skills do cover this. Problem solving for example involves bringing together a host of skills learnt in both formal and informal situations. This needs to be part of the way we teach students to think and learn. This will undoubtedly continue into their work lives but the foundations need to be set from primary schooling on.
- Re: Are generic skills up to the task?, posted by: Debbie E at Thu, 20 May 2004, 15:03
  Hello Mark,
  I like to think of the Mayer Key Competencies as the starting point on which we now (in the 21st century) will 'build' or add the higher order skills you are suggesting (learning to learn for example). The reality is that in TAFE, we have such a broad range of learners, the 'higher order' skills are appropriate to some but perhaps not yet as appropriate to others (thinking of disengaged youth as an example). I think we in TAFE have a responsibility/obligation to help develop these skills in our learners---in fact, the Mayers sit within Training Packages, and the other 'employability skills' are also critical for learners wanting to get/keep jobs----and for everyone who wants to participate in modern communities. Yes, people who have learned these generic skills have learned from a range of contexts---but surely one of those contexts must be TAFE programs. I think the emphasis on technical skills must stay of course, but I think there needs to be an additional (very explicit) focus for the future on generic skills.
- Re: Are generic skills up to the task?, posted by: David C at Thu, 20 May 2004, 13:55
  Hi Mark,
  Wow, that's a challenging position.
  Are the Mayer key competencies the answer for today?
  Well, they are not the complete answer, but they did get us started, and in my view, on the right track. In some ways they are limited. The Mayer committee deliberately took a cautious approach in deciding what would be acceptable as a key competency, and there were plenty of people at the time who wanted a much broader definition.
  There are other schemes that have tried to define what a 'useful set of generic skills' would be. this has been done in many other countries (US, UK, Canada, several European countries, South Africa and NZ). If you want to get into some really meaty arguments, try the OECD sponsored DeSeCo project. There is a book on this (Salganik, L. H., Rychen, D. S., Moser, U., & Konstant, J. W. (1999). Definition and Selection of Competencies. Projects on Competencies in the OECD Context. Analysis of Theoretical and Conceptual Foundations. Neuchatel, Switzerland: OECD.) Several of the original project papers were quite challenging.
  In Australia, two reports might interest you. Both are available online.
  One, written by Allen Consulting, suggested a set of skills needed in the emerging globally competitive environment (Allen Consulting Group. (1999). Training to compete. The training needs of Australian industry. A report to the Australian Industry Group. North Sydney: Australian Industry Group.)
  The other came from ACCI and BCA and did include a set of attributes as well as the more conventional thinking skills.
  (Australian Chamber of Commerce and Industry, & Business Council of Australia. (2002). Employability Skills for the Future. Canberra: Department of Education, Science and Training.).
  To go back to your original challenge, are the Mayer key competencies relevant today, I think you'll find that while the structure of the Employability Skills Framework (ACCI/BCA) is a bit different from Mayer, and they certainly added dispositions and attitudes, much of what the Mayer committee said is informative now.
  - Re: Are generic skills up to the task?, posted by: Mark L at Tue, 25 May 2004, 13:27
    Thanks to all who replied- good stuff. Lilian I agree, and the point you make re the need all education sectors to be involved from primary school on is am important one. How well are we doing this I wonder?
    On the other points made by Debbie and David -agree with what you have to say. They are a starting point and appropriate for many of our learners in VET. However, I would hate to see VET get focused on this level of generalist skills to the neglect of the higher order ones. AS many of the overseas reports and initiatives show, as mentioned by David, Australia, it seems to me has been a bit timid and not aimed high enough. Like many, I have concerns about the Employability skills, not so much as to what they are, but how is VET to respond. That is, what role for VET in the development of these skills? ( once again they seem more like attitudes, attributes and dispositions, rather than skills.) To move into the teaching of these 'skills' seems to me a very problematic task for VET to take on. Clearly, we need to be aware of what employers are telling us they need, but also cautious in light of the research that shows declining employer interest in their role of developing specific technical and cognitive skills. So like many of these things, its about getting the balance right.
    I still reckon by aiming higher we would get a much richer and integrated notion/model of what it means to be 'generically skilled'.
    - Re: Are generic skills up to the task?, posted by: David C at Fri, 28 May 2004, 13:00
      Mark,
      I think that you are right to say that these skills are "attitudes, attributes and dispositions", but not "rather than skills".
      I think that the separation of 'skills' from attitudes and beliefs and also motivations is part of the problem of teaching any concept. This division goes back to the 1950s when the first part of Blooms taxonomy was released. It entrenched the separation of 'cold' knowing and doing from the emotively warm 'feeling and believing'. Most people know that to do well at something you must like it and or believe that you can do it, so that you will invest the effort in it. So even very basic skills have a belief and motivation component. To ignore this is to ignore an important aspect of learners' approaches to learning.
      We could be more ambitious in what we define as generic skills (GS), if we are prepared to invest the time and effort into describing the constructs fully and clearly. I think people are timid about their GS proposals because they do not have confidence in the capacities of teachers to deliver on them. But if teachers AND learners are given descriptions of what they are expected to teach and to learn and if these descriptions include all aspects of them, including both the 'cold' and 'warm' bits, I think we could have greater confidence in the outcomes.
      
      David (slipping off the soap-box)
LINKup Tour – ‘INVESTIGATE’, link: http://www.tvtafe.com.au/linkup, posted by: Rob Denton at Wed, 19 May 2004, 10:47
I hope you enjoyed the DISCOVER section and the opportunity to ‘play’ with Key Competencies a bit … just to get you thinking!

Did you try the interactive Learning Styles Evaluation?
Did you agree with the result it gave?
Do you think you could use this free interactive online evaluation for other staff or students?
Let us know what you think.

Now its time to really ‘get serious’ and INVESTIGATE exactly what this Key Competencies Assessment is all about.

The LINKup Tour so far…
Go to the LINKup website (www.tvtafe.com.au/linkup) and click on ‘Tour’ travelling through
Stage 1: 'DISCOVER' and then to
Stage 2: ‘INVESTIGATE’

“INVESTIGATE Practical Assessment of Key Competencies”
This section contains resources and presentations to help you investigate the comprehensive LINKup Key Competencies assessment initiative.
LINKup recommends an assessment strategy called Validated Self Assessment (VSA) designed to offer formal recognition for Key Competencies AND to help people improve these important personal skills.

YOUR GOAL in this section is to answer these LINKup Objectives:
1. What are Key Competencies?
2. Why are Key Competencies so important?
3. Why ‘assess’ Key Competencies?
4. Who assesses your Key Competencies?
5. How are your Key Competencies assessed?
6. What are the 3 Performance Levels?
7. How can Validated Self Assessment benefit you?
8. What are the steps involved in Self Assessment?
9. What’s involved in Validation?

For those who just want to do the bare minimum you will find ALL these answers in the TWO items marked ** MUST SEE **.
For those interested in more detail, feel free to explore more of the resources in this section!
Resources are sorted into a number of categories to help you find the info relevant to you.

Please return and tell us what you think and ask any questions.
Thank you!
Rob
- Link up - who is it aimed at?, posted by: Lilian A at Thu, 20 May 2004, 15:26
  Hi Rob
  I have spent a bit of time this week (in a few quiet moments) browsing through your site. So much there to absorb. One question I have is about the intended audience. There are quite a few complex ideas and language in the site. Have you tried it with different cohorts of students and if so how have they managed the tasks?
  - Re: Link up - who is it aimed at?, posted by: Rob Denton at Thu, 20 May 2004, 16:06
    Lilian
    A good point. The site has developed out of action research primarily within the Electronics & IT (E&IT) program in which I work. It is currently fully implemented for our E&IT students. However, it has also been strongly influenced by other programs across my institute and also other research and university contributions.
    It has also been designed to be a resource for the workplace for employers, HR managers and employees to use for performance management etc.
    One client group that it has not yet been seriously tailored for is ESL or people with lower literacy skills.
    Student groups that have tried it include Cert 2, 3 & 4, Diploma, Adv Diploma, VET in Schools, employees in E&IT workplace, mature age students. All have proven successful with appropriate support provided for the particular client group and context.
    Having said that, I make the following points:
    (1) It is designed as a very COMPREHENSIVE explicit assessment strategy designed to nurture learning and development of Key Competencies. Many people choose to simplify or cut-out components of the LINKup resources to suit their context. By us making it a comprehensive strategy allows for it to be used at this level OR be distilled down to some other level of implementation.
    (2) The Validated Self Assessment sheets have evolved over a few years. The first generation incorporated very limited and simplistic and quite broad criteria. Feedback from students consistently indicated a lack of clear direction in what to do. They were continually seeking more specific guidance. The criteria was so broad it lacked specific direction. Subsequent development (including a research project conducted in collaboration with David Curtis, Researcher from Flinders University - and who is actively participating in this forum) led to the development of more detailed criteria. The current generation we believe reflects yet a further evolution towards something that is as clear and easy to use as possible without compromising the credibility of the comprehensive assessment process.
    (3) I have received some feedback from staff in a Vocational Preparation program saying that they have found the wording quite workable for their client group without significant change or explanatory clarification. School students have also successfully participated in the assessment strategy with guidance by the teacher.
    (4) The site can be adapted and customised and delivered in different ways. While the site is designed to accommodate self study by individual students and/or staff, it can equally be performed as a class activity or as a guided exercise whereby a mentor can be guided by the website but actually deliver it to a class by putting it in their own words and talking students through it.
    (5) Yes there is a lot to absorb in the site. That's the reason for identifying clearly the **MUST SEE** resources in the INVESTIGATE section, with everything else being extra info to satisfy individual needs or interests.
    (6) The biggest benefit in this whole process is to successfully engage students in the practice of guided reflection or self assessment of their performance in authentic activities. The fact that some of the wording they encounter may be misinterpreted or not fully understood is a secondary issue (which can hopefully be nurtured through personal mentoring or peer support etc). If we can get this important process started, THEN its a relatively easy task to refine the process and the assessment tools to accommodate the clients and contexts.
    
    Glen may be able to comment further on this, but I believe Alverno College consider 'self assessment' or guided reflection to be of equal importance as each of their 'abilities' (it is essentially the ninth ability). I agree - self assessment and reflection is a powerful skill for lifelong learning and performance in workplace and life generally - and that's exactly what generic skills are!
    Rob
- Re: LINKup Tour – ‘INVESTIGATE’, posted by: Fiona S at Thu, 20 May 2004, 11:21
  Dear Rob
  This was my first experience with anything online (shameful I know) I found the learning styles evaluation was spot on and actually fun!!!
  Thank you and I look forward to doing this more often - you have a convert on your hands!!!
  Bye for now
  Fiona Smith
  - Re: LINKup Tour – ‘INVESTIGATE’, posted by: Rob Denton at Thu, 20 May 2004, 12:26
    Hi Fiona
    I'm very pleased to hear that you took the time to go online and complete the learning styles evaluation. Glad you found it enjoyable and accurate.
    Also, I appreciate you making this posting to the forum.
    Did doing the learning styles evaluation also prompt you to reflect on your Key Competencies (generic skills) strengths?
    Thanks!
    Rob
- Suggestion for how to– ‘INVESTIGATE’, posted by: Glen Rogers at Wed, 19 May 2004, 15:27
  You might approach the INVESTIGATE step like you were a student looking for a validation for your understanding of this self assessment process and philosophy. The next TRY step has a cool assessment of your understanding, complete with feedback. Great fun. Glen
Student perspectives on LINKup Key Competencies Assessment, posted by: Rob Denton at Tue, 18 May 2004, 14:17
This is an invitation for students to give their perspectives on LINKup Key Competencies assessment from their direct experience. I will invite a number of students to participate in this forum.
STUDENTS: You may wish to respond to any of the following questions...
What are Key Competencies?
Do you think they are important to you?
Does self assessment help you understand and improve these skills?
Have you learned more about these skills through the assessment process?
Could you now talk more convincingly about your skills at a job interview?
What difficulties have you experienced?
Anything else you want to say...

FORUM PARTICIPANTS: Please feel free to take this opportunity to ask students directly involved in LINKup assessment what they think about it.

Regards Rob
- Re: Student perspectives on LINKup Key Competencies Assessment, posted by: Russell B at Tue, 25 May 2004, 16:03
  Hi there. Thanks Rob for all the work that you are doing for web site!
  
  In my experience with the Linkup Key Competencies - working with them over the past 5 months, I have found that they are both frustrating, yet beneficial. As a student studying electronics at TVTAFE, the questions asked in the assessments can be difficult, because the answer is often clear to see, but hard to prove. This challenges me to think through what I have been learning and doing, and to provide the evidence for it.
  
  Like all good things however, work must be done to produce good fruit. So I can see the assessments are beneficial because I become more consciously aware of what I have been learning. In addition, it is a very good thing to be able to say in a job interview, "Yes, I can work well as a team. I am sure of it, because I have been challenged to do so and prove it." etc.
  
  My opinion stands that I am all for the Key Competencies assessments.
  - Re: Developing understanding of the curriculum and student perspectives on LINKup, posted by: Glen Rogers at Thu, 27 May 2004, 13:36
    Russell, our research on Alverno students perspectives on their learning found that some students came to believe in the Alverno ability-based curriculum when they could abstractly conceptualize it. These were often some of the first to make sense of how the abilities were an integrated part of what they were learning rather than something "extra" added on. Most students developed such understanding, however, when they experienced the applicability of their abilities to internships or other contexts outside the classroom. I appreciate your actively articulating your understanding of the rationale for the curriculum.
    
    I also appreciate your recognition that what is initially difficult in the curriculum is a developmental challenge that you can learn from and achieve. That both speaks well of the curriculum and yourself as a learner. A valuable generalisable skill you seem to be learning is to connect evidence to a case that you are making, which in this context is your meeting the competency requirement.
    
    Thanks for giving a student perspective.
  - Re: Student perspectives on LINKup Key Competencies Assessment, posted by: David C at Wed, 26 May 2004, 11:01
    Hi Russell,
    How long does it take you to do a key competencies self-assessment once you have completed the technical part of your assessment task?
    What is the most difficult part of doing a key competency self-assessment?
    Regards,
    David
    - Re: Student perspectives on LINKup Key Competencies Assessment, posted by: Russell B at Wed, 26 May 2004, 15:31
      Hey David!
      
      On average, I would take approximately 30mins to complete one assessment form - to satisfy the questions and provide strong, clear evidence. We have to have two sheets approved before we can get one key competency 'ticked off' - so it would take at least an hour to finish one completely.
      
      Generally speaking, the hardest part to complete a self-assessment is the most critical part - showing and providing the evidence! Very often the answer is clear in my mind, but it's somewhat more difficult to put it down on paper and prove it. It's definitely a challenge, but it helps me to see exactly what I am learning. Also, trying to understand what the question is asking exactly can be frustrating; but once I started getting into them and becoming familiar with what it wants, it gets easier.
      
      Thanks for the questions! I hope these have helped.
      
      Regards,
      Russell : )
- Re: Student perspectives on LINKup Key Competencies Assessment, posted by: Gregg M at Mon, 24 May 2004, 13:52
  Hello,
  Whilst being a student at TVTAFE I also have the unique perspective of being an employer. It's interesting to see the contrast where on one hand I'm assessing my own skills and on the other, seeing the impact of self-assessment on the ability of others to function in a working (real or simulated) environment.
  
  From my personal experience, I have noticed several skills taken for granted and identified places where I can improve.
  
  From the perspective of an employer, I find that if people I hire can demonstrate not only academic qualifications but confidence in their own experience then these people will provide a quality contribution in whatever career they choose to pursue.
  
  Initially being forced to think about one's own skills and thing that are taken for granted subconsciously takes a bit of getting used to. I remember some great philosopher saying 'Know Thyself!' and after several key competency self-assessments, I can guarantee that both the good and the bad of 'Thyself' will become clear. Accepting the results of this reflection and making the necessary 'mental changes' is of course up to the individual student. I have found that the effect that this change has extends not simply to academic studies but to all pursuits undertaken.
  - Re: Self assessment skill and student perspectives on LINKup Key Competencies As, posted by: Glen Rogers at Thu, 27 May 2004, 13:20
    Gregg, your post reflects what I think are important self-assessment skills, which we have also found in our context often do need to be learned rather than assumed. Our research has identified taking a balanced view (e.g. strengths and weaknesses, rather than global evaluation) often is an initial skill and that abstracting one's abilities beyond the individual performance is another important skill, as well as developing the habit of focusing on steps for improvement. You have a clear and articulate sense of these. We have also found that in our curriculum at Alverno students learn to use and develop criteria for judging their performance. You do not explicitly mention this detail, but I draw no particular conclusion from that. Thanks for giving a student's view. Glen
  - Re: Student perspectives on LINKup Key Competencies Assessment, posted by: David C at Wed, 26 May 2004, 11:05
    Gregg,
    Have you tried to get your employees to develop their key competencies or to reflect on their current skill levels in these areas?
    Would you use key competencies in decisions about hiring new staff?
    Would you use key competencies in promotion or salary increment decisions about existing staff?
    
    Regards,
    David
    - how do employees demonstrate their skills?, posted by: Lilian A at Fri, 28 May 2004, 13:24
      Gregg
      We have heard much about business and industry's desire for employees with generic skills. As an employer how would you like prospective employees to provide you with this evidence? Portfolio, certificate etc????
      Regards Lilian
      - Re: how do employees demonstrate their skills?, posted by: Gregg M at Mon, 31 May 2004, 12:52
        My opinion about 'generic skills' is quite a bit different from what the current 'establishment' might view as 'normal'.
        Although analysing one's key competencies is a skill, actual ability to function in a work environment requires other skills. So I tend to split 'self-skills' and 'social/technical skills' into different categories.
        
        Tertiary education facilities such as TVTAFE and others combine the two and at least indicate that the person has learnt to 'think for themselves' and adapt somewhat to new ideas. Whilst a certificate or portfolio provides evidence of a persons ability to think logically, I always stress the ability to think laterally.
        
        A lot of businesses fail because people get scared of taking risks, their management push 'logic' and 'rationalisation' ending up being logically rationalized via bankruptcy.
        In my mind these are tools. If someone working for me uses them too often, things start going wrong and usually ends with the person joining the 'management attrition program'.
        
        I suppose everyone has a different view of what a perfect employee's skills will be, unfortunately my view of the current 'generic skills' push leaves me wanting a whole lot more. If only the bureaucrats running the education department could take a years course in key competency training!
    - Re: Student perspectives on LINKup Key Competencies Assessment, posted by: Gregg M at Thu, 27 May 2004, 10:33
      Getting employees to develop their key competencies.
      
      Yes but not a structured way. I've always believed if something goes wrong, you shouldn't blow your top but instead ask "What went wrong?, How can we learn from this".
      There's a difference between gaining experience and total stupidity. Unfortunately most people these days lean towards the "You did wrong, goodbye" school of skill improvement, which is why I see the learning of self reflection important.
      
      For key competencies to be really effective, it does require a combination of employers who are 'willing' to let the people who work for them to learn.
      It's my hope that perhaps people going through TVTAFE and other institutions that have a key competency focus will have their own self employment one day and that the freedom to learn in a real business environment will filter out into the wider community.
      
      I would most certainly hire someone who isn't afraid to make mistakes or pose the question "why are we doing it this way?" There's nothing worse than needing something done to a schedule and some tight minded degenerate saying "but we don't do it that way", "let's have a meeting to discuss it" or worse "it won't work" when they haven't tried a new idea before. As Edison said "I now know 100 ways NOT to make a light bulb."
      
      In relation to promotion or salary increment decisions, key competencies are always a deciding factor. Again my 'use' has never been terribly formal. Some people are good at motivating others. Other people are good at analysing a problem and coming up with solutions. Most of the time multiple key competencies are assessed, it's no good having a problem solver if they can't communicate their solutions effectively. There are rare occasions though where this can be overlooked, but only if the person's idiosyncrasies didn't affect morale or a product's schedule.
      Consistent display of key competencies 'should' always lead to a better position, or higher salary (within reason) but it's up to the individual concerned to ask for it.
      
      It's not good to reward someone with something they don't want. I've known many people who are quite happy at #2, they can't stand the bureaucracy within management - they'd rather do work!
- Re: Student perspectives on LINKup Key Competencies Assessment, posted by: Elizabeth N at Wed, 19 May 2004, 11:19
  Although I am a teacher, I am replying at this point because I think the student perspective is critical. But first Rob, congratulations on your website.
  Using small (trial) focus groups (TAFE) I have found that when students are presented with a list of skills and invited to nominate/describe specific situations where they have learnt/practised them during their studies, they are often able to give a range of examples.
  If students were to use these as the preparation for an 'enhanced CV' would it be important that their claims were validated?
  How can an institution best support "guided self-reflection" to the point that it is taken seriously by employers? Is this why validation is important?
  Are there real advantages in recording different levels of achievement?
  Am also wondering whether the student examples (and any blanks re particular skills) would be a good starting point for teachers to start evaluating their courses re their inclusion of employability skills.
  Hoping that I have mastered the technology and that this reaches you
  Bett
  - Re: Student perspectives on LINKup Key Competencies Assessment, posted by: David C at Wed, 26 May 2004, 11:16
    Bett,
    I agree with your point about the importance of student perspectives.
    I do think that we need to encourage students to build 'enhanced CVs' and to keep them up to date. It would be good if there was a device that students and others could use to create and maintain their CVs. It would need to be a combination of a data base and a series of documents, some electronic, others paper based.
    I think validation is vital. I cannot believe that employers will place much credence on a portfolio that only contains individually generated information. Such material is important, but it must be supported with credible evidence from other sources.
    I also argue that having several levels of achievement is necessary. Just how many I am not sure, it depends in part on the reliability of the judgments that are made about performance. Many skills, and especially generic skills, are developmental. Setting simple benchmark levels (not yet competent/ competent) simply cannot capture the growth in these important skills. A problem with simple competency benchmarks is that once someone has achieved that level, there is no reason to improve further. In addition, different roles require different mixes of skills, so it would be useful to identify multiple levels of generic skills and to match the skill profiles of individuals to the particular demands of specific job roles.
    
    Regards,
    David
  - Re: Student perspectives on LINKup Key Competencies Assessment, posted by: Rob Denton at Wed, 19 May 2004, 12:18
    Hi Bett
    Yes, I agree that students are generally able to identify examples of situations when they have demonstrated specific skills. The goal of 'guided self assessment' is to take this a step further to help students become aware of the 'processes' involved in performing those skills well. For example, in addition to being able to identify examples of times they have demonstrated 'teamwork' they would also be more confident to identify the good practice processes involved in their 'teamwork' performance - i.e. they find out more about HOW to perform effectively. By participating in self assessment of their activities raises their awareness of these processes and therefore increases the likelihood that they will grasp those good practice processes and apply them effectively in future activities and in different contexts.
    Just being able to identify activities involving teamwork doesn't enhance their performance as much as critically reviewing the fundamental processes involved.
    The intention here is to develop a framework of understanding of the Key Competencies to enhance their development and application in future activities.
    
    The 'self assessment' approach is crucial to raising the students self awareness of their skills. As you correctly stated, the 'validation' is important to provide quality control for the self assessment to give it credibility in the eyes of employers.
    
    Yes, I believe there ARE real advantages in recording different levels of achievement because the assessment process is designed to be a major strategy for 'nurturing the development' of these skills - not just 'recording their performance'. By incorporating developmental performance levels it allows students to differentiate between different levels of skill AND offers them a sense of what is involved to perform that same Key Comp at a different level.
    NOTE: All levels are equally important. Whilst the higher levels require more self directedness or initiative, the lower levels are equally important. It is important for students to develop an understanding of the different performance levels and be able to choose the most appropriate level to perform at according to the context of the activity. Sometimes it is appropriate to be very self directed and use lots of initiative (level 3) but sometimes it is crucial to follow instructions (level 1) to accomplish the task.
    Being able to perform at all levels AND being able to choose the appropriate level to perform at is important. The opportunity exists to develop your self directedness in the Key Comps by progressing from lower levels to higher ones. The explicit criteria for all levels helps people become aware of what is involved in performing at each different level.
    
    YES, using student examples to evaluate courses re their inclusion of employability skills is appropriate. I say that ANY conscious effort to make these skills more explicit for teachers and students is a healthy thing. The MORE students and teachers become aware of employability skills, example activities, good practice processes involved and developmental steps to nurture learning - the more they will benefit.
    Thanks for your posting - I hope my comments make sense to you.
    Rob
    - Re: Learning Principles and Student perspectives on LINKup Key Competencies Assessment, posted by: Glen Rogers at Thu, 27 May 2004, 13:07
      Rob, I appreciate your efforts to continually articulate the learning principles that underlie the curricular choices you are making. I think you make an interesting case for the competence inherent in being able to follow instructions. I might add that having the higher levels move toward increasing self-direction heads off the common complaint that focusing on developing this basic level of competence sends the wrong message to learners about what they should be learning.
Assessment as Learning and other burning questions!, posted by: Debbie E at Mon, 17 May 2004, 07:35
Hello Glen,

We at the University of Ballarat (Technical and Further Education Division) are in the throes of a year-long professional development project with teachers representing each of our teaching departments, with the aims, generally, to ensure that generic skills (what you call 'abilities') are 'embedded' and assessed in our learning programs. I have many many questions for you about how things are done at Alverno (and why you have taken certain approaches)---but I won't overwhelm you with all the questions at once. Where to start? I am definitely interested in Rob's first 2 questions (how you assess generic skills and the underlying principles)---I'm familiar with the validated self-assessment approach used at Torrens Valley---I suspect that we at UB will take an integrated approach to assessment (i.e., I doubt that assessment of the generic skills will be a separate process for us--but this is still under consideration).

I am also very interested to find out what learning strategies are used by academic (teaching) staff at Alverno to develop these abilities in their students---AND, how do you know that these learning strategies are effective? We know that learners develop these abilities in a range of ways, in a range of contexts, but employers in Australia are clearly saying that TAFEs have a responsibility to help develop these skills in learners, and the suggestion is that this is not happening successfully at the moment----I look forward to hearing from you.
Thanks for agreeing to be a part of this forum!
Kind regards
Debbie Eagles (TAFE Curriculum Officer at University of Ballarat)
- Re: Assessment as Learning and other burning questions!, posted by: Carolyn E at Thu, 20 May 2004, 11:25
  Hi Glen I am a nurse educator my primary teaching responsibility is the Diploma of Enrolled Nursing I find that one of our major issues is the difficulty students experience in their clinical placement in relation to existing practitioners and their reluctance to both facilitate students and provide them with opportunities to perform hand on tasks. We have constant ongoing issues with students on placement feeling that they are compromised in relation to meeting their required competencies. Some clinical venue facilities are reluctant to put forward the opportunities however they expect the outcome to be that the student is competent and confident to function in the workplace as an effective team member.
  - Re: Learning from placements!, posted by: Glen Rogers at Thu, 20 May 2004, 14:08
    Carolyn, One thought I have is that students might be prepared for the hands on aspect of the placements through some simulated patients on campus. Lower fidelity simulations may also be of assistance. Practitioners may find that they can find some legitimate tasks for students who are more prepared to jump in.
    
    A second possibility is to build up some reflective space back on campus, where the nursing students can discuss issues with academic staff/faculty. Where placements are scattered some facilitated on-line discussion groups may augment the learning. The competence and ability frameworks we are talking about here can provide a key part of the focusing and organizing structure for the learning.
    
    Michael Eraut at the University of Sussex in the UK has led some in-depth research on learning in the workplace. One of his conclusions from the field of nursing is that some units are persistently so busy that there is almost no time for reflection on action during work. Students initial placements may need to be in less fast-paced settings. But Eraut also finds that collaboration provides another form of learning that is fairly powerful in the workplace. It may be that there are yet avenues to working with the placement settings to explore how they are encouraging collaborative approaches and building teams.
    
    My heart goes out to you and I expect you are doing your best to find and build the best placements you can.
    
    Glen
- Re: Assessment Principles!, posted by: Glen Rogers at Tue, 18 May 2004, 07:42
  Debbie, I appreciate your asking about principles that underlie Alverno's approach to assessment. This is often a good place to start. Comparison of assessment principles across context can be an important stimulus to learning. And, the development of assessment principles within a particular educational setting can be lead to conceptual advances that support collaboration on achieving learning outcomes. There is a research document within the "Try" section of the LINKup web site where Rob Denton has recorded elements of a conversation he and I had several years back comparing Alverno and Torrens Valley assessment principles.
  
  Perhaps the most important macro-level principle is that teaching, learning, and assessment need to be integrated. As noted above, we expect that assessment will lead to learning. But, in the life of a academic faculty the question doesn't begin with assessment, but instead with the intended learning outcomes and how students are prepared to meet them. And it is important to begin where faculty and academic staff conversation begins. Thus, questions about assessment principles are best linked to learning principles and prior faculty discussion of teaching and learning. Here are some core assessment principles from Alverno: "if learning is to be integrative/experiential, assessment must judge performance. If learning is to be characterized by self-awareness, assessment must include self-assessment as well as expected outcomes and developmental criteria that are public. If learning is to be active/interactive, assessment must include feedback and elements of externality as well as performance. If learning is to be developmental, assessment must be cumulative and expansive. Finally, if learning is to be transferable, assessment must be multiple in mode and context" (From Alverno College Faculty, 1979/1994, Student Assessment as Learning at Alverno College, pp 18-19, excluding italics).
- Re: Assessment as Learning and other burning questions!, posted by: Glen Rogers at Tue, 18 May 2004, 02:07
  Debbie, I found a novel way yesterday to have my post evaporate into the ether, so I will take up your several questions about Alverno in separate posts. Thanks for asking, as it does help everyone to understand the various concrete settings as we converse. I hope you are able to take the Integrated approach at the University of Ballarat that you describe as that kind of infusion of ability and content enables the kind of reinforcement of skills across contexts in a way that they can become ingrained in how students think and act.
  
  At Alverno, there are eight general education abilities at Alverno that are developmentally sequenced in four levels each. There are also more integrated advanced outcomes for the student's particular major. Assessments are designed by academic faculty and embedded in the context of everyday coursework. Students are required to demonstrate that they have integrated their understanding of the disciplinary or multi-disciplinary content into their performance of the ability. Some beginning courses are interdisciplinary arts and humanities, some are math skills, some on communication, and some on disciplines like introduction to psychology. As students progress, the courses become more focused on a single discipline. Across all of these courses the "same" generic abilities are assessed, though what is required for successful completion is always also always somewhat different since the ability definition is integrated with the particular course content and a distinct assessment context that is designed to elicit performance that has some degree of fidelity with naturally occurring contexts in the broader world. For example a culminating assessment in a seventh semester business and management course requires students to submit a business plan for a new business to a banker, who serves as a co-assessor with the academic instructor. Some criteria for judging the plan would include "uses concepts, principles, and frameworks from the discipline to make sound business decisions" and "uses clear, concise business writing style that is appropriate to a business audience and meet effective writing criteria."
  
  There are many cases where co-assessors are drawn from the community, but most typically, academic faculty are the judges and most assessments will have a more interim quality, so that students are learning to meet the explicit performance requirements over time and are getting feedback on how they are doing and how they can improve so as to meet course requirements by the end of the course period. It is generally expected that they can be successful in meeting the course outcomes as they complete the required course assessments, but, if they do not, they are not validated and generally must then retake the particular course (though other courses may sometimes fulfill the requirements of their major or general education sequence).
  
  Although each student must meet explicit public criteria, we call it assessment-as-learning because the student not only always get feedback from assessors in relation to criteria articulated for the specific assessment, but will also generally required to complete a self-assessment in relation to these same criteria, usually in an open-ended format where the student is expected to cite evidence in their performance, analyze strengths and weakness, and articulate plans for further learning based on their performance. Faculty work hard on the self assessment prompts so that these prompts call-up a reflective approach to the skill, knowledge, values, self-image, and dispositions that underlie the multidimensional and complex ability. The expectation is that students will be able to succeed because of the multiple opportunities to learn and perform and the sustained feedback and self assessments over time.
  
  In order for students to learn to transfer the learned abilities across contexts, they will have to be validated for each level of the same ability in another course or in an external assessment. In an external assessment they generally have to demonstrate the ability to an assessor other than the course instructor at some distance from the course. I should point out that assessments are often designed so that students are required to demonstrate more than one ability at a time (say communication, analysis, and social interaction), especially as they advance in the general education. This integration of the abilities is explicitly required by the curriculum for the advanced outcomes.
  
  Some of the most important aspects of how assessments are evaluated include asking how well the assessment stimulus elicits a performance that fully expresses the intended ability. Also, do the criteria for the specific assessment when taken collectively give a full picture of the ability and how it is integrated with content? Is the mode of assessment appropriate to the ability? Does the instrument allow individual differences in how the ability is met? (Note that students are encouraged to exceed criteria and when they do are also given feedback on how they can improve).
  
  Debbie I hope this helps as an overview of assessment-as-learning at Alverno.
  - Re: Learning strategies, posted by: Glen Rogers at Tue, 18 May 2004, 09:15
    Debbie, your question about learning strategies at Alverno comes with the concern that although you are aware that learners develop generic abilities in a number of ways, Australian employers retain concerns about whether the TAFEs are being very effective in developing them. Employers in the U.S.A retain similar concerns about higher education on this side of the world. However, it is true that local employers see Alvenro graduates as having the sought after generic abilities.
    
    The key difference at Alverno is that the abilities are infused in learning, assessment, and course requirements throughout the curriculum. Students learn to look for criteria that would define effective performance and learn then to construct criteria for effective performance in ambiguous situations where the criteria given to them become more global and hence must be tacitly understood. Self assessment is important because students then become active participants in constructing an understanding of how the abilities frameworks can guide their performance. They become metacognitive about their performance.
    
    I hesitate to list the following concrete learning strategies a little, not only because they are not comprehensive and a personal construction out of my understanding, but also because they can encourage the idea of putting method at the forefront rather than focusing on the dynamic interaction learning goals, strategies, and assessments. We use quite a bit of group work to encourage the development of collaborative and social interaction skills. Faculty develop strategies for monitoring individual contributions as students will sometimes engage in social loafing. Students also give many speeches and multi-media presentations to assess their communication abilities. They will always have a designated audience, who is generally someone other than the instructor. Students have a video portfolio that they develop over time. As is common at many other colleges, they write papers, do lab work (in science), art shows (in art), engage in classroom discussions, and internships. Having the ability framework as an assessment/self-assessment tool makes learning in the internships more certain. When students are in classrooms, they generally sit at tables together. Lectures are generally brief and set up learning activities. Projects are common. Faculty are always striving to make assessment projects connect meaningfully to the lives of the students and high fidelity situations. Students may complete several portfolios during their student career, depending on the major. We have been developing a common Diagnostic Digital Portfolio (DDP) that enables students to reflect on their self assessments and faculty feedback on key performances over time. Some small video segments from assessments are now being added to the DDP.
    - Re: Learning strategies, posted by: Rob Denton at Tue, 18 May 2004, 12:22
      Glen
      You mentioned that "The key difference at Alverno is that the abilities are 'infused' in learning, assessment, and course requirements throughout the curriculum"
      I believe the 'abilities' are 'infused' yet very much 'explicit' as opposed to being 'infused and hidden'. In Australia we have experienced a mindset that Key Competencies are already 'built-in' to our Training Packages so we don't need to do anything else - hence they are all too readily forgotten.
      Our challenge is to make them more explicit.
      Our LINKup strategy strives for 'integrated performance with explicit assessment'. That is, infusing them into the whole process of learning and assessment but 'explicitly' assessing them against specific Key Competencies criteria.
      Rob
      - Re: Explicit criteria, posted by: Glen Rogers at Thu, 20 May 2004, 14:13
        Thanks for the clarification. Amen to explicit criteria.
    - Re: Effectiveness of Learning Strategies, posted by: Glen Rogers at Tue, 18 May 2004, 10:10
      Debbie, you asked about the effectiveness of the learning strategies. The Alverno Longitudinal Study (ALS) has provided some evidence about what works. The ALS followed two cohorts from entrance to the college through their student career to their lives as five-year alumnae. Longitudinal Perspectives Interviews of 100 students and alumna revealed that students internalized the ability frameworks and used it to carry their college learning with them as they adapted it to performance in post-college contexts. The internships often were critical for students as they experientially validated how the abilities constructs that they were appropriating from the curriculum were important to their being effective in other settings. It was important for students to develop a conceptual grasp of the integrated system, seeing it as putting knowledge into practice and encompassing assumptions about how people learn and what are important enduring learning outcomes.
      
      Collaboration and group work was sometimes seen as important. Five-year alumna performance on a Collaborative Thinking and Action ability factor measured through a Behavioral Event Interview was statistically linked to the breadth of their self-reported learning measured at graduation. A number of findings suggested that engaging diverse approaches to learning, engaging diverse views, and participating in a broad range of activities led to students taking responsibility for independent learning. Longitudinal growth on a battery of externally designed measures of abilities and holistic growth showed that students generally made gains in Critical Thinking and Moral Reasoning, with some gains in Integration of Self occurring in college and some after.
      
      But, beyond the longitudinal study, which has helped us develop our most formal models of learning and development, much of what we know about what works comes from faculty scholarship on learning in the classrooms and at the level of the programs. This is, of course, true at many institutions. Others sometimes find Alverno as relatively distinctive in how faculty collaborate together in their scholarship on teaching, learning, and assessment: (1) They meet for a week or more three times a year as a whole faculty to collectively discuss teaching and learning, (2) faculty members generally belong to an ability department that monitors and develops understanding of particular abilities, as well as belonging to the discipline department and serving on committees, and (3) faculty members participate in peer performance reviews where promotion to advanced ranks requires that they are, for example, effective as "resources to other teachers" and ultimately "provide direction in identifying and addressing significant teaching concerns." One of the things I have come to say about program and institutional assessment is that how it promotes collective responsibility among faculty for student learning outcomes is just as important as any findings that may be reported.
- Re: Assessment as Learning and other burning questions!, posted by: Lilian A at Mon, 17 May 2004, 10:00
  Hi Debbie
  You raise some very pertinent issues. My experience with TAFE staff is that many feel that this not their responsibility. So this means that we will have to sell the concept to them. If they do take it on board the next question is that they may not have the necessary skills, resources, teaching strategies to incorporate generic skills in an integrated way. I can see the need for a huge PD effort in training staff appropriately. Any thoughts?
  - Re: Assessment as Learning and other burning questions!, posted by: Rob Denton at Mon, 17 May 2004, 14:00
    Hi Lilian
    I agree that PD is a huge need.
    One of my responses to this need is the development of the LINKup Key Competencies website. I invite you to explore this website and see if you think it offers any useful solutions to the PD needs of your staff.
    Please see my posting below titled "LINKup Tour - 'DISCOVER'
    I plan to guide people through the 4 stages of this website throughout this forum - starting with the DISCOVER stage!
    Regards Rob
ASK GLEN !, link: http://www.alverno.edu, posted by: Rob Denton at Sun, 16 May 2004, 17:22
A special welcome to Glen Rogers all the way from Alverno College, Milwaukee, USA. Glen brings a wealth of experience and research about generic skills (Alverno calls 'Abilities').
I worked with Glen in 2001 to benchmark our LINKup Key Competencies strategy with the Alverno model (full report available in INVESTIGATE section of LINKup website).

Now is YOUR chance to ... ASK GLEN !

Please ask him your questions OR choose one of the following questions to ask him...
(1) How are generic skills assessed at Alverno?
(2) What are the most important principles of this assessment approach?
(3) Is it REALLY worthwhile? What does your extensive longitudinal research of your graduates in the workplace indicate?
(4) What are the benefits of 'self assessment'?
(5) What is 'assessment-as-learning'?
(6) What are your assessment instruments like?
(7) How important are developmental 'performance levels'?
(8) How do you suggest getting started with generic skills assessment?
(9) Would LINKup help?
(10) ... any other question?

If you are interested in the answers to ANY of these questions ... ask them now!
Rob
- Re: ASK GLEN !, posted by: David C at Tue, 18 May 2004, 14:13
  I would like to make a comment rather than ask a question, and I would be interested in reactions to this.
  Before we proceed to assess and report on key competencies or other generic skills, it seems that we should be asking: 'What they are?'
  
  This question has several levels. It can be taken to refer to a scheme in which a number of generic abilities are described. In this case, the 'What are they?' question is trying to get to their purpose and also to a related question: 'What should be included or excluded?' and perhaps also 'What priorities should we place on the skills identified?'
  
  I think that these questions are important, but in some of the debate in Australia, and also in the discussion of the DeSeCo generic skills project, it seems to have derailed development and implementation of the concept. This is unfortunate, but it seems we do need to be clear about what we are implementing and why. We also need to accept that we will not get universal agreement on generic abilities. However, if we examine the many schemes in which this concept has been articulated, it seems that there is quite broad agreement on at least some components -- most notably, communication, team work and interpersonal abilities, and problem solving. There are others, but these are a start.
  
  I would also like to address the 'What are they?' question at the level of individual generic abilities. In many generic skills schemes, there seems to have been a tacit assumption that 'we all know what we mean'. However, if we teach other areas of knowledge, we define the constructs that we wish to develop. Further, we also accept that the major constructs that we want to promote have several facets. We can talk about knowledge at several levels, declarative, procedural, conditional and conceptual. We also accept that there are skills components in most knowledge domains. Further, before we can expect success among learners, we need to develop a case for learners to engage with the constructs. We need to answer the 'What's in it for me?' question for learners.
  
  In short, I would like to see us develop a description of each generic ability (key competency, employability skill) that is accessible to learners. They need to know what it is we want them to learn, and why.
  - Re: Accessible abilities, posted by: Glen Rogers at Wed, 19 May 2004, 13:59
    David, These are all good questions and points. I think the LINKup site is intended to address most of these questions. For example, in the resources section under the INVESTIGATE step there is a brief description of abilities entitled "LINKup developmental levels" and a "Sample Validated Self Assessment Sheet" that breaks that further down for one ability. Do you see these and the site as accessible to learners? My thought in focusing on the site is that it provides a concrete context for supporting a conversation about the points and questions you are asking.
    For example, how well does the site answer the "what's in it for me" question?
    
    I liked your comments about the complexity of the abilities. Beyond various kinds of knowledge and skills components, I might add that performance is also supported by dispositions (e.g., toward achievement), values (e.g., toward collaboration), and self-perceptions (e.g., I am good at defining and solving problems). Ability models differ on whether they include these. Practically, whether as separate components or integral aspects of abilities they are important for educators to foster.
- Re: ASK GLEN !, posted by: Kay P at Tue, 18 May 2004, 08:56
  Thanks for this opportunity. The RTO I am involved with works with registered and enrolled nurses who are required to maintain their ongoing competency to be able to register each year. Self-assessment of continuing competency is very much the focus we want to take given that the nurses accessing the RTO activities are agency nurses (relief nurses) and self-assessment is also the approach being considered by our Nurses Board. Could you please relate your experiences with self-assessment, how you monitor/document/record self-assessment, the benefits you believe are inherent in this approach and what makes it work best?
  Many thanks
  Kay Price
  - Re: self assessment and continuing education, posted by: Glen Rogers at Wed, 19 May 2004, 14:29
    Kay, I am going to offer limited comments, as my experience does not extend into designing a self-assessment based system for monitoring continuing education experience. In general, I find myself agreeing with David's more extended reply and his point that external judges of performance seem like a necessary component. I might add, that self assessment in a continuing education system might play a central role in that the nurses may use self-assessment to identify areas where they need and/or want to pursue further development. At Alvenro much of the power of self assessment comes from how it interacts with feedback from assessors. I have found that self assessment gives key insight into how learners are constructing their performance and thus suggests avenues for powerfully engaging them. Ideally, feedback and self assessment are a dynamic conversation. This kind of dynamism may not be feasible in the system you are contemplating. I would like to disclose that my experience in the USA does not include familiarity with the acronym "RTO."
    - Re: self assessment and continuing education, posted by: Kay P at Thu, 20 May 2004, 11:19
      Thank you very much for your response. I agree with the view about feedback and self assessment as a dynamic conversation. Can I ask though what makes an assessor?
      RTO is the registered training organisation (TAFE's are public VET provider and I work with a private VET provider).
      - Re: what makes an assessor., posted by: Glen Rogers at Thu, 20 May 2004, 15:03
        Kay, In Alverno's ability-based curriculum an assessor is someone who observes the individual performance of a learner in action and judges them on the basis public developmental criteria and gives feedback to the learner. Your question about what makes an assessor also implies the evaluative question of what makes a good assessor. Professional expertise and experience in the field that is being validated can be prominent criteria in selecting assessors. There are also some generic skills that can be learned through training activities. For example, learning to focus on describing the observed behaviors and relating them to the public criteria rather habitually making higher level inferences that cannot be tracked back to the evidence in the performance. Educators can also learn strategies for giving good feedback, such as beginning with positive features of the performance and focusing on one or two areas for development rather than everything that ultimately needs improvement. Also beyond just expertise and experience in the field being judged, assessors may have broad backgrounds that contribute to making enlivened judgments about a particular generic ability.
        
        The views I have about what makes an assessor and what makes a good assessor are entangled with my perspectives on what assessment is and what makes for good assessment. I like your question because it highlights how much diversity of understanding may be embedded in a term that we might think we share in common.
        Re: what makes an assessor., posted by: Kay P at Fri, 21 May 2004, 12:06
        Yes, and the need to continually challenge the taken-for-granted. 'Making sense' of, or giving meaning to, what makes for, or makes a, good assessment or what makes for, or makes, a good assessor, is obviously entangled up with/in the extent to which we are aware of, or allow ourselves to reflect on/in, how we may continuously use our experiences of assessment or an assessor to transform these experiences into the meaning of assessment or a good assessor. The dynamic conversation an individual has with her/himself...
        Cheers
  - Re: ASK GLEN !, link: http://www.anta.gov.au/images/publications/fresh_thinking.pdf, posted by: David C at Wed, 19 May 2004, 12:09
    Kay,
    
    I am interested in the use of self-assessment as a component of ongoing professional registration.
    
    I have used self-assessment for two reasons. The first is that by making assessment criteria explicit to learners and asking learners to apply the criteria to their own performance, we expect them the engage much more deeply with the criteria than they would if they were simply informed about the criteria that someone else would use in making judgments about their performance. In most assessment, responsibility for making judgments is taken by a teacher/lecturer, and this absolves the learner of this responsibility. As Boud (2002) has pointed out, it also deprives the learner of an opportunity to learn.
    
    A second reason for using self assessment is that it should make assessment more efficient. I have not advocated self assessment alone. Rob has pointed out that it is part of a larger process, Validated Self Assessment (VSA). By getting learners to make a claim about their competence and presenting evidence to support the claim, the assessor does not need to spend time digging for the evidence in amongst, perhaps, a lot of irrelevant material.
    
    It seems to me in the assessment of professional competence, some form of validation is vital. VSA seems to make the process effective, in that it promotes learning, and efficient, in that it should reduce the load on assessors and therefore the cost of assessment. I imagine that, given funding constraints, this might appeal to RTOs and to VET sector leaders.
    
    Boud, D. (2002). Assessment: have we forgotten something? In Australian National Training Authority (Ed.), Fresh thinking about learning and learners: a blue sky project (pp. 42-45). Melbourne: Australian National Training Authority. (http://www.anta.gov.au/images/publications/fresh_thinking.pdf)
    - Re: ASK GLEN !, posted by: Kay P at Thu, 20 May 2004, 11:22
      Thanks David
      A couple more questions if this is OK - what do you mean by validation and what makes a validation valid? Also what is an assessor?
      Cheers
      Kay
      - Re: ASK GLEN !, posted by: David C at Thu, 20 May 2004, 14:19
        Kay,
        
        Some good questions, i.e., they made me rethink what I said!
        
        By validated self assessment (VSA) - there's a good description on the LinkUp site - I meant to say that the learner must seek an assessment from a recognised more experienced person who can corroborate what they have said, against explicit criteria, about their own performance. I think that this serves several functions. First, it gives credence to the self assessment. In most areas of learning and development, we want judgments to be made by people who are in some way authorised to make them. In schools we want teachers to grade students' work, in universities we want lecturers to provide informative feedback and to issue grades. In both cases we have systems in place to ensure that people who achieve a defined standard are awarded credentials.
        The second reason goes back to the point about continuous learning and improvement. I liked Glen's comment about 'powerfully engaging' learners in a 'dynamic conversation'. It is not just about whether the person achieved a set of criteria, but what the criteria mean in context. In a conversation between the self-assessing individual and a mentor, we see opportunities for significant learning.
        
        I used the term assessor because in the VET (Vocational Education and Training) system in which Registered Training Organisations (RTOs) operate, there is an assumption that each judgment about competency will be made by someone who is authorised to make the judgment.
        
        But I load the word validation with other meanings too. Above I have made reference to notions of credibility and authority. But validity means more. It also includes judgments about what the criteria mean in both the context of the assessment (you did this and this is how it compares with the specified criteria) and also the inferences that flow from the judgment. You did this today, and I think this means that you will be able to do a range of related things in a range of contexts that are more or less similar to the assessment context.
        I wonder if assessors understand the implications of these inferences.
        
        Re: ASK GLEN !, posted by: Kay P at Fri, 21 May 2004, 11:46
        And, I wonder how one (us) determines if assessors do understand the implications of these inferences and whether this is something a learner needs to know?
        Cheers
- assessing generic skills, posted by: Lilian A at Mon, 17 May 2004, 09:56
  Hi Rob
  I work at Swinburne TAFE and am working on generic skills passport options as part of my Flexible Learning Leader scholarship. We have come up against a number of issues when considering how to develop and assess generic skills. One key issue is managing the development and assessment over a complete certificate:
  - how do we ensure that a range of teachers encourage the process and manage the assessment
  - how do we record and validate the process in a credible way. Are there any electronic options available to do this?
  - Re: assessing generic skills, posted by: David C at Mon, 24 May 2004, 13:28
    Lilian,
    I am interested in your initiative and also in the issues that you have raised.
    First, you have referred to the need to assess generic skills. I believe that this is an essential step, and to many an obvious one, but not everyone has agreed to it. In the past we have had reports that have expressed a preference that we should inform teachers and lecturers about generic skills, and leave it up to them to decide how best to incorporate generic skills into their programs. Recent history has shown that this does not work.
    The second issue is reflected in the decision that you have taken to create a generic skills passport. That is, you have recognised the need to ensure that students' achievements must be recognised and reported. Further, I assume that the passport is endorsed in some way by Swinburne, that is, it is given some authority and credence. For employers to recognise a claim that a graduate has achieved a level of generic skills ability, it seems essential that the judgment is endorsed by a credible authority.
    Your question to Rob about an electronic way to record progress also raises some important issues. First, your question goes to an issue of efficiency across your institution. But, the idea of an electronic record also does something quite important. It opens the possibility that learners/graduates/workers -- i.e. everyone -- see the need to have an evolving record of achievement. A printed certificate, such as the passport that you are developing and the Statement of Attainment that Rob is using, is an important symbol. It is a statement that at a certain time, a learner had reached a particular level. This is what all certificates, diplomas and degrees attest. But they imply something more. They suggest that the person who holds the award has the capacity to continue to develop. I think that this is a key message that we have to impart to learners. So I take a second implication from your search for an electronic solution to recording achievement. Could we build the electronic record, or produce from it, a digital resource that students can take with them and continue to develop? That is, can it become an electronic portfolio?
    You also raised an important question: How do you get everyone to do it?
    Can I suggest that it must be mandated, and this cannot happen only at the provider level. It must happen and be supported (mandated, funded, monitored) at the national level.
    Regards,
    David
  - Re: assessing generic skills, link: http://ddp.alverno.edu/, posted by: Rob Denton at Tue, 18 May 2004, 09:46
    Lilian
    You may be interested to explore the "Diagnostic Digital Portfolio" tool developed and used by Alverno. This is a web-based tool that enables students access to their generic skills and learning performance portfolio wherever and whenever they like - including after graduation I believe.
    Here is the web address: http://ddp.alverno.edu/
    You may wish to ask Glen about it!
    Regards Rob
  - Re: assessing generic skills, posted by: Rob Denton at Mon, 17 May 2004, 11:37
    Welcome to this forum Lilian. Great to hear of your active involvement in the generic skills challenge through your FLL. Yes, there are many issues involved and we can gain lots from discussing and sharing with one another BUT ultimately you have to customise it to YOUR particular context. Other people's solutions are often inspirational but often must be modified to suit your context. With that in mind I will offer an answer to your questions based on my experience in my context of a very flexible learning (FL) environment.
    (1) Our FL environment is designed for 'team teaching' or 'team facilitation' so it lends itself to all teaching staff taking an active role in the processes and management of Key Competencies development and assessment. I can see how it becomes more difficult where teachers work more independently in the teaching process to ensure that a range of teachers share the responsibility for developing and assessing KCs. A computer managed resulting and reporting system accessible to all teachers may help - because it can make it explicit what teachers and what courses ARE achieving KC development and assessment - and those that need further encouragement to take responsibility or require more PD or assistance to help them get it happening.
    (2) A lecturer in our Electronics & IT program developed a comprehensive computer based student management system custom-designed for our FL purposes to record all student results including technical competencies and Key Competencies. The program is called VSMART and is now a commercial product sold nationally. It has been designed to record all KC results and generate a portfolio of evidence that students can use for their own purposes such as job applications and interviews etc.
    If you have an interest to learn more about this program feel free to contact me directly (email or phone).
    Rob
    - Re: assessing generic skills, posted by: Lilian A at Mon, 17 May 2004, 15:35
      Hi Rob
      Thanks for the info. I would love to see these applications when I come to Adelaide. Sounds like it will meet some of the problems we have identified. Is VSMART a system that both students and teachers can log into? Is it a portfolio that students can take with them longer term?
      - Re: assessing generic skills, posted by: Rob Denton at Mon, 17 May 2004, 17:21
        Yes Lilian, VSMART is accessible by staff and students. It is an absolutely essential tool for management of our flexible learning program - for staff AND students. Without it we would have no idea of student attendance and progress because they are able to operate in a fully self-directed mode and attend the college whenever it suits them (anyway, that's another long story).
        Staff use VSMART to enter results or review records and progress for all our students. Students have access to their own record and can view their results and course details at any time.
        As for the Key Comps portfolio, it is a matrix showing how many Key Comps and what Performance Levels have been assessed and validated. It also comprises a log of all activities that the student has used throughout their course to demonstrate and assess their Key Competencies. It records the name of the activity, date, the particular Key Comp assessed, the Performance Level assessed, the number of successful assessments already achieved for that KC performance level (because the guidelines state that a KC at a particular performance level must be assessed on two different occasions for it to be formally recognised) and finally some comments from the assessor about the activity if appropriate. So it is a log of evidence indicating the range of activities throughout their course in which they have demonstrated Key Comps. This is just a text file and can be saved by the student for them to continue to build upon or to incorporate into their CV folder etc.
        Difficult to convey the whole story in a few words like this. Definitely something that is best seen in the flesh - which you can do when you visit us here in Adelaide.
        Regards
        Rob
LINKup Tour - 'DISCOVER', link: http://www.tvtafe.com.au/linkup, posted by: Rob Denton at Fri, 14 May 2004, 12:18
Welcome !!
If you have any interest in 'authentic learning' and 'generic skills' then I'm sure you will find something here to interest and engage you.
I hope you will feel comfortable to participate.

We are very privileged to be joined by Glen Rogers, a researcher from Alverno College, Milwaukee, USA - the ‘world leader’ in implementation and assessment of generic skills or what they call ‘Abilities’.
Alverno runs workshops annually (since 1976) for educators from around the globe based on their highly successful ‘Ability-based learning program’, so Glen is very familiar with answering questions just like YOURS, from educators just like YOU.
So don’t feel shy to take this unique opportunity to communicate with him.
Alverno website: http://www.alverno.edu/ (look for ‘Ability-based learning’ and ‘student assessment-as-learning’)

LET'S EXPLORE THE LINKup KEY COMPETENCIES WEBSITE...
Join others around the nation taking the 'LINKup Tour' to explore and discuss this new website and how it could help you.

LINKup is designed to:
(i) ‘inform’ you about Key Competencies (Employability Skills)
(ii) offer a ‘proven practical approach’ to assessing and developing these skills
(iii) give you a free ‘complete package’ of resources for this assessment strategy
(iv) help you ‘experience’ this learning strategy via an online simulation.

Proposed itinerary for this LINKup Tour:
May 17 18 Stage 1: DISCOVER
May 19 21 Stage 2: INVESTIGATE
May 22 25 Stage 3: TRY
May 26 28 Stage 4: REFLECT

NOW START THE TOUR…
Go to the LINKup website (www.tvtafe.com.au/linkup) and click on ‘Tour’ which takes you to Stage 1: 'DISCOVER'
We commence the tour by just 'breaking the ice' and getting to know a bit about Key Competencies, ourselves and the other people on this tour via the discussion forum.
Everyone ENJOYS this stage. It's really easy, engaging and entertaining.
It includes video clips, online learning styles evaluation and an outline of the 7 Key Competencies.

'DISCOVER' is designed to start you thinking and reflecting about Key Competencies and about yourself.
You will benefit from this stage even if you decide to go no further!
Go and DISCOVER! ... then return to this discussion forum to ask questions and have your say.

WEBSITE NAVIGATION…
Use the buttons at the BOTTOM OF EACH PAGE to continue the Tour.
The 4 circles at TOP LEFT of every page indicate what stage of the Tour you are in.

Please return and tell us what you think.
Thank you!
- Re: LINKup Tour - 'DISCOVER', posted by:Di P at Thu, 20 May 2004, 11:38
  Dear Rob
  
  i completed the online learning styles evaluation and agree with the results completely.
  
  as a practicing midwife and educator and qualified counselor
  i am very much a social learner and practical learner.
  
  i need to be around people and interact.
  
  i enjoy seeing learner develop and move on
  from public activities such as antenatal classes to more formal classroom environment. but i teach with a plan in mind but can also go with the flow as the class progresses
  
  thank you for the opportunity to do this assessment and look forward to interacting with you further
  
  regards
  
  Di Perkins
  - Re: LINKup Tour - 'DISCOVER', posted by: Rob Denton at Thu, 20 May 2004, 12:05
    Hello Di
    Thank you for your comments about the learning styles evaluation. You seem to have a clear perception of your personal preferences and I'm glad these were affirmed by the results of the evaluation.
    I would be interested to hear whether you think the Key Competencies that I have associated with these learning styles also match your preferences and strengths. I'm sure you probably have fairly strong competence in all the Key Competencies - but do you think you have particular strengths or preferences for the following Key Comps:
    Social Learner...
    Working with Others & in Teams
    Communicating Ideas & Information
    Practical Learner...
    Planning & Organising
    Using Mathematical Ideas & Techniques
    Using Technology (any technology - not specifically computer technology)
    
    Thanks!
    Rob
- Re: LINKup Tour - 'DISCOVER', posted by: Glen Rogers at Sat, 15 May 2004, 10:25
  Rob, one of the key ideas in the Discover Section of LINKUP is under "Key Competency and Performance Level." Sometimes competency frameworks only envision the skill or ability at the end of the program but not the steps for getting there. I am glad to see that articulated so clearly here.
  
  Competency frameworks also do not always envision the integration of skill and disciplinary (or technical) knowledge content. What is the relation between content and ability as it is validated in your context? How is required content knowledge selected, specified, and defined? Is it any different than in a traditional technical education curriculum? Glen Rogers
  - Re: LINKup Tour - 'DISCOVER', posted by: Trevor P at Tue, 18 May 2004, 12:51
    Hi Glen,
    I am one of Rob's colleagues and I refer to your second point to which I think the following relates:
    
    Last week I had an example of a student who correctly identified clear evidence of meeting the Key Competency assessment criteria although the technical content that he was basing his assessment on was weak.
    This (for me) was a first! Generally, if there is a problem with a self-assessment, the technical requirements are correct without doubt and any confusion is in addressing the evidence requirements for the Key Competency.
    In this case, as he had (weakly) met the technical requirements, I was able to validate his Key Competency assessment. Had he not met that technical requirement, then I would not have been able to validate his Key Competency self-assessment because it would have been outside of the context in it was made and therefore not relevant.
    My point is that the required technical content is specifically identified to meet requisite learning outcomes. The student then identifies how the knowledge/activity relates to the Key Competency self-assessment he/she is seeking to prove.
    Trevor Parrott
    - Re: Integration of content and ability, posted by: Glen Rogers at Wed, 19 May 2004, 14:50
      Trevor, thanks for the clarification and sharing your assessment experience. In my context, I find it helpful to encourage the integration of disciplinary knowledge and broader abilities in performance. This might be obvious in for an ability to effectively gather information, but also may have implications in VET for an ability like communication, where knowledge might, for example, need to be represented to a client in a way that they can comprehend. Do you see a deeper applied understanding of the technical knowledge when students demonstrate the generic personal abilities or is there no positive relationship?
      - Re: Integration of content and ability, posted by: Trevor P at Tue, 25 May 2004, 17:46
        Thanks for your response Glen. Sorry I've taken so long to get back, but got distracted to the mainstream.
        My answer to your question re "deeper applied understanding" etc is a resounding YES. I am firm in my belief that students who understand how and what they are doing with respect to the Key Competencies, CONSCIOUSLY apply the principles to their studies and IMPROVE in the process. It is a continuous development process that feeds the development of technical knowledge. Students exhibit an improved ability to more finely discern the essential knowledge required from their study and to be able to make connections across or between knowledge areas. In other words, they seem to more rapidly mature in their study habits and develop technically at a faster rate.
  - Re: LINKup Tour - 'DISCOVER', posted by: Rob Denton at Sat, 15 May 2004, 16:56
    Hello Glen
    Yes, integration of skills and technical knowledge is paramount to effective performance.
    Fortunately, the competency based training methodology of Vocational Education & Training (VET) in Australia is very much a 'performance-based' approach to learning rather than a 'knowledge-based' approach. There is a close relationship between skills and knowledge as specified in the Competency Performance Criteria within our 'Training Packages'. This is largely the result of strong industry involvement in the development of Training Packages providing a focus on student 'performance' rather than just 'knowledge'.
    
    HOWEVER, assessment and reporting is overwhelmingly focused on the technical component of a student's performance and NOT the Key Competencies (generic skills).
    This LINKup assessment strategy makes Key Competencies much more explicit - especially to the student. LINKup supports students through a process of deliberate reflection and self assessment of their Key Competency skills against developmental criteria - helping them better appreciate these valuable skills they have and accelerate their development. AND it offers formal recognition and reporting of these skills - helping students to secure employment and assisting employers in the recruitment process.
    Rob