Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Empfohlen
Empfohlen (20)
Online Assessment trough Peers
- 1. BY ERIKA POGORELC ONLINE ASSESMENT TROUGH PEERS Peer 2 Peer University
- 2. NEED A NEW DO?
- 5. There Is More than One Path to an Answer Model Concepts of Quality and Community Norms Supporting Learners Continued Growth and Evolution
- 6. Encouraging Conversations and Building Social Presence Built in Reflection and Self Assessment
- 10. YOUR MESSAGE
Hinweis der Redaktion
- Hi I am Erika, my occupation is a software engineer and I am working with Peer 2 Peer University as a developer of the software, which helps people teach and learn in a peer to peer fashion. I like to start by sharing my story of how I went from a hairdresser to a software engineer. After finishing primary school I enrolled in a hairdressing school. And when I tell this to people they often try to subtly hint that they need a new haircut. Don’t get me wrong I am honored by them wanting to trust me with their hair, but honestly I would never allow me to cut my hair if I was them. I am just not very good at it.After I finished school, I have never really worked as a hairdresser, so before I enrolled in the University at my “tender age of 26”, I had been trying on a few professions, such as assistant to accountant (my mum) I even explored the option of helping young addicts back to the right path as an assistant to psychologists and social workers in a non profit. But wherever I worked, I had always been the one who got under the desks, to fix mysterious computer glitches in the office and (re)install that elusive software, because I have a big interest in technology over all, and computers are really an interesting devices, that can do plenty of stuff, especially if you know how to “talk with them”. So I thought to myself, I might as well go and learn about this properly. And this was back when the internet was not very accessible, so my only option really is to go and study as a full time student at the university. And I did.
- I vividly remember my first lecture at the Faculty of Informatics and Computer Science here in Ljubljana, as I was so proud to be a part of it, but my peers were 7-8 years younger than I was and most of them had a pretty good foundation to learn about computers, since they mostly came out of computer secondary schools. I was really the only one in my generation who came from a hairdressing school. This experience was to me almost like I walked to some sort of star trek movie set or something, a lot of “layered architectures”, “linear recursive equations” and “b trees”. I remember at one point even asking myself, do I even speak the same language as them. The jargon was just so foreign to me. But I was thought in my life to stuck with hard things and this one was pretty up there, but I did stick with it.So with time my anxiety has decreased and being who I am, I immediately started to forge ties with some of my then peers, but now the more appropriate term would be a lifelong friends. In the end they turned out to be the lifeline to my graduation.Even within this traditional atmosphere of higher education, there was a massive amount of peer learning involved for us. Through all of the study groups we had organised during all those long nights that we studied together, we did in fact learned from each other, not only the subject in question but we gained soft skills, for example how to communicate with each other, especially when there is a difference of opinion, and performing tasks as a team. These things are invaluable to know now, in our professional lives. Every exam was a project on it’s own, a project we did together. And with that experience, I believe, with all my heart, that this is how we all, all of us, learn, improve and better our lives. This is how we get that deep lifelong knowledge.
- Nowadays when learning materials are so much more accessible, people are solving big problems more easily than ever and this results in all of us being able to start and finish very complex projects. The learning curve for most of these projects is growing with time of involvement in them. Interaction with peers that provides us with conversations and feedback is giving us a better insight in the way we are learning.For example in my field of software development , it is a fact of life that there is almost never “one right way” to solve a particular problem, so learning how to solve problems is valued the most. Getting a feedback at the end of the learning cycle can be particularly damaging to the whole working process, and can result in some very poor practices. Early feedback coming from an online community is very valuable to me as engineer and can be beneficial to end-users of systems that I help build. With that realization, our community had developed a lot of endpoints, where feedback can be received or given, for instance websites like Stack Overflow and Quora where we are asking and answering questions and are learning from each other along the way.
- With this approach I realize that assessment also needs to shift. And what better way of getting recognition and credentials for the newly gained skills than through peer-to-peer assessment.For the knowledge we as people are trying to gain it is very beneficial to let community decide what is acceptable. Getting feedback from of the community on your own project, provides with good quality and lifelong learning experience.The actual coreskill in such a community is feedback, weather it means a coment on a post, resolving an inquiry or suggestion on a project.Communities that are considered successful in this type of learning share some of the similar principles:Recognition there is more than one path to the answer:Traditionally assessment is very binary 0 or 1, but working on a complex projects where learning is iterative understanding only grows in time. For example, urban planner with a task to build a new park could find many solutions for the problem. optimizing for greenery, or social interaction, maybe ecological impact and all of those solutions are relevant. The fact is that there is no one right answer to the problems people face on a daily basis. Model concepts of quality and community norms:Usually Learning communities are self managed, so they determine acceptable approaches and practices on their own. Community moderators and facilitators are shining a light on things that could be perceived as quality and beneficial to the members within the community but more often than not people who gain a status of an “expert”, are a good trust source as well if not even better, to show the level of trust and share their opinion about stuff.Supporting learners continued growth and evolution:For a member to become this kind of trust source starts with a simple place like e-portfolio which shows growth of knowledge of a member through time, as well as their ability to gather and share resources available to them. Progressively members get more and more visible within the community, and so expectation to take more responsibility and give more quality feedback is greater through time as well. It is such members that usually maintain the norms of the community. They are seen as experts by others.Encouraging conversations and building social presence:Another principle that emerged is fostering deep conversations among community members which is an action to improve the quality of community projects and knowledge of their members. For example:You decide to make your own “hippie” pants, but you have never done this before. You find some instructions on the internet and even ask for some directions within a community that has exemplary expertise of tailoring and sewing a simple pants together. You would present your brand new pants to them and their feedback will likely teach you two things:one, making your next pants more stitch perfect, maybe even adding some pocket or a zipperand two it would model for you how to participate within this community, the very next day, you are the one to give instructions to another person who wants to make hippie pants.Built in reflection and self assessment:Another thing that proves to be very beneficial for the community as it evolves and grows, is to ask members to diagnose their past work, to reflect upon it and self assess. Thinking about the learning process during the development of the project means gaining skills to use this knowledge in future projects, for themselves and other members.
- Encouraging conversations and building social presence:Another principle that emerged is fostering deep conversations among community members which is an action to improve the quality of community projects and knowledge of their members. For example:You decide to make your own “hippie” pants, but you have never done this before. You find some instructions on the internet and even ask for some directions within a community that has exemplary expertise of tailoring and sewing a simple pants together. You would present your brand new pants to them and their feedback will likely teach you two things:one, making your next pants more stitch perfect, maybe even adding some pocket or a zipperand two it would model for you how to participate within this community, the very next day, you are the one to give instructions to another person who wants to make hippie pants.Built in reflection and self assessment:Another thing that proves to be very beneficial for the community as it evolves and grows, is to ask members to diagnose their past work, to reflect upon it and self assess. Thinking about the learning process during the development of the project means gaining skills to use this knowledge in future projects, for themselves and other members.All of this is great, but real problem with all of this is to make newly gained skills recognized outside of a community. The biggest challenge in fact is the to make them visible to the formal institutions and broader career ecosystems, since peer learning is mostly recognized as a support to a formal education. Reality is that most formal system does not recognize this skills simply because they are not accredited and sadly because of this person's whole story about their skills is lost.
- One of the offered solution to this problem that we at Peer 2 Peer University, looked into are digital badges. They can be displayed as credentials on learners e-portfolio, and help make the learners story more whole.The model of the badges was taken from the Boy and Girl Scouts groups, where a certain badge is awarded for the correlating achievement. In this way members are recognized for their strong points.Transferring this to an online learning community, displaying of achievements in this way could be presented to the key stakeholders as a demonstration of individuals capacity. In this way badges are playing crucial role in making learned skills more pragmatic and mobile, thus more impactful. Badges can be rewarded for a potentially limitless skillset of an individual regardless of where each skill was developed, and they can practically serve as an virtual resume of competencies and qualities for potential employer, schools or any other stakeholders.We are also realizing how web is affecting the way people learn (learning is more incremental, feedback based), and that there is a need of offering an alternative to the traditional assessment and credentials. We also wanted to bring some examples of how these ideas could work in action, so we went to action.
- We built this kind of system that would enable it’s users to create, give and earn badges. An individual's portfolio of badges would display their constant growth and evolution of of their knowledge. Saying that, we do realize that badges are not a new concept in online learning. Even we had used badges before in our online learning platform, which were automated. So user got a badge as a “reward” automatically after finishing a task successfully, and the only feedback they received was the badge. But with this project we wanted to move badges away from that kind of gamification, and for that reason we put a lot of emphasis on feedback. Feedback in a form of writing down a few points that stand out in a project must be given before the badge can even be awarded. There is no automatization of badges anymore and we like to believe that achievements that got badges awarded to them, were a non-trivial ones, and had a great deal of work put into them. We believe that this type of badges are much more credible as credentials.Because we wanted to move away from gamification, we did not use any of the existing systems for badges as assessment, but we built our own from scratch. We have not coupled it with our online platform, so projects that get assessed by them do not need to be involved with P2PU at all. This is exploratory/learning project, so our first priority was to make design of this system qualitative, learner-centred, project-based, but mostly feedback driven. And as such it is an alternative to some of existing more gamified and automatic systems out there. We were already able to change some peoples minds about badges, by walking them through this different approach that we took.So, central idea of our system is to allow anyone to make a badge and therefore they are the ones who decide on the requirements and criteria for the badge, and how should understanding of these requirements be interpreted.Badges created by learners are intended to develop communities around projects, and to develop relationships based on giving feedback.So to sum up the core design principles of this system:we had been optimizing for meaningful feedback, encouraging iteration and improvement of submitted projects and the most important thing was to nurture mentorship between learners.
- With these principles in mind we started to carve out the foundation for the system. First we drafted the user experience mockup and presented that to our own online learning community. Their feedback helped steer the development all the way, but throughout of development we did stuck to three we believe to be most important things for building such a system:We were always building in short iterations, a small pilots and then testing them with the help of who we believed to be potential users, which would provide us with early and valuable feedback, upon which we decided on the next iteration of development. With this we avoided long and tedious development that is in the end not satisfying the users needs fully, because of assumptions of what would be needed instead of the real feedback that we got.Recruiting and sustaining small but vibrant badge communities even during the development was the key. It turns out that the power users of the system are best marketers.We also tried to reach out to more traditional folks to help make badges count.This system went out of its beta state and was first presented at the DML (Digital media and learning) conference last year in Chicago. And I am proud to say that from then we have recorded 407 projects that applied for 280 badges. With this dataset we are observing who are the main beneficiaries of the system and in what way are they benefiting from it. Until now the conclusion is that the main group of users are organizers of informal learning groups, such as School of Open, all sorts of MOOC organizers (ConnectedLearningMOOC, DLMOOC,...), and even creators of project #walkmyworld. This was a bit unexpected at first, because we initially thought that anyone might navigate to our system and find a badge they qualify for. And with the initial badges, this was in fact the case, but it very quickly turned into more learning groups assessment tool. The reason for that could be that the discoverability of badges is not really implemented as this is the feature that is still waiting in the backlog.Another finding emerged with our observation of the system, that is the fact that these organizers of informal learning groups, had to quite often buy into peer-based, learner-centered approach. Before they would integrate these badges in to their learning design, many times organizers struggled with the idea of sharing their “expertise” among learners. And many of them struggled to perceive a badge as a tool paired with assessment. Only after we have implemented a feature of being able to send ones badge to Mozilla’s backpack open badges system, these badges gained the credibility as they could be showcased in more ways than one. And it turned out that users give each other meaningful feedback regardless of them not being creator of the badge. When they earn it, they often perceive it as their own, so they are not prepared to just “give it away”.Like I mentioned before, this is mostly experimental project, and while our assessment experiences are evolving there is still some issues we want to address. For example we are still unclear about how exactly we should approach expertise in an online learning community, since the term “expert” implies hierarchy.Another example of possible improvement of the system that we concluded would be beneficial for users is to enable a sample project for the creator of the badge. Right now the person who creates a badge is automatically an expert, but that might not always be the case. So opting for a sample project would really be ideal.So in other words we are still thinking about how to treat those who have more experience in an equal way than to novices or professional experts.And in addition to this, users has also expressed that they would like to be able to create the badge image within the system, because right now they have to have the image prepared and are only able to upload it when they are creating a badge. In addition the implementation of LRMI is also still in the backlog.
- But thinking about it constantly we know we will get there, because we believe the start is right. Everyday there are more badges created, and even if we were at first expecting too much too quickly, with this project we learned that the patience is the key for a change.
- I invite everyone to take a look at the system, try it out in your learning experience and share your thoughts with us in our community place.
- Thank you!