I had hoped to get this post out last week, but the dissaggregation 2 post came out instead and here I am in the middle of week six trying to combine a post that addresses both evaluation and success… and then i realized… that kinda makes sense. The problem with creating an evaluation model for a PLE is that it will inevitably have a strong impact on the success of the PLE. If the PLE is essentially about emancipation (which Scott Leslie tells me everyone believes in the comments of the previous blog post) then the scaffolding applied to allow for evaluation seems like the other end of the counterbalance.
This post is as much a simple reflection on my own practice… I hope you find it useful.
Presenting The Challenge
In the passed several blog posts we’ve talked about how the PLE can contribute to people committing to learning in a different way. To learning as practice… as a side effect of the work that they do everyday. In a PLE modelled course all the work that you do can contribute to the overall work that you produce, it can interact with work that you’ve done before and, most importantly, it can connect with other people doing great work out there as well. Those connections between your work and the work of others is where the real magic, where the network in the uh… network… really starts to pay dividends.
But wither the facilitator… It’s all fine for people to go out and do ‘real’ work on the internet, but how am I supposed to know that they’ve done it. Sure they can tag it… but what about comments? Sure they can repost in their blog… but that takes it out of its normal context. How do you evaluate work done in a PLE? How do you make sure that you are out there connecting and helping students make connections? How do you provide enough scaffolding to ensure that ‘nervous’ or ‘resistant’ students are coming along? How do I help provide comments? How do i find everything? I mean… in my Moodle course I just press on someone’s name and i find all their work.
Building the perfect system
So… off I went to try and build the perfect system. I started years ago using html, moved to moodle, elgg and drupal (all content management systems of one kind or another).
phase 1. I taught my first online classes on invisionboard, a discussion board software. I had 200+ writing students all working collaboratively in a discussion forum, editing writing, talking shop… it worked pretty well. But I wanted more! I had tried to cobble invisionboard together with a variety of other pieces of software, you know, calendar, assignment tracking… and then I found Moodle.
phase 2. Oh wow, I thought, here’s a system with all of the things I want in the same place. They had forums, and wikis and assignments and a gradebook. Piles and piles of toys. Except all those things were in the same place, and I started finding things constrictive… Where were my students going to put their content when the course ended… I had students coming up to me wanting to continue the conversation. Did i just delete these courses where people were feeling like they were connecting with each other when they were done?
phase 3. And then I found ELGG. It lets people have their own identity. They could blog, they could… uh… manage files. Lots of cool stuff. You, yourself, can live in your own location, you can write to whom you would like, be open or not. Seemed kind of ideal really. But it was kinda hard to use, and people got lost in trying to make connections, they kept expecting the connections to be there… and they weren’t. At the time (elgg has changed a bit since then) there was NO real way to start in ELGG. You couldn’t easily just ‘go to the home page’ and this made people crazy.
phase 4. By the summer of 2008 I thought i had it all sorted out. I would build a system out of drupal, loosely modeled after the interesting work that Funnymonkey had done for/with us for the http://livingarchives.ca project. It had a blog, that could be public or private. It had a very simple organization, by date, of the people in your group who had written reflections on their blog, reverse sorted to allow you to comment on those people who hadn’t been commented on yet. It was all in one place, I had feeds coming out of it in a dozen ways. The other main piece of functionality is that I had an ‘almost-wiki’ where students collaboratively built a textbook from the course. http://edugrids.org. It’s still there. No really… I still do the updates on it. I still have it.
so…
There are others I haven’t mentioned, I made some detours down the wikis paths, and down some other ways. Suffice it to say that I’ve tried a few things on my way to trying to set one of these things up. Here’s what I’ve learned.
Building from scratch – writing your own html is annoying
Using a VLE – moodle (blackboard etc…) a blackbox, no matter how you open it up, its still designed to be top down, it’s not for empowerment, it’s designed to keep work tidy.
Using ‘social software’ – While some of the projects we ran with ELGG (youthvoices comes to mind, now transferred to drupal) were successful, it was difficult for people to get started in it. It was committing to a new country without knowing if you wanted to be there… tough to make that ‘yours’
Building your own – You have to own it. and when you own it… it doesn’t belong to your students.
Overall – These things are about the teacher. Not the students. I don’t know of a way to design a system that changes this.
Why I gave up trying to build a system
So then this summer I looked back at the 2008 course site i was still managing and said NO! I’m NOT doing it again. Everytime i throw away a system, a webpage, a moodle course or something like that I’m breaking up a community (even though the students have likely long left) that I tried to make my students believe was important to build. I’m breaking connections that others may have to my students work… The whole thing works against my philosophy. So i don’t do it anymore.
The internet is my PLE
In the course I taught this summer, I had the students go out into the wild with their work. I asked them to set up a blog on wordpress.com. I asked them to get a twitter account. I told them that they would need to keep track of each and everything that they wanted me to evaluate and put it in a googledoc. I said “I”m not going to look for it, I want you to interact with people, keep track of it, and tell me why it was important.” You can see the full plan here… http://wikieducator.org/User:Davecormier/Books/Educational_Technology_and_the_Adult_Learner I wont belabour the point… those interested in such things can wander over there and take a look at it. I stopped building systems, and I’m happy about it.
So… what can this mean for evaluation
There’s no way of knowing whether my summer course will replicate itself, but i had the most satisfying evaluation experience of my professional career. My students each handed me in VERY different pieces to evaluate, each reflecting their own style and their needs after the course. They all did 10 or so blog reflections, which fit to a format. They each sent me in a ‘reflection and collaboration report’ which included a grade for them posting in comments they had made and explaining how those comments for formative in that context. They also handed in a “learning network plan” which included things like links they hadn’t followed, things they found challenging they might go back to, interesting people they could follow… things like that. I really enjoyed going through them.
So what can this mean for whatever success might mean
My goal for their PLEs was that they should be as individual and as focused on their own context as possible. I also wanted to make sure that the content was in a format that they would be able to use after the course as easily as possible. Their blog posts are still out there (though some people may argue that this isn’t necessarily great) but in collecting and choosing how to organize all their work, they each seemed to create something that was personalized enough that they could see it as a valuable go forward reference if nothing else. Maybe, for some of them, it was the start of getting networked enough to be able to carry on with the work on line in as professional way as possible.
So your PLE isn’t “the web”, it’s a personal selection of tools *from* the wild world wide web (and, I bet, a bunch of non-web based things, digital and not) that you could (and have) share with others who want to build their own. You could, if you wanted, even draw out a graphic representation to help people who aren’t familiar with your particular selection understand their interactions…
Which is exactly what I’ve been preaching for a while, preaching that has brought me no end of grief for reasons I still don’t understand.
But I’m with you. I think.
Dave,
I love your methods here, and I am going to steal that whole idea of the students selecting their own, best work and make a “portfolio” of it, or a new site. very cool, thanks for this.
And what’s more, is I love Chris Lott, and I am beginning to love both is no incompatible. These are strange days 🙂
@chris
I don’t particularly care which tools, just which kind, although i guess that is still a personal selection. I think I’m with you as well, and I’m very happy to have the return of the ‘omg chris has commented on my post i wonder what he’s going to say’ feeling that has been missing up to recently. The goal, eventually, is to not care at all what tools they are using, as long as they collect their work together to send it in to me for ‘looking at’.
@trackback
I’m not sure what to think about BIM yet, I’ll keep looking… i’m not against VLEs automagically, i just find they don’t work for most of my uses.
@Jim
I’m not sure if I’m being made fun of or not… so I’m going to guess that I am. Yes, the idea of portfolio is not new at all, for me the realization was that there was no sense in me trying to create it programatically, that it made far more sense, and did far more from a literacy perspective, to have them make it up themselves. That… and it’s easier and doesn’t leave me holding websites that no one uses anymore 😛
Dave,
To be clear, I was not making fun at ll. I love the self-selection idea for a portfolio (and whether it is new or not doesn’t matter, cause it struck me as new when I read it here). I must be losing all control over my tone on the web, and that scares me.
@Jim,
I don’t think it was your tone, it was more that in a sentence you wrapped up what i was trying to say in 1300 words, and made me think – “hey, maybe it is that simple”. That and I’m paranoid 🙂
Also Dave,
if I could freakin’ write it might make it easier, what I meant to say was:
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
One day i will become one with the keyboard.
@Dave as long as you stop squealing like a little girl every time I step out of my Ed Tech Stretch Limo…
I think the key word in your last bit is “eventually” because the “eventually” applies to the learners too, whether they are literal students or faculty or what have you (I belong to the what have you category, I think). There are choices being made at every stage… it’s just a lot harder to make a meaningful choice when you have no idea of the qualities or benefits or drawbacks or even the basic function (or functionality) of what you are choosing from.
Which is why I don’t understand the resistance I’ve seen from various quarters to providing some elementary help, whether it be a kind of map or examples for wayfinding to help learners feel confident in exploring and finding their own way through what can be a bewildering thicket. The best way to learn this stuff is to learn over the shoulder of others, to get those aha! moments when you think to yourself “oh, *that’s* how they did that” or “ah, those X things can actually be woven together,” etc.
And I can’t see that it hurts to help others understand how the pieces and parts we’ve chosen at any particular moment interact with one another, sometimes to enrich, sometimes to aid in efficiency, and in the most awesome confluence, both. Which is why what I refer to as the PLE isn’t anything like the LMS (or the VLE as a I understand the term in use) and why when I talk about it I talk about it in terms of helping people do things they want to do and examples of how others are doing those things.
I do think there are deep complexities and subtleties when it comes to providing patterns and practices in scripted ways and the way those can lead to more power for learners, just as there are counter-intuitive paths to freedom in providing practice through artificial constraints, but I’ve droned on long enough.
I want learners to be able to send stuff to whoever they want, when they want, and retain ownership, pride, and the intellectual currency of attention. Giving them means to both create their own ad-hoc portfolio as a part of their learning and the tools to pull pieces together into more literal portfolios as they want and need to.
*squeal*
I’ve been resistant to the idea for a while now, and i think it is the complexities you speak of that got in my way… i needed (just checked) about 8000 words of blog posts in order to work out why i was concerned by the concept and how i felt about it. I’ve been taking long looks at tony hirst’s work on mapping looking for ways to produce some of the ways of helping people understand that you are referring to… I’m also going to steal your last paragraph, which i think sums things up nicely (better than that ‘Jim’ fellow).
*/squeal*
@Chris,
it totally agree with you in this, andactually cited you a few times when this argument flared up again recently. I’m not sure why some folks seem to get hemorrhoids every time the acronym PLE is mentioned, but I see its uses, as well as its limitations. And the way you frame its uses here, makes it a pretty damn good way to frame these various tools and spaces one can engage in online. But you have said all this before, what I really want to know id why you didn’t react to my statement of love for you? I’m deeply hurt, my PLE is in my stomach right now.
@Jim the response I meant to post was a witty observation on the Bava being able to love the one he wants AND the one he’s with, but then I got caught up trying to understand the aporia of Stephen Stills’ lyrics and then I think I was enlightened because I woke up face-down on my keyboard which had made an impression on my forehead that looked JUST LIKE JIM GROOM!
@Chris,
That is true love, thank you.
@cormier,
Watch yourself, the bava knows.
Excellent article. I think it is a perfect example of migration from the PLE to the PLN. It is essentially two approaches: the centralization and the Internet. But is it always justified is the use of students of public services? For example, when a student is studying several courses, he will need to adapt to the different PLE-teachers and vice versa.
Dave,
I would very much like to do this and evangelize for it where I am. However……
…I’m still trying to understand how this approach, which lends itself to reflection and big picture thinking, works with rudimentary material.
As an instructor I teach introductory French. I probably teach my students the same sorts of things Oscar is learning now, and really struggle with how an individual PLE functions when the typical piece of output in the first term is (with a handful of exceptions) a few sentences long.
I’d really like to find out your thoughts on this.
Dave, this seems exactly right – asking people to self-select what it is they want to best represent what they’ve learned rather than giving them a predefined target of what that looks like seems the logical correlative to the network learning apporach.
The issues that of course raise their hoary heads mostly come from the formal learning side – how is it “fair” if people are assessed on different types of submissions, and, as you’ve maybe encountered, the real investment of time it takes to authentically engage with a mass of different submissions as *individual* pieces.
Neither of these seem like legitimate reasons for giving up on the idea, but for this to get more purchase in institutional settings (if indeed that is what we want – not clear that is so) there needs to be some recognition of the challenges, why they are worthwhile, and what can be done to ameliorate them.
Anyways, I apologize if my last set of comments felt dismissive; I think what is obvious to some is not to all, and I too often assume everyone is at the same point in the conversation. You were posting partly in the context of a course that has people at all sorts of places in their understanding, bringing all sorts of different contexts, and in that regards it was an important distinction you were making.
Cheers, Scott
@scott I post these blog posts for my own learning as much as anything else. Some people are looking at the distinctions and learning from them and others, like you, force me to constantly think a little better, a little deeper and more effectively. I’m always happy to see your name show up and get your feedback. I know you well enough to understand how to take it.
the oftener you return the better i’ll be pleased 🙂
You make a good point about the challenges involved. I need to keep better track of what they were for me and start soliciting others experience (not my strong point) to try to make a more coherent argument. I think I’m going to pull together my PLENK posts into an article of somekind following the course and hopefully that reflection time (whenever that will be :P) will give me a chance to make a rounded argument.
YES (I think) but may need a ball of virtual string to find my out. Going to share this with my online self-paced ESL study group ~ personal learning network from customized plan for self-study is an obvious step. They get this, more or less. Active collaboration and sharing is slower coming
Thanks for sharing this reflection on your practice Dave. I’ve been wrestling with this for some time now.
In K-12, I’ve used various systems to track student’s progress over time (Hyperstudio / First Class / html / Powerpoint / Google docs). That work around portfolios as a way of documenting learning over time led me to Concordia’s ePearl http://grover.concordia.ca/epearl/promo/en/epearl.php
I think a system like ePearl or an opensource tool like Mahara is still appropriate in K-12. That, of course, doesn’t preclude students opting to set up other accounts, but I’d want to make sure there was a measure of privacy and equity of access for my students.
Now that I’m working with practicing teachers in the graduate diploma program, I’m rethinking what makes sense. I’d like to use Mahara because we have to maintain confidentiality around their field work. There is an ethical consideration to posting fieldwork. My student’s learning statements draw evidence from these field studies, so their portfolios have to remain private. I would like them to be able to collaborate with each other and like the ability mahara gives them to share data with specific users. But the conversation doesn’t end there. I need them to speak with a wider audience of educators. I need them to share their reflections as you have done here, without referencing the students they work with but sharing what they have learned from their inquiry. So far, I’ve asked them to create accounts on WordPress and we’ve been exploring Google and delicious. We’ll work our way up to twitter 🙂
Thanks for making me think Dave. Always enjoy reading your posts.
Tom