After Cheggification – A way forward (Part 1)

In 2015 I had a random conversation with a representative of the ‘school help’ site Brainly at a conference. The company had 50 or so million (now 350 million) registrations and I was told that any student could go on the website and ‘exchange answers’ to questions.

You could call it cheating. Students going online, pasting their questions into a search bar and finding an answer from another student, somewhere else in the world. Or you can see it as part of a broader systemic challenge that comes with the Internet. (or both) One way or the other, they were getting answers to classroom assignments from the Internet.

This week, Michael Feldstein wrote an excellent piece about how Chegg, another version of this answer exchange, has not only setup ways for people to find answers together, they actually bought some of those answers from the textbook companies.

The students not only (literally it seems) have the teacher’s copy of the textbook they have the means of getting answers to anything else you throw at them. My students tell me that the answers to questions on exams can sometimes be posted on services like Chegg minutes after the exam has started. These sites aren’t magical. You can get ‘wrong’ answers on Chegg. There are also times when getting answers to those questions take a little longer, some posted questions from students go unanswered… it’s not a perfect system. But it does work. If a student is working with access to a computer, and your question has a correct answer, they can find it.

So I’ve been asking students and faculty what the implications are…

The faculty response

Throughout the pandemic, I’ve trained and worked with 60 or so students who were hired to support faculty and students. It’s been a few years since I worked with University students in this way and I’d forgotten how straightforward and honest the relationship is. We’ve talked about how they are living through the pandemic, about how their friends are doing and, increasing as their grasp of education design increases, how education has been changing in the last 12 months. Interestingly, their perspectives match up almost perfectly with the conversations I’ve been having with clients and colleagues from around the world – except for the students, it’s more personal.

You see, faculty know that Chegg & friends are out there. Moving their courses online (thereby removing in class exams) has removed the only secure check or balance they had that could guarantee that Chegg wasn’t doing assignments for their students. They are responding, and it’s not making things better for anyone. I’m going to leave virtual proctoring out of these responses as that is primarily a school based response, and is a terrible solution.

Response 1 – Make the exams harder

The most common response that I’ve heard is to make the assignments and exams themselves harder. Whether by asking more questions, giving less time to answer the questions, making the questions themselves more difficult, or some combination of the three, the idea is to try to make the event of the test so challenging that students wont have time to cheat. It does make one wonder… if last year’s exam was fair, and this year’s exam is harder… doesn’t that make this year’s exam unfair?

Response 2 – Entrapment

There are a variety of ways this plays out. Some faculty are lurking on Chegg trying to see when their assignments/tests get posted. Some are making very small changes in their questions from pervious years and calling out the students using previous years answers. Lots of ways to do this. Doesn’t necessarily make for friendly feelings all around.

Response 3 – open/take home exams and assignments replacing high stakes exams

Some faculty are taking a different route. They are saying ‘hey, i know you have the exams answers, I’m going to create these really involved exams and give you 24 hours to do them.’ Alternately, they are throwing the exams out the window entirely and giving students assignments and projects to replace those exams.

So what’s the problem?

Well. The first problem is that each of these solutions makes the course experience more difficult for the student. The student experience of education is not, usually, a one course experience. It is a cumulative experience from across 4-6 courses. Options 1 (make exams harder) and 3 (make more assignments) increase the stress and workload on students, leaving them with less time to do all their work. Less time leads them to, you guessed it, look for help from places like Chegg. Faculty are creating a scenario that is pushing students towards the very solution that they are trying to design themselves out of. Cheggification leads to more cheggification.

I’m also worried about how combative the whole thing seems to be getting. Faculty feel like the core social contract of the education system is being betrayed by students. I’ve spoken to long-tenured, student loving faculty members who are getting really frustrated. I’ve also spoken to students who I firmly believe to be students who are trying to do the right thing who suggest the ONLY way to get the grade in the current situation is to get answers from Chegg. It’s not healthy all around.

Well-structured/ill-structured problems

Underlying this conversation is the idea of there being a right answer to a question. In order for Chegg to work as a cheating tool (rather than, say, a tutoring tool) the assignment that the students is looking up has to have an answer that is ‘correct’ that they can ‘find’. And you might say ‘yes, dave, that’s what a test is’. Ok. I hear you. But it doesn’t ‘need’ to be that way. There’s a distinction in cognitive education research between a well-structured problem and an ill-structured (or ill-defined) problem that tells a story that I think is interesting. (for long tenured readers of this blog, it is similar to Cynefin’s complicated/complex)

A well-structured problem is what Chi and Glasser call a classroom problem. That’s a problem with a clearly defined question, clear inputs and a well-defined answer. Think of any math problem, any definition you’ve ever been asked to remember, or even a writing piece with a super-well defined rubric. They are problems that have been created with the express intent of teaching someone something. They are, necessarily, artificial.

An ill-structured problem is what Spiro might call a real-world problem. I kind of like Herbert Simon’s definition of an ill-structured problem, “all the things that aren’t well-structured”. If it’s hard to formulate the question, or the inputs could be different for different people, or there isn’t only one answer (or maybe it doesn’t even make sense for there to be an answer) you’ve got yourself an ill-structured problem. If it helps, read the excellent 1973 Dilemmas in a general theory of planning by Rittel and Webber… they know an ill-structured problem when they see one.

Why do we want well-structured questions?

So, if you have well-structured questions, students are going to use Chegg (or something like it) to find a way to answer those questions without ‘doing the work’ that you want them to do. If you try and fight against it, you make the situation worse. So. That seems easy. Lets just get rid of them. Ill-structured questions For The Win.

Well… maybe not so fast. Maybe it was William Farish who started this whole ‘lets assign a grade for right answers’ business, and maybe it wasn’t, but we’ve sure been doing it for a long time. We might want to consider what we’d be losing by getting rid of well-structured problems. I mean. They’re also called classroom problems. What do we lose if we get rid of them?

A certain kind of equalness

I didn’t say equity and I didn’t say fairness, because i don’t think it’s either, but at least with a well-structured question it’s easy to grade. You either chose (b) or you didn’t. You either did this thing that i specifically laid out in the rubric or you didn’t. It ends a lot of arguments. I know I’ve talked to a ton of high school teachers and even some university professors over the years who’ve moved to this approach just to settle arguments with parents calling for higher grades for little Johnny.

Maybe it’s better for novices?

The research I’ve seen on this is mixed (like all educational research). There are some suggestions that giving people simple answers to things like ‘how is that cell constructed or how do you write a good essay’ is a good way to start people on the learning journey. Novice learners tend to look to the superficial answer, and maybe this helps them develop some basic language or something, that makes it easier for them to dig in later on? (more on this in a future post) I have also seen it argued that it sets people on the wrong path to believing that there ARE simple answers to questions… something they need to break as they move towards being experts.

It’s easier for grading

I mean. Obviously. grading for multiple choice or with a clear rubric is way faster.

Why should we be using ill-structured problems?

Well. that’s going to be the next post. They are ‘real-life’ problems, and, well, it does seem like learning how to deal with real-life might be useful. The big issue though, is that our entire educational design and planning system is designed to work on well-structured problems. There are HUGE implications to change it. Lots more to discuss.

Coupla citations.

Chi, M. T. H., & Glasser, R. (1985). Problem solving ability. In Human abilities: An information-processing approach (pp. 227–257). W. H. Freeman & Co.
Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169. https://doi.org/10.1007/BF01405730
Simon, H. A. (1973). The Structure of Ill Structured Problems. Artificial Intelligence, 21.
Spiro, R. J., Feltovich, P. J., Feltovich, P. L., Jacobson, M. J., & Coulson, R. L. (1991). Cognitive Flexibility, Constructivism, and Hypertext: Random Access Instruction for Advanced Knowledge Acquisition in Ill-Structured Domains. Educational Technology, 31(5), 24–33. http://www.jstor.org/stable/44427517

What do I actually want from my PhD?

In all the 2020ishness (i don’t have another word for it) of the last 12 months i’ve not really put a lot of thought into this question. I finally decided to jump into a PhD program after several attempts because, well, this was the easiest way to do it. Foreign residency at faraway institutions sounded great, but the logistics of kids and jobs and whatever just never really made them possible. And the cost. The cost of this particular run is mostly covered by my institution. So, the answer to the first and easiest part of this question “why now?” is answered -> because it was easy.

But that is not really an answer to the question that my supervisors are interested in. My relationship with them has been excellent so far while I’ve taken the first two mandatory courses of the program. They’ve been supportive and hands off. Encouraging, but not in such a way that would complicate my position in the courses I was actually taking. So far so good. But we’re getting into the meat of the thing now. The place where we actually have to come to some kind of agreement about where this thing is going. And, given that they are committing their time to this process, i don’t think it’s an unfair question. What do I want to get out of the PhD?

As my supervisors are going to read this: you guys have been great. This post is not directed ‘at’ either of you. It’s just me thinking about what I want (and occasionally, don’t want).

A little preamble

A little preamble here, in case people coming across this post are not long-time (read: patient) readers. I’ve had some opportunities to participate in the conversation about education in a few different contexts. I had a string of a couple of years where I was very fortunate as to the time and place where I showed up, and managed to do a few things that gathered a little notoriety (like, in edtech circles, not like, in regular places). Through the people I met and the communities I worked in I managed to learn how to write articles, to do research and participate in the conversations about education that I thought were important.

I’ve also had some jobs (both job-type-jobs and as a consultant) in community colleges, k12 and university that have given me access to the other side of education. The planning, the policy and the strategy side of education have been a particular study of mine. As I am fond of telling my students (i’m not sure they’re fond of hearing it, but I do like the sound of my own voice) I have made mistakes all over education.

So. I’ve done some speaking. Lots of teaching. Some writing. Some admining. I am, as it were, a ‘mature student’. I am coming to a PhD exactly like dozens of those people I met all over the web talking about education. And. Well. They mostly hated their PhD. Not all of them. But most of them. So I came into the PhD with a certain amount of caution.

Research methods

I have things I could learn about different research approaches and the way they can be applied to different situations. Being an alt-academic for so many years, i have played to my strengths in any number of situations. I LOVE progressive coding for instance, and have stuck to it, taught it to other people and fallen back on it time and time again. I realize it’s not the only way to code data… but I like it.

Having switched into the cognition stream of the PhD program (note: i STRONGLY distrust the impact of cognition research on education), I have been opened to a pile of new ways of looking at research. I didn’t like all of it, but I did like some of it. I also enjoyed learning about the stuff I didn’t like. So that’s good.

But, i mean, I don’t REALLY want to learn LOTS about research methods. Everyone else in Higher Education chooses methods they like and stick to them, and I’m pretty much going to do the same. I’ve got a quant course this summer. Am I suddenly going to love quant instead of a 15 year commitment to the value of story in changing education. I mean… probably not. Doing the quant course will improve my anti-quant rants… which I’m looking forward to. It’s not that I don’t believe in quantitative research… I just think some of it is bad for education. Not all of it. But people like to count things, and I don’t think that learning is a counting noun.

So. Research methods… I’m probably good. I’m going to do some kind of story based research in my thesis or go back to it as soon as its done. I like story. I trust it. I want to pick one story based research method, nail it for the thesis, and wander off into the sunset. You could say “hey, you should get out of your comfort zone” and i would reply “really, how many researchers actually do that? Why wouldn’t I use the approach I believe in?”

I would love support on nailing that research method and doing it well.

Community

I’m never really ever working on my own. People who’ve worked with me will tell you that I basically can’t think with my mouth shut. I have been fortunate in finding smart and patient learning friends.

And I’ve really enjoyed getting to know the people in my courses last summer and this fall. They are, some of them, new nodes in my broader community of learning and knowing, and new nodes are always good for learning. While excellent to meet new people, I did not come to a PhD to ‘find’ a community. I am lucky to have several that are willing to have me.

Learning about ‘being an academic’

This is a bit more of a touchy issue. The experience of my colleagues entering the PhD process has often included a couple of myths about higher education that seem mostly reserved for the PhD process (speaking mostly for North American PhDs). I’ve certainly heard it from some faculty over the years, and during a very long meeting over the summer, heard it from an instructor who told me I would never get published if I didn’t start ‘thinking more like an academic’. Whatever that means.

The pure beauty of higher education and, I would say, its strongest strength, is the diversity of approaches that it contains. We could use lots more diversity, that’s for sure (and WAY more acceptance of it), but even as it is, it’s kind of amazing that all those people work in the same location. You can go to two consecutive classes and have one person claim, with absolute certainty, that if you can’t count something, it’s not worth researching. Or, as i heard it once put by a Stanford Researcher -> it will be in the quadrant that the NSF wont fund. You can then cross the hall and be in a room where someone (say me) tells you that learning is mostly about people, and people being different, you can really only learn about people.

I mean. That’s amazing. But it also means that there aren’t any ‘rules’ about how to ‘be’ an academic. There are rules about getting tenure… but not like… rules… about what is the right way to do it. It’s like the myth of the sanctity of blind peer review. I once left a peer review journal committee where someone said something like “well, we all know who really wrote that, so we’re pretty much going to have to publish it.” If I get sent a rhizomatic learning paper to review, there’s a 50% chance I know who wrote it after skimming it. I’ve been asked to review stuff I totally don’t understand (and thereby given the authority to judge it). So… yeah.

I’m not saying I have a better system. I love the chaos of it. To me it represents what knowledge is… a big mishmash of differing beliefs.

I don’t want any nonsense about purity in higher education research. It’s a simulacra. Doesn’t exist. Never existed.

I mean, I might want a job in an academic department someday, but I’m not really sure about it. I’ve worked with enough departments to understand how they work. I currently have an ancillary faculty position. I like it. I might go in for an admin job. I dunno. But what I need for those jobs is a credential. So that’s something I want too.

What am I willing to do?

I totally get that this is someone else’s process. I chose to sign up for this gig, and I knew what i was getting into. I’m going to have to do the lit review, and the comps and the whole writing and defence business. I’m wary of putting my existing work into it (rhizomy stuff), because i’m not crazy about putting it in for a ‘yes/no’ judgement. I can just imagine the struggle around “defining rhizomatic learning” something I’ve happily refused to do for 15 years.

Happy to do the work though.

What do I really want?

I’d like to complete the research that I’ve started in the PhD so far. I want to attach the cognition literature on ill-structured domains to how faculty learn to teach online. I think there’s a really interesting pattern there, and it seems to answer some questions I’ve had for years. I think there’s a story to be told that could help people see how the digital has changed whatever relationship they could have had to information and knowledge. Maybe it has made it clearer? I want to tell that story.

And maybe do a little future of higher ed strategy. But that’s probably a bridge too far, and I can always do that in my spare time anyway.

I’d like to work with peeps (you guys) and my supervisors to make that research project as good as it can be. I want to tell a story with more than my voice. I mean, I’m probably going to try and recruit half of the 12 people who read this post. One of the suggestions that my supervisors made was to do some blogging to get feedback from the community on the process. This… is a start.

I’d like to figure out a good way to do it (three paper, interpretive dance… i’m open to it) that will make the research useful for people. Useful. For people.

That’s what I want my PhD to be. Useful.