AI and Assessment: A New Reality for Faculty With Dr. Phillip Dawson

Full Transcript

Cristi Ford (00:00):

Is AI in assessment something we think we can fix, or is it really something we need to rethink entirely? In this episode, we’re gonna explore way- why AI may be less be puzzle with the right answer and more of a wicked problem as our guest describes it, which is full of trade-offs and competing priorities. And as we explore assessment, we would be remiss in not talking about students in the equation. What does it mean for students to truly use feedback well? And how all of this is reshaping the way educators think about assessment in an AI-shaped world.

Emma Zone (00:37):

Welcome to Teach & Learn, a podcast for curious educators brought to you by D2L.

Cristi Ford (00:43):

Each week we’ll meet some of the sharpest minds in the K-20 space. Sharpen your pencils, class is about to begin. All right. Joining us today from sunny Australia is Professor Phillip Dawson, co-director of the Center for Research and Assessment and Digital Learning. That’s CRADL for Short at Deakin University, the world’s leading higher education assessment research group. He’s internationally recognized for his work on feedback, academic integrity, and AI and assessment. Phillip, I am so excited to have you here with me today.

Phillip Dawson (01:14):

Oh, it’s so great to be here, Cristi. Thanks.

Cristi Ford (01:17):

Absolutely. So let’s just jump right in. I wanna really kind of set up around some of the great work that you’ve been doing, but before we dive into the published paper that you’ve written along with your colleagues, uh, called “The Wicked Problem in AI and Assessment,” can maybe we start off by sharing with our listeners some insights into what AI and assessment really is, and why is this topic graining so much traction, particularly at this moment in time?

Phillip Dawson (01:44):

So, I think that AI and assessment has really captured all of us because, uh, there’s a fundamental assurance of learning problem that it throws up at us, which is if students use AI to do the task, how can we be sure what they themselves are really capable of doing? Uh, it’s not the only problem that it throws up. So, in Australia, we’re really trying to balance that assurance of learning problem with the need for assessment that prepares students for a world where AI is just gonna be everywhere. And, you know, in my conversations with educators, there’s so many other problems that throws up as well, you know, our identities as, as

Cristi Ford (02:21):

Teachers-

Phillip Dawson (02:22):

Yeah. … concerns about its impact on our own jobs and job security. It’s, it’s so, so many interesting challenges all at once.

Cristi Ford (02:32):

Yeah. I, I, I resonate with that, around that. As I talk to educators across the globe, this is just such a, a inflection point right now in terms of how we’re helping to shape the future of learning. Um, and then I think, um, coupled with that, shape the future of work because we know that we are preparing learners to go out into the workplace and what does that mean in the era of AI? And I’ll just tell you, I have really been looking forward to this conversation because last fall when I had an opportunity to really delve into your paper around the wicked problem of AI and assessment, I really felt like there were some, some tangible discourse that you were offering and some examples to really be able to unpack that. So maybe we can talk a little bit about that now and, and that, you know, I’m finding that the industry is treating AI and assessment like a jigsaw puzzle, like there is a right answer or a silver bullet, a better policy or a better detection, uh, or how do we AI-proof assignments and tasks?

(03:33):

But your paper argues that we’re misdiagnosing this problem, that AI and assessment really behaves, like you say, as a wicked problem. Like there’s no single definition, no stopping rule, no true, false, but really an opportunity to think about what are the trade-offs, what are the consequences, and how do you continuously adapt? Can you share a little bit more behind the impetus of this paper and how you think about this work?

Phillip Dawson (04:00):

Yeah. So this paper came out of a lot of work we’ve been doing with educators at my university, talking with them about the real things they’re doing to try and address AI and assessment. It quickly became obvious to us that there’s not one formulation of this problem. You know, I, I’ve often framed this predominantly as an assurance of learning problem… around how do we know what students can do, and I walk around and I try and sell that formulation to people, and not everyone buys it. Um, because, you know, if you are in a discipline like, uh, computer science, my home discipline, you’re now seeing that AI is there just doing stuff for people. People are using Claude code to write their code. It’s fundamentally changed how work is now, and that might be the framing of the problem for you. It might be, how do we make sure assessment represents the worlds that our students are gonna go into?

(04:59):

Um, in addition to sort of many formulations of, of the problem, those formulations shape what appear to be acceptable solutions. So I can think I’ve got a really great solution to the problem in my own context, but for you and yours, it doesn’t solve what you believe to be the problem. And, you know, I could, I can go on about this paper so much, but I guess one, one other piece of it is that, it’s actually down to the people with skin in the game to have their own solutions, to be allowed to be wrong… uh, to be allowed to try something, have it not work, and try again, and build something better, and know that we’ll never get to a perfect state. Yeah. And we, we need to be allowed to do that.

Cristi Ford (05:45):

Yeah. I, I love that. And as I’m listening to you, I wonder if maybe we can ground some of this for the listeners, that when you talk about wicked problem, there’s a conceptual framing around that. So maybe we can share a little bit about what that conceptual framing is, and then what are the top characteristics that most clearly show up in assessment right now?

Phillip Dawson (06:05):

Yeah, okay. So, uh, you know, we are building on a literature around wicked problems that have been, has been used in many different contexts. And there’s, you know, various characteristics. For me, one of those features that really shows up is there is no stopping rule with wicked problems. So what that means is in a lot of problems we try and solve, we get to the point where we say, “Yes, the problem is now solved. We can move on. We know we have a clear definition of what that means.” With assessment in a time of AI, we might have something that’s good enough for now, but six months from now, we’re gonna have to change it.

(06:47):

Um, and also we, we have to decide when we get to that point where it’s, it’s, it’s good enough. Some of the, the literature on sort of learning and teaching talks about a thing called satisficing, which is where you don’t try and come up with the perfect solution, you try and come up with something that meets the criteria to a, to a good enough point. So I think for, for me, that’s probably one of the, the really important ones. And I guess that other piece that the way a wicked problem is framed shapes the solutions that seem, uh, appropriate. I find that one really fascinating. Yeah. And I, I guess the idea that outside experts can advise, but they ultimately can’t be the ones to have to do it. A colleague of mine, Thomas Corbin, who’s, who’s the lead on this wicked problems paper, uh, and I have, and, and some colleagues have a paper, actually it’s a chapter in a forthcoming open access book-

Cristi Ford (07:51):

Okay.

Phillip Dawson (07:52):

… where we are talking about what we call wicked solutions. And this comes from the experience of Thomas and I going out and trying to, I guess, workshop with some of these academics approaches to address their wicked problems. And our solutions were always wrong and they were always bad. They were sometimes useful. They were what we’re kind of calling wicked solutions that, we could, we could never come up with something that actually satisfied the people we were working with.

Cristi Ford (08:25):

And is that because context matters and, and those experts or those faculty members just had a different lived experience with what, what they were really looking for and achieving and, and assessing learners around, like, or, you know, what, what was the crux of that in terms of, as you talk about those wicked solutions?

Phillip Dawson (08:42):

Yeah, that’s really interesting. So, I’ve, I’ve been working in assessment design research and sort of how faculty members do it for, I don’t know, about 15 years. And the AI context does seem to have changed things a little bit in that people are trying to solve a problem that fundamentally can’t be solved. So, you know, in, in my past experience doing this sort of work with faculty members, um, there have been rare times when I’ve been able to say, “Hey, we’ve talked through five things, but what if you try this sixth thing?” Right. <laugh> And they’re like, “Oh, that’s it. That’s the one. Yep.” Yeah. In, in this context, everything was broken. There, there is no fix. However, there were, there were useful little tidbits people could pull out and make their own thing. So context matters and has always mattered. But I do think there’s a, a fundamental, unfixable nature of, of AI and assessment that we have to become comfortable with.

Cristi Ford (09:41):

Yeah. I, I agree with you there. And I will say the one piece that when you talk about the wicked problems that I really enjoyed from the paper and shout out to Thomas Corbin, Margaret Bearman and I believe, uh, David Boud. Is that how you say his last name?

Phillip Dawson (09:53):

That is correct. Very good. Few people pronounce it correctly. <laugh>

Cristi Ford (09:57):

Your other co-authors on this paper, I really enjoyed the focus on that there are no right solutions, but there are trade-offs. And in the paper you talk, there’s a faculty member who talks about really being able to find alternative assessment solutions, but those, uh, trade-offs then also made for more work and more time and intentionality to the grading process. And so there were some really good opportunities to, to grapple with, to your point, how do we really figure out what is good enough to be able to evaluate learners in today’s realm that may just completely be different in next, the next six to nine months. And, and then really researching that. I think the other piece about this, I’m off of script a little bit here, but I think this is, you’re the right person to ask about this. Researching this when the tools and technology are changing so quickly, how do we keep up with the, the research to be able to track what’s happening with these tools and these LLMs that, you know, six, six months ago were very different from what they are today.

(11:02):

I, I do struggle with the tension around, around that opportunity there.

Phillip Dawson (11:07):

Yeah, yeah. I think that’s, that’s a really fascinating one. So I’ve been using OpenAI’s GPT, which is like sort of the underlying technology behind ChatGPT- to write student assignments since 2019. And the, the fundamental idea that this technology can produce possible things for student assignments has been with us for a surprisingly long amount of time.

Cristi Ford (11:32):

Yeah.

Phillip Dawson (11:33):

And I, I think one of the, the shifts that we’ve had to make that I, I’ve really been trying to push people through is to stop trying to design assessment that AI can’t do.

Cristi Ford (11:44):

Mmm

Phillip Dawson (11:45):

That’s a real short-term view. Uh, I always say like, whatever your list of things AI can’t do is probably on some developer’s whiteboard of things that they want AI to do… and some of those things are ticked off and in beta. So- Yeah. … it’s a shift from designing in a way that I’ve, I’ve figured out something AI I can’t do and I, how clever am I, to accepting that if you’re in a context where the assessment is unsupervised, AI is likely to be used and it can likely do many things you can’t even imagine.

Cristi Ford (12:22):

Mm-hmm. Yeah. I, I think you speak a lot to the tension that I’m seeing in the field and, and, and really understand. I really appreciate you focusing on giving guidance around trying to stay away from trying to AI proof things. I think there’s a lot of conversation still in the field around AI detection. And, and, and, and I think to your point, it’s, that is the wrong conversation, um, that we should be having. But what tensions did you see teachers name most often when you were working through this paper? Was it integrity versus learning? Was it authenticity versus workload, work relevance, skill foundation? What were the kinds of things that really, um, that you as the researchers really saw that, with attentions for the, the educators you were working with?

Phillip Dawson (13:07):

Um, everything that you’ve said probably. <laugh> Um, uh, I guess, look, if I can talk briefly to another paper that we’ve done- Yeah. … broadly in this area, again- Absolutely. … it’s another Thomas Corbin led paper. He’s an amazing researcher in my center. Um, this paper’s titled, uh, the start of the title is, “Where is the line? It’s an absurd line”. And-

Cristi Ford (13:30):

Mm.

Phillip Dawson (13:31):

… that comes out of a direct quote from a participant talking about, “I don’t know where the line is between acceptable and unacceptable uses.” And I think that still seems to be coming through as the real challenge that a lot of people are having. What should I allow? What shouldn’t I allow? And how do I know that the sort of lines that I can set around acceptable and unacceptable uses of AI and assessment are the right lines, uh, lines that students will respect are enforceable, all of that. So the, the kind of boundary work- mm-hmm. … I think is probably where a lot of us are at.

Cristi Ford (14:12):

Yeah. I, I appreciate that focus and I’d love to be able to link, uh, in this episode, that article as well for our listeners. As I’m listening to you, I, I hear chasing the right answers is, is not the right approach. It’s exhausting educators that there is this ambiguity around the white line. Where is the line? So how should educators really and institutions be, what should we be doing instead concretely? What are the things that, that we can give folks as a call to action after listening to this podcast today?

Phillip Dawson (14:43):

Okay. So <laugh> this is gonna sound funny, but-

Cristi Ford (14:46):

Yes.

Phillip Dawson (14:47):

… I’m, I’m going to plug another Thomas Corbin led paper. <laugh> Um, now we have a paper with Danny Liu from University of Sydney, uh, it’s led by Thomas and in it, we argue that structural changes are needed to assessment design.

Cristi Ford (15:05):

Okay.

Phillip Dawson (15:06):

So, uh, what we mean when we say structural changes to assessment- mm-hmm. … are changes that go beyond discursive changes or talk. So often we’ve tried to solve this problem of assessment and AI by telling students, “You can use AI in these ways, but not in those ways.” Mm-hmm. Or you have to make a declaration about how you used AI. Mm-hmm. Or we have traffic lights or, or whatever else. Yeah. And those are useful pedagogical approaches. They’re really helpful. We do need to guide students in what acceptable use of AI is, but those are not the same as structural changes that actually, uh, set the conditions in which students do the assessment. We can’t rely on telling people what to do. Uh, I think that probably applies in a lot of fields. Um, so a structural change to assessment is something like, we will have you do the take home task like normal, but then we’ll have a brief conversation with you afterwards and that’s what we’re actually gonna grade.

(16:12):

And I’ve, I’ve done this before in my own teaching. Mm. Um, in computer science, we noticed a lot of students were getting through first year into second year and they couldn’t do the things they’d been assessed as being capable of doing. Mm-hmm. Mm-hmm. So we changed our labs, we would have students do their assignments, their tasks at home, and we would sit down with them in the labs and just talk through, “Why did you make this choice? What would’ve happened if you’d have done that instead?” You know, those sort of conversations, so we’re really sure the student can do what we say they can do. That’s a structural change. You can’t, it’s not just, we tell you what to do. Yeah.

Cristi Ford (16:51):

Mmmm hmmm

Phillip Dawson (16:51):

We’re the context of assessment.

Cristi Ford (16:54):

I really resonate with that because one of the things that I talk about often is our need as a industry structurally to think about things like co-construction of learning and how do we do that as educators when we have historically been the ones who are the, you know, the holders of knowledge and we are imparting knowledge and expertise to students. And so what I’m hearing you talk about is giving students a reflective practice and opportunity to do that at home assessment, but then to be able to have this conversation, it’s a, it’s a constructivist conversation or, or, you know, a co-constructed learning experience where you are then being able to evaluate, you know, the, the, the mastery or competence of your learners around these concepts or topics in a very different way. So I, I mean, I love that. And I, I wonder how do we get more faculty ready to take on that challenge when we’re in an era and age where a lot of times faculty are asked to do more with less, but that structurally we have to make these adjustments to be able to really, uh, evaluate student learning of the future.

Phillip Dawson (18:06):

Yeah. I think it’s really important that we consider academic faculty work when we’re talking about any sort of change. And, you know, the, the example that I’ve given of interactive oral assessment- mm-hmm. … is one of the time consuming ones. It’s also one of the logistically challenging ones. Yeah. Uh, in Australia, it’s common to have class sizes of, you know, 1,000 or 2,000 students.

Cristi Ford (18:34):

Yeah.

Phillip Dawson (18:34):

The logistics of doing that, having great interactive oral assessment are not easy. I guess my push to faculty is to say one of the best uses of class time that we have is talking with students about their work, and students will agree with, with that. Uh, often students can get through a, an entire degree and never have a one-on-one chat with a faculty member about this thing that they’ve uprooted their whole life to go and study and do. So I, I think we can’t put it in the too hard basket. We do need to find ways. And I find a lot of educators have a lot of success just getting rid of a week or two of content. I know we love content so much. <laugh> We do. But I would al- I always try and sit down with educators and say, “All right, is there, is there a week in here-

(19:30):

that fits? Where instead of talking to the students about it, like talking at them, we can get them to tell us what, what they can do with it.

Cristi Ford (19:39):

“Yeah, that’s a really hard conversation. I remember having a conversation with some business faculty and accounting, and there was an accounting one faculty and accounting two faculty, and then accounting two faculty said, “You know, students are coming from your class and accounting one, but they don’t, they don’t remember anything that they’ve learned.” And the accounting one faculty says, “Well, I’ve taught them all of the material.” And so to your point, are you covering the material or I, or are they actually gleaning the core concepts and competencies that they need to be successful foundationally to move on? Um, but is, is, is it a hard trade off when you have structural institutional systems that are programmatic review and evaluation and accreditation that’s all built on, you know, this house and this schemata that you must be able to align all of these things around.

(20:27):

So I completely agree with that.

Phillip Dawson (20:29):

I, I might jump on that a little bit as well-

Cristi Ford (20:31):

Yeah. …

Phillip Dawson (20:31):

Because I think, uh, you’ve, um, addressed here too that it’s a team sport. It’s- mm-hmm. … not something that any of us can kind of do on our own. And the problems often don’t arise until it is the accounting one and the accounting two faculty members talking to each other. Uh, when, when people sort of say to me at the end of a talk or whatever, you know, “What, what’s the number one thing I should go and do?” I say, “Go have a coffee with your colleagues and have a, have a chat about this. ” Yeah. Because we need to all be working together. Ideally, as a whole program team, all the faculty that teach in a particular degree are the owners of this particular problem. Yeah. If we all just try and take it in our own strange individual directions, we, we kind of don’t get there.

(21:18):

Yeah. Um, I’d, I’d point to a, a paper by, uh, led by one of my colleagues, Kelli Nicola-Richmond, uh, the first part of the title in that one is “It Takes a Village” and it’s a paper about how do we do this assurance of learning work when, you know, no single one of us can do it. And also it’s not just faculty members who need to do it. Yeah. We need to work with a whole range of sort of professional staff with learning designers with lots of people.

Cristi Ford (21:46):

Agreed. It, it, it does. I like that. It does take a village and it’s about that community of practice to be able to work on, um, you know, these kinds of challenges. We, we need multiple cohesive strategies to do that and, and multiple diverse thinking around some of this work to be able to make that happen well. So, um, I really appreciate that. You know, we’ve been talking a lot about assessment and AI, um, and the, the construction of evaluation and assessment, but you do a lot of work on the other side of the output. And I wanna spend a little time talking about your research on feedback. You know, it’s one thing for educators to create an evaluation and assessment, to evaluate that assessment, but it’s another thing to create, um, feedback opportunities and then quite another for learners to accept it. And I think this is where, um, you know, that old adage of you can, you can lead a, a horse to water, but you can’t make them drink.

(22:40):

What is changing about feedback in this AI-mediated learning environment and what are we at risk of misunderstanding as you think about your research, uh, if we treat, uh, faster feedback as better feedback?

Phillip Dawson (22:55):

Ooh, this is great. So yeah, I, I do a lot of work on feedback, um, and I’m currently leading an Australian Research Council grant where we’re looking at how can we develop students’ feedback literacy. So that student’s ability to seek out, understand feedback, make use of it, and work with their emotions throughout that process. And that was already a hard challenge that we had years of work ahead of us on, and then now AI is here. I think one of the interesting things is that AI is really blurring those boundaries between, um, a feedback process and sort of more of a co-authorship process. Um, a, a real challenge is that feedback, uh, AI will offer to provide you with some feedback and it will also offer to action that feedback for you without any real thought in the process. So I see that as a really interesting challenge that we have.

(23:57):

Um,

(23:57):

My group’s currently doing some work on sort of feedback as an ecosystem, so that students are inside of their own sort of feedback ecosystems that include a lot of different parts all working together, and now we have AI in that feedback ecosystem as well. So we’re trying to understand, well, what, what do students do with AI feedback and how does that sort of connect with feedback from teachers? How are teachers using AI to provide feedback information as well? We know that there’s a lot of, um, above ground, known processes with that, and likely much more underground feedback with AI processes with faculty members.

Cristi Ford (24:42):

Yeah. I, oh gosh, I’m gonna really try to be restrained here because I could go off on a huge tangent, um, thinking about some of the cases that have been brought up by students around, you know, faculty’s use around AI. But for those of our listeners who are hearing feedback literacy for the first time, um, I imagine you can probably provide a, maybe a little bit of an explanation that it goes beyond just being good at receiving feedback. Can we just maybe unpack that a little bit before we move forward?

Phillip Dawson (25:12):

Yeah. So we’ve often assumed that if we wanna improve feedback, the best thing we can do is write better feedback comments. Um, now the problem is a very significant portion of feedback comments never get seen by students. Um, if we look into things like logs on learning management systems and those sort of things, studies that have been done on those have found that anywhere between 30 and 70% of student feedback comments never get read. So we spend our weekends and evenings writing these comments that go nowhere. Now, to be a bit radical, uh, we define feedback as a process in which learners make sense of information about their performance and use it to improve their work or their learning strategies. And that’s really building on some definitions from Michael Henderson, from David Carlis and others. Um, in that view, in a process view, all those comments that went nowhere aren’t actually feedback.

(26:14):

Mm-hmm. They’re just kind of what Royce Sadler would call dangling data. <laugh> They’re, they’re left there. Now, Dylan William has this great quote that I love, which is, “Feedback should be more work for the recipient than it is for the donor.” Mm. And I love that as a way of sort of reframing our feedback work. It’s then, so there’s then been a shift towards, okay, we need to set feedback processes that require students to engage in these really good feedback habits. So things like, rather than give students feedback at the end of a class at, at the end of our semester, we try and bunch a lot of that feedback earlier and then have revision tasks where students have to apply the feedback, which is great. Yeah. But then we get to feedback literacy, which is, okay, so we’re asking students to do all these extra things.

(27:06):

Yeah. Can they do them? Yeah. Can they, you know, work with the feedback? Can they understand it? Can they apply it? Uh, can they hold onto it and then find useful places to deploy it later on? And also, how are they emotionally through this whole process? Because I, I don’t know about you, but some of the best and worst moments I’ve had in, in education as a student have been emotional ones around feedback.

Cristi Ford (27:31):

Oh, absolutely. Absolutely. And as I’m listening to you, gosh, I have so many questions. I wonder as I’m listening to you talk about the amount of feedback from the research that is pretty much left on the cutting room floor that students never see, how we tie that to helping students to do some sense making around the feedback that they, that they do consume to the most effective design moves that educators can make. You gave one example in terms of providing feedback and then having learners to be able to utilize that for revision work, but are there, are there effective design moves that educators can make to build into everyday learning that will be, help them be successful in providing that right opportunity for feedback literacy to be developed by their learners? Yeah,

Phillip Dawson (28:19):

Yeah. So I think the big shift firstly is to identify feedback as a learning activity that students do. It’s a process, it’s a thing. And like all learning activities we’re trying to design, we wanna provide scaffolds and supports. So I, let’s say I’m working with one of my doctoral students. We’re working on a paper and all that, and they’ve got comments from reviewers and they have to address them. I need to be there to provide the scaffolding and saying, “Hey, actually, when I’m doing this sort of work, I build a revision table and I have, I break down the long text comments into discrete actionable things and I write down here’s the comment from the reviewer and here’s what I’m gonna do and then here’s, here’s where I’ve done it.” And, and that’s providing them with a bit of a scaffold

(29:12):

… to reduce that, that challenge to allow them to actually do it. In another paper, uh, uh, with some co-authors, I propose an idea called authentic feedback, which is perhaps not what it might immediately mean to some people. It’s drawing on ideas like authentic assessment.

Cristi Ford (29:31):

Mmm hmm

Phillip Dawson (29:31):

Authentic assessment is assessment that in some way represents the world beyond the university.

Cristi Ford (29:36):

Yes.

Phillip Dawson (29:37):

Authentic feedback represents the feedback practices beyond the university, the feedback practices in the discipline that students are gonna go into. So we wanna look to where are our students gonna go when they graduate, how does feedback work there and how can I scaffold in some of those productive feedback processes into my teaching so students get to practice the, the ways of doing feedback in their likely future.

Cristi Ford (30:06):

I like that. It’s like taking the anagogical principles that Malcolm Knowles sets out and then puts them on steroids in terms of discipline, specific opportunities to be able to see those real lived experiences around that. Um, that’s, that’s really, really helpful. Before we sign off, one of the things I’d like to do is to ask my, um, guest to finish a sentence. So I’d like to ask you, if I were to say to you, the future of good learning in an AI-mediated world depends on our willingness to, how might you finish that sentence?

Phillip Dawson (30:37):

It depends on our willingness to have a lot of conversations. So those are conversations with our students. And I, I think, uh, interactive oral assessment is very much on the rise and it’s a really great thing because a lot of what’s left for people to do in an age where machines can do so much more is relate to each other. So talking with students, but it’s also having conversations with each other because none of these are problems we can solve on our own and it’s counterproductive for us to all go off in all sorts of contradictory ways as colleagues. We wanna work together.

Cristi Ford (31:17):

I really appreciate that. I talk about learning being social a lot, and I think that that epitomizes our opportunity to do that. You know, you were working on so many things. I wanna make sure before we sign off, are there any final thoughts, any other, uh, future directional research or any other things you’d like to share with our audience here today?

Phillip Dawson (31:37):

Um, I guess, look, if I can have one sort of plea to people-

Cristi Ford (31:41):

Yeah.

Phillip Dawson (31:41):

… it’s, it’s probably around the, the feedback stuff. Okay. It would be to really evaluate your feedback, not on how good your comments are, but on the effects that it has. And to really get in there, be a bit curious and find out the feedback designs that you currently use, how are they working for you, and how might you be able to take a kind of learning design sort of lens to changing what students do with the feedback you, you currently provide? Um, we’ve got a lot of work going on at the moment around feedback literacy, and in particular, behavior change. So we’re building on, uh, concepts from sort of health behavior change, the sorts of things they’ll use in a quit smoking campaign or something to help people exercise more. And some things I’m learning from that body of work, uh, you know, if we want to change people’s behavior, for example, we wanna get students to work better with feedback.

(32:45):

There’s a lot of, a lot of stuff we can do to help make that happen. It’s not just telling people what to do. For example, you wanna make it easy for people to act on feedback. You wanna make it obvious. So you wanna make it that the feedback’s easy to get. It’s obvious where to use it. Um, you wanna build in those reminders of when to do it. So yeah, try and be a bit curious with how your feedback’s working and think, how can I maybe try and change behavior there? So we’ve built a scale that people can use to try and understand their students’ feedback literacy, but it’s also really fascinating to sit down and fill it out for yourself. So working with a bunch of feedback experts, we’ve come up with, uh, just some real simple questions you can answer. Uh, you then get some scores at the end, but you also get some advice tailored to how you’ve scored about how you might be able to improve what you do in feedback situations for yourself.

(33:42):

Now, we’ve got that for the being on the receiving end of things. We’ve also just very recently, on the same website, got a teacher feedback literacy instrument that you can use to kind of self-assess your feedback practices as an educator. So a couple of things you can try out, reflect on, maybe identify some areas you might want to improve.

Cristi Ford (34:04):

Professor Phillip Dawson, you’ve given me and, and us a lot to think about. I really wanna thank you for being here, and we will happily accept any and all feedback you may wanna share with us, uh, about this experience. It’s been really great to have this conversation. I’ve been really looking forward, uh, to talking with you around your work and, and happy to have some time with you here today.

Phillip Dawson (34:23):

Thank you so much, Christie. It’s been really great.

Cristi Ford (34:26):

So I wanna thank my guest again, Philip Dawson. You can find more about Philip, uh, and the work he’s doing at Deakin University at experts.deakin.edu.au. I wanna thank our dedicated listeners and curious educators everywhere. Remember to take some time and follow us on social media. You can find us on X, Instagram, LinkedIn, or Facebook @D2L, and take a moment to subscribe to the D2L YouTube channel. You can also sign up for the Teaching and Learning Studio email list with the latest updates on new episodes like this one, articles and masterclasses. And if you like what you’ve heard, remember to give us a shout out, give us a rate, a share, or, and remember to review this episode, uh, and subscribe so you never miss anything that we have in store. And, uh, thanks so much for joining us today. You’ve been listening to Teach & Learn, a podcast for curious educators brought to you by D2L.

Emma Zone (35:16):

To learn more about our K through 20 and corporate solutions, visit d2L.com. Visit the Teaching and Learning Studio for more material for educators by educators, including masterclasses, articles, and interviews.

Cristi Ford (35:30):

And remember to hit that subscribe button, and please take a moment to rate, review, and share the podcast. Thanks for joining us. Until next time, School’s Out.

Resources Discussed in the Episode

• The wicked problem of AI and assessment
• ‘Where’s the line? It’s an absurd line’: towards a framework for acceptable uses of AI in assessment
• Talk is cheap: why structural assessment changes are needed for a time of GenAI
• It takes a village... Program-wide approaches to redesigning assessment in a time of generative artificial intelligence (GenAI).
• The Feedback Literacy Behaviour Scale (FLBS)