Regan Gurung and John Dunlosky are shaping how today’s educators think about learning science, effective studying, and the craft of teaching.
In this conversation, they join Thomas to explore why strong research makes for better teaching, how to identify the most effective study strategies, and what it really takes to improve student success.
Drawing on their shared experiences as researchers, authors, and instructors, they discuss everything from overcoming common barriers to learning (like poor note-taking or misconceptions about memory) to designing courses that build in retrieval practice, spacing, and timely feedback. They also tackle how teacher training can benefit from a deeper understanding of cognitive processes, examine the promises and pitfalls of AI-enabled feedback, and address what personalized learning could look like when done well. Whether you’re teaching in a K–12 setting, designing higher-ed courses, or simply trying to study more efficiently, you’ll come away with powerful insights for turning learning research into real-world classroom practice.
Recorded 09 December 2024
Subscribe: Apple • Spotify • YouTube
Thomas Thompson: Hello, everyone, and welcome to this episode of Learning Stack. Today, I have the distinct privilege of chatting with Regan Gurung and John Dunlosky. Dr. Gurung is a professor of psychology at Oregon State University, where he was just awarded the school's highest teaching honor. We'll get into that, thus perhaps implying a link between good research and good practice in teaching. I think all the more stressing the importance of a rigorous and creative scholarship of teaching and learning, something I think Dr. Gurung might also agree with. Absolutely.
As a researcher, Dr. Gurung has published over 130 articles in peer-reviewed journals and has co-authored or co-edited 15 books. His most recent book, Study Like a Champ, was a collaboration with our second guest, Dr. John Dunlosky. As someone who has only recently begun my study of this field, many of Dr. Dunlosky's papers have really formed the basis of my approach to teaching and my understanding of how learning occurs and how that may differ from my perceptions of how my learning is progressing or not. He describes his research interests as understanding the interrelated components of self-regulated learning and that these components are metacognitive, meaning that they concern people's cognition, their thinking or beliefs about their cognitions, again, their thinking and that by studying metacognition, we may begin developing techniques to improve student's learning and achievement across multiple domains. Gentlemen, thank you for joining me today.
John Dunlosky: Great to be here.
Regan Gurung: Absolutely.
Thomas Thompson: So, as I said, the newest book is Study Like a Champ. You can buy a copy on Amazon. It is cheaper on Amazon than on ThriftBooks, so I recommend going the Amazon way. I bought a copy in preparation for this episode, and it's very accessible.
It's very applicable. It seems like something undergrad students, perhaps parents, and certainly teachers could all use, not just read, use. And that's striking because typically I come across these recommendations of how to study. I read them. I like them, yet I don't use them. Why is just reading about how to study not always good enough, and how does this book get around that?
John Dunlosky: You know, I think one thing that we set out to do is exactly what you'd experience. Develop a set of recommendations that are usable. So what makes our book, I would say, a little bit more approachable than others of its kind is that instead of giving students a list of the hundreds and hundreds of ways that they could go about studying, all the various strategies you could use, many of which are actually quite effective.
What we did was just narrow it down to a couple of the most effective, widely applicable strategies and then provide relatively intense descriptions about how students could take those strategies, again, very few of them, and apply them specifically to their own daily lives. So we're not going to give you the top 100 most used strategies by students, and then you have to figure out, well, when would I use most of these? You could just take any particular chapter, which are each meant to be kind of one lesson on their own—right—read that chapter, and then immediately, through a little guidance that we give in the book, apply that to your own particular studies.
So, we think it's just a very hands-on tutorial on how to take the first step toward studying like a champ. Obviously, there are other things students could do that go beyond our book, but this is really the first foray into what I would tell students to do as they start out on this journey. I'd even go as early as junior high school for these kinds of strategies.
So that's kind of my perspective and why I think this book works. Regan might have...
Regan Gurung: Yeah, you know, what I'll add to that is, in stark contrast to a lot of resources that just tell you what to do, which is the, okay, here's the science, you should do this, you should do this. The big difference here is that we explicitly, from the get-go, said we want to reach out and talk to students. This is going to be in a student voice.
John and I were adamant with our publishers and editors that we would write directly to students. So you will notice that. I mean, this is not the jargony, stuffy, old-school, snobby, hey, we are the professors; do what we say.
This is very much about understanding the student perspective. I want to add that working with John is absolutely wonderful. I want to tie this to something you said in the introduction, Thomas, John Dunlosky is the man to cite for cognitive science. He's been an idol of mine for a long time and, thankfully, a friend. If you care about teaching and learning and haven't cited or read Dunlosky et al. 2013 and the many others, you know you're missing something. But that's not all. And I think that's what makes the book different is that John and I are both teachers. We teach. We are in the classroom, not just researchers. Although both of us do a lot of research, we're not just sitting in the lab doing research. A lot of our research is in the classroom and we are teachers. And I think that really changes the flavor of the book. And I think readers notice.
Thomas Thompson: Do you notice your research and that kind of work influencing your teaching, or is it the other way around, which, or is it kind of this persistent relationship?
John Dunlosky: That's a great question. I think some of the strongest research programs in all times are connected to real-world questions, right? Regardless of whether you're in physics, psychology, chemistry, what have you. So you're just not kind of inside your own little ivory tower, so to speak. And where I've always looked for my questions are real student learning and performance. So I'm teaching it at the same time, seeing how my students are struggling, which leads me to come up with kind of hypotheses, our best guesses about how to really help them improve. And then I take those kinds of questions like, will this work or will that work? I take those back to the laboratory and then I evaluate their efficacy. Do they actually really work?
Unfortunately, and I say unfortunately, because this is the really difficult part. After you see that this stuff works in the classroom, then I take the next big, most difficult leap, is bringing it right back to the classroom, right? Because now we know it works in the laboratory, but now I have to see it in real terms.
So it is this reciprocal kind of like teaching leads to other questions about how to improve student learning and happiness, right? Investigate that laboratory, but then go back and bring it to the students. That's what gets me up every morning when I was really into researching and teaching is knowing that the two work hand in hand and that what I'm doing in the laboratory, at least some of it, will show promise for helping students really kind of make some advances on their own.
I don't know. You have the same sense, Regan?
Regan Gurung: You know, absolutely. And I think what you're getting at is for the longest time, Thomas, there's been this disconnect between lab work and the classroom. And for many, many years, there was great lab work and particularly, and this can be said in general, but particularly in cognitive science, right?
So all these great studies and people often joke, oh, yeah, just because a study was done on memorizing Swahili words doesn't mean it's going to apply to an intro psych class, right? But what changed a lot, and John alluded to this in his comment, is researchers started doing what we now call translational research, right? So not only was there translational research saying, look, it works here, will it work in the classroom, and vice versa? So it's this loop that keeps going round and round. But then we're now at a point where there is a whole area of research called implementation science, where, you know, it looks at how do you implement what's going on? Which context will it work on? Because that's really been the big problem, I mean, we can name a list of strategies that will work in one context and not in the other. But the problem was, and is actually, it still is to some extent, is somebody will read a published article and go, oh, look, this is going to work.
I'm going to put it in my classroom and then be surprised when it doesn't work. Well, behavior is complex, right? Studying is complex, classes are different, you know?
And I think that's what we really tried to capture in the book, is sprinkled throughout it is, but remember, if it's this kind of class, you may have to do that. And parts of the book that I love the best, and John and I, you know, wrote the whole thing together, is we thought about where our students stumble. And this gets at, you can't just plug in some research finding and expect it to work, and it's not going to work with everybody.
So when some of our students said, oh, you know, we tried retrieval practice, and it's not working. Well, we've had those conversations to say, what did you do? Why did you treat it this way? And that finds its way into the book, which is something I think you don't get to see a lot in other sources.
Thomas Thompson: So is that why, say, a lot of advice on studying tends to be ineffective because it's framed as this universal advice when it ought to be very specific to certain situations?
John Dunlosky: To some degree, I mean, some of the advice is meant to be universal and, quite frankly, is relatively general. But let's take any particular strategy, like Regan said, retrieval practice. It has a general positive benefit for any students who use it appropriately. But the word appropriately is important there because for any particular approach to study, and that's effective, there are a lot of barriers to success, right? So I can tell you how to, say, memorize, not that anyone would want to do this, a long passage or book or something. Well, I give you the most effective strategies to do that. Well, there might be a lot of barriers to them actually being effective for you because A, you might decide, I don't have enough time to use these. Well, that's totally appropriate. Students are overwhelmed with all the things they need to do.
So just saying, well, this particular strategy could help you, but without realizing that, in fact, many students won't have the time to use it, is a major barrier, okay? Now, there are specific barriers to some of the strategies that Regan and I love, like retrieval practice. Let me just give you one example that's some of my favorite that I've dealt with in my classroom teaching is I'll have students set up really nice flashcard decks to help them learn basic concepts in class, and they'll set these flashcards stacks up using their notes, which is totally appropriate, yet they tend to struggle, even though they're doing something that I think, wow, that's a really good approach to learning this. And then I look at their notes, and then I look at the flashcard stacks, and I realize that, in fact, the difficulty they're having is not using this really effective strategy, that is retrieval practice, but it's in taking good notes. So what they have now are flashcards that are littered with incorrect information.
So they're learning that, but in fact, it's incorrect because they didn't get it from the classroom correctly, which in some cases could have been my fault. Maybe I wrote something down wrong, or maybe they just didn't quite understand it in the class. So there's lots of barriers to any particular strategy working well. And part of what we do in this book is not only provide like, hey, here's a really good strategy, but we try to talk about some of those barriers and how students can like sidestep them, right? Don't, you know, try not to fall prey to this mistake, or when you start using the strategy, you know, you're going to feel as if you're not making a lot of progress. Don't let that get you too upset or frustrated because it turns out the pathway to success, there's always a bit of frustration and tripping, because if it's worth learning, it's probably difficult to learn, and hence you're going to trip up. But don't blame that on the strategy. Blame that on how the mind is set up, and it's just not a sponge or a video recorder. You have to know how to work with it, right?
So lots of barriers to success, but quite frankly, still a lot of success to be had by students. Just use a variety of simple strategies.
Thomas Thompson: I guess the follow-up to that would be, let's identify some of those major factors that influence our learning, and then contrast those with the factors that influence how we think about our learning, right? You can imagine a student who spent the last five hours studying, they made flashcards, they penned a summary of the chapter, they re-read it. Some of those I threw in there for you. These factors influence their assessment of how much they learn. Like, oh, well, I spent the last five hours doing this. I feel pretty confident, or I don't feel confident at all. And then maybe some of these things at least influence their learning. Where do these concepts converge? Where do they come apart?
Regan Gurung: So, you know, I think I want to start with something you said a couple of times. They've spent the last five hours studying. That, I think far too often, students, all of us, use how many hours have I been doing this versus what have I been doing?
You know, so that's an important starting point, right? It's not just how long, but what you've been doing. Then we start going into what are the kinds of things you've been doing, you know?
And before I forget, I want to say, John mentioned note-taking. I mean, this is why we have a whole chapter on note-taking in the book, because we've both seen so many problems with note-taking, right? So then, I mean, I'm going to go from note-taking to go, how are you using those five hours? Well, there are some students, and this surprises me and scares me, who will spend two, three of those hours taking notes on the textbook, right? Now, I think there are a lot better ways you can spend your time than taking notes on a textbook, because now you've got the textbook, you've got your notes on the textbook, and your lecture notes. Well, that's a lot of content, right?
Instead, you've got to be allocating your time to help you process more deeply, to help you test your recovery. So whenever we, you know, when you said there are lots of those factors in there, when John and I work with students and we do this, right? They come in after an exam and they go, oh, Dr. Dunlosky, Dr. Gurung, you know, I studied so hard, and that's almost exactly what they say. I studied so hard. And my favorite is, look at all my flashcards, and show me the big stack, but I'm still only getting a C. And so then we talk about, all right, let's talk about exactly what you're doing.
So the question becomes teasing apart these things, and I'll tell you why this has been a problem. And for many years, I've been guilty of this. For umpteen years, I thought I was doing my job because I went into class and said, here are the best ways to study, right? You know, the cognitive scientists say this is the best way to study. And I thought by just saying these are the best ways, I'd done my job and they could pick it up. How I have evolved dramatically over the many last years is not just seeing it, but designing my courses so that they use those strategies better, and taking the time in class, the first day, the first week, to unpack what these are. We should not be taking for granted that students understand, even without the jargon, what retrieval practices, what space practices, right? And those are the top two. I mean, we got to start there. How do you get students to spread out their studying, right? Now, again, we can tell them, we can even show them some really neat data on, look, you know, uh, when you spread it out versus cramming, here's how you remember, we can show them. But the missing element that was missing in my pedagogical life was that I did not fully use the ability to design the course.
And then I said to myself, wait a minute, I'm the instructor, I'm designing the course. If I know retrieval practice and space practice and interleaving, and all these things are good, well, I can design my course, literally how I set up my learning management system, what I put into the syllabus, I can set it up so that students will use it better. So this goes all the way back to where we started, where you said, oh, five hours. Well, if I've designed my course right, if I've spent the right amount of time in class, explicitly talking about these things, those students are going to spend their studying time better and more effectively. So embedding those effective practices into the process itself.
Thomas Thompson: How does that then apply from, say, students and learning techniques to teachers and teaching techniques? You know, I have my first teachers who are my, their first three years, I sit them down in a professional development and say, hey, you should do this. It's a really effective method. Are we going to run into the same issues we would run into if we tell students, hey, these are the effective study methods? You know, is there some overlap on that?
John Dunlosky: Well, I mean, certainly there are. It doesn't underestimate how difficult it is to apply the simplest strategy to a new context. I mean, I run into this, I'm a beginning guitar player and people tell me strategies. And the first thing I think, well, I'm not going to use that. That'll never work for me.
I mean, I'm sounding exactly like my students sound right. So I kind of need more of a crutch or scaffold to show me how to use these things, because I could give someone a set of flashcards with all the content that I know that they're going to be tested on. Yet I can tell you five or six different ways students could use flashcards. So they'll learn almost nothing. In fact, research done by Katie Wissman here at Kent State showed that in fact, many students report using flashcards in ways that are suboptimal.
So here they have the tools available to them. Yet, since they don't quite know how to apply it, like, you know, you said five hours of study the night before an exam, they're really not going to benefit as much as they could from some of the strategies. And unfortunately, in your case, Thomas, you said, how does some of these pieces come together?
The student who comes after me to talk to me after a five-hour study session is frustrated, whatever, thinks they're thought they knew it, but yet did poorly. I also have something to admit, 30 years ago, when a student came to me like that, I thought they were lying to me. They didn't study hard. It's like, there's no way you could study hard and get a D in my course, right? What I came to realize pretty quickly is that, yeah, you could study really, really hard or think you're studying really hard and not learn much at all. So it was a double whammy for these students. They really were trying hard. And what they were doing was giving them these illusions that they had knowledge that they yet hadn't gained. So part of it's convincing the student who really does want to work hard to learn content to get them to realize, well, part of that, the working hard, which is the number of hours, needs to involve the right kinds of practices that might be frustrating, right?
That might trip you up on your path to learning, but yet eventually will lead you to a really good place. So I really feel for students who are out there struggling and not learning because many of the things we do give us a sense that we know when we fact don't know. So a lot of our book is about like, hey, let's break these bad practices and instead replace them with some relatively simple ones that can help you get to really achieve your learning goals. And I think excel in school, which is something that any student has the capability of doing if they have the time and the energy to utilize the most effective practices.
Regan Gurung: And Thomas, you know, there was something in there in your, in your comment that I have to pick up on too, is, is you said, but how about when you're talking to teachers, right? So, so two things I want to touch on, number one, and this is building on what John said, we're all humans and we all fall prey to the same kind of issues, right? And of course, one example there is I can be giving a talk on the perils of divided attention, and the teachers to teachers and the teachers will be there, some of them on their phone scrolling Instagram. And I'm, and I have said, and I've pulled them out and I've said, this is what we're talking about right now. You have just shared your frustration with your students doing it, and take a moment and notice that some of you are doing it, you know? So that's, I think the nice thing about being, you know, having many years under the belt is I don't feel bad calling some of those things out. And it's just a simple case of, oh yeah, we all do that. So that, that's the more, yeah, of course thing, but here's the bigger thing, right? Then, I studied so long and I only got a D, right?
I think, I think we would be remiss if John and I didn't underscore the fact that there are many instructors who may not have gotten training in how to do assessment, in how to write exams. Well, if there is a disjoint between your assessments and what you're having your students do, that's another whole reason for them not doing well. So obviously we wrote a book on studying like champions, but I think we would be remiss if we, if we mentioned that that indication of being a champion is the assessments that we teachers design, right? And that is so important to not let go of because there are so many higher ed faculty and instructors who just haven't been given the training on how to teach, on how to write exams, and how to do assessment. And I think that sort of belongs in this conversation too.
John Dunlosky: It's like if Thomas is taking my class in cognitive psych, well, I'll just randomly choose questions because what I'm trying to do is evaluate whether he's got all of the knowledge, right? So if I randomly take questions from all of the content, that's a really good, right? Well, it's a horrible idea as, as a teacher, I should decide, no, these are the core concepts that Thomas needs to learn. And out of my parade of 10,000 concepts that I could teach in this class, I've got to be honest with myself, only five are important. So now what I need to do, right, is restructure my course so that I'm really pushing and repeating and doing long demonstrations on just those core concepts that are most important and telling the students that they're important and then assessing the students on just that content. Doesn't make it necessarily that much easier on the student. At least they know where the target is per se, which I love. You know, one other barrier too with teachers, Thomas, that you asked the question, and this is also difficult, and I do not know how to sidestep this.
Many teachers do not understand how the mind works and they bring a lot of misconceptions to the table themselves. One of the major ones that I've noticed throughout different areas, especially STEM, is that memory is bad and they'll just reflexively say, well, you can't use flash cards, memory's bad. And then you probe a little further to try to get what they're saying about that.
You realize they don't understand what memory is. They don't realize that you can't understand anything without a memory for what you're trying to understand, that kind of stuff. So, and it's not as if Regan or I would say that, look, flash cards or memorizing content is the panacea or the end all of learning, but it certainly can be the beginning of learning to give students the sense that they're making progress. And the teacher's goal then is to take all the knowledge the students are accruing and to give that life through demonstrations and through examples so the students truly understand it. So there's this, so there are quite a few barriers to break both on the teacher side, right? They don't always understand how things work well, but yet teachers often are very confident that they know what works well and what doesn't. And all the barriers that are on the student side, many of which are not understanding what works best, but many are because they're students and they're very busy and they have lives too that they're trying to coordinate with their, basically with their studying. So there are ways they need to approach that so they can have the best balance between their classroom work, their studying, and everything that's outside of that too. Because I think I could speak for both of us, Regan and I, that we want students to be first and foremost, happy, productive people, and you have to have a really good balance and regulate your own behavior to do that.
Thomas Thompson: Okay. Many follow-ups from here. So how much of some of these, I guess, bad practices, we can call them, come from a place of ambiguity or maybe ignorance might be the word related to, you were saying like a teacher saying memory's bad, right?
But that same teacher might go, you know, it's really important that my students are critical thinkers, however, and it's like kind of a, you're picking up on a misconception here, which is you don't realize that it's all about, you know, being able to apply your prior knowledge in new contexts effectively and efficiently, thus knowledge and memory in this case is relatively important in that endeavor. Is it a lot of just confusing terms and goals, and you don't have a clear goal, and you don't know clearly which methods support that goal? How would you rate this?
Regan Gurung: Yeah. You know, my first thought to that is it's, there are two things going on here and I'll be very honest and say, and blunt and say, my first response is there's so much variance in teacher training. All right. Classic case in point is, you know, year after year, there are studies of K through 12 teachers and you'll still see a high amount of, a high number of teachers still believing that learning styles is a thing. Okay. And, you know, psychological science has debunked the learning styles thing many, many years ago, but you know, many of our kids are still coming home from middle school and high school being asked to take a learning styles inventory.
Right. That's part of the issue is that, is that there's the teacher education. And again, you know, I say K through 12 because there's actual data on this. There are studies that show these, this is not some sort of bias, but I'll also say you know, I think the focus on higher education, there's the same issues there. I mean, I spent five years as the executive director for the center for teaching and learning with a lot of faculty who were very open in sharing that at no point in their careers did they get some of this information. So I think that's the big part.
I think the second part, but I do believe it's secondary, is the ambiguity and the confusion. Right. So I think, you know, I wouldn't, I think ignorance may be too strong. I would say just a lack of knowledge. That's the first part, the teacher training part. But the reality is, and the second part is some of these terms are confusing, you know, and I love the example John used, which is, you know, before you can do some critical thinking, before you can be interdisciplinary, you need some disciplinary knowledge.
You need some knowledge to work with, you know, you can have a really smart individual, but if they aren't aware of some of the basics, well, it's very hard for them to continue and do more. And so I think it's exactly that kind of work. Now, again, I also want to say there is some nice research showing that when you think about something like Bloom's Taxonomy, even if you jumped to the analyzing and the application and so on, you don't, an active, smart student can do a fair amount, even without a lot of that understanding and memorizing, but obviously it gets much better when you have it.
Thomas Thompson: So it's a breakdown in the logical sequencing of material over time, where, you know, I find it sometimes in like pre-packaged curriculum documents, I'm teaching U.S. history, yet I'm jumping all over the place, and the concepts may not be neatly related to one another. And it's like, how do you progressively build student capabilities in using this material when the material, in some cases, doesn't even relate to each other from one day to the next or one week to the next?
Regan Gurung: Well, you know, and that's great because I was just recently at the NCSS convention in Boston and actually chatted with a lot of history teachers and, you know, secondary school history teachers. And it was so interesting to hear about how they dealt with content, right? And something, you know, a distinction we make very often in teaching in psychology when we teach is, look, there's knowledge and then there's skills, right? And especially in the intro psych class, psychology as a field has done a great job of saying, look, a psychology major and even in an introductory psychology class is building a lot of skills and you need some focus on those skills and study techniques and how to study and how to understand material and process material. That's a skill that you can apply across the board. So I think it's that, and it's even more important in places where, you know, there is a lot of content. I think we as instructors have to really think about, let's think about the psychology of learning. Let's think about the science of learning and how people learn. And, you know, this is where, you know, engagement and hope and telling a story and all those kinds of things come together.
So, you know, we've ended up actually saying quite a bit right now about how much we teachers need to do, but a lot of it is what we can do. And I guess that's my way, a really long way to answering your question of, I think in most cases, the students are not to blame per se. It's just that no one's taken the time. No one's taken the time to say, here's the best way to study.
Thomas Thompson: Which, how much do you think that a lot of those effective learning and study techniques, pretty much boil down to deliberate practice with immediate feedback, or how much of it is something else than that? Well, you see a lot of people saying it's just persistent practice with clear guidance, and I need to get back to the students really quickly. You know, you did this incorrectly, tune up on this and just, you know, kind of this coaching, I suppose, throughout.
John Dunlosky: You've got to be the coach. Definitely students need feedback to correct, but just one correction isn't going to lead to an epiphany of knowledge, right? So, and again, it's figuring out, well, what content do students need to remember or understand for a long period of time? Not just for the content, just not for the time of the course, but beyond the course. Those are the things where you have to ramp up, right? Your exposure to the materials.
So, after you provide them with correct feedback, you come back to the content again later on. So it's, again, making really difficult decisions as an instructor about what really is most important so that you and during your training and that students, while they're studying, can use the most effective strategies. For instance, one that's, I think Regan and I would both applaud is space. Where you're studying the same content in more than one session across time, well, students who cram just the night before really can't use this particular strategy effectively, right? And it's a very simple strategy to use, like many of these others, but it's students, many won't be able to use it or take advantage, right? Well, they certainly won't be able to take advantage as a teacher when I introduce new content the day before the exam, like if I'm like, Hey, I'm going to lecture on a Tuesday about all this new content. And by the way, get ready.
There's a test on Thursday. So you have to learn this too. Right now I am exhibiting a teaching practice that undermines students' really use of strategies and ability to learn the content. I might as well be just telegraphing. I don't care if you really learn this stuff. I just have to tell you about it because it's part of my job. I mean, let's stop that, right? And start engaging in practices, both the teacher and the student, almost like having a contract. I got to tell you what's most important to learn. I'm going to give you the time to learn it in the context of this class and all the other courses you have. And let's work together to develop the best pathway to success. And again, it's just a simple technique to use, but again, many teaching practices can undermine the use of these really relatively effective practices.
I'm not sure if I answered your question there, Thomas.
Regan Gurung: I love the way that you mentioned feedback because I think that's so important and something I say to students and I say that to my kids, that I mean, actually my 16-year-old and my 18-year-old. It asks for feedback on your studying. Ask for feedback on your performance because we know, I mean, there have been recent meta-analyses that show, and I'd love to share this one to put some teachers at ease. It's not the speed of the feedback necessarily, but the quality of the feedback. That's really important. And, you know, we've just said that many students don't know what to do. Well, I think it's really important, especially early in the class when a student does get some form of assessment to ask for feedback. And I say ask for, because not every instructor builds in feedback, you know, and we should.
Thomas Thompson: Can you build out assessments, particularly with that in mind? I mean, I guess these would be practice tests. Are those an effective way?
Say, you know, you're in Dr. Dunlosky's class. He says, all right, I'm going to lecture on Tuesday, but you're going to have a practice test on this on Thursday or on Friday. And then you'll continue to study and, you know, space the practice out and relieve new concepts throughout.
And then there'll be a high stakes test in the future, and you'll know when, but the practice test is Friday. Is that an effective way of thinking about this? Is that a good mechanism for feedback? Say you do it online, students get scores back immediately.
Regan Gurung: Right. Yeah. It's not, it's not just an effective way of thinking about it. I know John and I actually both do this in our classes, you know? Yeah. Right.
John Dunlosky: So, yeah, I think it's a very effective way to do it, but don't, part of the issue too is how we set up classes. Most, most faculty don't want to think they're punishing their students, don't want to give a cumulative exam, right? Well, the cumulative exam gives students a chance to relearn the most important concepts, right? So as far as giving these small exams or practice exams, students must also be told that, yeah, you're going to be tested again in the future on this. The difficulty with generic feedback, both with respect to some research and just all our own experiences as teachers, though, is that sometimes students don't even engage with the feedback. So here's one practice that you could use easily, potentially on a high-stakes exam with feedback, because usually, unfortunately, teachers high-stakes exam, I gave it. Now I put a number on all the students that says how smart they are in this class. And we move on and forget that you even had that content. And instead think about now you're going to give them back the exams and they can look up the correct answers. All they do is the, on their exam, they write down, right? Why they missed one and what the correct answer is. And if they do, it's a contract with the teacher that they get some of their credit or that they'll, they get something for actually engaging with the feedback in a way that can help them learn versus just look, oh man, I missed that one. Or, oh, I thought it was that. So there's a way to actually provide feedback that at least increases the chances that students engage, because at least early on, and this is all anecdotal. I'm not sure if there's evidence out there.
Maybe Regan knows. When I first started, I would just give feedback and I didn't realize that students weren't even looking at it. So where's their learning opportunity there?
For me, those high stakes exams should all be really low stakes. It should be a high-stakes learning that is supported by the feedback that they get from these low-stakes exams. But I'm not sure if anyone does that, but it would be a great way to go, decrease the pressure on students, but increase the expectation for real learning, right? Which usually means less is more, right? Less content, more learning.
Regan Gurung: Yeah. I had to smile when you said, you know, students weren't paying attention to feedback. In days of yore, way, way back when I started teaching and students would write, you know, 300 word or, I mean, two to three page papers. And this is intro psych, large classes. And they would write, you know, these two to three page papers, everybody would. And there was a week of the end of the term that all I did was read papers and took pains to put all these comments on it, right? And I can still remember, I would then, you know, and then I would give it back. And so I decided to figure out how many people are, let alone reading it, how many people would even care to pick up their paper with the comments on it? Okay. So I would leave this whole stack or take it, A, take it to class, B, leave the stack outside. And there was such a large number of students who did not even pick up the papers that I quickly changed that giving feedback on the papers notion. Now, I think the key point here is it's how do you give feedback and do you like you make it a developmental exercise, right? For example, what I moved to is saying, look, I'm going to give you written feedback on drafts, but on the final papers, I'm not even going to jot a single thing down because I could see behaviorally that it was what's my grade. All right. Onward.
So, by just moving all the feedback to the draft paper for those who wanted to do a draft, and I'll also let you know that a lot of 70% of the class turned in a draft. So I still had a lot of papers to write comments on. But these are students who said, yes, I'm taking trouble to do a draft because I want better feedback.
So I want to add one more little thing. How do we how do students how do students know what they're doing right or wrong? Right. And a term we use right at the beginning of this was self-regulation, right? Self-regulation. And I think in the context of feedback, one of my favorite things that I have students do and I advise instructors to do is right after the first exam, right after the first exam, I ask students to write down what they think worked, what they think didn't work and what they are going to change for the rest of the course as far as their studying goes. So it's this very explicit.
What did you do? What should you be doing? And then we talk about in class, how can I help them do those things that they don't feel like they can do? And it's that level of don't just assume it's happening. Don't just assume it will take care of itself, but actually taking class time with students and for students, putting that break on and saying, wait a minute, I've just had an exam. I just got feedback on a paper. What do I need to do differently? And anything we can do as instructors to help students have that self-reflection, the better off I think the student will be and the learning will be.
Thomas Thompson: How important is who's giving the feedback in that equation? Does it matter if it comes from the instructor? Does it matter if it comes from some automated system? What's the how do we rate where the feedback is going? Because you can imagine I'm playing a game, right? And, you know, I do something wrong. My character dies. I respond. Okay, I made a mistake here. I need to fix that. Can you create game spaces in which the victory or win conditions are tied to success in this particular learning objective? Thus, losing is feedback and you can improve your performance to win the game, thus achieving success in whatever the objective is, right?
These are just some examples, but does it matter that it's the instructor giving the feedback or can feedback come from many different places, peer feedback, AI feedback, game-based feedback loops, things like this?
John Dunlosky: It, a lot of this has to do with a student's belief about the quality of the feedback. So if the student believes that the feedback is appropriate, it is quality. I'm not sure, especially with today's cohorts and the amount of AI and tech that students grow up with, I'm not sure they would necessarily care where that feedback's coming from as long as they know it's also tied to the course. So I wouldn't listen to feedback that my teacher doesn't endorse or something like that. Why would I? Because it's now just external, but I don't think it would matter, but I could be wrong. I'm just speculating on that, but it's the quality feedback that's critical.
Regan Gurung: And I think it's the modality of the feedback too. I'm glad you mentioned AI because that's what everybody's looking at right now, because AI could give you feedback a lot faster. And I'll actually, I'll be, I'll tell you just earlier last week, just last week, one of my grad students and I, Joe Slade does a lot of work on AI. And so a quick tangent to say, we are running a lot of studies in my lab right now on AI and learning and stuff like that. But Joe sent me this great example. He said, he said, here's this question a student had, and I read the question and I could not understand what the student was asking. Just not. It was about statistics and research methods. The question was so convoluted. I looked at it and I thought, what are they saying? There were terms used wrong. There were multiple terms juxtaposed. And Joe did something pretty neat. He popped into chatGPT and chatGPT provided feedback that I was, I was astounded at. I was like, wait a minute. In one, in three seconds, or probably two seconds, ChatGPT has taken this word salad and provided feedback to the student that read really well. Right? So this is an anecdote, but it says to me, there are some affordances with AI here that I think may really help out. Now that said, there are other variations on the team, right? You said instructor. Well, there's research that shows that even pure feedback is better than no feedback. So know that there are other questions I think to keep in mind, which is, is it written feedback? How about video feedback or audio feedback? Because in many online classes, it's a lot faster to record a quick bit of feedback than type something out. And, you know, none of what my grad students are working on that as her master's right now is comparing these different modalities of feedback.
You see what makes a difference in terms of student motivation and engagement, and learning. So, really how you give feedback and doing it efficiently, I think, and effectively is, and both those words are important, effective, but also efficiently. Yeah. Three hundred people, you know, got to come up with a good way that works, that still makes sense.
Thomas Thompson: The only piece of multimedia feedback I ever received was from one of your coauthors, Dr. Christopher Devers. Oh, yeah. Because he would give other feedback through Flipgrid. So it would be a short 30-second video of him talking to me, typically very directly.
So a persistent topic in educational technology is that of personalized learning. I see the term thrown around a lot. Very rarely is it defined well. And sometimes when it is, you hear concepts like learning styles being used to illustrate its importance. Do both of these ideas just need to be thrown out? What's the difference between them? How should teachers be thinking about this? You know, where do we lose effective methods in like hype and buzz and stuff like this?
John Dunlosky: Let me answer this first because I'll have the least informed answer. And then we'll let Regan answer this. But all learning has to be personalized, period. Right? If I'm a self-regulator, it's probably the best way to personalize it, right? Because it's my current knowledge state. And when I think about it, that's going to drive me to the next best situation. If I'm really good at evaluating what I know and what I don't know, my personalized learning is going to take me to the next step, right? Ideally, for an AI system now, it's just going to keep track of where I am as a learner. And in the same course, Regan and I could be at very different places, even though we've experienced the same things. And to the degree to which any piece of technology can get a good assessment of where I am at and where I need to go next. That could be an advantage.
So I can't imagine any learning. Well, I can imagine some learning being effective, even if it's not personalized. But I can tell you with students, again, being so overwhelmed with all the content they need to learn, some form of personalized learning is going to be really critical for that individual student to be most efficient, happy, and effective. Whether that could be offloaded to some technology would be awesome, right? Or whether that individual needs to take the track themselves to understand how best they can personalize their experience to gain the most is going to be essential. Now, basically what I'm saying, Thomas, is that all learning needs to be personalized, but that's way too general to be useful.
Hopefully Regan will have something useful to add to this.
Regan Gurung: So here's the reality. The reality is, higher education and K-12, and especially K-12, have been doing or playing with personalized learning since the 1970s. All right. We can go back a little further. Good old psychologist B.F. Skinner came up with teaching machines that were personalized learning. So, you know, we've known personalized learning makes a difference. I mean, it had its heyday, 70s, 80s. It pops up and down and it's, you know, and even literally called personalized learning systems, right? But here's the issue though. It's the technology now is getting better, right? So from a social psychology standpoint, right? The moment, and a personality, I guess, but you know, the moment we feel that that information is directed at us, we feel better and taken more seriously, right? And the reason I say social psych is because you can actually write a very generalized statement, right? Think horoscopes and people think it applies to them, right?
So, and that's, I think the scary thing is even the semblance of personalization seems to help. So for me, as I think about it, what's more important, and I make this comment in the context of AI very specifically. I think the big reason that large language models today are so different from all the other stuff that we've seen before is that because they are large language models, because they are scrubbing human text, they sound human. Those responses, your interactions with, with ChatGPT, they sound human, right? And automatically that conveys a kind of relationship, a kind of supportive something that I think is a big part of the teacher learner interaction, right? We've got to build comfort. We've got to build a caring environment where the student is comfortable in learning. And that's from a rapport. And when we build rapport with the student, right? That student pays more attention, feels more motivated, wants to learn, right? And that's what personalization is doing. It's almost like creating this, this rapport. And I think for me, that's where I'm looking is to what extent, right? And, and John said, which I agree with, technology may be able to make this much more efficient. My big question is how human, I don't even know, will that seem, and will it feel the same as an actual live instructor versus a non-live instructor?
And I'll just give you two, two points to support this. Number one, many textbook companies right now, and their learning management systems, with one switch of a toggle, you can send your students feedback every time they finish a quiz. And you can edit that feedback to make it sound like you. And from talking to many students, when they get my automatic feedback in the middle of the night, even though they know it's automatic because it's my voice and my words, they still feel a connection. And of course, some of them even actually think I was up at 3 a.m. when they took the quiz, that they respond. And I say, so, you know, I've said that, so I don't have to be up, but, but they still go, but they still appreciate that, right?
That's exhibit A. Exhibit B, this one is scary. Joe Slade, who I mentioned previously, and Elena Heick in my lab ran a study where they had an AI avatar, give a 15 minute lecture on operant conditioning and had a live person give the same lecture. And then students took a quiz on it. Learning scores were the same, whether it was an AI avatar or not. And maybe that's not, you know, that's not that surprising because the content was identical, right? Here's what really gives me pause and, you know, intrigues me is when students were asked, how connected do you feel to the instructor? How, how much rapport did you have? How likely would you be to go ask for help? No differences. One was an AI avatar because the AI avatar looks human. It's not a pencil. It was a person, but you know, AI, they still felt it. And that's why, so intrigued by this humanness feedback. Well, you know, that's where I come back to the feedback is, yeah, I hear you. Yes, some feedback, no feedback, but I think there's that humanness and I'll, and I'll close this little parade as it were by saying so many of our students are experiencing so many mental health issues, such a need for support that they are hungry for connection, even if it's an instructor giving them feedback, it's connection.
Thomas Thompson: That's interesting because I mean, a lot of large, large language models are their phrasing is selecting for coherence and fluency. These are things that it rates pretty highly, not factual accuracy. So you end up with statements that are statistically correct, but factually inaccurate, which is a strange misunderstanding.
We call it hallucinations, which I think anthropomorphizes it too much, but that's a different conversation for a different time. But what you were getting at with the AI avatar and the human instructor is quite interesting. I mean, there's been some recent studies that found in blind testing, you know, people prefer AI art over human generated images and largely couldn't tell them apart, even if they state that they dislike AI art still were rating AI art more highly than they were rating the human works in the selection. It's the like form of communication and of coherence, you know, beat out on the actual conveying of meaning that we think we're doing is a tough question that we're probably going to have to address at some point. We've talked a lot about methods for teachers, for students. One danger I see for many teachers is seeing man with a hammer syndrome.
That is, they have a set of evidence-based teaching or learning practices. They adhere to them like a checklist, using them everywhere like a man with a hammer who sees everything as a nail.
They see every instructional situation as calling for their pet methods, the ones that they like to use most frequently. What would you share with that person to make them a better teacher? And all that said, should they still read Barack Rosenshine's 10 principles of instruction?
Regan Gurung: Yeah, you know, so it's, I'm going to say, let alone the 10 principles, there are so many different collections, Chikring and Gamson, you know, so on and so forth. There are so many of these. And I'll say this, that it's very hard to come up with an empirical set of best principles for teaching. I'll say that. I think the biggest thing that's important for me is for teachers to be aware of what's out there, because whether it's instructional methods or study techniques, but especially instructional methods, you've got to find the instructional method that works well with your philosophy of teaching and your style. I think that's very important.
Now, how you can, but regardless of what instructional method you use, I think we've got to do, we, higher education, education, we've got to do a better job of sharing this, the science of what works for studying. We've got to do a better job of sharing it with teachers as well. And what John and I have done with this book is, I think we've been pretty successful as based on the feedback, but sharing it with a lot of students, right? And interestingly enough, I'm going to share this with some more teachers and say, look, this is the kind of stuff we need to get out to more people. So the 10 principles aside and all that, I think there are some really key principles for how to study and how learning works that we've done the hard work for you and put it into this little book. That's easy to read.
John Dunlosky: Exactly. I think what I would tell these teachers is that they should consider themselves students. And what we tell students is that they have goals for your class.
If you have set goals where you want your students to obtain these goals, that as a teacher's goal, my goal is that they learn X, Y, and Z, right? Well, if I see semester after semester, many of my students are failing to learn X, Y, and Z, it's time to change my practices, right? To look for another technique to use to help students obtain their goals.
That said, if you're a teacher, you do have goals. The students are learning what you want them to do. I mean, if it's not broken, don't fix it.
To some degree, my guess is where Regan is now at his teaching, many things that were broken 15 years ago, he's fixed. They work well and you go for it. That said, if what you're doing year after year doesn't work, you have choices to make, either this is a sport that afford, but either continue to be the Cleveland Browns and lose year after year because you refuse to make any changes or decide to be a winning team and it's like, this is not working, right? Many of my students are still failing and know that it's not your fault that they're failing, it's not their fault. It's the joint contract that's not working out. You need to find ways to help them learn.
So, I guess my, it's more of a generic thing. Hey, teachers need to go set their goals, evaluate whether students are getting there and if they are thumbs up, and if not, it's time to pick up study like a champ and other how-to books and maybe try on a couple of different more teaching principles to get you where you want to be and where you want your students to be.
Thomas Thompson: Okay. I have one more long form question, a couple of rapid fire, and then we'll get you out of here on time. I think this opens us up to a meta question. The rationale for the show is that one, a podcast is a useful form to explore ideas, which is really just a benefit for me. I'm deeply engaged in a single task. I get to hold two really intelligent people hostage and ask them a bunch of questions for an hour, right? That's great for me. The second rationale is my belief that a podcast is an interesting medium for learning, and maybe informally, but do you disagree with that statement? You know, people often multitask while listening to a podcast. Does that limit their ability to effectively get something out of this? I'm doing the dishes and I have a podcast in my ear. Are these two cognitively taxing tasks? Am I switching between, you know, the pots and pans of the podcast? Is that having an effect on how well I'm engaging with you?
John Dunlosky: Of course it is. But you have to think about the context, right? If I'm listening to this podcast, my goal is not to memorize it all, not to go, blah, blah. I want to get excited about something. Hey, there were a lot of interesting things that I learned from that, or like that I know about now that I didn't, that I can write, go and find out more about, like grabbing a copy of Study Like a Champ or something. So yeah, a podcast should be casual. It doesn't come on all experiences in life. You don't have to learn something from, you just want to experience and have a good time. So podcasts like this, if someone's engaged doing dishes, I would probably be playing my guitar while I'm listening and not doing well at either of the things, but at least I would pick on, right, pick up on snippets.
So by all means, you know, wash your dishes, you know, even have the TV on mute and listen to a podcast, I think you can get a lot out of it. And then if there's something you think you missed that's good, go back. You can listen to it again, right?
That's the key.
Regan Gurung: Well, I'll take it a step further and go, there's some really intriguing evidence that the way we learn and pay attention is different through auditory channels versus through visual channels and reading. You know, Marianne Wolfe has done a great job, Browsed in the Squid, of talking about, of demonstrating how, you know, the human brain wasn't, didn't, wasn't, isn't necessarily designed to read, you know, dealing with symbols. That takes work. That's why reading is so hard to learn. But, but on the other hand, listening may provide a different channel. And so I think, you know, in addition to what John said, it's, you know, it's better to get exposure to this in this way than not get exposed to it at all.
And I'll add to that by the fact that reading takes a different kind of attention, different kind of focus, and the whole mentality is different. So I love this. You know, it's the classic Trojan horse approach.
You know, let's get your, you know, get your foot in the door and get, get, get the excitement going. And with some luck, they'll listen and want to find out more because nothing's more powerful than being able to inspire somebody and motivate somebody to want to know more. So kudos to you for doing it.
Thomas Thompson:
And if you want to find out more, go in the show notes. Study Like a Champ is available, like I said, on Amazon, real cheap. It's a great resource. If you're teaching and you're listening to this, you should definitely pick it up, as well as to some other studies. We'll link all that in the show notes along with full transcripts. All right. Last segment here, rapid fire questions. I'm going to present a concept and you'll tell me if you think that concept is overrated or underrated. Doesn't have to be binary. It can be appropriately rated. But hot takes are welcome and encouraged. Ready? All right.
Highlighting texts in textbooks.
John Dunlosky: Underrated.
Thomas Thompson: Underrated?
John Dunlosky: Well, I mean, it's just, it's often misused. You highlight and then you turn those highlighters into flashcards. It's the beginning of the journey. So as a learning technique, it's overrated. As a skill, it's probably underrated. I don't know. There, that's how, how's that for nuance?
Thomas Thompson: Scholarship of teaching and learning (SOTL)
Regan Gurung: If done well, underrated. Significantly underrated. Not enough attention paid to it.
Thomas Thompson: Discovery learning.
John Dunlosky: Appropriately rated? I mean, it's, it's important, but any, most things that are really difficult, a mentor needs to be involved at some point. So initial discovery learning, then with scaffolding from, but now, at least in many of the things that we learned, we don't need to be on an academic campus to get the scaffolding. Right. So yeah, I'm all for discovery learning, but when your discovery takes you to a dead end or to a barrier, it's time to go out and discover who can help you past it.
Thomas Thompson: The apprenticeship model, as opposed to, say, like full group instruction, right? You have someone who is new in a field working directly under someone getting direct, immediate, timely, specific feedback on their work, right? This kind of thing.
Regan Gurung: Great when you can get it.
John Dunlosky: Yeah. Well, I think, you know, it's a, I would say it's openly rated because you need, you need an apprenticeship when you get to levels of knowledge that require a master to understand that you don't understand. That's why you go to grad school.
That's why Regan and I are so valuable to our grad students, because we have knowledge that they don't, and we can evaluate and help them take that next step. We are their kind of help in their personalization of learning. So I don't know.
I think mentorships are important, but we have to understand that they're mentorships, right? At some point, the mentees outgrow that, move on.
Thomas Thompson: Certainly a lot of boundary conditions there. People stick on a mentor too long or try to emulate them too much. And it kind of stunts their growth overall.
Okay, um, Grit.
...
Regan Gurung: While John pauses, I'll say, you know, I notice so, so often my answers are so nerdy, but I'll give you the nerdy one. I mean, the data is so, I mean, a nice concept, but, you know, the operationalization has been, has been troublesome. So, yeah.
John Dunlosky: So I'll say it, overrated, overrated, but not as a, everybody needs grit to succeed. Yeah. But if I tell you, oh, you've got a lot of grit for calculus. Well, you're not going to succeed without the right strategies and all the other things you need to succeed, right? I want to be a virtuoso on a violin. I can have as much grit as you want. It's not going to happen without the right mentorship, so it's going to be common. However, to think of any one factor, like grit or self-efficacy, where the term originally came from, is one piece of a very difficult and very large elephant that all has to go together to work well. So on its own self-efficacy, thank you. And not all of its stolen versions subsequent to then, uh, there's a, there's a place for it, but it's overrated.
Thomas Thompson: Only a few more left. Duolingo. As an application for spaced repetition, overrated, underrated, appropriately rated. I
John Dunlosky: don't know enough about Duolingo to give a confident answer.
Thomas Thompson: I'm seeing some nodding on that one.
Regan Gurung: My wife likes it. And I've seen some data on its efficacy that looks pretty promising.
Thomas Thompson: I mean, is it more of like I'm doing the semblance of learning and being entertained mostly, or is it a supplement? It shouldn't be the source of me trying to learn Japanese, like do something else and have Duolingo in the background.
Regan Gurung: My sense is, and again, I haven't, I've only dabbled with it a little bit, is it opera, you know, it, it provides a practical way for retrieval practice, you know, as one of its components. It does other stuff too, but, uh, I think, I think you did say it's that fun part of it. So yeah, this is more of the, I haven't looked enough into this particular product to know, you know, to know one way or the other.
John Dunlosky: I'm going to have to pass on that.
Thomas Thompson: Last three, multiple choice questions.
Regan Gurung: Yes. Appropriate if written well, more often not written well.
John Dunlosky: Yes. Both over and underrated at the same time.
Thomas Thompson: Let me pick one more here. I think I have too many: peer instruction and group work.
Regan Gurung Peer instruction and group work. You know, teaching is, you know, teaching is learning twice, is, you know, and so lots of strengths of, of peer instruction. Uh, and of course, Eric Mazur and his work out, you know, out these show that peer instruction can be pretty solid. Uh, so enough really promising work coming out of there to go probably appropriately rated.
Thomas Thompson: Dr. Grung, I guess this one would be for you. Open stacks, open access textbooks.
Regan Gurung: Yeah. So, you know, I've done a lot of work with, with, uh, OERs and open educational resources, and, and here's my bottom line on it. Uh, there's a lot of variance in quality and here's the most important thing, just an open educational resource without the training on how to use it well, either as a teacher or an instructor can lead to lesser learning outcomes.
Thomas Thompson: I fear we're going to fall into many of the same pitfalls we fell into with OER with AI. I can foresee the same thing where you're going to overhype like cost savings. You're going to overhype time savings at the expense of focusing on methods.
Regan Gurung: That’s right, and this is why the hard work of learning and developing some expertise is important to use OERs or AI. If you just use it blindly, you know…
Thomas Thompson: Last question. What are you each working on next? And practicing guitar is perfectly a fine answer.
Regan Gurung: Well, let me say that John is really good at the guitar. I love how he's always modest about his guitar playing. I am working on a sequel to studying like a champ, which is teaching like a champ.
John Dunlosky: I love it. In fact, I'm excited for Regan's next step. I'd love to read that book. Yeah. My next step, I'm currently emeritus. I just retired recently. So my next steps are basically finalizing some of my last research on successive pre-learning for those in the studio who aren't in the studio audience. I'm like doing a little crying here. It's like, I cannot believe the end of this journey is almost here. But yeah, it is wild, wild, right? Regan and I went to grad school together, by the way. So he's, but how could I be so much older than you?
Thomas Thompson: Gentlemen, thank you both for your time today. Thanks for joining us on Learning Stack.