Mary Loder: - Welcome to Course Stories,
Mary Loder: produced by the Instructional Design and New Media team
Mary Loder: of EdPlus at Arizona State University.
Mary Loder: In this podcast, we tell an array of course design stories
Mary Loder: alongside other ASU online designers and faculty.
Mary Loder: On today's course story.
Andrew Maynard: - This is an incredibly powerful,
Andrew: but an open-ended platform
Andrew: where it's as good as the creativity you bring to it.
Andrew: And helping students understand that
Andrew: within the limitations of the system,
Andrew: they can play and they can innovate themselves
Andrew: and they can learn for themselves
Andrew: how to get the most out of it.
Andrew: That then becomes transformational.
(lively music)
Mary: - Hi, I'm Mary Loder,
Mary: an instructional designer from ASU Online.
Ricardo Leon: - I'm Ricardo Leon, I'm a media specialist
Ricardo: at the same place.
Mary: - Yeah, we work together.
Ricardo: - Let's get on with the show. Mary: - Okay.
Ricardo: - Hey Mary, and hey Liz.
Liz Lee: - Hey. Mary: - Hello.
Ricardo: - How are you guys today?
Mary: - Good. Liz: - Pretty good.
Ricardo: - What's going on today?
Mary: - We had a few great people at the university
Mary: around generative AI join a conversation today.
Mary: Generative AI is like a very hot topic.
Mary: This will be a highly listened-to episode, I have a feeling.
Ricardo: - And so what's the course
Ricardo: that we're focusing on this episode?
Mary: - Well, FIS 394 is an introduction to
Mary: basic prompt engineering using specifically ChatGPT.
Mary: So Andrew hosted this class for the very first time,
Mary: and I think it's the first of its kind that was hosted
Mary: in like a really large public university this summer.
Mary: There were quite a few enrollments,
Mary: not only of like students who are fully enrolled here,
Mary: but also staff who we're gonna be hearing from
Mary: in the episode as well,
Mary: because we have access to these classes as ASU employees.
Ricardo - Yeah, so who is Andrew and who are these people
Ricardo that we're gonna be talking to?
Mary: - Andrew Maynard is, well,
Mary: he's who I previously called the king of podcasting
Mary: until it offended you, but he is a wonderful--
Ricardo: - Until I defeated him at podcasting.
(Mary laughs)
Mary: - Yes. He's a wonderful human being who has lots of insight
Mary: into how technology integrates into our future.
Ricardo: - And who else did we talk to?
Liz: - We also talked to Mary Ann Naumann,
Liz: she works with the ASU Library.
Liz: And she has a really unique perspective being a librarian,
Liz: especially her views on ChatGPT, AI and research,
Liz: how it can help and how it doesn't help,
Liz: which is fascinating.
Liz: And she also took not only Andrew Maynard's course,
Liz: but she also took the GenAI course developed by Tamara,
Liz: who we also spoke to today.
Mary: - And Tamara worked with lots of people
Mary: across the entire university to put that course together.
Mary: She was an excellent project leader,
Mary: did a ton of work herself.
Mary: This is a gorgeous course for faculty and staff at ASU.
Mary: And while it's not available to everybody,
Mary: we will put a link in our show notes.
Mary: 'Cause if you haven't gone into it,
Mary: or if you need to reintegrate
Mary: 'cause it came out just during a really busy time.
Ricardo: - Yeah, and it was all hands on deck.
Mary: - Oh, yes.
Ricardo: - We were involved in it,
Ricardo: we were creating video content for it.
Mary: - Writing content, revising content,
Mary: helping others who were doing parts of it.
Mary: I mean, it was just,
Mary: anybody who was interested in helping was, you know,
Mary: invited in to help.
Mary: And then those who were helping, it was like,
Mary: that was our sole focus of our life for the entire summer.
Ricardo: - You guys got invitations, I was voluntold?
Mary: - You kind of get voluntold to do a lot of things
Mary: through the media team because you know, you're offered up
Mary: because you are so good.
Mary: That's what happens when you're good. (laughs)
Ricardo: - There's a punishment for doing a good job
Ricardo: or something like that. Mary: - It's not a punishment.
Ricardo: - Well, there's like a term, there's a.
Mary: - An elevation of work.
Ricardo: - Okay, that sounds good.
Ricardo: And well, so yeah,
Ricardo: we're gonna listen to a really good conversation,
Ricardo: I wanna jump right in.
Ricardo: But also I wanna billboard that
Ricardo: the end of this episode, we've got a special treat,
Ricardo: a special use of AI that Liz is going to talk about.
Liz: - Yeah, so stick around if you wanna hear more about how to
Liz: deceive your in-laws using ChatGPT.
Liz: I'll just leave that teaser.
Mary: - We got tips from the experts,
Mary: that was a good call to ask them those questions
Mary: while they were here.
Liz: - I made sure to join this podcast
Liz: specifically to be a part of this conversation
Liz: so that I could get some good prompt engineering feedback.
Mary: - We're all rooting for you.
Liz: - Thank you, I'm gonna need it.
Mary: - All right, welcome you guys, gals.
Andrew Maynard: - Good to be here.
(group chuckles)
Mary: - We're gonna have a whole discussion
Mary: around generative AI today,
Mary: but specifically from a very unique perspective
Mary: 'cause we have Andrew Maynard, Mary Ann Nauman,
Mary: and Tamara Mitchell, who are all major contributors
Mary: in different ways for generative AI around the campus.
Mary: And not only do we have faculty perspective, with Andrew,
Mary: we have designer perspective with Tamara
Mary: in a very unique way,
Mary: and student perspective with Mary Ann
Mary: because she was a student in both of the courses
Mary: that we're talking about today.
Mary: So, welcome, I'd like to have you guys
Mary: just briefly introduce yourselves,
Mary: tell us who you are officially at the university
Mary: and anything else that you'd like to add.
Andrew: Yes, my name's Andrew Maynard,
Andrew: I'm a professor of Advanced Technology Transitions
Andrew: in the school for the Future of Innovation in Society,
Andrew: which basically means that I study how we get from here
Andrew: to the future without making a complete mess of things.
Andrew: And that covers everything from AI
Andrew: to any other technology you can imagine.
Mary: Super important perspective
Mary: when we enter the world of generative AI,
Mary: and we'll talk more about that.
Mary: And Mary Ann. Mary Ann Naumann- Yes.
Mary Ann: My name is Mary Ann Naumann.
Mary Ann: I'm an undergraduate instruction
Mary Ann: and assessment librarian at ASU.
Mary Ann: The group of librarians I work with
Mary Ann: primarily work with new students
Mary Ann: or those who are new to the academic journey,
Mary Ann: figuring out how to find and evaluate sources,
Mary Ann: so that's my specialty.
Mary: Lovely. You've built a wonderful site
Mary: that we'll plug later in the show notes as well,
Mary: if you don't mind.
Mary Ann: Sure, thank you. (both chuckles)
Mary: And Tamara, my course design bestie.
Tamara Mitchell: I'm so happy to be here.
Tamara: I'm Tamara Mitchell,
Tamara: I'm a senior instructional designer with ASU Online.
Tamara: And I have the privilege of working with
Tamara: a wide variety of faculty
Tamara: who are wrestling with the opportunities
Tamara: and challenges of AI.
Tamara: I also feel like this is kind of a full circle moment for me
Tamara: because I would have to be about 10 years ago, 7 years ago,
Tamara: I developed a self-paced course
Tamara: in partnership with one of the leaders at IBM,
Tamara: who is part of the core team for developing Watson.
Tamara: And he said, "Watch out, things are coming."
Tamara: 10 years later, we kind of see the ripples of that.
Tamara: And I know Watson isn't even the generative AI
Tamara: that we're talking about,
Tamara: but it's a shift toward the future and I think
Tamara: we're experiencing another shift toward the future.
Mary: Absolutely.
Mary: Now let's talk first with Andrew about
Mary: the course that you developed this summer.
Mary: So tell us about the course,
Mary: tell us about why you decided to host the course.
Andrew: Sure. So this was an online course
Andrew: that taught students the basics of how to
Andrew: get the best out of ChatGPT specifically.
Andrew: We called it, I forget what the exact name was,
Andrew: but it was basics of prompt engineering using ChatGPT.
Andrew: And the prompt engineering here was not the coding
Andrew: that we used to think about
Andrew: as prompt engineering 6, 12 months ago,
Andrew: but it was how you write those ordinary language prompts
Andrew: that enable you to get the platform to
Andrew: do what you want it to do.
Andrew: And we put this together actually in quite a hurry
Andrew: because we realized earlier this year that
Andrew: this was massively accelerating technology.
Andrew: It went from 0 to 1000 miles an hour
Andrew: almost overnight it seemed.
Andrew: And we were in a position where our students
Andrew: were already using it and we were scrambling to understand
Andrew: sort of what that meant.
Andrew: But more importantly, it seemed very, very clear that
Andrew: as they graduated, our students in order to thrive,
Andrew: needed a set of skills around generative AI
Andrew: that we simply weren't teaching.
Andrew: And they also needed confirmation
Andrew: that they had those skills,
Andrew: so they needed something on their transcript.
Andrew: So we rushed to put together a course
Andrew: on teaching them the basics of how to do this
Andrew: and taught it for the first time this summer.
Andrew: But the course had a number of
Andrew: really intriguing aspects to it,
Andrew: one of them being, that to formulate it fast,
Andrew: I literally sat down one evening in front of ChatGPT
Andrew: and said, hey, I wanna teach a course
Andrew: about how to use ChatGPT, what should we teach?
Andrew: And ChatGPT produced the bare bones of a syllabus,
Andrew: and we actually built it out from there.
Mary: That's amazing.
Mary: That's actually very similar to how Tamara and I worked,
Mary: and many of the others worked
Mary: when we were developing the Generative AI course, right?
Mary: ChatGPT was our first draft best friend.
Mary: So, fabulous.
Mary: Now, the first draft was never the final draft,
Mary: as Tamara can tell you from her hours of revision work.
Tamara: So many revisions. Mary: But it was a starting point.
(group chuckles)
Mary: And Mary Ann, you were a student
Mary: in this class with Andrew.
Mary: What was your first impression coming into the class?
Mary: And also, why did you take it?
Mary Ann: Yes, so as my role as a librarian,
Mary Ann: I'm really interested in when I saw the technology come out
Mary Ann: and seeing how it was impacting potentially writing skills,
Mary Ann: critical thinking skills, research skills,
Mary Ann: that one of the things that is really not
Mary Ann: where there was a huge gap was
Mary Ann: in the aspect of prompt engineering,
Mary Ann: how do we formulate searches in this area?
Mary Ann: There were a lot of people who immediately jumped online
Mary Ann: and started to kind of pontificate about
Mary Ann: what prompt engineering was,
Mary Ann: filling that void for clicks and clout.
Mary Ann: And so I was really excited when I saw this course come up
Mary Ann: because one, I knew it was gonna be taught by somebody
Mary Ann: who was thinking about it from the
Mary Ann: ethical perspective as well,
Mary Ann: but who would be able to guide us in that way?
Mary Ann: So I was really excited to take the course because of that.
Mary: Awesome, awesome.
Mary: So what was your favorite assignment in the class?
Andrew: That's a bit of a pregnant pause there.
Andrew: Were there any favorite assignments?
Mary Ann: It's all of them.
Mary Ann: That's hard to say because there were a bunch of times,
Mary Ann: but I know there were a couple times
Mary Ann: where we were really getting to prompt engineering
Mary Ann: where we were talking about constraints,
Mary Ann: ambiguity reduction,
Mary Ann: some of the different strategies in creating templates
Mary Ann: where it started to click because you were using it
Mary Ann: in a more advanced way more frequently.
Mary Ann: So that was kind of the aha moment where you recognize that,
Mary Ann: okay, I'm getting this,
Mary Ann: I'm getting better output from the system,
Mary Ann: as well as then the modules about
Mary Ann: how do I assess what I'm getting out of it?
Mary Ann: How can I change my prompt based on
Mary Ann: what I'm seeing the output and the response from the
Mary Ann: generative AI system.
Mary Ann: So those were my favorite just because
Mary Ann: it was that aha moment of,
Mary Ann: I think I'm kind of getting this
Mary Ann: where I'm doing a better job at querying the system
Mary Ann: and getting better responses.
Mary: So Tamara, you and the Generative AI course
Mary: for our faculty and staff.
Mary: You also were very focused on prompt engineering,
Mary: how to help those who are in that prompt engineering space.
Mary: What did you do in that course design consideration
Mary: that you think would benefit
Mary: maybe outside of our faculty and staff or,
Mary: I don't know, maybe there was an artifact created
Mary: that we could share somewhere?
Tamara: So I think in the Generative AI course,
Tamara: Teaching and Learning with Generative AI course,
Tamara: that was our module four,
Tamara: and that was the module everybody wants to skip to
Tamara: because everybody wants to play
Tamara: with generative AI, don't they?
Tamara: They want to get into the trenches, they want to experiment,
Tamara: and they want to get some really important things
Tamara: from that work.
Tamara: It's funny though.
Tamara: It's funny because we have so many people who will go to
Tamara: ChatGPT per se and put in a prompt and say,
Tamara: and have it, you know, spit out something
Tamara: that's nothing like what they wanted.
Tamara: And they'll say, "Oh, the tool isn't great."
Tamara: And so it's kind of like somebody
Tamara: walking over to a guitar that they've never seen before,
Tamara: playing a note, realizing it didn't make a song,
Tamara: and then putting it down because they don't like it.
Tamara: And so I think the first and most important thing
Tamara: I would say is that it's a skill and it's a literacy
Tamara: and it's something that we have never had to develop
Tamara: in the past.
Tamara: And because we've had to develop it now,
Tamara: one of the biggest,
Tamara: and I think the best tips that I've seen in
Tamara: Andrew's course where I lookie-lood,
Tamara: and then in the Teaching and Learning Generative AI course,
Tamara: yes, I stalked you Andrew,
Tamara: in the teaching and learning.
Tamara: Can you say that? (chuckles)
Mary: Yes, you should say that.
Mary: We are observers,
Mary: if you wanna use the professional name in canvas.
Mary: But yes, we were full on stalkers. (chuckles)
Tamara: Professionally observed and learned from you.
Tamara: But I think one of the most important things that
Tamara: is taught, and I think that would benefit everyone,
Tamara: was number one, treating it like a conversation partner.
Tamara: If you can't show up with good information,
Tamara: your partner can't respond.
Tamara: Being clear, number two,
Tamara: and number three, one thing that Andrew does quite well
Tamara: that we actually used with his permission
Tamara: in the Teaching and Learning with Generative AI course,
Tamara: was actually evaluating your prompts,
Tamara: and I'm sure we're gonna get to get that a little bit later.
Tamara: But if you can do those three things,
Tamara: then you can refine and iterate
Tamara: and you can get better results
Tamara: and you'll have a better experience with generative AI.
Mary: That is a really good perspective.
Mary: Now Andrew, how did you articulate that to your students
Mary: when it came to prompt engineering in your class?
Mary: Was there like a framework that you gave them
Mary: or some step-by-step instructions,
Mary: how did that work?
Andrew: There was.
Andrew: To go back a step, we actually had a systematic approach
Andrew: to helping students understand what ChatGPT is
Andrew: and how to begin to use it.
Andrew: So we built things up, first of all,
Andrew: by ensuring that pretty much everything they did
Andrew: was within the ChatGPT environment.
Andrew: So they were building up their ChatGPT mental muscles,
Andrew: if you like, around it.
Andrew: But then we started off by giving them assignments
Andrew: that gave them a sense of what it can do, what it can't do,
Andrew: what the limitations are.
Andrew: We then went on to looking at the basics
Andrew: of how you sort of create prompts
Andrew: and how you create prompt templates.
Andrew: But the evaluation bit was really interesting
Andrew: because it was very clear that once
Andrew: they developed those skills in how to develop
Andrew: and how to write a fairly sophisticated prompt,
Andrew: they needed a framework with which they could assess
Andrew: the quality of that prompt and the quality of the response
Andrew: and they could iterate around to make things better.
Andrew: So as we were developing this, again,
Andrew: I was working very closely with ChatGPT.
Andrew: I asked, well, how do we do this?
Andrew: We've gotta evaluate this.
Andrew: What's a really good way of doing it?
Andrew: And it came up with this framework,
Andrew: which we ended up calling the RACCCA framework,
Andrew: R-A-C-C-C-A.
Andrew: Please don't ask me to say what that stands for.
Mary: I could read it off if you need to. (laughs)
Andrew: Oh, thank God I've got two experts here.
Andrew: But the key thing is that ChatGPT came up with these
Andrew: six criteria, which I'll let you read off in a second,
Andrew: which made sense.
Andrew: So we worked around sort of how we would apply that.
Andrew: But then the thing that blew me away was,
Andrew: in this conversation,
Andrew: ChatGPT started abbreviating it as RACCCA
Andrew: without any prompting.
Andrew: So it became the RACCCA framework, which stands for,
Andrew: now you can read it off.
Mary: Well, let's have Mary Ann read it off
Mary: because she might be able to do it rote.
Mary Ann: R for relevance, we've got A for accuracy,
Mary Ann: C for completeness, C for clarity,
Mary Ann: C for coherence.
Mary: I always forget the third C.
Mary Ann: And then A for appropriateness.
Mary: Awesome, and we'll put all of that
Mary: into the show notes as well.
Mary: I think you created a video on RACCCA,
Mary: would you be okay with us sharing that?
Andrew: Oh absolutely, yes.
Andrew: So we have an introduction video to the RACCCA framework.
Andrew: And yes, it actually ended up being a very useful framework,
Andrew: I would say.
Andrew: You can see how poor my memory is,
Andrew: which is why I need ChatGPT.
Andrew: It took me about 20 takes with that video
Andrew: even to sort of get the sequence right on it.
Andrew: So I can talk about RACCCA,
Andrew: but it's very hard without working with ChatGPT
Andrew: to remember the specific things until you're beginning to
Andrew: apply that framework.
Mary: Well, it's common to know how things make you feel
Mary: without being able to recall all the specifics
Mary: of how you got to that feeling.
Mary: So that's very resonant.
Mary: Thank you for letting us share that.
Mary: 'cause I think it's probably one of the better artifacts
Mary: for both courses because it's something that
Mary: we're all looking for,
Mary: like how do we mindfully use generative AI
Mary: to properly, ethically associate this within our world.
Andrew: And I would say on top of that,
Andrew: so the RACCCA framework isn't the only one
Andrew: that there is out there, or that you can use.
Andrew: But the critical thing is bringing that process
Andrew: to things where you evaluate what the output is like
Andrew: and the outcomes are like,
Andrew: and you engage your own brain in determining
Andrew: whether it's fit for purpose and if not,
Andrew: how you're gonna change that.
Mary: Absolutely.
Ricardo: Okay, so this is an acronym being used.
Ricardo: Is it RACCCA?
Liz: R-A-C-C-C-A, I think. Mary: Yeah, three C's.
Ricardo: Can you guys unpack that for me a little bit?
Mary: So in our review of key terms in the Generative AI course,
Mary: this is how it comes across,
Mary: and by the way, this was written in part by ChatGPT,
Mary: not only because that's how Andrew made the framework,
Mary: but because then we use ChatGPT also to reiterate it
Mary: and give it to us in as concise way as possible.
Mary: So the acronym represents a method to evaluate
Mary: generative AI outputs created by Andrew Maynard
Mary: at ASU in partnership with ChatGPT.
Mary: And so R stands for relevance
Mary: and that is to the extent which a response
Mary: directly addresses the issue or question.
Mary: So, is it relevant to what you prompted?
Ricardo: So like, really, let's break this down.
Ricardo: So, what does that mean?
Ricardo: In terms of my prompts?
Ricardo: I'm gonna consider the relevance,
Ricardo: so I'm gonna make sure that--
Mary: Well, the output relevance, 'cause this is evaluating.
Ricardo: Oh, this is evaluated, okay.
Mary: Right. So is what you put in there,
Mary: did it get you what you expected back in reference to,
Mary: does it make sense for what you were trying to accomplish
Mary: around your issue or question?
Mary: And then there's accuracy, that's A,
Mary: the degree to which the response provides accurate,
Mary: reliable, or fact-based information,
Mary: and that was actually part of
Mary: why Mary Ann was taking this class,
Mary: was because she wanted to make sure
Mary: if we're using ChatGPT in that student space.
Mary: Them using it as a research tool
Mary: was probably not an accurate usage
Mary: but it can give ideas on
Mary: how you might go about your research
Mary: or using research skills
Mary: that the librarians can help you learn.
Mary: You can then gauge the accuracy
Mary: using really basic digital competencies,
Mary: like who's actually saying this message,
Mary: where did this come from?
Mary: And with ChatGPT, you don't get citations.
Mary: I mean, you can ask for them and get the wrong ones.
Mary: They're not actually sourced that way
Mary: in the large language model.
Mary: So you need to go and actually fact check and double check
Mary: that the same thing that ChatGPT is telling you
Mary: is being said in other spaces
Mary: that are reliably tracked and cited, and you know.
Liz: Yeah, this methodology for evaluating
Liz: kind of reminds me of some of the other methodologies
Liz: that already exist in higher education
Liz: SIFT, other ways that we can evaluate
Liz: the research that we're looking at,
Liz: the sources that we're evaluating.
Liz: I work a lot with our English 101, 102 students,
Liz: that's a huge part of doing the research,
Liz: is evaluating what you're looking at for its accuracy,
Liz: for its relevance.
Liz: But it also reminds me of the big scare of Wikipedia.
Liz: When it first came out, people were like,
Liz: oh, you can't trust Wikipedia.
Mary: People still say that.
Liz: They do, but I feel like much like Wikipedia,
Liz: ChatGPT is gonna turn into a ideation,
Liz: beginning the research spot.
Liz: And I think that this RACCCA methodology
Liz: structure is gonna be a helpful way to
Liz: help students learn how to work with the robots,
Liz: not against them.
Ricardo: And that's one of the things
Ricardo: that Andrew kind of kept going back to,
Ricardo: because we have this new tool
Ricardo: doesn't mean that we still need to scrutinize
Ricardo: what we're looking at.
Ricardo: And he mentioned that, you know,
Ricardo: just in any technology and also just in people in general,
Ricardo: you still have to scrutinize and kind of vet
Ricardo: the information that you're getting from anyone.
Liz: Yeah, seems like critical thinking around
Liz: what you're hearing, what you're learning
Liz: is more important than ever.
Ricardo: Yeah, what are those three C's now.
Mary: What you're learning is more important than ever,
Mary: that's a good transition to completedness,
Mary: so the completeness of the response.
Mary: Does the degree to which the response was given,
Mary: does it cover all the essential aspects of the topic
Mary: or question being asked?
Ricardo: Right, 'cause I put in prompts before and I thought,
Ricardo: well, you didn't reference this other thing
Ricardo: that I asked you to reference.
Ricardo: So that is--
Mary: And you need bring it up again, yeah.
Mary: So it's a constant refinement process, right?
Mary: And then the clarity,
Mary: how easily the response can be understood
Mary: by the intended audience.
Mary: So I've had that, right?
Mary: Give me a writeup for this letter,
Mary: but then I didn't actually articulate who the audience is.
Mary: And then when you articulate,
Mary: no, the audience is a bunch of five-year-olds.
Mary: Okay, well it's gonna be said very differently
Mary: than how that first draft was.
Mary: So priming the model is very important as well,
Mary: making sure that it understands everything
Mary: that you're looking for before you're trying to get it.
Mary: So back to completeness,
Mary: clarity of what you're looking for
Mary: helps you also accomplish completeness.
Mary: And then coherence, the extent to which
Mary: the response is logically structured and well organized
Mary: and flows smoothly from one point to the other.
Mary: I think that's pretty nice.
Mary: In ChatGPT, there's like a conversational history.
Mary: So, that coherence is pretty easily built in
Mary: through an ongoing conversation,
Mary: but it does have to be ongoing to continue to refine it.
Mary: And then the last one's appropriateness,
Mary: how well the response aligns with the intended audience
Mary: and context and is suitable and respectful
Mary: in tone and content.
Mary: Because that bias exists,
Mary: and so knowing what the biases are
Mary: and how individuals take certain things, like,
Mary: the model itself, its burstiness and perplexity,
Mary: like how it actually talks is very wordy, right?
Mary: So if you're not refining it for certain audiences,
Mary: it gets very liberal with its language flow,
Mary: like, is that normal? Ricardo: It's kind of a bad writer.
Ricardo: In my experience, it's kind of a bad writer.
Mary: Yeah, it's like patting the audience.
Mary: Like, anytime I get something
Mary: that's way longer than it needs to be,
Mary: it's fine for a first draft,
Mary: but then refining it.
Mary: The appropriateness, yeah, you don't need to, you know,
Mary: try to sweeten up the message for every single person
Mary: you're talking to, that's inauthentic.
Ricardo: Okay so you get a response,
Ricardo: and now you've applied these criteria
Ricardo: to see if it was a good response,
Ricardo: but you need to make some refinements.
Ricardo: What are you doing when you're making that refinements?
Ricardo: Are you telling it good on these points, bad on this?
Ricardo: How do you refine?
Ricardo: Are you just coming up with a new prompt?
Mary: Sometimes I'll say "thank you very much"
Mary: because we should always thank our overlords ahead of time
Mary: before they take over and be very polite to them.
Ricardo: I think about that all the time.
Ricardo: Someone who yells at their cell phone for being an idiot.
Mary: Absolutely, I yell at Siri a lot,
Mary: and I'm sure she remembers.
Ricardo: Oh yeah.
Ricardo: I'm always polite to my.
Mary: I should be more polite to Siri,
Mary: that's a good reflection for me.
Mary: But with ChatGPT, like I just, honestly,
Mary: I do put that human aspect into it.
Mary: And I know there are people
Mary: who don't believe you should do that,
Mary: but it's just like a common way to respond.
Mary: So anyway, I'll start it with, thank you very much,
Mary: that was a great response.
Mary: However, I was actually looking for this.
Mary: And then it'll be like, oh, I'm so sorry,
Mary: 'cause it's also super polite,
Mary: and will come up with the refined approach.
Mary: So yeah, that's really common.
Mary: Is that what the question was?
Ricardo: Yeah, just how you're, you know,
Ricardo: so you're giving it a new prompt rather than
Ricardo: telling it's a build off its last prompt.
Mary: Sometimes I'm asking it to build off it last prompt.
Mary: Like, for this course when we were building them,
Mary: Tamara did a great job of laying out like A, B, C,
Mary: this is in every single module, this is the stuff we need.
Mary: So I copy and paste it,
Mary: her template in there to prime my ChatGPT.
Mary: And I'm like, and then this is the topic.
Mary: Give me some potential opportunities
Mary: to like fill in the blanks of the template,
Mary: and it would every single time.
Mary: And then if it got it a little bit better
Mary: in one of the other prompts, I'd be like,
Mary: refer to this specific one when doing the rest of these,
Mary: and then it refined it every single time.
Mary: So yeah, absolutely building on the conversation
Mary: and building on the prompt and priming.
Ricardo: Just say chat 'cause you said chit.
Mary: Chit?
Mary: ChatGPT, my words.
Ricardo: You're not prompt,
Ricardo: your response is--
Mary: Someone needs to prompt my brain with food, I'm hungry.
Mary: How many students did you have in your class?
Andrew: 72 Students, plus lurkers.
Mary: And there were a lot of us lurkers, right? 628 Mary: 00:23:54,927 --> 00:23:56,670 I kept adding people like, Andrew,
Mary: is it okay if I just keep adding as many as we want?
Mary: So from that, from the 72 students,
Mary: how many assignments did you have to grade?
Andrew: A crazy amount.
Andrew: I thought I was putting together an easy course.
Andrew: There were individual assignments across those 72 students,
Andrew: a little under 3000.
Andrew: Out of those, there were a little over 2,300 conversations
Andrew: between individuals and ChatGPT
Andrew: that I read through.
Mary: You read them all?
Andrew: I read them all, well, I skimmed them all.
Andrew: But the thing was, yes, it was a lot of work,
Andrew: but it gave me an incredible insight
Andrew: into how people were engaging with the platform
Andrew: and how they were using it.
Mary: And how were they using it?
Andrew: They were using it in many, many different ways
Andrew: that I didn't expect.
Andrew: And what I saw through many of those conversations was
Andrew: how ChatGPT sparked their curiosity and critical thinking,
Andrew: rather than it just dulling their brains, it lit them up
Andrew: and you could see it in terms of the questions they ask.
Andrew: It goes back to this idea of conversations.
Andrew: The conversations were rich, they were deep,
Andrew: they went in directions I didn't expect,
Andrew: they covered topics that I didn't expect.
Andrew: I had students having long conversations around
Andrew: how to synthesize different types of plastics
Andrew: and the chemistry behind them.
Andrew: Stuff that I didn't know
Andrew: but stuff that they actually got a lot
Andrew: of detailed information from where they could validate
Andrew: and test that information then carry on the conversation.
Mary: Interesting.
Mary: So what was the validation mechanisms in those conversations
Mary: that you noticed that you thought
Mary: that was a good way to do it?
Andrew: Much of it was typical approaches to critical thinking
Andrew: where if you get a piece of information,
Andrew: you work out what feels like it makes sense,
Andrew: what aligns with what you already know,
Andrew: what maybe doesn't align.
Andrew: And then the next question you ask is based on
Andrew: where those alignments are or disalignments are.
Andrew: So you had students push back a little bit or say,
Andrew: can you explain this a little bit more?
Andrew: Or what is your reasoning behind what you said?
Andrew: And the really interesting thing about this platform is
Andrew: that it can simulate reasoning,
Andrew: it can tell you why it comes up with a response.
Mary: Interesting, and sometimes it'll tell on itself, right?
Mary: Like it'll say, I had to tell you something. (chuckles)
Andrew: Oh, it does, yes.
Andrew: It's very endearing the way it sort of
Andrew: either sort of corrects itself or apologizes or says,
Andrew: well, this is what I was told to say.
Tamara: It also has some pretty interesting safety features.
Tamara: Over spring break, I was showing my youngest daughter to
Tamara: use ChatGPT to do kind of an educational scenario
Tamara: that I had her feed in as a prompt,
Tamara: and it was based off of Harry Potter.
Tamara: So maybe there was copyright there, I don't know,
Tamara: but it's personal.
Tamara: But she was trying to convince the ChatGPT to let her go
Tamara: chase the Dementors into the dark forest.
Tamara: And it kept telling her it wasn't safe
Tamara: and it wouldn't allow that action.
Tamara: And so she had to reason with ChatGPT to explain,
Tamara: and she learned how to reason through
Tamara: what was safe and what was not safe
Tamara: because of the conversation, which was an unintended result
Tamara: but it was a nice safety feature that it has, so.
Tamara: Not a bad thing. Mary: I love that story too.
Mary: You're such a fun mom, I love that.
Mary: From the perspective of the student, Mary Ann,
Mary: were there any conversations that you had with ChatGPT
Mary: from the outcomes of the assignments you were given that
Mary: surprised you or where you learned deeper skills?
Mary Ann: So most of my focus in the class was very, very narrow.
Mary Ann: So I looked at it as to inform my position
Mary Ann: and my role as an instruction librarian
Mary Ann: and working with first-year students
Mary Ann: who are learning how to do research
Mary Ann: and learning that it goes beyond finding information
Mary Ann: to exploring and building knowledge and understanding.
Mary Ann: So I focused mostly on that.
Mary Ann: And when I got to some of the more sophisticated prompts,
Mary Ann: as I got better at it,
Mary Ann: I was starting to see
Mary Ann: where it could be very useful for students.
Mary Ann: So I would start to get responses that would
Mary Ann: approximate what talking to a librarian
Mary Ann: or talking to your instructor might be,
Mary Ann: and so I got very excited about that, so.
Mary: You created an artifact that you shared
Mary: in our Generative AI course, would you talk about that?
Mary: This was a result of the work you did in Andrew's class,
Mary: is that correct? Mary Ann: Yes.
Mary Ann: Sure, so one of the things that is lacking out there
Mary Ann: at this point are really
Mary Ann: what I thought were decent academic-focused prompts
Mary Ann: to help students or learners through the research process.
Mary Ann: Rather than asking ChatGPT to write a paper for you
Mary Ann: or to find you sources,
Mary Ann: what is the role it could play
Mary Ann: in being a brainstorm or being a partner.
Mary Ann: And so I started to develop prompts in that way,
Mary Ann: focusing in on those composition classes
Mary Ann: that we work very closely with,
Mary Ann: kind of identifying where students get stuck in the process,
Mary Ann: whether it's from needing to do a lot of research
Mary Ann: before jumping into academic sources,
Mary Ann: maybe that's learning what kind of words
Mary Ann: or jargon they might need to use
Mary Ann: when they're going into databases
Mary Ann: to search for academic sources.
Mary Ann: That was my focus.
Mary Ann: And you know, it was really interesting to come at it
Mary Ann: also as an expert in search, but a novice in this,
Mary Ann: and having that intellectual humility.
Mary Ann: And I think it's super important
Mary Ann: and being able to kind of embody that space
Mary Ann: where students are, where they're learning how to
Mary Ann: use the tool for the first time.
Mary Ann: So I'm not sure if that kind of gets to your question.
Mary: Definitely gets to it.
Mary Ann: But you know, I really looked at the assignments
Mary Ann: and the kind of work that students would need to be doing
Mary Ann: and tried to generate prompts that people could just
Mary Ann: pull from that would guide them through that process.
Mary Ann: So rather than asking basic questions,
Mary Ann: it would lead students to asking ChatGPT, for example,
Mary Ann: the questions that would have it respond back
Mary Ann: and kind of guide some conversation, so.
Mary: Do you plan to share those prompts on your library kit?
Mary Ann: - Yes.
Mary: Is that how we should be referring to it,
Mary: or library guide? Mary Ann: Library kit, yep.
Mary: Yeah, it's a fabulous little site
Mary: that you've put together.
Mary: I think a great resource for all instructors
Mary: to share in their courses,
Mary: especially if they need a little assistance
Mary: with how they might use ChatGPT in their course.
Mary: It might be good to review that resource.
Tamara: I just wanted to say,
Tamara: I really love what we're talking about when I hear
Tamara: Andrew mentioning, you know,
Tamara: how he can help his students actually learn more
Tamara: with these prompts.
Tamara: And then we have this wonderful library guide
Tamara: to prepare students for research.
Tamara: And that's kind of, I think the best part about that,
Tamara: I think it goes back to Mary,
Tamara: maybe a conversation that we had,
Tamara: I think at the end of last week where we were talking about
Tamara: there's a lot of concern about using generative AI,
Tamara: that students will become, you know,
Tamara: overdependent on the tool
Tamara: or that they won't think critically,
Tamara: when in fact, it actually requires to use the tool properly,
Tamara: you have to be on the higher level of blooms
Tamara: and have critical thinking and be on the evaluate level
Tamara: so that you can actually look at your response
Tamara: and then change the way you communicated,
Tamara: fill in the gaps of the information
Tamara: that you should have provided anyway,
Tamara: and then make decisions about refining what you're doing.
Tamara: And I think that's exciting for educators because that means
Tamara: that we're capable of doing so much more in classes,
Tamara: helping our students to go from just rote memorization
Tamara: to the identify level,
Tamara: all the way up to more of a creative level
Tamara: and an evaluate level with fewer resources
Tamara: and with lower course costs.
Tamara: So it's kind of an exciting time to be in.
Mary: Absolutely. I do wanna highlight that
Mary: I think it would really work well at scale
Mary: based on the model that you used, Andrew,
Mary: and the prompt refinement questions that you included
Mary: in some of your assignments.
Mary: Could you talk about that?
Andrew: Yes, so with a proviso here.
Andrew: So absolutely, this is a system that could
Andrew: help us to scale up learning massively
Andrew: if we create the right environments.
Andrew: And certainly what we did with prompt refinement,
Andrew: we helped our students understand how you start off
Andrew: with a simple prompt and see what you get
Andrew: but then how you begin to either refine the prompt
Andrew: or refine the conversation
Andrew: so you get much richer information coming back,
Andrew: specifically in that realm of brainstorming ideas,
Andrew: developing your own understanding in a partnership.
Andrew: So that's where this now becomes
Andrew: a personalized learning environment
Andrew: that you can put out there at scale.
Andrew: But the proviso is students have gotta be able to understand
Andrew: when to trust what they get, when not to trust it,
Andrew: and when to push back.
Andrew: If they don't have that level of digital literacy,
Andrew: they're not gonna, well, A,
Andrew: they're not gonna get what they need outta the system,
Andrew: but also the system becomes potentially dangerous
Andrew: as it pushes them in directions which aren't that helpful.
Andrew: So then the question comes,
Andrew: how do we develop that level of digital literacy,
Andrew: that generative AI literacy in our students
Andrew: so that they can thrive with these tools?
Mary: Any suggestions?
Andrew- Well, starting to teach courses like this
Andrew-is a really good way of doing it.
Andrew-But beyond that, I think this isn't just a single course,
Andrew-this is a set of competencies that should be universal
Andrew-in higher education these days.
Andrew-If we are gonna equip our students to either learn in class
Andrew-using these tools or learn on their own
Andrew-just through their own curiosity,
Andrew-they've gotta have that toolkit.
Andrew-And that means both incorporating it
Andrew-into a wide array of courses,
Andrew-but also giving them the tools
Andrew-that where they can independently develop these skills.
Mary- Any suggestions on what these competencies could be?
Tamara- I was just gonna say, you know,
Tamara- something that's really interesting about that is
Tamara- we are constantly telling our students ever since the
Tamara- onset of the worldwide web
Tamara- to challenge what they read online.
Tamara- Now we have to challenge and ask critical questions about
Tamara- what we read in the newspaper or what we read on our phones.
Tamara- And so what's interesting is,
Tamara- is we're not asking students to do anything new,
Tamara- we're asking them to do it at a higher level
Tamara- and we're asking them to do it in a better way.
Tamara- And so we're using ChatGPT or other generative AI tools
Tamara- to not only teach them to do something
Tamara- that we need them to do anyway in a digital
Tamara- and technologically advanced world,
Tamara- but we're also giving them the tools to support that.
Tamara- So I think that's what's exciting.
Andrew- I would absolutely agree.
Andrew- And if we approach it in that way,
Andrew- helping students understand, well, first of all,
Andrew- breaking the barriers down
Andrew- so they're comfortable actually using these tools,
Andrew- but then helping them understand
Andrew- how to navigate what they get outta them
Andrew- and how to have those conversations
Andrew- then becomes very empowering.
Andrew- It's almost like, almost but not quite,
Andrew- like when students first come to university
Andrew- in their first year maybe
Andrew- and they have access to all these incredible instructors
Andrew- and yet they don't know necessarily how to engage with them.
Andrew- They don't know how to go and ask questions
Andrew- and learn through those conversations,
Andrew- and that's a skill that they need to develop.
Andrew- They need to do exactly the same with generative AI.
Mary- Agreed, and this really gets to the new design principle,
Mary- right, principled innovation.
Mary- So how we engage, why we engage
Mary- is very important to consider.
Andrew- Absolutely.
Mary- There was recently a release of
Mary- an internal generative AI option through ChatGPT,
Mary- the one that you were talking about,
Mary- the enterprise integration potentials.
Mary- You wanna talk about that?
Mary- 'Cause that's kind of new,
Mary- I mean it might not be new
Mary- when this episode comes out in a few weeks.
Andrew- No, but as of today, it's brand new.
Andrew- So despite all of the huge potential of not only ChatGPT,
Andrew- but other large language models, there is a problem,
Andrew- and that is a problem of accessibility.
Andrew- And so if in education and learning we embrace these,
Andrew- we run the risk of only those students
Andrew- that have access to the systems
Andrew- really being able to benefit.
Andrew- So for instance, in my course we use ChatGPT Plus,
Andrew- which is the paid version,
Andrew- which meant every student needed to be paying $20 a month
Andrew- to complete that course.
Andrew- And we did that because it felt like
Andrew- this was the equivalent of a textbook,
Andrew- but not every student can afford that.
Andrew- So now immediately, if you have classes where
Andrew- some students are using advanced generative AI
Andrew- and some aren't, there's a digital divide there.
Andrew- And the way to get round that,
Andrew- or one of the ways to get round that
Andrew- is for places like ASU to ensure that
Andrew- everybody has access to these powerful tools.
Andrew- Up until now, that wasn't possible with ChatGPT
Andrew- because they didn't have an enterprise model.
Andrew- That has just been announced,
Andrew- which means that higher education institutions
Andrew- can buy into it in ways that both protect their students,
Andrew- they protect their privacy, they protect their data,
Andrew- but they give them access to incredibly powerful tools
Andrew- across the board.
Andrew- So you no longer have that digital divide
Andrew- because people can't afford to access the system.
Andrew- There is still a digital divide in terms of
Andrew- the mentality where students
Andrew- will or won't engage with these technologies,
Andrew- and that's where digital literacy comes in.
Andrew- But at least we are removing
Andrew- potentially one of those barriers.
Mary- So that brings us back to the
Mary- digital literacy competencies.
Mary- What competencies would you label
Mary- that students need to have,
Mary- that everybody really needs to have, not just students?
Andrew- So I think we're still feeling our way here,
Andrew- this is very early days.
Andrew- But I would certainly have one of those understanding
Andrew- the limitations and the benefits
Andrew- and the power of the system.
Andrew- So, students go in with open eyes and realize that
Andrew- systems like ChatGPT will lie to them,
Andrew- they will hallucinate, so they can't trust everything.
Andrew- At the same time realizing that
Andrew- through the process of conversation
Andrew- they can glean new knowledge from it,
Andrew- that's an incredibly important skill.
Andrew- And of course that ties in with critical thinking,
Andrew- how you evaluate everything you learn and hear.
Andrew- But beyond that, I think there's also a critical skill
Andrew- around curiosity and experimentation.
Andrew- This is an incredibly powerful but an open-ended platform
Andrew- where it's as good as the creativity you bring to it.
Andrew- And helping students understand that
Andrew- within the limitations of the system
Andrew- they can play and they can innovate themselves
Andrew- and they can learn for themselves
Andrew- how to get the most out of it,
Andrew- that then becomes transformational.
Mary- I love that curiosity and exploration
Mary- is now part of our digital competencies.
Andrew- They are, yes.
Mary- That's great.
Mary- Mary Ann, how about from your perspective,
Mary- from the research perspective?
Mary- Did you come away with that course going,
Mary- okay, now this is how I'll articulate a literacy that
Mary- I need to ensure our students have?
Mary Ann- I think I actually came a little bit more hopeful
Mary Ann- that it could be used in the research process
Mary Ann- than I was initially going into it,
Mary Ann- playing around with ChatGPT
Mary Ann- or playing around with Bing or Bard.
Mary Ann- It's not a great tool for research at this point.
Mary Ann- The fact that you cannot trust accuracy
Mary Ann- of the sources that you're getting,
Mary Ann- that there's no way to know what the training data is
Mary Ann- that the system is based off of,
Mary Ann- knowing that there are inherent biases
Mary Ann- underlying the training data.
Mary Ann- knowing that there are huge issues there that, as a academic librarian,
Mary Ann- I kind of get a little bit nervous about
Mary Ann- students relying upon technology that hands them answers
Mary Ann- or allows them to think that the system is neutral
Mary Ann- when it's not.
Mary Ann- So I have, I mean,
Mary Ann- it goes back to basic information literacy,
Mary Ann- basic critical thinking about
Mary Ann- what are the sources that I'm getting,
Mary Ann- can I trust this being skeptical,
Mary Ann- trust but verify, so.
Mary- Trust but verify, I like that.
Mary- So let's talk about that bias for a moment.
Mary- What have you heard, what have you noticed,
Mary- what have you heard through the grapevine
Mary- exists around bias?
Mary- What should our students know?
Andrew- They should know that just like talking to people,
Andrew- there are inbuilt biases to ChatGPT.
Andrew- Some of those biases arise from the data it was trained on,
Andrew- and they reflect human biases and human foibles
Andrew- in those big data sets.
Andrew- And we're actually finding out that
Andrew- because some of those data sets are skewed,
Andrew- that actually enhances the bias.
Andrew- So great care needs to be taken there in understanding
Andrew- what comes out of the system
Andrew- or what those implicit biases might be.
Andrew- There's actually a second level of bias
Andrew- which comes from the training.
Andrew- So OpenAI have trained ChatGPT to respond in certain ways
Andrew- and put certain guardrails in place, which is good,
Andrew- but it also still brings a bias to the responses.
Andrew- That to me is okay because everything around us
Andrew- has those implicit biases,
Andrew- but you've gotta be aware of them,
Andrew- you've gotta be sensitive to them and understand
Andrew- either how to counter them or navigate them.
Andrew- And that comes back to another set of skills
Andrew- that the students need.
Andrew- If they just absorb without thinking
Andrew- what they get outta ChatGPT,
Andrew- it will put them in a worse place
Andrew- rather than a better place.
Mary- I like that.
Mary- So now digital literacy also includes consciousness.
(group laughs)
Mary- Coming out of your unconscious bias
Mary- and being more conscious of your bias.
Andrew- But this isn't a bad thing
Andrew- and it extends way beyond just dealing with generative AI.
Andrew- That ability to sort of test and evaluate what you hear
Andrew- and not necessarily reject it in a black and white way
Andrew- but understand sort of what the influences might be
Andrew- in terms of what you're getting
Andrew- and actually using your own cognitive power and ability
Andrew- to untangle the threads there and make sense of things.
Mary- Excellent.
Mary- Tamara, did you have anything to add
Mary- in reference to digital literacies there
Mary- or even the things that you've noticed from your own use
Mary- or from observing the use of others?
Tamara- That's a good question.
Tamara- First, I love listening to these two experts because
Tamara- I'm thinking, oh, we should say this, and they said, so.
Tamara- I here and I agree.
Tamara- But I would also just kind of take a different perspective
Tamara- on the limitations.
Tamara- And I think that that might be a bit in line 1059 Tamara- 00:42:13,290 --> 00:42:15,753 with what Andrew was just saying.
Tamara- This tool was created by society and for society,
Tamara- and because of that it has society's limitations,
Tamara- limitations that we're not proud of
Tamara- in terms of social justice, in terms of bias, in terms of,
Tamara- you know, not necessarily exchanging factual information.
Tamara- And these things are existent
Tamara- and we do need to deal with these anyway.
Tamara- And so having some sort of prompt literacy nowadays
Tamara- is just about having social literacy.
Tamara- It's not something that we can untether from who we are
Tamara- and what we need our students to be
Tamara- to be thought leaders and innovators and pioneers
Tamara- in the future to making a better society.
Tamara- And so, I think sometimes these tools like generative AI
Tamara- elicit and bring up all of these issues,
Tamara- but they didn't create them,
Tamara- and we certainly need to address them.
Tamara- And so I look at it as an opportunity to address them.
Tamara- And you know, I think, one last thing,
Tamara- I've seen students learn really well from generative AI
Tamara- when they do take the approach that
Tamara- this is an initial way to
Tamara- begin to think about information
Tamara- and to see what's out there and challenge it
Tamara- and then do fact-finding or fact-checking on their own.
Tamara- So I think that if you pair natural skills
Tamara- valued by higher education in society 1087 Tamara- 00:43:51,990 --> 00:43:54,660 with the use of these generative AI tools,
Tamara- I think you'll get further faster.
Mary- I love that, and I love the,
Mary- because this is like the age of aquarius, right?
Mary- That is unknown will be known,
Mary- and now we have to deal with it. (laughs)
Mary- Sometimes it's hard to deal with our issues,
Mary- but it's important that we do,
Mary- which again goes back to principled innovation
Mary- and our moral responsibility, right?
Mary- And then considering the civic duty
Mary- of understanding our cultures, understanding our students,
Mary- how they're going to use these systems
Mary- is extremely important.
Mary- I will share the principled innovation link
Mary- in the show notes as well
Mary- 'cause I think it's a fabulous resource
Mary- that was created by ASU.
Mary- It would be great for anyone to read through
Mary- before you start thinking about how you might use ChatGPT
Mary- or any other generative AI platform in your course,
Mary- or integrated at the enterprise level, right?
Mary- There's a lot of conversations around digital trust
Mary- and how are we as a large university,
Mary- one of the largest public universities going to
Mary- model how this is used outside of our university
Mary- 'cause it will be influential.
Andrew- And I would say that
Andrew- taking that principled innovation approach also begins to
Andrew- address the elephant in the room around cheating.
Andrew- And of course, as soon as ChatGPT became a big thing,
Andrew- there were all these concerns about students
Andrew- seeing it as a shortcut to surviving and thriving courses
Andrew- without putting in the effort.
Andrew- I think the evidence is that
Andrew- that isn't actually as big an issue as people thought it was
Andrew- for a number of reasons.
Andrew- But it also is something that I think
Andrew- that we're being forced to think about.
Andrew- And what intrigues me is, it's forcing us as instructors
Andrew- to think about what we're trying to achieve
Andrew- through education, and what learning actually means
Andrew- and what the outcomes are that we want
Andrew- versus just sort of ticking the boxes as we teach.
Tamara- I think we'll be able to achieve more
Tamara- through education because of these tools,
Tamara- which is very, very interesting.
Tamara- I don't know about you guys,
Tamara- but you probably heard concerns about academic integrity
Tamara- and cheating, and things like that.
Tamara- And it's very interesting because they're accurate,
Tamara- but also generative AI,
Tamara- cheating did not start and academic integrity issues
Tamara- did not start with generative AI,
Tamara- they just could be exacerbated by it
Tamara- if we don't teach our students correct use of tools.
Tamara- And so I kind of wonder what everyone thinks,
Tamara- Mary, you too, about when it comes to addressing
Tamara- academic integrity concerns,
Tamara- how you teach students and how you help faculty understand
Tamara- a proactive way to address these things.
Mary- I guess I'll start with the
Mary- "Cheaters Never Prosper" pamphlet that we have online,
Mary- which we'll also share.
Mary- And that really gets down to,
Mary- where there's a will, there's a way.
Mary- So don't focus on those who have the will necessarily.
Mary- Maybe figure out why they have that will.
Mary- Is it too many assignments or, you know, cognitive overload?
Andrew- See, and that to me is critical.
Andrew- Again, if we think about education as a series of
Andrew- rewards and punishments,
Andrew- of course people are gonna work out
Andrew- how to get the rewards and avoid the punishments,
Andrew- it's a really bad way of teaching.
Andrew- On the other hand, if we think about
Andrew- what learning are we trying to give our students,
Andrew- what do we want them to be able to do at the end of it,
Andrew- that opens up the pathway to seeing the way forward
Andrew- without thinking about how are we gonna punish them
Andrew- if they don't do what we want?
Andrew- And what we are finding, not only myself,
Andrew- but others with the use of generative AI,
Andrew- is if you create a pathway toward learning
Andrew- where it's actually easier for them not to use generative AI
Andrew- to cheat than it is to actually do the learning,
Andrew- it's a win-win situation.
Andrew- And just to give you an example of that,
Andrew- in the ChatGPT course,
Andrew- most of the stuff was based in ChatGPT
Andrew- but we had a few assignments where students either had to
Andrew- write reflections or create videos.
Andrew- And I was worried at one point that
Andrew- because they were well versed in ChatGPT
Andrew- I was gonna have a load of written reflections
Andrew- that were just generated by ChatGPT.
Andrew- No, not at all.
Andrew- And I didn't, and I'd love to sort of see
Andrew- whether this matches sort of what it was like as a student.
Andrew- But my sense was students wanted to have a voice
Andrew- and they wanted, when they were writing their reflections,
Andrew- for those reflections to reflect
Andrew- what was going on in their head,
Andrew- they didn't want a machine to do that for them.
Andrew- So the path of least resistance was for them
Andrew- not to use ChatGPT, and that worked incredibly well.
Mary- And that kind of gets to a resource
Mary- that Tamara and some of our other colleagues
Mary- very intentionally produced this last session,
Mary- and I think is actually in the Generative AI course.
Mary- Tamara, do you mind talking about that?
Tamara- Oh, it's the academic integrity risk reduction guide.
Tamara- And it's a process and it's a series of questions
Tamara- that help you to determine what the risks are in your course
Tamara- and how to use course design decisions to address them
Tamara- even before students enter the course.
Tamara- We have received feedback that it's quite long,
Tamara- so we need to emphasize that it's a--
Mary- There's a lot to talk about. (group chuckles)
Andrew- Did you have people cheating
Andrew- to try and get to the end of it?
Tamara- So for the next two hours
Tamara- we'll be talking about this guide.
Tamara- No, I'm just kidding.
(group chuckles)
Tamara- No, we do like to say that this is a reference resource
Tamara- the same way you wouldn't sit down
Tamara- and you wouldn't read the dictionary from cover to cover,
Tamara- you would look in alphabetical order
Tamara- and find the word that you need,
Tamara- learn the definition and move on,
Tamara- this guide was intended to be comprehensive in a way
Tamara- that would allow you to look at a categorical issue
Tamara- that you're expecting, experiencing,
Tamara- ask yourself a question,
Tamara- and then have probably 10 different solutions
Tamara- for addressing that one single issue
Tamara- that you could easily try in your course.
Tamara- And then you can kind of use that
Tamara- as almost like the scientific method.
Tamara- You've got a hypothesis, you think this is the issue,
Tamara- implement this one strategy, does it address the issue?
Tamara- Does it not address the issue?
Tamara- It did, that's fantastic.
Tamara- So it's a wonderful tool
Tamara- because it assumes that we are constantly learning
Tamara- and that our courses are constantly changing
Tamara- and the needs of our learners are changing.
Tamara- And so to make course revisions to help our learners to be
Tamara- more engaged in the course studies
Tamara- and reduce academic integrity risks
Tamara- is the whole goal of the guide, so.
Tamara- And then at the end it teaches you--
Mary- I do wanna plug those six categories too,
Mary- because that, I think if people know like,
Mary- this is what you cover, they might just go to that category
Mary- and then it's not as overwhelming.
Tamara- That's very true, that's very true.
Tamara- Do you want me to plug the six categories, or?
Mary- Absolutely, or I have 'em up I can read 'em, either way.
Tamara- Yeah, do you wanna read 'em? That's fine.
Mary- - So the first one's course materials,
Mary- which I think everybody's a little bit worried about
Mary- with generative AI,
course tools, course Mary- activities and collaboration,
Mary- assessment design, that'll be the most visited space,
Mary- I have a feeling, I wish we could track that.
Mary- Course expectations and instructions,
Mary- and then feedback and student support.
Mary- So you were really very comprehensive
Mary- in your consideration of all of the different ways
Mary- that generative AI,
Mary- well, not just generative AI,
Mary- 'cause this is not just generative AI focused,
Mary- I think we should say that as well,
Mary- but many ways that academic dishonesty can show up
Mary- in a course, very intentional.
Tamara- And that it can be exacerbated by using
Tamara- higher level technology if you're not careful.
Tamara- So definitely the consideration was for first and foremost
Tamara- the academic integrity overall
Tamara- and then also addressing it with technology,
Tamara- which is nice because the last part of the guide is
Tamara- specifically designed to harness the power of generative AI.
Tamara- I think we've got an AI versus AI webinar coming up,
Tamara- and I think that the idea that you can fight AI with AI
Tamara- by using it properly,
Tamara- I think that's kind of a nice approach.
Mary- And it's possible because of the magic of podcasting
Mary- that this podcast will come out after that webinar,
Mary- but if you do want access to that,
Mary- just reach out, we can get you access.
Tamara- Sorry. (chuckles)
Mary- No, it's fine.
Mary- I'm excited for your webinar.
Mary- It's gonna be a fabulous opportunity
Mary- to have conversations with faculty on the real concerns
Mary- that are happening,
Mary- but also talk about how to use this risk reduction guide
Mary- in a very meaningful way to help alleviate concerns
Mary- and maybe put people in that explorative,
Mary- experimentive, curiosity space around how
Mary- these generative platforms are going to change our society
Mary- and therefore change the way our disciplines
Mary- will interact with society
Mary- and what we need for student readiness
Mary- before they leave our university.
Ricardo- I was really like on the edge of my seat the whole time,
Ricardo- waiting for the negative impacts.
Ricardo- From your perspectives and your experience
Ricardo- working with these things,
Ricardo- what do you guys anticipate would be the,
Ricardo- 'cause I feel like this course,
Ricardo- the Generative AI course, was a response to
Ricardo- faculty kind of being worried about the kind of
Ricardo- negative impacts of this technology.
Liz- Yeah, I think the big surprising part,
Liz- not only about using and exploring ChatGPT,
Liz- but also this conversation was that there wasn't actually,
Liz- there was a lot of hesitation going into it
Liz- and into explorations with ChatGPT.
Liz- There was kind of an initial response to ignore, avoid,
Liz- try to shun the robots from coming as a part of our
Liz- higher education landscape.
Liz- But I think everyone actually found that
Liz- it was more helpful, that it was actually,
Liz- that students were using it for good, not for evil,
Liz- they were using it to get ideas.
Liz- And that afterwards when reflecting that they were
Liz- more excited to share their own thoughts and opinions
Liz- and they weren't actually using ChatGPT
Liz- for academic dishonesty purposes,
Liz- and that it's more of a help than a harm in a lot of ways.
Mary- And honestly, early on we were hearing from faculty that
Mary- those students who were using ChatGPT
Mary- were actually telling their faculty members,
Mary- hey, I use this, is that okay?
Mary- And some of them are like, I don't know if it's okay.
Mary- But it really comes down to instructor decision making.
Mary- And I think that's what I really love about
Mary- Arizona State's model here is that,
Mary- yes, there's some syllabus language
Mary- that the provost office was able to provide,
Mary- yes, there's this Generative AI course,
Mary- but it really takes deep internal reflection on
Mary- what does this new way that
Mary- society interacts with technology mean
Mary- to all kinds of spaces,
Mary- the disciplines that we teach,
Mary- the jobs that are being created,
Mary- the jobs that are potentially disappearing
Mary- or changing dramatically because the mundane work
Mary- can be done in different ways.
Mary- And it's gonna make us better,
Mary- like I think that was actually said in the conversation,
Mary- it's gonna make us better,
Mary- but primarily the things that I'm hearing about
Mary- that are worrisome are the ways that the models
Mary- have been fed information.
Mary- So one of the examples
Mary- in the generative AI InScribe course,
Mary- so if you haven't gone in, you can see this,
Mary- was an example of a woman who was like,
Mary- I need an image of a professional.
Mary- And it ended up being a white woman,
Mary- even though the person prompting it was of Asian descent.
Mary- And so it's like, that is a concern that our models,
Mary- especially our image models,
Mary- have limited data based on bias perspectives
Mary- of what we think certain people look like.
Mary- And so feeding those models a diverser set
Mary- is an important thing.
Mary- So having the opportunity to feed the models
Mary- more intentionally, curate the models more intentionally.
Mary- If we get access that way,
Mary- I think it's gonna change the game
Mary- and it also will change, and this was said too,
Mary- the social dynamics of like how we see things
Mary- and correcting the historical way
Mary- of how things were reported.
Liz- I think that's so interesting
Liz- 'cause that kind of goes back to good prompt engineering.
Liz- We need to feed the information
Liz- so that we can give better prompts
Liz- and feed the robot with more information
Liz- so that we get better, more diverse results
Liz- that are actually reflective of real people
Liz- versus what the internet thinks it is.
Mary- And real occurrences,
Mary- because it will give you a citation if you ask for it,
Mary- but it's very likely and almost always not accurate.
Mary- We had this example in the
Mary- School of Historical, Philosophical, and Religious Studies.
Mary- Jeff Watson came out of that college,
Mary- he also gave a lot of content from a conversation
Mary- he had with DeAnna in the Generative AI course.
Mary- But he and many others at his school
Mary- hosted this session last semester.
Mary- And one of their examples for their fellow peers
Mary- that were in attendance was,
Mary- one of the guys who was in there,
Mary- he writes on a specific topic and they were like,
Mary- find one of the books from this individual.
Mary- And then it came up with a title
Mary- that seemed very relevant to his study.
Mary- And he's sitting in the room and he is like,
Mary- "Wow, that sounds like a book I would've written,
Mary- but I did not write that book."
Mary- And it was just such a good moment
Mary- for everyone to go "yikes."
Mary- So citations is a place where it will not do well.
Mary- And if we could again, curate intentionally
Mary- and provide citations, then what a great collegial resource
Mary- these kinds of experiences could be
Mary- if that's how the models were intended to work.
Liz- Yeah, I think it's a great stepping stone
Liz- for hesitant students.
Liz- I think a lot of people in our conversations,
Liz- a lot of people that I've either talked to
Liz- outside of our podcast today
Liz- are actually more excited to integrate it
Liz- and tell students how to use it
Liz- and almost train students on the best way to utilize
Liz- the tool while also maintaining academic integrity.
Liz- So I think that the more people look into it,
Liz- the less concerns they have about academic integrity are,
Liz- and the more they're excited to see
Liz- how this can help students be better.
Tamara- We've gotten a lot of good responses from faculty.
Tamara- Some of them have been through our survey
Tamara- with high levels of numbers, you know,
Tamara- out of one to four, rate this, with four being the highest.
Tamara- We have so many fours, which is wonderful,
Tamara- and so much good constructive feedback.
Tamara- But then we also have things being shared
Tamara- outside of the university.
Tamara- And this is one of my favorite quotes
Tamara- that's being shared publicly outside the university.
Tamara- It's from a professor who said,
Tamara- "I have to admit that as someone who has experienced
Tamara- academic integrity issues,
Tamara- I had more of a negative feeling
Tamara- toward generative AI at first.
Tamara- And since I've been gone, I felt overwhelmed
Tamara- in terms of teaching."
Tamara- And she said, "The only thing I did know was that
Tamara- generative AI was transforming higher education,
Tamara- which was the most powerful transformation after that
Tamara- due to COVID-19.
Tamara- However, after going through this course
Tamara- and many other resources and discussions,
Tamara- I experienced how powerful generative AI can be
Tamara- and already have some ideas that not only integrate it
Tamara- into the curriculum, emphasizing student critical thinking
Tamara- and the iterative and collaborative process
Tamara- with generative AI, but also to utilize it
Tamara- in various aspects of my own teaching
Tamara- from developing and refining my curriculum
Tamara- and designing assessments,
Tamara- or even to just draft messages and feedback to students
Tamara- to generating images for slides.
Tamara- So now I'm really very excited about the potential
Tamara- and all the opportunities that it brings."
Tamara- And so that to me,
Tamara- if we want one person to get something out of the course,
Tamara- that's what we hope they'll get out of it,
Tamara- that generative AI is something to address and be aware of,
Tamara- but it's exciting.
Mary- Well, I do wanna say thank you for the opportunity
Mary- to talk to everyone.
Mary- Before we go, does anyone have anything they wanna plug?
Mary- We have the library guide, is there a short link for that?
Mary Ann- Not that I can think of right now.
Mary- Okay, that's okay.
Mary- We're gonna put it in the show notes.
Mary- The short link is, go to Teach Online.
(group chuckles)
Mary- Andrew, how about you?
Mary- You have a lot of things to plug.
Mary- Please, the whole list. Andrew- Oh, goodness me.
Andrew- Yeah, so if anybody is interested in the exercises
Andrew- we do in the ChatGPT course,
Andrew- most of them are available on my website,
Andrew- which is just andrewmaynard.net.
Andrew- So you can go and check them out there.
Andrew- But the other thing I would say,
Andrew- and you see this through all of my writings around
Andrew- ChatGPT and generative AI,
Andrew- is if you think you know what it's about
Andrew- but you've never tried it, you don't.
Andrew- You really have to experience it before you can understand
Andrew- what its benefits and power and limitations are.
Mary- That is a very good call to action
Mary- for those who are listening that are like,
Mary- I was just listening to hear that
Mary- I shouldn't use it in my class.
Mary- But you did enjoy your student-based class.
Mary- Are you offering it again?
Andrew- Oh yes, we're offering it in the fall, Fall B.
Andrew- It was a wonderful experience.
Andrew- Not only important, but it surpassed my expectations
Andrew- in terms of what the students learned
Andrew- and what they came away being able to do.
Mary- Are you gonna change anything this time
Mary- that you learned from last time?
Andrew- Yeah, there were a couple of assignments
Andrew- that didn't work so well.
Andrew- Multi-stage prompt templates was a disaster.
(group laughs)
Andrew- We're gonna be changing that assignment.
Mary- And then Tamara, I know that you have
Mary- a little bit to plug,
Mary- we've already talked about the risk reduction guide.
Mary- But anything else to say as well
Mary- in reference to your experience
Mary- in building the Generative AI course for faculty
Mary- and staff at ASU?
Tamara- I think I'll just kind of describe it really quickly
Tamara- and what it was.
Tamara- The Teaching and Learning with Generative AI course
Tamara- was a way to gather some of the most
Tamara- innovative folks at ASU who are using generative AI
Tamara- effectively in their courses
Tamara- and experimenting with generative AI
Tamara- and put that expertise into one course
Tamara- to help faculty and staff know how to not only
Tamara- harness the power of AI,
Tamara- but also address issues that might arise with AI.
Tamara- As we said, Andrew is a wonderful voice
Tamara- throughout the modules in the course,
Tamara- he's almost this guiding voice.
Tamara- And then we have so many of our other faculty experts
Tamara- as well as staff experts who've contributed to this.
Tamara- And I love this part about the course,
Tamara- one of the things that I love about the course is that
Tamara- it is a catalyst for other work to be able to showcase it.
Tamara- But also we have this wonderful page
Tamara- that talks about accessibility related to generative AI
Tamara- on the university website.
Tamara- Because of this course, we have tools that are ASU-created
Tamara- that have passed our VITRA security review process
Tamara- that leveraged the powers of ChatGPT
Tamara- that were created and showcased.
Tamara- They weren't created because of this course,
Tamara- but they were showcased.
Tamara- We've got wonderful syllabus statements.
Mary- Would you mind naming what those are, by the way?
Mary- Sorry to interrupt you, but those are really great tools.
Mary- What are those tools that were shared?
Tamara- Those are incredible.
Tamara- We've got the ClipGist,
Tamara- which takes the transcripts of a video
Tamara- and summarizes it and actually asks questions.
Tamara- So if you have an interactive video tool and
Tamara- you want to quickly make some interactive questions
Tamara- for your video to pause and talk to students
Tamara- and assess where they're at.
Tamara- And then we have the question pool generator,
Tamara- which takes either your learning objectives for the module
Tamara- or a question stem
Tamara- and it generates anywhere from 5-30 questions with
Tamara- anywhere from two to five or six responses.
Tamara- And it sends it over to faculty or whoever used the tool
Tamara- in a PDF so that they can look at those and use those to
Tamara- refine, revise, start again.
Tamara- But it's a wonderful way to get ideas.
Tamara- It's actually a great way to combat the issues
Tamara- with academic integrity, isn't it?
Tamara- Because we notice a lot of the questions that are asked
Tamara- in courses end up on Chegg or Course Hero
Tamara- or something like that.
Tamara- And so, it's hard to continually update your questions.
Tamara- So you are assessing knowledge
Tamara- and not if someone can find the answers online,
Tamara- but this is a nice way to harness
Tamara- the power of generative AI,
Tamara- so I'm really excited about that.
Tamara- And one thing, I wanted to tell you guys about
Tamara- an exciting fun fact about this course.
Tamara- Right now, we have, I think I had already mentioned,
Tamara- we have about 624,
Tamara- and when I was looking at it, the 625th person was added,
Tamara- which was wonderful to see the numbers go up.
Tamara- But in the first day of the course launch,
Tamara- we had 3,460 page views.
Tamara- So people were in it and experimenting.
Tamara- I know, I'd never seen a number that high in any course
Tamara- that people were excited about going through it.
Tamara- So, hopefully there's a lot of benefit,
Tamara- and it is riding on the shoulders of people
Tamara- who have already done this work.
Tamara- Like I said, Andrew Maynard and so many of the other people
Tamara- who are experts in this Generative AI course as well.
Mary- And while this course is only available
Mary- for ASU faculty and staff,
Mary- we will still put the link in the show notes
Mary- 'cause if you're listening to this from ASU,
Mary- that'll be an easy way for you to get into the course.
Mary- If you haven't already, or to go back into it,
Mary- if you started to get into it
Mary- and then got distracted by the start of the session,
Mary- which is normal, because there's a lot of great content.
Mary- You were saying the syllabus,
Mary- I think you're gonna give me some other parts of it.
Mary- Did you have anything else you wanted to add
Mary- on reference to that course?
Tamara- Oh, so many things came out of this,
Tamara- which was really exciting.
Tamara- Yeah, the syllabus statements were wonderful.
Tamara- Oh, actually, Mary,
Tamara- you developed kind of an innovative solution at ASU
Tamara- with using InScribe to develop a community where we can--
Mary- I didn't develop that,
Mary- but I definitely put things in there for the course.
Mary- But this is the Learning Experience team
Mary- that created that space.
Mary- Well, I'm happy to do the work where I can.
Tamara- You were fantastic.
Mary- But the InScribe community is awesome,
Mary- it is a great place to share and to learn.
Mary- So in the course there are many opportunities
Mary- to share your artifacts with the community of ASU.
Mary- That's actually how I found Mary Ann
Mary- is because I'm a lurker in all spaces,
Mary- and I saw her artifact get posted there.
Mary- So if you have any interest in how could I use this,
Mary- what does that look like when other people interact with it,
Mary- there's a lot of opportunity to lurk.
Mary- You don't have to do anything other than lurk,
Mary- you could be like me and just watch people from the outside.
Mary Ann- My imposter syndrome's waning just enough for me
Mary Ann- to actually follow with something.
Mary Ann- So I do have the lib guide that has some basics on
Mary Ann- citing generative AI, et cetera, and it's growing.
Mary Ann- Another resource is the prompt engineering,
Mary Ann- like having some templates.
Mary Ann- But I just wanted to plug, I have wonderful colleagues.
Mary Ann- One of the things I've done in this class
Mary Ann- and Dr. Maynard's class was to really practice
Mary Ann- and look beyond my own,
Mary Ann- like, the students that I work with.
Mary Ann- And so I worked with some of the health sciences students
Mary Ann- to create a specific template for the health sciences
Mary Ann- that would lead students through
Mary Ann- the design of a PICO method question.
Mary Ann- So please do feel free to reach out to your librarians.
Mary Ann- I'm happy to work with you if you're looking to see
Mary Ann- how can ChatGPT or generative AI
Mary Ann- be used in the research process
Mary Ann- that doesn't equate to finding sources,
Mary Ann- that gets them to the place
Mary Ann- where they can start jumping into academic sources.
Mary Ann- The librarians are here for you,
Mary Ann- and I'm happy to collaborate, so.
Mary- There you go, wide world,
Mary- ASU, you're very supported,
Mary- you have lots of great experts here at your fingertips.
Mary- Don't be afraid, jump right in, get curious.
Mary- And if you're not from ASU,
Mary- a lot of the resources we're sharing
Mary- are still viewable to you.
Mary- And Andrew's Substack is absolutely viewable to you,
Mary- so go visit that.
Mary- There's lots of great material there.
Mary- And thank you again for coming
Mary- and talking through this experience with us.
Andrew- Thank you. Mary Ann- Thank you. Tamara- Thank you.
(lively music)
Ricardo- That was a great conversation,
Ricardo- such great experts in the room here.
Mary- Truly.
Ricardo- And so we decided to ask them to stay here
Ricardo- because Liz has got a scheme.
(group chuckles)
Liz- So I am working on a little bit of a side project,
Liz- I thought it could be a fun side project
Liz- to trick my loving in-laws or soon-to-be in-laws
Liz- into thinking I know something about football.
Ricardo- Right, integrity's now out the window.
Mary- Which you also call sports ball,
Mary- just to set the tone.
Liz- I know nothing about football except that
Liz- Larry Fitzgerald was my hero until he retired.
Liz- I drafted him for my team but he retired,
Liz- so he got me zero points that season.
Liz- So what I decided to do as a part of my little scheme,
Liz- and I think it's really relevant to the conversation
Liz- we've had about prompt engineering
Liz- 'cause I need some help on prompt engineering for,
Liz- I'm gonna use ChatGPT to draft and run
Liz- my fantasy football team this season
Liz- and hopefully get some points
Liz- and show my future husband's family that
Liz- I know something about football.
Liz- So I can tell you a little bit about what I've done.
Liz- I've basically gone to ChatGPT
Liz- and I've given it some,
Liz- what I've started doing is asking you questions like,
Liz- can you help me with this?
Liz- And then it will give me a response,
Liz- and then will tell me, yeah, I can help you with this
Liz- if you give me A, B, C information.
Liz- But like what would your be advice for like
Liz- prompt engineering for something like fantasy football,
Liz- or are there things that I should be keeping an eye out for
Liz- in terms of accuracy when I'm working with my?
(group laughs)
Andrew- Yeah, don't.
Andrew- So your biggest challenge is that ChatGPT, A,
Andrew- knows nothing about football and it knows nothing beyond,
Andrew- when is it, September, 2021.
Andrew- So anything after that is a problem.
Andrew- However, I think it can help you begin
Andrew- to formulate your approach to things.
Andrew- So you can first of all ask it general questions about
Andrew- what sort of characteristics should you be looking for
Andrew- as you put your fantasy team together.
Andrew- You can even give it basic information
Andrew- about the sorts of people you are interested in.
Andrew- So you can sort of create notes about potential players
Andrew- and upload them and say,
Andrew- hey, what do you think about this team?
Andrew- So now it begins to look like a conversation
Andrew- where you give ChatGPT information,
Andrew- you may not know what you're doing,
Andrew- but at least you can put notes together
Andrew- and it'll help you sort through those
Andrew- and make some sort of sense of them.
Ricardo- So how can I con my in-laws is not a good prompt for?
Andrew- Well, you know, knowing ChatGPT,
Andrew- it will probably start off by saying
Andrew- that that is not a good idea.
Andrew- It is a very moral machine.
(group laughs)
Liz- That's disappointing.
Tamara- It's illegal.
Liz- I'm trying to cheat at this fake game
Liz- and it's not gonna help me.
Tamara- One of my neat tricks that I like to do with things is
Tamara- I like to leverage plugins in GPT-4
Tamara- and I like to use PDFs.
Tamara- And I discovered this because
Tamara- we have a lot of board games that we play in our family,
Tamara- and there's always a little bit of a conversation,
Tamara- a heated discussion about which rule somebody broke
Tamara- or which rule somebody used.
Tamara- So what we did is we took our board game rules
Tamara- and we put it in a PDF and we use the ChatGPT for PDF plugin
Tamara- to say, here's a PDF, remember these rules,
Tamara- and we're going to ask you questions about the rules,
Tamara- and then it references that PDF.
Tamara- So you could do something similar to what Andrew was saying.
Tamara- You could put this stats
Tamara- for key players for last season in a PDF.
Tamara- Use the ChatGPT for PDF plugin,
Tamara- and then you could say,
Tamara- based off of the information in this PDF,
Tamara- which are the top players according to this?
Tamara- Which would be a balanced team according to that?
Tamara- And which should be my first,
Tamara- second and third picks, and why?
Tamara- And if you want to really trick your in-laws,
Tamara- that's probably the best way to do that because you can say,
Tamara- well, actually in the third quarter of last season,
Tamara- so-and-so did this and he had this number of losses
Tamara- but I would say he's only a second round pick.
Tamara- That would impress them, so.
Mary- Absolutely, I love that you're using it
Mary- as a gaming referee.
Liz- This is really good
Liz- 'cause I've already asked it like, who should I,
Liz- it's given me some advice on like a strategy for drafting.
Liz- Our draft is actually tomorrow night,
Liz- so I am gonna have to do some research
Liz- and some uploading today.
Mary- You got some chatting to do.
Liz- So it's good to know that
Liz- it only goes back so far 'cause it has already given me
Liz- some thoughts on who my first pick should be.
Liz- And I do get first pick
Liz- 'cause I was the big loser last year,
Liz- so I get to pick first this year.
Mary- Maybe then Google, have they retired?
(group laughs)
Liz- Honestly that was the biggest disappointment
Liz- of like two years ago, so.
Mary Ann- I think if I were trying to trick my in-laws,
Mary Ann- what I would do is also have it ChatGPT quiz me
Mary Ann- to see what my knowledge is so that I can respond quickly.
Mary- Next level. Mary Ann- That's great.
Liz- Yeah, my plan was to have my laptop open with one window
Liz- with the draft information
Liz- and another window with ChatGPT
Liz- and I was just gonna feed it.
Liz- So here's who's been taken, who should I pick next
Liz- based on these players being taken?
Liz- But I'll have to upload like an entire roster for this year
Liz- it sounds like if it doesn't work for 2023.
Andrew- But actually that's not too difficult to do.
Andrew- The information in that PDF or whatever you upload
Andrew- doesn't have to be perfect because it can make sense of it,
Andrew- you just need the information there.
Liz- This is great news.
Liz- So I'm trying to keep this a secret from the in-laws
Liz- and have them be surprised that
Liz- suddenly I'm a wiz at fantasy football.
Liz- They've already been making fun of me with memes
Liz- in our family chat about how I'm terrible at picking people.
Liz- I pick based off of fun names,
Liz- I don't know anything about their stats.
Liz- But I like, oh that's a fun name.
Andrew- Just whatever you do, don't put money down on this.
(group laughs) Liz- I'm not.
Ricardo- All right Mary, if anybody wants to chastise Liz,
Ricardo- how can they reach us?
Mary- Well, they can reach out to corestories@asu.edu.
Mary- Come to Teach Online, we'll post Liz's football draft team.
Ricardo- We'll keep you updated this season.
Liz- Yeah, we'll see how it goes.
Mary- Yeah, we'd love to get an update.
Mary- In fact, if you wanna publish your conversation,
Mary- we can also put that on there
Mary- 'cause that's how easy it is to share these things.
Mary- Like, there's a link that you can go to
Mary- people's conversations and see what these students, faculty,
Mary- Liz, our producer, are doing with ChatGPT.
Liz- That's good to know.
Liz- Plus I'm gonna try to record some of this on TikTok
Liz- is like a little diary entry for my experiences.
Liz- I thought it'd be a really fun way to explore ChatGPT,
Liz- explore prompt engineering,
Liz- and maybe hopefully come up with some great ways
Liz- for people to use it in the future
Liz- for good or for evil, in my case.
Tamara- You might've just done some work for anybody else.
Tamara- If you share that link, they can build on your link
Tamara- and create their own fantasy football teams too.
Ricardo- Oh yeah, you're Prometheus, bringing down the fire.
Liz- I'm gonna one by one,
Liz- I'm gonna take down fantasy football.
Liz- Two years of being embarrassed,
Liz- I'm gonna burn the whole thing down.
Andrew- Take the ethics module first.
(group laughs)
Ricardo- Course Stories is available wherever you listen to podcasts.
Ricardo- You can reach us at corestories@asu.edu.
Ricardo- Course Stories is produced by the
Ricardo- Instructional Design and New Media team at EdPlus
Ricardo- at Arizona State University.
Ricardo- If you're an instructor at ASU Online,
Ricardo- tell us your core story and we may feature it
Ricardo- in a future episode.
Ricardo- Thanks for listening.