From Framework to Reality: Operationalizing AI Implementation

Summer Heat Cold Open

[00:00:00] Chris Van Wingerden: Hot time in the summer here, gang. Woof. Getting overheated.

Muppet Impressions

Chris Van Wingerden: And what the folks can't see is that, I'm rather more like a Muppet, so the person below the screen with the pulleys and the sticks and stuff, they're really out of breath too.

there's nothing... I don't exist below, you know, below my elbows. I'm actually- There's, I'm just, I'm made of felt. and y- the giveaway is the googly eyes.

Megan Torrance: But can you do the la, like a Muppet?

Chris Van Wingerden: [00:01:00] Hi-ho, Kermit the Frog here- Ooh ... for Muppet News

Megan Torrance: Definitely a child of the '70s

Chris Van Wingerden: Yeah, no kidding.

No kidding.

Megan Torrance: Well done.

Show Intro Sponsor

Chris Van Wingerden: Hey folks, believe it or not, w- we're here for Instructional Designers in Offices Drinking Coffee, not two, Gen Xers talking childhood memories.

Although, you know, that's good too. and remember, as always, Instructional Designers in Offices Drinking Coffee is brought to you by the team here at dominKnow, makers of dominKnow ONE, helping L&D teams to develop, scale, and deliver learning that maximizes employee value. Friends, neighbors, Romans, countrymen, how did that speech go from Caesar?

Lend me your... Lend us your ears.

Meet Megan Torrance

Chris Van Wingerden: We have one of our rockstar guests back with us, Megan Torrance. Megan, it seems hard to believe, but there are maybe some folks joining us today who have not encountered you at conferences or other spaces.

For those folks who haven't, met you before, give us a bit of an introduction to yourself.

Megan Torrance: Absolutely. There's lots of people who don't know me, which gives me lots of opportunity to meet new friends, um, and [00:02:00] dance, right? So I am the CEO and founder at TorranceLearning, and we're a, we do what everybody who's probably listening to this does.

we're a full stack learning, and performance enablement company. So we offer design and, and consulting services, a lot of custom design and development of learning and performance experiences, and then the infrastructure and architecture to make it all work, to deploy it, to track it, to know what's going on, and use that information to feed back in.

And then we spend a ton of time helping our colleagues in the field do exactly that, and do an amazing job at that. So we share how we do our agile methods, we share how we do data, we share how we do AI, and all sorts of other things. So we are really passionate, not only about helping learners, but helping our fellow professionals in the field.

Chris Van Wingerden: Yeah, for sure. That sharing and that passion has included, a few books, including, your most recent one, talking about the AI, AI implementation [00:03:00] canvas and, and helping teams. 'Cause my gosh, I don't know if you know this, but there's quite a few people talking about this AI thing lately.

Megan Torrance: I have heard

Chris Van Wingerden: that. Yeah, weird.

Megan Torrance: Yes. Yes.

Chris Van Wingerden: Uh, clearly. Um,

Megan Torrance: it tends to

Chris Van Wingerden: come up.

Megan Torrance: Yeah.

Chris Van Wingerden: I'm not bragging, but you know, obviously we hear from all the influencers. So maybe the general public hasn't heard about AI yet, but, in our space.

AI Hype Reality Check

Megan Torrance: Interesting you say that

though, right? So I mean, if you were to read LinkedIn, right, for any amount of time, you'd think that's the only thing in town, the only thing that anybody is thinking about, and that everybody has already gone full agentic, otherwise you're square and behind. and that's where it's interesting, right?

Because the influencers are not necessarily representative of the general population, have different size organizations in which they can operate at. Some of them are mega organizations with gobs of budget and lots of focus on this, and some of them are solo folks that for which governance and [00:04:00] implementation, you know, messiness just aren't the same kind of issues.

So there's a lot going on and that's why I'm doing some research with the Guild, just to figure out what's going on.

Chris Van Wingerden: Yeah. and we'll drop in a, a note into the various places, about that research.

AI Canvas Explained

Chris Van Wingerden: So you've got this canvas then. and so, I mean, are you using oils? are you using encaustico techniques? Tell me a bit more about the canvas.

Megan Torrance: Right, 'cause watercolor doesn't really work as well on canvas.

Yeah. No, actually, like, funny you should say, 'cause I have

Chris Van Wingerden: Holy cow ...

Megan Torrance: I, I have a big one. This is for conferences. It's so much easier to, like, talk about this when, part of the point of the canvas is that people talk to each other.

Chris Van Wingerden: Mm.

Megan Torrance: So this is a great, a great prop for people to be talking to each other.

so here's how it works, right? you'll notice it's all blank, right? There's no answers in it. The whole point is to get together with people and ask questions around 14 different planning dimensions [00:05:00] when we bring in any new AI tool. Not just- Yeah ... AI in general, like any new AI tool or new process.

We look at, like, what are the, wh- where is the data coming from? Where is it stored? What's the architecture? What's the user experience? Why are we doing this? How are we measuring it? How are we gonna scale? There's all sorts of questions, and the whole point is we have these conversations so that we have a much more thoughtful implementation of any AI tool.

And 14 is a lot, and a lot of folks miss some.

Chris Van Wingerden: That was, that was my thought, was 14. that's a bigger number than I expected, and it's also a very specific number. Like, 14 ...

Megan Torrance: it was almost 16.

Chris Van Wingerden: Oh.

Megan Torrance: And, and the instructional designer, drinking coffee in my office- ... um, in, in me knows that that's a lot.

So that's, that's a, a, a concern. Mm-hmm. It's chunked, right? So there's a chunk around strategic foundations. There's a s- a chunk around data and [00:06:00] experience architecture. There's a, a chunk around, right, notice good instructional design. I'm chunking. There's a chunk around, scaling and experimentation and measurement, right?

And how do we know it's okay to go from pilot to full scale? and then there's a chunk around people, their, their AI literacy, the bias in human protection, the change management, what that means to the workflow. And so all of that is, yes, 14 is a lot.

Chris Van Wingerden: Mm.

Megan Torrance: the other two I realized were baked into everything.

Chris Van Wingerden: Ah.

Megan Torrance: Like risk, right? One of them was gonna be risk. Well, actually, there's risk everywhere, so.

Chris Van Wingerden: Very cool. Yes. I, I mean, in our own experiences, as here at dominKnow as we've rolled out and added on different AI facilitating tools, et cetera, to improve how people can do stuff in dominKnow ONE, every time the larger organizations amongst our client base have very rigorous processes that they go through.

lots of paperwork, lots of diagrams, you know, all of those sorts of things, which is the nature of big organizations. [00:07:00] They're, you know, they are always looking to, to mitigate risk. and then I think of some of our smaller sized client teams and who a- almost in a sense don't have that, I'm gonna call it a luxury really in a sense, right?

maybe there aren't resources to, whether that's time or, or budget or teams of people who can do these kinds of things. But there's a lot to be thought of. and as you've noted, like you're suggesting it for everything, not just AI as a blanket thing, but looking at every different kind of a tool that, you know, that you're looking to bring in.

Risk Data Vendor Costs

Chris Van Wingerden: you mentioned the word risk. What kinds of things, 'cause that's usually the word that gets used in all of these. They are risk assessments, at least from our experience. you know, what kind of things, pop up the most, do you think, when people think about risk and AI?

Megan Torrance: there's a lot of talk around risk of, well, what happens if I put my personal data, my company's secrets, like the recipe to the Big Mac sauce and, and, and stuff like that in, right?

Um, where does that go? And what, what do I, but I have PII about people, right? Now, one of the things in, in [00:08:00] learning, we're not quite like HR, but we have a lot of people data. We have a lot of data about people, right? And, and is that okay to, to put in? So that's often the first conversation that people have, and it's one thing to say, "Well, yeah, like we've got ChatGPT and we've got a private accountant and they take care of our stuff," but- That's just that one tool and that's just that one space Mm-hmm And e- every other tool, are they using the same models?

Are they using your same models? And so all of the, the, you know, all the, the magic AI button that's in every single piece of software that you have right now. Like, where is that data stored and how, how is that protected?

What is the risk when all of our data and all of our processes are all stuck with one vendor? Because the more data and the more processes that a particular tool or vendor takes on, the more fantastical the tools are. We have a project management so- software, and it is great. It's got [00:09:00] fantastic AI capabilities, and the more we put into it, the more amazing those AI capabilities are, but the more at risk we are of having everything in with a single vendor, and that single vendor may change its policies, may change its mind, may get acquired by somebody with different ideas, right?

So that puts us at risk. So there's that vendor risk, and that scale risk. There's risk of what happens when I no longer have junior staff learning around basic tasks and I then lose my pipeline five, 10 years from now, 'cause I plan on being around for a while. What happens when I lose my pipeline of talent because I'm not cultivating those younger people?

So there's lots of risk. There's cost risk, right? All of these tools operate on somebody else's model. Now, many of these tools operate on somebody else's model, and the cost structures for those models are going up, and where it [00:10:00] used to be, oh my gosh, it cost me a fraction of a penny to do this amazing work, and now it costs a lot more than that.

We're still talking about pennies and, and dollars, but at scale- That's a lot of money.

we very quickly realized that we hit our relatively conservative max because we had four project teams all hitting that tool very hard to come up with some super cool stuff. So yeah, there's, there's risk. That's the thing. There's risk everywhere in the canvas, absolutely everywhere.

Chris Van Wingerden: I mean, the pricing alone, what strikes me, Netflix used to be really cheap and you used to get everything,

Megan Torrance: Yeah.

Chris Van Wingerden: Yeah ... and now it's, it's not nearly as cheap, and, you know, other services have been, you know, put up, and so now you have multiple services, et cetera. So it's, you know, y- you turn around and realize that, oh, you look at the credit card bill at the end of the month, you personally, how much did you just spend?

and oh, it was great originally, like, you know, we cut the cable costs right out of our family budget, and now you're back up to the, the same level of [00:11:00] prices. And you hadn't really noticed until one day you went, "Holy crow," and looked at things, so.

Megan Torrance: But imagine, right, so imagine if your entire business process or if you're at one of these tool vendors, right, your entire business model changes out from underneath you- Mm-hmm

while your pricing model to your customers may not change as, as dynamically and as quickly. Mm-hmm. Like, it makes us think very differently about business models. it's a fantastic time for innovation and entrepreneurship, but also it's a, all, there's a lot of untested small companies spinning up software really, really fast, and that is, is both exciting and sometimes worrisome.

Chris Van Wingerden: Yeah.

All 14 Dimensions

Chris Van Wingerden: so y- you, you've noted risk underlines are sort of a thread throughout all 14, but I feel we'd be remiss if we didn't at least itemize the 14 in total. We, we won't have time to talk about them all, maybe all today in any level of depth, but the 14.

Megan Torrance: 14. Okay, so here we go. [00:12:00] In strategic foundations...

I don't know, I'm trying to zoom. I don't know

Chris Van Wingerden: how this- Also, so I, I take comfort that they're actually in four groups. There's 14 in four groups. That's reassuring. That's- Yeah. Yeah ... that, that lessens the, uh... Nice. Nice work.

Megan Torrance: Yeah, you don't have to revoke my instructional design card.

Yeah, so in strategic foundations, we're really looking around, right? Everybody says that you start with a business alignment. Where's that? Right? workforce impact and, and talent planning, I actually have that as part of strategy, right? I think that's really, really important. Regulatory and legal environment.

Oh, Margaret Spence is writing a new book. It's out just now on Amazon. It's called When HR Breaks the Law, or When AI Breaks the Law. Super, sup... Like, I've, I've read some of her early stuff. Really, really cool. Um, and then AI readiness. Now, here's the thing. I put AI... Where'd it go?

I'm doing this backwards. That's okay. I put AI literacy over here. Okay? Down under the human-centered adoption and change, this is gonna be the weirdest, LinkedIn [00:13:00] Live session anyways. I'm doing my thing anyway. I think of AI readiness as not just AI literacy, but really, like, what else is going on in the business?

is our data ready for this? do we have a governance structure? Do we have processes in place to, to take on this kind of change? It's not just AI literacy. And so, you know, we think about the amount of change that is happening in organizations. At what point do we wanna take on different things?

So that's, that's strategic foundations. the one that everybody kind of jumps to and thinks about, or at least the technology folks do, is the technology and experience infrastructure, right? So systems governance, we have our data architecture, what models we're using, but also what's the user experience and the workflow?

Mm-hmm. Right? Am I alt tabbing over to some other tool, or is there a tool that just know... Like, I'm swimming in the AI software, right? I'm swimming in the environment, it always knows where I am. I was talking with somebody the other day, right? 100% context aware, like where I might be in the middle of, of [00:14:00] adding a graphic on a PowerPoint, and the AI says, "Oh, I don't think you'd wanna do that."

Right? Are we ready for that? What's that user experience? Um, so that's a, a big piece. There's some really interesting stories in, in the book on like, when we, we don't do the user experience well, what does that look like? Mm-hmm. I think that part will get better as this gets to be a more mature space, right?

So I, I, I think that'll get better. over here then we have design and implementation enablers. So how do we experiment with this, and then how do we scale it, and how do we measure that? And measuring, there's lots of conversations on LinkedIn, right? Measuring is more about like, oh, 100% of our people logged in.

The goal isn't that people complete the training The goal is that people do their jobs better. The training just happens to be one of the things that might enable that. So When people say, "How do you measure AI use?" I'm like, "I don't care. I'm measuring output of what it, whatever it is," right? I mean, I do care about AI [00:15:00] use, but, so, so that's, that's important. And then the final section is, this human-centered adoption and change,

So we talk about social engagement, we talk about bias and, and mitigation, and how do we protect people. And those people include not just the people using AI, but the people who are affected by that, right? Change management, how do we roll this out? You know how many times I have heard stories, Chris, of people who are like, "Yeah, we didn't have AI, we didn't have, uh, AI, we didn't have AI," and then one day they sent us an email and said, "Yeah, you got Copilot now."

And that was the rollout. Come to a lunch and learn, like a- and, and, and just kind of plop that out there. And then finally we get to AI literacy, which is not just AI literacy, but that's a whole other conversation, right? There's layers and depth in there and, and lots going on, and that's a space where L&D professionals are absolutely playing a place But there's more we can be doing there.

Chris Van Wingerden: On, on the measurement, what popped in my mind was, something I saw on my socials. a [00:16:00] software... I think it was a software developer who said the company, you know, gave everybody AI and then was measuring your success with it by the number of tokens, if you didn't use up all of your tokens.

So he just set his AI to play video games. So he used up his tokens and everybody thought he was doing a great job. Right. Is there truth to it or is it just a, a facetious tongue in cheek post? But it's, it's an example of something. and the whole juniors, you know, juniors developers, junior makers, junior roles pipeline is something that very definitely, concerns...

I- is a concern for me. you see it, as a concern in a lot of the software space too, you know. Great, we've got senior developers who can understand what the AI is doing and why it may be coming to certain outputs, et cetera. but how's a junior supposed to understand that if they haven't actually walked those paths themselves already?

they don't have that actual, un-AI'd experience and, and what does that bode for the future of, of who's going to be, or how are we going to make the senior developers or, or in our case, the [00:17:00] senior instructional designers, et cetera? because we are definitely a space where most of us take a path, at least initially, through, you know, osmosis or, or by apprenticeship.

Learning by the folks around us- Mm-hmm ... in a sense when we start into this space too. So, so it is definitely something, that crosses my mind as I think about the changes in our industry because of AI.

Megan Torrance: Yeah. Absolutely. Absolutely. I think we're in, in, to sound, to take a very trendy word, we're in a liminal space, right?

Mm. Where we using it now have grown up the analog way, right? The digital analog way. I don't know what's the best way, right? But the next generation will not have had that experience. Mm-hmm. And how's that gonna go for them? And maybe this is just old people complaining.

Chris Van Wingerden: It may be.

Yeah. You know,

Megan Torrance: back then I climbed the... You know, I, I had to climb that PowerPoint uphill and- ... waste heaps snow back and forth, you know, all day long. You know, that might be something that turns out to not be as relevant-

Chris Van Wingerden: Yeah ...

Megan Torrance: in a future world.

Chris Van Wingerden: [00:18:00] Well, and something else will come along and bump it all over anyway, so

Megan Torrance: Okay, that's the thing.

This is not the last thing.

Chris Van Wingerden: No.

Megan Torrance: So let's learn how we do it and get better because this is also not the first time we've ever done technology change, right? But this way, um, m- may not even be the best thing.

Um, and it's why, throughout my book, I actually have a, a few chapters that get kinda nerdy and wonky and historical, but, like, this is not new, folks. Like, there, people have been studying how we adapt to technology innovation, and how we, we drive this through organizations, and how we drive this through societies, and how it works for people, how it works economically for decades.

Probably even longer, right?

Chris Van Wingerden: I've been seeing machine learning art related articles for 20 years in publications and such, so. Yeah. Um, just the context of it, has just, you know, exploded outwards, rather than- Yeah ... being more focused like it used to be. Yeah.

What Teams Miss

Chris Van Wingerden: So o- of the 14 things, like, what are the ones that, that you [00:19:00] think aren't top of mind for folks when they first get into this but are really important?

Like what's the thing that they're missing a lot when they get started?

Megan Torrance: I think we're doing really well, for as much as I talk about measurement, right? we're measuring, doing things that are strategically aligned. Remember when ChatGPT was first released to the public, everyone was making funny limericks? we're not spending time doing that anymore, right? We're much more purposeful around it. I think that the depth of care around, bias and kind of people, risk mitigation is something that might be easy to skip over.

And, having people on the team who are like, "Not so fast. Don't think I'm comfortable with that," is exactly the people you want on your team, right? so bringing them in. And this is really a collaboration tool, right? what are the conversations we have when we all come together so that the technologists and the eager beavers don't just race right ahead, right?

This slows us down to have those conversation. I don't often see a lot of conversation around the user experience [00:20:00] Which is interesting, right? And, and there's the user experience for end users, and then the user experience for the build side of things. And, and it, it gets a little bit into scaling, but one of the stories in the book was a, a, an organization, they had bought a fantastic role play experience from a, a vendor.

It was for sales training, and, um, they were really excited about it. They, they saved it, like, they, they were able to kinda level up and, and, and, make decisions about their sales force in a much more... It, it was actually more equitable because you weren't being judged by individuals that had different scales and everything.

Like, it was, it, there was lots and lots of goodness. And they said, "We want more of this." And the vendor said, "Great, here's the bill for more of it," because there wasn't a u- the interface to build your own. So the vendor had to build custom from scratch every single time, which meant scaling just didn't work, right?

Um, and so that, that what happens after the [00:21:00] demo when they actually go to do it for real. Actually, our team just did some work. It was really, really interesting. We evaluated a number of, I'm gonna say docked course, uh, experiences where, you know, the, I'm, I'm taking some source material, it might be standard operating procedures or white papers or whatever, and I am somehow creating an e-learning course out of that software.

And some of the things that our team evaluated, it was a very robust evaluation, were the end user experience for the developer in addition to the user- Mm ... and what does that look like? what should a tool at scale be able to do? It's some really, really nice work, that, Chris King, and Lou Khan, and Catherine Steele have been doing.

Chris Van Wingerden: I mean, middle, middle ground then. The things that people maybe are knowing about, but maybe not doing either the best job that they could or, or, or could use a little boost on then.

Megan Torrance: one thing that's top of mind for me, and it's because of the role I'm [00:22:00] in and there are others who are agencies or consulting firms or support or in their work support a number of different clients, is making sure that your tool can very strictly parse what it has access to, right?

we don't turn our tools loose on our SharePoint environment. Right? Because we have different clients, and we need to keep their data separate from each other. and that's really important, and that's something that, I'm cautious, this is a particularly a small firm risk, I think.

Mm. Um, to make sure that that data is, is, is secure, across tools. I think that's really important.

Hackathons Experimentation

Megan Torrance: I love it when I hear about how organizations are experimenting- Mm ... and how they pilot. And, I've talked to a number of organizations who are hosting hackathons, who are doing cool things, and, you know, it, it, two years ago, hackathons, absolutely.

But now that the tool sets have changed, now that the capabilities are different, now that we're trying to roll out AI use [00:23:00] to, a wider population within the organizations, a great additional time to do more hackathons. I'm actually working with, um, ATD to create their, um, AI intensive this fall. What we did last year, we did last, last July, we did, a three-day, the intensive program is three Thursdays.

They're intense Thursdays. sequential, and, we did a hackathon during that last year, and we had a big Miro board and over 100 people all, like, zooming around on the Miro board and making projects. We had a dozen projects got started. About six of them actually presented at the end. It was fantastic.

It's kind of like the xAPI cohort, just in a very compressed timeframe. And, and it was a great learning experience, so we're gonna do another one. I had people coming up to me at, like, DevLearn and Core4 in the fall saying, "I did the hackathon, and it was great because I got to do this thing."

Right? and I think that's really important. It [00:24:00] also gives people who might be a little bit more reluctant to try a chance to get in in a safe, playful space, but working with other people who have more skills. And so that's, it, it's less about I'm gonna sit down and take a course- Mm-hmm ... and more I'm going to experience with my team.

that was actually one of the things we did at our onsite. We had four teams working. Wasn't exactly a hackathon, but, we had four teams working, and I think at least three of them used AI somehow in their solutions. And but that included people who didn't have a lot of AI use under their belt, and they got to see what could be possible and have some fun at the same time.

Y- you can think of some of the the larger organizations. We think of them as button-down, serious. So the idea of, of a hackathon, playful experimentation, all of those sorts of things almost feels counterintuitive for some of those organizations to, to consider.

Chris Van Wingerden: Almost feels like a risk to me for them to think about in [00:25:00] a sense, or, or it might feel or present to themselves as, as risky.

Megan Torrance: You know, I don't know because they also need and want innovation.

Chris Van Wingerden: Mm.

Megan Torrance: I'm trying to think if I know any large, large organizations with pure on hackathons, but certainly medium-sized organizations, which they just said, "Hey, everybody, let's come up with lots and lots of ideas and, and then weed those ideas out."

let's start with 1,000 ideas and then move forward on 250 of them and then have a re- and then move forward on 50 of them and then 10 of them." Right? And so they're, they're, they're choosing the best ideas. I think one of the risks with a hackathon is the governance risk actually. So somebody goes out and gets a different tool like, "Hey, you told me to play."

So we need good boundaries around that. but, and, and that's where having- A more playful use case can help, right? And that's, and that's the nice thing about the ATD Hackathon is [00:26:00] that you can come in, you can have a playful use case, you can work with other people and, and, and it's a little bit less risky, with sensitive data.

Chris Van Wingerden: Yeah.

Governance Guardrails

Chris Van Wingerden: You mentioned the word governance there. What kinds of things fall into that bucket? What kind of things do people are you seeing as implemented and, and, and what does that really mean? It's, it's just a... It sounds like such a somber word. Like you said at the beginning, governance.

Nobody two years ago would've wanted to even talk about that, but, fill us in on, on your perspective on that.

Megan Torrance: Yeah, I mean, governance is really the process and it comes out of and looks a lot like the security review process that software has had to go through for, probably the last 10 or 15 years in order to be adopted.

Um, but it's really around how does an organization set rules around what tools are going to be used and how they're gonna be used. there's multiple layers of governance. It's, from a, you know, we're gonna use this tool or this tool, or we're gonna use both of these tools so we don't have that risk, that one of, you know, something bad happens to one of them.

Um, [00:27:00] but also rules around what data can I enter into this, right? So AWS Guardrails allows us to say, "No, you, you can't even say certain things." Like, it just kind of doesn't, doesn't compute, right? and it prevents certain pieces of information from going into, the models and, and, and even being processed.

so there's, there's governance at that level. There's also governance at the level of I have an existing tool and they release an AI function Right? How do I approve or disapprove of that new use within that tool? or if that tool is gonna push it out to everybody and I don't have the ability to turn it off, I may not have access to that tool anymore, right?

My organization may say, "Nope, we don't like how you're storing data. We don't like how you're communicating about how you're storing data and where that data is being used. And therefore, since you can't be trustworthy about that, we're just gonna say no." Okay, that's fine.

sometimes we see governance in terms of how many different tools are we bringing into the [00:28:00] organization? Who can go buy a tool? Mm. Who can go buy a tool or a service, right? and what are those rules? And then I'm now seeing in some more mature organizations, the governance lead is the one who is leading the, are we getting the value out of this?

Is it actually working the way we thought it was going to work? And being the one who's coming around and checking and evaluating and measuring what's going on. And governance also is which laws and regulations are we adhering to and what, the, and are we subjected to?

So, actually Margaret Spence, who I mentioned already, is, is spinning up a fantastic tool in site that said, you know, you put in where you're located and what you're doing with your software, right? And it says, "Oh, you're subjected to this law in California, and this law in Connecticut, and this law in the, you know, EU.

And you've got 17 days until this other law in this other state comes out," right? And to really look at, what those data protections that are [00:29:00] required are, and how you're validating that you are protecting that data.

Chris Van Wingerden: Yeah. That data protection is usually, as I mentioned with the organizations that are doing, they do risk assessments on, you know, the stuff that we roll out.

Data protection is pretty much the first thing on that list for sure. and even just in general conversation, "Okay, what happens to our stuff?" Yeah, yeah,

Megan Torrance: yeah.

Chris Van Wingerden: even from casual users too. So it's very top of mind for sure.

Whats Next Pragmatic AI

Chris Van Wingerden: what do you think is next?

Megan Torrance: I feel like, we're in a spa- where, you know, if you, if you think about that, the, the Gartner hype cycle- Mm

right? Where there's peak of inflated expectations. Right? Everybody's all excited, and then that falls into a trough of disillusionment, and then it kind of levels out at a plateau of productivity, right? we are somewhere between the peak, I think, like, we're, we're, we're pretty peak. it's hard to get much more hyped,

but we're finding out that some of the real work is, is hard, right? You're seeing some, companies [00:30:00] hiring back people. You're seeing, you know, lots of conversation about the human in the loop. we built a tool called MADGE that is basically a dashboard for both an AI and a human in the loop to work visibly, and, and to be very, very, transparent about how we're reviewing what AI does.

I'm hoping that we'll see MADGE at Demo Fest at DevLearn this year. But, the getting down to the hard work of making things really work, right? It's very cool to vibe code something fancy and flashy, but those tools tend not to scale or to be terribly robust- Mm ... right?

And so how do I get from the excitement of being able to vibe code to being able to actually ship reliable software, right? And, so there's all of that space, and it's a fantastic learning space, right? a near term next, Chris, is I think an opportunity for learning professionals to go back and look at [00:31:00] every other- Course, right?

So we've spent a lot of time spinning up AI literacy courses, right? But to go back and look at how does leadership change when leaders have AI tools? How does sales change when your customers can research everything? How does, customer service change when your customers are using AI? how does safety change when you have the ability?

How does quality control check, right? So all of the things that we teach and skill people up to do on the job now change. Now, would we change- would we just update all of our e-learning modules? No, we would do something different, right? We would update our e-learning modules, of course, but perhaps add some additional supports and, and extension within the organization in order to make sure that what I learn in this new updated module then gets, reinforced and supported two, four, six months out in onto the job so that I'm doing that [00:32:00] consistently and well.

So I think there's a lot of opportunity and I'll... Two things. So I've... Kind of my fall, seems hard to believe, right? It's like 80 bajillion degrees outside, and that's Fahrenheit. Um- But, uh, it's not 80 bajillion degrees Celsius, everybody. That would be a bad, bad idea.

Chris Van Wingerden: But- Heaven forbid it's Kelvin too. That would be even

Megan Torrance: worse.

That would be more. We would just be like fried, right? So for me, fall, fall conference season has started, and I'm seeing two conversations come right in alongside the AI conversation. One is data. Super interesting, right? So I wrote a book on data. It came out, it's fantastic timing, like two months after ChatGPT came out-

to the public, right? Nobody cared about data. I mean, data was supposed to be like all cool, and I spent all this time writing this book, and then nobody cared about data, because they were writing limericks with ChatGPT. Yeah.[00:33:00]

Now my data friends, my m- data and measurement friends, we're all seeing this upsurge, right? How do we measure what's going on? How do we measure actual performance? how do we measure whether people are doing things differently with, with AI, right? And so d- data and analytics is actually really, really hot right now.

And then the other is performance support. How do I personalize that moment on, like, you know, Bob Mozer and Conrad Gottfredson called the moment of need, the moment of apply, right? In that moment when I need to do something, how am I supported? How am I enabled to do that? And those two conversations, right, don't compete with the AI conversation.

It's all the same conversation, right? Because I can use AI to do both the things, things, and I have the opportunity to do both of these things, and the pressure to do them now that we have, gen AI tools kind of like in, in that space. So it's super exciting. There's so much good work to be done.

Chris Van Wingerden: it's interesting, how it's sort of fanning out from here's what AI is to more practical things, as you say data et [00:34:00] cetera, that it's still the AI conversation, but the tone of it's changed. We're hitting the pragmatic stage.

Megan Torrance: Okay, so Chris, 'cause we've been at this for a really long time. It's like xAPI, right? We used to spend a lot of time talking about xAPI. Well, I used to spend like an inordinate amount- ... of time talking about xAPI, right? But, but, but that wasn't the, the, like, that was a moment of time. It's a tool. It's not the only tool.

and ideally we move past talking about the tool or the data standard, and we move on to what do I do with it, right? Mm-hmm. How do I document competency and skill and activity in meaningful ways that reinforce the learning and the business processes, right? So that's the interesting conversation. and, and the what do I do with this thing.

Chris Van Wingerden: Moving away from, every xAPI statement has a subject, a, a verb and an object, and what it is- Yeah ... et cetera, to ooh, who cares about some of this stuff? this is the place where the stuff really actually proves beneficial value.

Megan Torrance: But also to get there, there's hard work.

Chris Van Wingerden: [00:35:00] Hmm.

Megan Torrance: There's hard work and there's like hard wonky work like data profiles- ... Like, how are we actually going to document the timestamp on a particular type of thing, right? Mm-hmm. Like, somebody's gotta do that, right? and I am very happy that those people do that because- ... it, that, that's what makes it useful.

And I think we're gonna have that same kind of, kinda practical productivity phase for our gen AI tools too.

Chris Van Wingerden: Yeah.

Wrap Up Links Outro

Chris Van Wingerden: Megan, as always, thank you so much for joining us here today. lots of links folks. If you haven't noticed, there's been lots of links in the chat. both links to, Megan's books and resources and some other stuff too.

So I got my reading list set out for me for the, for the rest of the evenings this week for sure. Um, as always, folks, remember, instructional designers in offices drinking coffee, drinking hot cocoa, drinking cold coffee, what the heck, it's summer, is brought to you by here by the team at dominKnow, makers of dominKnow ONE, helping L&D teams to develop, scale, and deliver learning that maximizes employee value.

Also keep in mind, hit our website up. Our State of Learning Content [00:36:00] Management 2026 report available from, our website. I'm sure our team is also gonna drop a link into the chat on that if you want to, delve into some of that particular data too.

So all right. Hey gang, let's dance on out. Catch you guys in two weeks.

[00:37:00] [00:38:00]