Learning Engineering: Data-Driven Mastery in Practice

Welcome and Guest Intro

[00:00:00]

Chris Van Wingerden: There we go. Hey. Hey, folks. Welcome to instructional designers in offices drinking coffee, #IDIODC. As always, brought to you by the team here at dominKnow One, helping L&D teams develop, scale, and deliver learning that maximizes employee value.

We are joined for this week, by Jim Goodell. Jim, you've been with us before in the past, but there are probably folks who haven't caught that episode or maybe heard you speaking at, conference sessions, et cetera.

So, introduce yourself to, [00:01:00] to the folks joining us here today.

Jim Goodell: I'm Jim Goodell. I am the founder of INFERable, a public benefit corporation. But a lot of my time is spent volunteering as the chair of the IEEE Learning Technology Standards Committee, and I'm heavily involved in IEEE ICICLE, which stands for the International Consortium for Innovation and Collaboration in Learning Engineering.

Last time I was on, I was talking about the book that I co-wrote with 28 other people called The Learning Engineering Toolkit. Mm-hmm. I also work on projects for the US Chamber of Commerce Foundation and the Gates Foundation and, do a bunch of other things. So I wear a lot of hats, but, I, I think, I, I'd like to, talk about learning engineering and, some of the things we're doing with INFERable, if we can today.

What Learning Engineering Means

Chris Van Wingerden: You've mentioned that phrase a few times, learning engineering, and it might be something new to, to some of our folks. I mean, we have instructional design, you know, those sorts of [00:02:00] phrases, but learning engineering might be something new. So let's, let's start with, with there.

What do you describe it as, and how is it maybe different from some of the other phrases that we use to describe what we do or what we ought to be doing?

Jim Goodell: Okay. Well, Chris, I'm gonna share a secret with, you and our listeners. And, that's that-

Chris Van Wingerden: Keep it under your

hat, folks. Yes.

Jim Goodell: The secret is that you can be, have the title of instructional designer, but be doing learning engineering, because learning engineering is more of a verb than a noun.

And so listen to the official definition and think about what you do. Learning engineering is a process and a practice that applies the learning sciences using human-centered and engineering design methodologies and data-informed decision-making to support learners in their development. So if you're doing, human-centered [00:03:00] design as part of your job, and if your team is using data-informed decision-making and using some engineering principles then you might be doing learning engineering.

And it's something often done as a team.

Chris Van Wingerden: You just

didn't know it.

Yeah, you just didn't know it yet.

Jim Goodell: Yeah, you didn't know it yet, right. And you don't have to take my word for it. This definition was created by, IEEE, the largest technical professional organization in the world, and the same organization that brings you standards like Wi-Fi.

Chris Van Wingerden: A lot of us in our space, we learn what we're doing, sort of by an apprenticeship model. We get hired for a role, and we learn from the person in the next cubicle beside us what w- what's always been done in the organization or other documentation, you know, et cetera.

What strikes me is that, that definition with its, i- in, it... I don't think the word is expansiveness, but it's completeness, et cetera, certainly sets out a few expectations [00:04:00] and, and, and gives us maybe, some, some things to think about that maybe if we've done that learn by, learn by apprenticeship, maybe we didn't get to, to pick up on along the way.

If we're just doing the things that other people have always done, it does strike me that there's some, some key things there that, can maybe, keep us between the, between the lines on the road in a better form.

Jim Goodell: Right. We all should be lifelong learners.

Chris Van Wingerden: Mm-hmm.

Jim Goodell: And we all should be curious about what else can I learn that will help me do my job better, or, um, what can I learn about what other team members do so I can communicate with them better, and kind of have that common language.

Often on learning engineering teams, there are people coming from completely different, domains, and sometimes they use the same words to mean different things. So a little bit of cross-training goes a long way.

Chris Van Wingerden: Yeah, yeah. Set yourself up a, a lexicon or a glossary of terms- Yes ... so that everybody can point to the same thing and, and be [00:05:00] speaking the same, speaking the same meaning, not just the same language.

Human Centered Design Basics

Chris Van Wingerden: One of the things that, I mean, so you mentioned, you know, human-centric as part of that, that initial definition. Maybe just give us a couple of examples of that, so people can understand what that means.

Jim Goodell: Sure. Um, human-centered design starts with, developing empathy for the end users, in this case the learners.

So, if we don't really understand the learners, we're not going to be able to design solutions and content that, is going to meet their needs. And learners, not every learner is the same. Learners are different. So, understanding, the different characteristics of the learners that you're trying to serve is really important.

And, one way that human-centered design does that is to develop personas of, of the different kinds of learners that might be served. Mm. But there's some great resources such as IDEO, the design [00:06:00] company, has, a free toolkit for, design thinking and in- and, human-centered design.

Chris Van Wingerden: When you think about applying this and we think about maybe the landscape of what gets produced a lot as learning events, whether those are e-learning courses or in-class things, a lot of those events, and, and we've always talked about this over and over again it seems here on IDIODC, but there tends to be a tradition anyway of here's a whole pile of information.

There, we've given it to you, and now you go away and you know things or, uh, or you can do things, et cetera. Um, and I think, one of the things that, that you've been focusing on is, is, I don't, I don't want to say breaking that model, but making sure that we do a better job than, than just that in what we do.

Forgetting and Duolingo Insight

Jim Goodell: Exactly, and it's not just ... And here's another secret: people forget things.

Chris Van Wingerden: Oh, wow.

Jim Goodell: They forget how to do things.

Chris Van Wingerden: Crazy, eh?

Jim Goodell: I started in INFERable because, partly with the research for the Learning [00:07:00] Engineering Toolkit, I was, trying to discover who was doing learning engineering and what was learning engineering back in 2017 and 2018.

And, one of the places I visited was Duolingo, the language learning app, and, I spoke with their chief scientist, Burr Settles, and a colleague, who's now the, lead scientist at the organization. And, Burr asked me, "When is the best time for someone learning a language to practice a word?"

When is the best time? And I said, "I don't know. Maybe the morning?" And then I thought, oh, well, for my kids, they're younger people, maybe later in the day. And he said, "No." What, well, what do you think, Chris? When was the best time?

Chris Van Wingerden: When it's needed maybe.

Jim Goodell: Yeah, you're, that, that was, that was the answer that I- You're be- you were better than me.

Woo. It's [00:08:00] right before you forget it. So, so they have some AI behind Duolingo that can predict when you're going to forget a word, and they, they use transactions from millions of other users to determine which words are easier to forget or harder to forget. Hmm. And then they use your own data to personalize it, to personalize the prediction.

And I thought when I, at that time, "Why can't every learning platform have this kind of intelligence? And ho- why can't we transition to not only do the traditional course formats that really test short-term memory, to, to... And, and in addition to that, have, this kind of mastery practice like Duolingo?"

Chris Van Wingerden: Yeah. Short-term memory. We've, we've all ha- done it. We've, we've all crammed for a, a night before an exam, gone in, [00:09:00] written the exam, and then boom, stuff is gone, later on. Yeah. Doesn't sit in. It doesn't s- switch over to long-term memory. It doesn't create new pathways to reinforce that.

So, yeah. And yeah, giving people a, a half hour or a one-hour e-learning course, with a 10 question quiz typically, you know, amounts to about the same it seems.

Jim Goodell: And that's the kind of the compliance model.

Chris Van Wingerden: Mm-hmm.

Jim Goodell: Is we, we have people, go through a s- short amount of introduction to a topic or instruction, and then take a little quiz.

And then the lawyers can say, well, we trained them in that safety procedure,"

Chris Van Wingerden: Y- yes, you can run the report and say that the boxes were checked and we have data showing that we did what we needed to do. Yeah.

Jim Goodell: But, six weeks later or six months later, who knows?

Chris Van Wingerden: Yeah.

INFERable Skills Practice Dashboard

Jim Goodell: So what we've developed at INFERable is a plugin that, allows any instructional designer to create a dashboard of skills [00:10:00] practice. So in addition to the course, every learner can have their dashboard, and on the dashboard it's very simple. There are, there's a widget for each skill that they might wanna practice.

There's a gauge that gives a prediction of success on the next attempt using the previous practice attempts as the, source data for that inference, and there's a practice button. So at any time the learner can go in and practice something, and if they see, "Oh, well I'm at 60% of, being able to succeed on this task, maybe I should practice this o- this one a little bit more and get that gauge up

Chris Van Wingerden: So, so the, the gauge that's predicting for them is actually alerting to them that maybe they should do some more work before they give it a whirl.

Jim Goodell: Or it could be in between maybe there's an annual, refresh, recertification for a, a particular area of [00:11:00] skill. But like, like we said earlier, every learner is different. Maybe a year is too long for some people, and maybe a year is too short for other people. So maybe the, to keep the lawyers happy, we still do something every year.

But in between that time, there's an opportunity for practice. And, and that gets to not just compliance, it gets to performance.

Chris Van Wingerden: Mm-hmm.

Jim Goodell: So the, individuals can perform better, and then the organization, the company can perform better, because of that collective skill upskilling or reskilling.

Chris Van Wingerden: And I mean, spaced repetition, which is, you know, this is leveraging, Yes ... it, it's one of the oldest, y- strategies. Folks have been talking about it, researching it. Will Thalheimer has a great PDF if anybody needs to- Yes ... to, to brush up on that. He offers it free, from his website.

Super awesome thing for him to have done. And [00:12:00] I've relied on it multiple times. But, you know, the idea of, of, making sure that you ... Well, I guess it ties into that just before you forget it model that you were describing from Duolingo too, that, i- if you can provide people with opportunities to relearn and reinforce something, that it does, especially if that is sometimes done in different ways, not the same thing over and over again, so it gets, maybe seen as redundant or, or repetitive or whatever, you know?

But different approaches to that helps reinforce, that learning and help actually seed it into longterm memory so that it can become something you can draw on, later on without having to go to the job aid or heaven forbid, review the LMS course.

Jim Goodell: Yeah. In learning engineering toolkit, we kind of go into the history and, the, researcher over a century ago was looking into f- forgetting curves and- Mm-hmm

how quickly people forget things. So it's, there's a lot of science behind it.

Chris Van Wingerden: Yeah. And, most of the time we don't [00:13:00] even get given that when the person in the cubicle next to us is te- teaching us how to make courses for, for our company, right?

Jim Goodell: Yeah. Yeah.

Chris Van Wingerden: Uh, that, that background, that seems to all often be something we have to find out and pursue on our own.

So yeah.

Jim Goodell: That's another, bringing us back to learning engineering, one of the tools of learning engineering, are, is, making a log of decisions as you go along, and that can, you just kind of prompted a thought that, recording those decisions help the team later understand why certain design decisions were made.

But it also can be a learning tool for the team as you're working with the person in the c- cubicle next door. Mm. And you, do something a certain way, that can be recorded in the running log of design decisions.

Chris Van Wingerden: Right, and if it leads to something successful or whatever, you could pull it out from that and turn that into a practice, a standardized practice that can move forward and be, put into [00:14:00] practice on the next sets of projects too.

Yeah. 'Cause gosh, I've had times when, you're looking at something going, "I know there was a reason we did this, but, but nobody wrote it down," so.

Jim Goodell: And I'd encourage listeners to check out IEEE, I-E-E-E, icicle.org in that there are some free resources if you connect with that community, they have, some tools that you can use for, like, recording design decisions, and, s- kind of seeing, where the work that you do fits in with learning engineering.

Chris Van Wingerden: Nice. Yeah.

xAPI Integration and Setup

Chris Van Wingerden: Um, so, back to the dashboard, et cetera, that you created. I mean, you guys have done this in a way that it's, it's leveraging xAPI is what I understand?

Jim Goodell: It is leveraging xAPI, and, I'll mention that dominKnow is a great platform for xAPI.

One of the earliest, tools that fully embraced xAPI, and so we actually have a dominKnow template- Yeah ... that is a way [00:15:00] that makes it really easy for instructional designers that use dominKnow to implement, our, our plugin and our inference service. And Moodle is going to be, launching a marketplace probably next week- Hmm ... if everything goes well. And our, our plugin will be on that marketplace. Oh, right on. But if you use dominKnow, just go grab the template and, and try it out.

Chris Van Wingerden: If you're a dominKnow author and you go to make a new project, you can go to the marketplace. When you go there, switch over to projects rather than bundles, and then just type in infer, and you'll find it easy there.

And there's an import button, looks like a little arrow pointing to a server kind of icon. Click that, and it will be brought in as a project, and you can start exploring it. There's a very helpful, video at the start, and then a guide that's part of the early pages as well to help you understand what it's doing.

Inference Algorithms and AI Signals

Chris Van Wingerden: So, Let's talk about, 'cause you used the word infer or, or inference, et cetera, and, and the organization is INFERable. [00:16:00] What's doing that inferring then? Y- xAPI is capturing information, I guess, and sending it to an LRS to be collated. But what's the next step that's happening then?

Jim Goodell: The inference service is looking at the xAPI data, and it's using an algorithm, Our initial algorithm is, based on Bayesian knowledge tracing. It's one of many knowledge tracing algorithms that can predict things like success on the next attempt. It also factors in things like if someone, answers correctly, maybe they just guessed.

Chris Van Wingerden: And

Jim Goodell: if, if someone answers wrong, maybe they really know it, but they slipped up. But there are other knowledge tracing algorithms out there, and there are also forgetting curve algorithms. As we, progress, we'll be, updating and improving the models. And AI is making it possible to get more and more precise models, [00:17:00] and then predict other kinds of things like- Predicting frustration and boredom or disengagement from a learning activity.

And you can imagine that can be really valuable if, someone is using l- in your course or in your micro-learning activity, and they're getting frustrated, they're about to throw the laptop out the window. If you knew that ahead of time, you could adapt that instruction and, maybe scaffold the experience a different way.

Chris Van Wingerden: Hmm.

Jim Goodell: And, so we're hopeful to get our disengagement detector out there soon to help with that kind of problem as well.

Chris Van Wingerden: Yeah. Sometimes the experience that you set up is itself also a, a great conduit for disengagement. Yeah. You know, thinking of a s- those so common e-learning courses where it's words and text and, and maybe a button or two.

And, there's a, a Simpsons episode, I think I've mentioned this in the past on a, on other IDIODC [00:18:00] episodes, but there's a Simpsons episode where Homer, ends up becoming, overweight and can't go to the nuclear power plant. Yeah. So they give him a computer to work at home, and all he has to do is type yes or no, on a wire.

A yes or no or just a Y or an N, I guess it is, on whether to engage the safety systems or not. And, he ends up setting up one of those drinking birds to, to type. And, uh, and, and of course, uh-

Jim Goodell: Everything goes wrong ...

Chris Van Wingerden: hilarity ensues after that. But, but I sometimes think of that as the model that a lot of people just do.

It's, it's the, if you could get the drinking bird to just hammer the next button,

Jim Goodell: Right ...

Chris Van Wingerden: you can get through it, and then you guess C on all of the 10 multiple choice questions, you, you typically have a good chance of, of just getting through it. And meanwhile, you can do other things in the background.

Um-

Flow of Work and Better Experiences

Jim Goodell: I'm gonna put on my futurist hat, hat now- ... because I think, the world is changing because of AI, and every job is going to change. And, I think our industry needs to change as well. [00:19:00] So we need to be thinking more about kind of the blending of performance enhancement with instructional design and L&D.

So I can see instructional designers and learning engineering teams working more on learning in the flow of work and, using AI and job aids- As part of what we do- Mm ... to help improve performance.

Chris Van Wingerden: We in the past have had, Konrad Gottfredson on, of course created the framework, The Five Moments of Need.

And four of those moments aren't e-learning. You know? They're, they are about that flow of work, when information is needed to do something new, something, you know, different, make a decision, those things too. A- and I believe his team is doing stuff also with AI around amplifying that, for sure.

And it does seem, I mean, even pre-AI, that was just, that's just such a sensible [00:20:00] framework to, to look at and think about for moving things beyond just a, a one-off experience through to something hopefully lining up closer with mastery, and performance improvement.

Jim Goodell: And the other thing now is with the AI tools for instructional design in learning engineering, we can create more engaging content.

We can turn static content into mini simulations.

Chris Van Wingerden: Mm.

Jim Goodell: Different kinds of puzzles or different kinds of more engaging experiences.

Chris Van Wingerden: Yeah. One of the flags for, for making sure that something is going to be helpful is to make sure that it is as real world applicable as possible.

Sometimes people say, "Well, can we make a Jeopardy type game?" And I say, "Yes, you can. But should you?" Are you teaching people to memorize statics and facts, or are they, or do they need to do things? And is a Jeopardy style game actually, you know, contributing to a positive [00:21:00] goal of maybe changing behavior, et cetera, so-

Jim Goodell: I volunteer on the, tutorial subcommittee of the I/ITSEC conference, which is a, a big modeling and simulation conference connected with, Department of War and other corporate, entities like, like Boeing and, that needs flight simulators and high-risk environments like oil and gas that need-

Chris Van Wingerden: Hmm

Jim Goodell: simulated environments for training, s- to kind of lower the risk of people training in the real environment. And I think, um, simulation is one of those areas where, we can get close to the real environment and there are much lower class ways of doing simulation now, because the cost of the devi- like devices like VR have

come down. But all of it should be instrumented with xAPI, because if you're not collecting the data from the experience, if you're not observing the learner in the [00:22:00] learning conditions, then you really don't have the data you need to, both adapt the experience for the learner but also find out what's working and what's not working so that you can improve whatever the experience is.

Chris Van Wingerden: I think about one project, and I've, I know that I've told this story on other IDIODC episodes, but it, it, has stuck with me for, for so long that it remains a, a, an important touchstone for me. One of my first projects that I did early on anyway, was, a simulation of interviewing, like a, a know your client kind of, process.

Yeah. And a conversation and the, the, the learner was making choices of what questions to ask, and so it was branching and took you down different paths. And about a year and a half later I got, and I always use this word, the privilege, to be able to, do follow-up and make revisions and changes based on the experience and some of the, the feedback were that, the, there was one branch that took you a fair piece down but ended up being a negative branch, and [00:23:00] people-

Jim Goodell: Mm.

Chris Van Wingerden: People really resented that. The, the learners had a limited amount of time, and then they're told that they have not succeeded. They didn't ever go back and do the right pa- path to, to correct themselves in that way. So being able to understand xAPI could have really helped surface that, maybe a lot sooner, those kinds of patterns and, and understanding those sorts of things, it would've been quite valuable, where SCORM just looking for that completion, the- Yeah

The bell ringing in the LMS, so.

Although, xAPI is also something that a lot of teams go, "Oh yeah, we'd like to, but..." And the buts include the sheer technology, knowledge that you, you typically need. You need to understand xAPI, you need to understand statements.

You get an LRS, you need to then understand how to make the LRS do what you want it to do, 'cause You need to be pretty savvy, and that can all be very, very daunting as well, especially for teams that don't have in-house resources with that level of technical skill.

We're, like, more than a decade since, xAPI was officially launched, let alone the years of Tin Can before that in [00:24:00] development, et cetera, and, and it is something that we still hear people, "Oh yeah, we'd like to get there, but,

Jim Goodell: yeah, I would say it's less of an excuse now-

because tools like dominKnow make it pretty easy, and you can ask, let's say you have a learning experience that's running in an, a, a single page web application, something like that. You can, dump it into the AI and say, "Add an xAPI, capability to this, and I want to capture these kinds of events," and it does it for you.

Chris Van Wingerden: Mm-hmm.

Jim Goodell: All you need to do is put in the, the URL to, and the sign-in credentials for the LRS, and you're good to go. And there are hosted LRSes so you don't have to set up your own.

Chris Van Wingerden: Yeah. It's an amazing world, The way things are, are changing, and at speed, for sure. Yeah.

Learning Engineering Education Paths

Chris Van Wingerden: back to the, you know, the learning engineering, does the IEEE or other bodies, like, i- is there a formal program that one could enroll in to become [00:25:00] qualified or, or upskilled in that front?

Jim Goodell: There are some universities that are-

Chris Van Wingerden: Okay ...

Jim Goodell: launching programs. John Hopkins University just launched a program that I think is a master's or micro masters- Okay ... in learning engineering. There are, some resources that you can kind of get a s- start to get a sense of what competencies are included or a range of competencies for a learning engineering team, available through IEEE ICICLE there's a learning engineering program at ASU, and there's a...

If, if you're really interested in the data analytics side of it, and really kind of want to get into that in a big way, Carnegie Mellon University has-

Chris Van Wingerden: Okay ...

Jim Goodell: their METALS program, which is a master's in, education learning analytics, kinds of things. [00:26:00]

Chris Van Wingerden: Awesome.

Jim Goodell: I also would point out that there are, micro-learning experiences, folks like Megan Torrance. Check out, Meghan Torrence's site. I know she has something coming up, I think early fall-

Chris Van Wingerden: Okay, very

Jim Goodell: at one of the events. We're, focused on learning analytics. but I'll... if I think of others- Mm-hmm ... I'll, send them, your way so we can add them to the show notes.

Getting Started With Analytics

Chris Van Wingerden: there's a question from a LinkedIn user here in the chat. Would data analytics class be sufficient, or do I need someone or maybe perhaps something, you know, more specific?

I mean, if you're trying to understand that stuff, where, you know, where would you suggest folks start?

Jim Goodell: Well, I, I'm biased, but I would say start with- ... Learning and Tooling Toolkit. We have a chapter on instrumentation and a chapter on, learning analytics, and that kind of gives you a real high level of what is the process.

Okay. And, basically, I'll tell you what it is so you don't have to read the chapter. [00:27:00] So the, the learning analytics process is you start with the questions that you want the data to answer, and then you ask, "Do we have the data needed to answer the, that, those questions?" And if the answer is no, then you need to instrument it.

So maybe you need to add some xAPI capability. but if you do have the data, you need to translate the questions into data questions, and what I mean by that is, if the question is, should we give this learner a certain scaffold or additional experience, that's really more than one data question.

So the... It's one data question is, will the learner - what's the probability that the learner will benefit by this additional experience? So you can use one algorithm to find out that probability, and then you use another algorithm to set a threshold. So let's say you have a, limited number of learners that [00:28:00] you can offer this to, so you need to pick which one.

you look at, the needs, and you use the second algorithm to determine whether you actually do give the extra experience to that learner. and then you get into the heavy data analytics part, where you need to pick which algorithms you're going to use for those two things, and that's where, I'd recommend e- either a formal program like METALS or

There are some free, there's a free MOOT, Massive Open Online Textbook, that goes over some of the learning analytics methods.

Chris Van Wingerden: Hopefully that, is helpful for the audience member who asked that.

I think there's sort of a follow-up version of the same question, wondering if there's stuff in Coursera, for example, that would, help get them started. Yeah.

Jim Goodell: Oh, that's true. Yes.

Chris Van Wingerden: Cool. I mean, start somewhere. That's usually the first ... That's usually the simplest path, is start somewhere and then see where, see what else you, what you want to add on, you know, after that.

[00:29:00] the, the toolkit, that you've mentioned a couple of times, where do folks find that if they wanted to locate it?

Jim Goodell: the publisher is Routledge.

Chris Van Wingerden: Okay.

Jim Goodell: So if you go to the Routledge site and, search for Goodell Toolkit, it will come up.

Chris Van Wingerden: Nice.

Jim Goodell: There we've, open access some of the chapters, so there- Cool

you can get the, the first few chapters for free.

Chris Van Wingerden: Right on.

Jim Goodell: And if you're an academic, you always can get the book's evaluation copy for free. and over time, I'm, paying Routledge to open up more and more of the book for free.

Chris Van Wingerden: Very cool ...

Jim Goodell: but I think it's worth paying for as well.

Chris Van Wingerden: So many things are free, but so many of those free things come maybe with unknown pedigrees, unknown sources, et cetera.

So sometimes paying for something when, is, is actually an assurance of quality. It's an assurance of giving you confidence, I guess, in the actual info. So yeah.

Jim Goodell: And the, the, what I didn't remember was, [00:30:00] um, it's Ryan Baker is the author of the free MOOT.

Chris Van Wingerden: Okay.

Jim Goodell: posted at, learninganalytics an upenn.edu.

First Steps Toward Learning Engineering

Chris Van Wingerden: If you could suggest one or two things that people should do differently or, add to their practice, et cetera, to maybe move from, let's call it the standard instructional design model. We've kind of referenced a few times what that might look like. but to be able to just start being and, and start improving.

'Cause sometimes something called learning engineering might sound like a whole thing, right? It might be seen- Mm-hmm ... as more of a mountain. but if we could give folks a few mole hills to, to get on that path, what would be some of the things that, based on stuff that you've seen a lot, what would be some of the first steps that you would suggest to people to start getting onto that learning engineering path?

Jim Goodell: Yeah, I would say it really depends on where you're coming from. If you're coming from, traditional instructional design, maybe it's learning a little bit more about what is possible [00:31:00] with the data to help inform what you're doing. I think everyone should be thinking more about, from kind of this monolithic course mentality to more micro experiences-

Chris Van Wingerden: Hmm

Jim Goodell: with everything tied to skills. So we have in IEEE a couple of standards that go along with xAPI for encoding skills into, a machine-readable format. It's called a shareable competency definition, and we also have a standard for, what is a well-defined competency definition. So that's, IEEE 1484.20.2 and 1484.20.3.

So, not only knowing what skills you're trying to teach, but also, tagging your content to those skills, and then you have assessments [00:32:00] tagged to the content and, and the xAPI statements tagged to the skills that the items are teaching or assessing. and when you have those data links, you can do some really interesting things, in analyzing whether what the learner is experiencing is actually resulting in the skills that you want.

Chris Van Wingerden: Imagine that. We're doing something that maybe doesn't end up resulting in what we expected it to do. Mm-hmm. That, that never happens in our space, does it? Ah, good times. Well, good times that we can joke about it, or make wisecracks about it, so yeah.

Jim Goodell: I would just say, so that's for individuals.

I would say look at the definition of learning engineering and find something that you don't, you, you're not an expert in yet, and learn about it. It's, it's-

Chris Van Wingerden: Yeah ...

Jim Goodell: just natural curiosity. And then for organizations We have a, generalizable [00:33:00] maturity model for the adoption of learning engineering that, a bunch of experts published a couple years ago, and it was a paper at the I/ITSEC conference.

but that can be used to kind of look at your organization and see, what things might you do to make small changes or big changes within your organization so that it becomes more of a learning engineering organization. And it might be something like if you're a company that outsources the production of courses, to think about different about that procurement, that you're not just outsourcing a one-off and, and just hoping that it works, but you're thinking more iteratively, that you're going to hire someone to do multiple iterations and use the data to make improvements.

And that's, that would be a great investment because if the training is [00:34:00] working better, then the workers are performing better.

Chris Van Wingerden: Sure.

Jim Goodell: So instead of training just being a cost center, it becomes a profit center and increased revenue for the company.

Chris Van Wingerden: Yeah. That ties back to a conversation theme that we end up hear a lot.

You know, how do you demonstrate value as a, a, an instructional design or a learning engineering team? And our job is to change behavior, our job is to improve things, or reduce risk or, you know, all of those sort of high level factors. and truthfully, what we often do doesn't target that very well or doesn't actually have demonstrable connection to that, and that's an injustice for us as a, you know, for all the effort and time too.

It's not just about not actually making changes, but it's also about getting, more respect and being able to be part of the bigger conversations and, helping the organization overall. we do have, following up to this and related in some of these tactics

Webinar Invite and Wrap Up

Chris Van Wingerden: You're joining us here at dominKnow for a webinar on September 13th at [00:35:00] 11 Eastern, Making L&D a Driver of Performance, Not Just Compliance. Join Jim and I again for a, a deeper dive, maybe in a little bit of s- little bit of overlap to here, but, other things as well.

Jim Goodell: And I think we'll be able to give a demo of,

Chris Van Wingerden: Very cool ...

Jim Goodell: how we've implemented the, mastery practice as something not to do instead of, traditional and compliance courses, but something that can be quickly and easily implemented to give that little extra, opportunity for everyone in the organization to practice the skills-

and to stay fresh.

Chris Van Wingerden: I'll also mention fr- here from, our team here at dominKnow, we have a research report that we've recently released. the State of Learning Content Management 2026 report, looking at insights where learning value is found or lost with the content that folks and teams and organizations are making.

Jim, thanks for joining us this week. Been such a fascinating, conversation. it, it feels like in one conversation we've woven together so many of the things that we end up talking about in, in so [00:36:00] many, episodes here of Instructional Designers in, in Offices Drinking Coffee, which I will mention again, is brought to you by the team here at dominKnow, uh, helping L&D teams develop, scale, and deliver learning that maximizes employee value.

Check us out at dominknow.com. Jim, as always, thank you so much for joining us. Thanks, Chris. Looking forward to catching up with you again, on that webinar in September. and folks, if you have questions, you know, most definitely check out, the INFERable site to learn more about , the mastery, widget, plugin that, you know, his, that team has developed.

It's, it's really fascinating and, and I think it can make a big difference for a lot of folks for sure. So gang, we'll catch you guys next time.

​[00:37:00] [00:38:00]