Trusted AI for Associations: Turning Knowledge Into Real-Time Member Value With Rob Barnes and Andrea Holovach 

Full Transcript

Bill Sheehan (00:00):

Welcome to Learning by Association, a podcast brought to you by D2L, where we delve into the ever-evolving world of associations and the challenges they face in navigating the currents of change. I’m Bill Sheehan, and I’m thrilled to be your host. Join me in our guests as we explore the role learning plays in driving associations forward and how it can impact every part of your organization, from recruiting to engagement and renewals, to staff development, business strategy, and more. So let’s dive in. Hello, everyone. Bill Sheehan here, global head of association strategy at D2L. And, uh, I’m on a. I’m having a, a podcast today with, uh, one of my favorite people in the association space. Somebody I have been dying to get on here to talk about the latest and greatest things around AI. And, uh, so I’m, I’m happy to have Rob, uh, here from Betty.ai and Andrea from, um, ISAs here.

(00:52):

But I would do, would not do you guys justice if I didn’t allow you to introduce yourselves. Tell us a little bit about your role, what your co – you know, what your company is and Andrea, what, what ISA does and, and its importance within the industry. So with that, Rob, let me kick it over to you, uh, for an introduction.

Rob Barnes (01:09):

Oh, so when you were saying one of your favorite people you’ve been dying, I thought you were talking about Andrea. And I’m just as. I’m ju – you know, because I’m, I feel the same about her as well. So, uh, no, thanks, Bill. Great to see, uh, Andrea, good to see you as well. Um, so Rob Barnes here, I’m CEO and a co-founder with Betty AI. Um, we are one of the world’s leading AI companies specifically designed for associations. So, uh, from our humble beginnings in, uh, 2023, uh, as a, as a company to today, you know, we’ve grown to about 150 a- association customers around the world, uh, and doing some pretty neat things that I think that add significant value to the work of associations, the purpose of associations. I’ve been in the association space my entire career, my first job when I was 19.

(01:58):

I’m now a little bit older than 19, as you can tell. Uh, and so it’s all about associations for us. The whole company is built around full-purpose organizations. And so it’s exciting to be able to talk about it, and exciting to be able to talk about it with one of our favorite customers, and one of the more successful and courageous organizations in the association market in, um, in the International Society of Automation, uh, and a good friend of mine. Andrea, over to you.

Andrea Holovach (02:26):

Thank you, Rob. Um, Andrea Holovach, um, as Rob said, with the International Society of Automation. Um, we are, we are a professional association for engineers, technicians, um, and management working in industrial automation. Um, really lay our claim in, um, setting standards and offering training. Uh, our mission really being to, to build a better world through automation. Um, my role is governance and membership, so I definitely make sense being on the AI podcast. Uh, but, uh, have known Rob for just a few years. Um, we’ve been partners with Betty since 2024. Um, been a, been a fun and exciting journey with them, and, uh, happy to be here.

Bill Sheehan (03:12):

Great. Well, thank, thank you both. And, uh, obviously, AI is the talk of, of association space, right? It’s the buzzword. And Rob, you and I have had conversations, uh, late into the evening, early mornings about, um, you know, do, do they really understand what AI is? Um, it’s been around, as you know, forever. Um, but it’s really starting to make, make, you know, become, uh, pervasive within the association space. There’s both fear, hesitancy, um, other types of words you could use where associations are not readily, uh, accepting AI within, in their, um, uh, you know, within their ecosystem. And I know some associations, particularly in healthcare and legal, maybe finance, you know, they got their, they gotta have guardrails. But I can’t think of anything as important as, you know, that supply chain, that industrial training, uh, for, uh, for industries. I, my background has been in associations for, you know, I’ve been in the association space for 30-plus years.

(04:11):

Most of them on the trade side, so they were manufacturing, so very familiar with ISA. You guys did, did great things. But, but Andrea, what, what was it that, that, you know, where you found Betty.ai? Because I’ve known them, they have a great reputation throughout the industry. H- how did you come across, Rob? I mean, what, what were you looking to do as an organization that you said, “You know what? I gotta call, I gotta call Rob and betty.ai.” What was that, that catalyst with – He didn’t

Rob Barnes (04:37):

Need to call me, actually. I was right there. This was a good story.

Andrea Holovach (04:42):

He didn’t need to call. So in, in all fairness, um, and I, I tell people this all the time, I’ve, I’ve known Rob, uh, 20 years-ish.

Bill Sheehan (04:52):

Indeed.

Andrea Holovach (04:53):

Um, which is crazy to say. Um, and not always while Rob was at Betty, obviously. Um, so we were really fortunate. We had a board, um, still have an executive board that was very interested in this AI journey and, and really challenged us as a staff to, um, to get, to get out there, figure this out. Um, so how we get to Betty is, um, as I will describe serendipity. Um, Rob was in town and we met to have a beer. And I was like, “What is this Betty thing? I don’t. What is this?” Um, and he started to explain it, and I’m quite certain I think I was texting my IT director under the table, um, saying, “You should probably check into this.”

Bill Sheehan (05:39):

What was that, um, uh, concept or, you know, when, when you were looking to bring AI, uh, Betty, that AI in, you know, Rob, what was that, the need, the initial need? You know, kind of walk us through those associations who are, you know, the conversations you had with, with Andrea, I’m sure other associations are having those questions. You know, where, wh- where, what’s the best way for you to kind of come in and say, okay, AI, a lot of them use it for marketing, I guess, communication types of things. But there’s other ways, you know, data and the like. Wh- when you talk with associations, typically, what’s the, the, the, the one need or want that they’re, they’re looking AI to solve?

Rob Barnes (06:18):

Uh, the one need, the one thing that everybody almost consistently says is our members still complain they can’t find it. They can’t find what they need when they need it most. That’s kind of where the problem statement starts. Our contention when we started the company, so again, kudos, Dre McFarland and Thomas Altman, anyone that’s listening, these are two of the smartest people and the nicest people you’ll ever meet. They are computer scientists, but they can talk like humans. Though it’s, they are an incredible. So listen to them, listen to that. But they had built the initial stages, and they’d been working in machine learning and AI for five years prior to launching Betty. Um, the, the premise of the company, though, was associations are the best placed organizations in the world to benefit from generative AI. So keeping in mind that AI has become a very, uh, ubiquitous term to tr – as a coverall for a whole lot of technologies.

(07:19):

Then generative AI came to be, and it came to be in the public domain when ChatGPT was first launched. Um, and we were already out in the market, you know, uh, with, with Betty, the initial v- versions of Betty by that stage. But this kind of catch-all. So our premise was, again, associations were the best placed organizations in the world to benefit from generative AI for two reasons. The first being, they are the largest repositories of knowledge for an industry or profession in the world. And they, uh, as organizations, um, we have spent millions and millions of dollars producing knowledge, knowledge assets that lives in these repositories, largely untouched. It gets published, it gets old, it sits in a folder somewhere, never gets used.

(08:11):

So that’s one thing, large repository of knowledge that’s inactive. The second part of it, though, was the, the way that knowledge was created. And it was created through systems and processes that have governed associations for over a hundred years. Peer review, ideation, collaboration, publication. It goes through these checks and balances that often drive us all mad when we, as Andrea is, she’s in charge of governance, right? We have technical committee after technical committee after executive boarder. But these systems and processes produced this body of knowledge that was already trusted by the industry or profession. And that was the exact thing that the world needed when the ChatGPTs of the world came out and used all the crap on the internet. Uh, notice I didn’t swear there, Bill, so, uh, but all of.

(09:06):

Yeah, on the internet to create these responses that were grounded in rubbish, right? And so people started to mistrust what these GPTs, what these tools were saying, which was awesome because associations were then automatically placed in this, in a, in, again, in this position of strength as the trusted source. So we applied a technology to an existing asset for an industry that was needing to be repositioned as these industry experts, these industry trusted sources again, um, to meet the knowledge needs of the people who needed it most. And so this idea was not just about, I can’t find what I need when I need it. That’s the complaint. But it was, I can’t trust what I can find when I use these public tools. And so Betty was designed specifically to serve that need.

Bill Sheehan (09:58):

Yeah. And, and, and Andrew, did you find that, that kind of that application effect group?

Andrea Holovach (10:04):

Well, it was, it was 100% of the application we were looking for, right? So when our strategy was increase awareness and use of our content, right? To, to Rob’s point, we have spent 80 plus years developing content that is trusted and developing content to make the world safer and more secure. But how do we surface it? And, and how do you find what you’re looking for? Um, and that’s really where, where Betty met us exactly where we needed to be. And we wanted to surface it in a way, and this was an added element of our strategy where we could protect our intellectual property, right? We didn’t wanna surface all of our content to jet – to ChatGPT, right? But we wanted to meet our members and our customers where they were, right? We wanted to give them the experience they were used to having. Um, and having Betty allowed us to not only make our content more accessible, um, but it, it helped us ensure our content was also safe and protected.

Rob Barnes (11:05):

So –

Andrea Holovach (11:05):

Bill,

Rob Barnes (11:06):

H – Bill, Bill – We took

Andrea Holovach (11:07):

Two strategies in one.

Rob Barnes (11:08):

Yeah. The other part of this, Bill, is our experience still is today to some degree, a little bit less now in 2026 than it was in 2023 and 24. But typically, people’s perception of the value of AI was driven around productivity efficiency for me as a staff. How can I do my job better with a little bit of help from an AI? And that, that’s still a very pervasive kind of thought process, right? Which is fine. There are plenty of free public generative AI tools, and now there’s embedded ones like Copilot and stuff that can help you do those things. That’s great. But what wasn’t being solved at the time that we believe we have solved really efficiently and effectively, and now we’ve repeated it hundreds of times, is how do you make that, um, solution or that experience publicly available or member-facing, right? How do you put that in the hands of a, of a professional member of a society and say, “We’re here to help you do your job today.

(12:03):

What do you need to know?” And do it at a level that’s, that created so that it’s a scalable knowledge need, right? For the most part, for the 20th century and the early 21st century, we talked about personalization, right? And personalization is cohort-based. We make decisions and presumptions about a group of people. We lump them all in together and we say, “We’re gonna communicate to you in a way based on these attributes.” But it’s all presumptive, right? It’s all just, we think that this is the way we can do this in a way that, that our current software tools allow us to scale some of the outreach that we do, right? It’s all the marketing tools were built like that. Create a cohort, create a persona, talk to everyone like that. Well, what we realized very quickly in the early stages of, of, of Betty and Mimo, ISA’s Mimo is a classic example of this, Bill, is that we deal now in the personal knowledge needs.

(12:59):

And I’m talking about what I need to know at this moment to do the job that I need to do right now at scale. I think in Mimo’s first month, or certainly couple of months online, Mimo answered 4,000 individual knowledge queries in nine languages.

(13:22):

And so that was personal. There was not one conversation with Mimo, and there’s not one conversation with every per – every version of Betty that’s out there in the wild that is exactly the same. Because everybody comes at their knowledge need from a different place, from a different. And they don’t even have fully formed knowledge needs, right? This is what we’re learning from this process, is that an organization in aggregate can look at all of these conversations, but look at them at a personal level, not a personalization or cohort level. And the insights that come from those conversations in aggregate are deeply impactful in ways that no association has ever experienced before this technology was available to them. And so what we love to see, uh, and I think what, what I’m hoping Andrea will be able to explain to our listeners that, is what does this actually mean when you’re running an association where member value, where member experience, where has an impact?

(14:25):

Um, because our vision was that this would be able to meet the very personal knowledge need of a person at the point of time when they needed it, when associations typically aren’t available to help them, or it’s really costly, or it takes too long to respond or whatever it might be. And we’ve proven that that’s possible. And that is incredibly exciting to see what we though could be possible with Betty actually happen in real life.

Bill Sheehan (14:55):

Yeah. A- Andy, I, I, I’m, I’m assuming, you know,

(29:56):

In today’s, in today’s association marketplace, particularly with the younger generation, they do want a personalized experience. Uh, uh, that’s no secret. They want that show me you know me, the, from the association. You know, what do I need? Can you provide that information when I need it right now? Because then it, it’s not only. And I’m trying. And, and Rob, you, you, you told me that let’s take the word relevancy out and let’s start talking about meaningful impact, right? So I’m quoting you and I’m, I’m using that throughout the industry now. Let’s forget about having deeper relevance as an association. Let’s talk about meaningful impacts. And I think that gets down to the individual level. So Andrea, I’m, I’m wondering too, as the members started coming on, I’m, I’m assuming you’re seeing increased engagement and maybe a, a ma – perhaps maybe even a longer duration of engagement on the site and, and the like.

(29:57):

Did you. Was that kind of what, what the outcome of that was? Yeah. And, and it’s funny you say as, as we saw members start coming to Mimo, um, as, as Rob alluded to, this was not a slow start. This was a Mimo went live, we have 4,000 questions. It may be dipped for a month or two. And I think at this point, are we two years in? We are averaging 4,000 questions to Mimo, individualized interactions with Mimo a month. Wow. Um, and I think we’re now up to 11 languages. Um, so there were a couple of things about that unique experience. The language piece has been impactful in ways we couldn’t even have dreamed of. Um, we produce our technical content in English. Um, it is very difficult from a governance standpoint to provide technical specifications in other languages. Um, but Mimo, our re – our buddy, um, responds to those technical questions in the language that it was asked.

(29:59):

Um, it links to the source documentation, which is huge. What a, what a massively different experience than I get with ChatGPT, right? I don’t get to know where ChatGPT dreamed up that answer. Yeah. Right. The links back to our content repository, where that question came from. Um, but really being able to allow our, our younger members, and I will say even our older members, get to know our technical content in their own language, um, is an experience we never could have provided them any other way. Um, and, and that, that interaction, it’s not just our members. We actually see people join. We’ve tracked it. People that come in, we built a different experience in Mimo for our members than for our non-members. We didn’t wanna hide our content from non-members. Um, but we have seen people interact with Mimo, not as a member. Mimo encourages them to join if they want more.

(30:01):

So you’re – We have actually seen them join and go back to Mimo and continue the conversation. So the conversion rate has been huge, right? And unexpected. Not will be intended. We love to see it. I’ll take it all day. Yeah. That’s, that, that’s great. Let me ask you a question from that to, to, wa- was there, um, and just from a logistical standpoint from the staff, th – were you promoting this out to, to the membership too, that this is coming? Right? Or was it just they found it and lo and behold, there it is? So I, I, I joke a lot. This was an incredibly easy implementation. We were so ready to launch that we kept looking for things wrong. And we just kind of started whispering about Mimo. Um, and then the members just found it. Now, we did, uh, our marketing team, I will give them kudos all day long.

(30:02):

They were on this journey with us from the beginning. This was not a, we went and developed a product and built it and then said, “Oh, by the way, marketing, can you go launch this?” They were at the table with us from day one. Um, and so Mimo has a mascot. Mimo attends our conferences. Mimo interacts with our members and our online community. Um, it’s, it’s a really. Mimo’s become our mascot. It, it’s, it’s part of our brand identity in a way that I would say we didn’t plan, except our amazing marketing team absolutely did plan for it. And it’s been great. Yeah. I, I’ve heard similar things. And, um, uh, Rob, you, you know, you know, uh, there’s a few associations. I know one in particular that has betty.ai in the org chart, right? It’s like part of the org chart. Um, which I think is, is, is terrific, uh, for that.

(30:04):

You’re reading now a lot of associations putting up guidelines and policies and procedures about this. And some of them are doing, doing it without even having AI as part of the program yet. It’s almost like you’re setting guardrails before. And, and Rob, I, I don’t wanna date you, but I was around when the inter – I was in the in – uh, association space when the internet came on, right? Worldwide Web. I remember I was there. So arery. I was there with you, buddy. Yeah, I was there when the fax machine was really cool. Um, and they started putting up, same thing, policies and procedures about this. Um, did, did you have to. Was there any of that going on, Andrea? Like, you know, do you really, you know, have to sit down? And the reason I’m asking this is I, I think the one thing that betty.ai can do probably better than anything else is tear down the silos that associations tend to find themselves in, right?

(30:05):

Communication department sometimes doesn’t know what marketing’s doing. And marketing sometimes doesn’t know what government relations is doing. And government relations sometimes doesn’t understand what, uh, you know, the membership’s doing. And, but I think because of AI, because it is so pervasive within the organization, it just kind, it, it pulls from everything. It kinda tears down those silos. And Rob, I mean, are you kinda seeing that when you’re talking with organizations? It’s not just marketing or IT, for example. People line up at ISA conferences to have selfies taken with Mimo. This is the impact that it’s had. But what’s important about this is one of our core premises at the company and for the product was the mitigation of the erosion of trust. 80 years, the ISA has been developing assets and knowledge and a reputation that is trusted. The last thing you need is a new piece of technology coming in and eroding that trust.

(30:07):

And the ero – the public erosion in, of trust in AI was already happening because the public tool’s making stuff up. Um, and so it was important to us that, and that the way we built the technology was that it protected the trust the organization had already earned and promoted ongoing trust. Citation of sources, the fact that it has guardrails, it’s, it’s trained to do its job. Betty is trained to do the job that ISA needs Mimo to do, and only that job. Anything outside of that, Mimo will politely de- decline and offer you some other a- assistance. This is, so this is about trust. It’s governed, scalable, trusted technology. That’s why that’s important. But the manifestation of that is what Andrea described is at events. People trust ISA, they trust Mimo because it can have technical conversations in 11 languages and people can get on with their job, which is exactly why they’re a member of ISA.

(30:09):

So, you know, that, that then leads into Bill, like what you’re talking about, you know, is this, is this idea that it’s gotta be beyond just productive, right? It’s gotta be trusted throughout the organization. And the challenge is that associations and, and, you know, we’ve worked in them our whole careers, right? I used to run the National Fitness Industry Association in Australia. And what happens, you try to build these structures which end up becoming departmentalized. You know, as you grow and succeed, you need more people, so you end up with these departments which can become siloed. Um, but the knowledge need of a member, of a person, a professional is not siloed. It’s, I s – my first question might have been about how do I register for the conference? But my third question could be about a policy initiative or an advocacy project or a mentoring program.

(30:10):

Association’s support, the way we were structured, didn’t allow for that. Yeah. And if I can, if I can add to that, we did not initially plan to have Mimo trained on our website, right? Our members showed us what they needed. They showed us by the questions they were asking that they didn’t just want the technical content they wanted. How do I renew my certification? When is registration due for this? And so Mimo has become more and more useful as our members teach us what they need. We’re not having to guess at what questions they have. Mimo’s telling us what questions they have. And we’ve able, been able to change that. When you talk about the organization, it changes the way we talk about what we’re doing. Um, I know associations have always said we’re the, we’re the trusted knowledge resource. But when you have, um, a sort of living, breathing tool like Mimo, every conversation turns to how do we continue to make Mimo better?

(30:12):

How do we continue to make Mimo smarter? Our content focus is more on building Mimo out to be what our members are telling us they need. We’re able to identify topic areas that we don’t have covered, that they need information on. Um, we’re able to tell questions they can’t find answers to on our website. Uh, so it, it, it’s been interesting. It doesn’t just change how we think about what we do every day. It also has allowed us to get to know our members better. Yeah. I think that’s, that’s so, that’s so, I think, powerful. And, and Rob, you touched on something too. Remember early on when I was talking about not only increased engagement, but increased duration of engagement. So going from 50, 60 seconds, right, bouncing around to six minutes to 15 minutes. To me, that, that immediately says that the, that person, that individual is finding value, uh, within this or – you know, within the, the organization.

(30:14):

And I think that just reinforces that. Andrew, have you seen, um, I mean, uh, what I’m, what I’m wondering, and this may be a little bit of a detailed question, but, you know, a lot of associations, they’d love to have engagement from 100% of their membership, right? We want everyone to be involved. And some people will say, you know, how are we measuring engagement? How are we tracking engagement? And I think that can be very, you know, subjective, um, in, in, in essence. But are you seeing. I mean, do you have the demographic that says, “Hey, you know, we’re seeing a much wider, you know, uh, swath of our membership coming in, uh, and, and, and engaging with the association that we didn’t see before, or couldn’t even track before. But now you can actually literally track engagement, you know, because I think that could be so, you know, tr – I don’t know.

(30:15):

It can be elusive. The, the, what does engagement mean?” Are you kinda seeing that, that it, you know, a, a much wider array of folks engaging with – I wish, I wish I could say we, we could see it. Um, to be honest, it’s, it’s one more data point, right? So I think if we really sat down and focused, we, we could probably see it. Um, but to your point about engagement being so elusive, um, Mimo isn’t all things to all members, right? Anymore than all members wanna participate in an online community or all of your members wanna go to your conference, um, or can go to your conference. Um, it, for me, it’s about mem – meeting our members in all the different ways that they want us to meet them. And whether that be through their local chapters, whether that be through Mimo. Um, do I think we have opened ourselves up to a, a completely different audience because of this, because of the language?

(30:17):

Absolutely. Absolutely. Yeah. I, I mean, that’s, that’s great because what, you know, D2L being a, a learning company, um, a- and we’re, we’re passionate about getting folks involved and engaged. And we’re not necessarily so focused on the technology. Yeah, our technology’s great. Award, all that stuff. What we’re really basing the success on is, are we seeing increased, you know, membership renewals, membership acquisition? Are we seeing dura – you know, increased duration? Are we seeing other opportunities that learning has? ‘Cause education is, I think, right now, probably the largest growth engine for associations right now. Everyone wants education to advance their career, get smarter, uh, so they can get paid more. But we spend a lot of time. We have teams of, of people. You know, we have, we have a group of just, of 300 people just on the learning and creative and strategy side that focuses on that, that, that going beyond the technology.

(30:18):

Are you, are you seeing, um, uh, you know, uh, utilizing, uh, uh, Mimo on that educational learning side as well? Sort of like, uh, uh, trying to identify any of the missing gaps that might be there in education or learning or upskilling or, you know, even professional development? Does it go down that path as well? I think it can. Um, again, I think we’ve, we’ve really just scratched the surface of what we are learning out of Mimo and what we can do with it. Um, it has certainly helped us identify topical areas where, where our members are seeking more information, uh, which just leads us to what, what training, what education we need to develop. What’s been amazing is our members that come to us and say, “You know, I, I was asking Mimo these questions and I wasn’t getting good answers.” And we can say, “Great.

(30:20):

Do you wanna write something about that?” And they say, “Yes, I do.” Um, and we just immediately are able to fill in those gaps. Um, you know, how, how can we expect our members to know all the content we have and what’s missing? A- a- absolutely. And, and, and on the, on the other side too, it, how can the, like, the association staff know everything the industry wants? That’s not, that’s not why the association was formed, right? I mean, it’s basically there to protect, promote, and advance that industry, right? Or e – or even, Bill, what the, what the association staff don’t know is what they’ve published. When you train, when you train Betty, when you train Mimo on 80 years of documentation, there is no human on the planet that knows everything that the organization has ever published. And so the gap analysis that Betty, that Mimo, that, or that, that is able to do is e- equally important.

(30:22):

People think that, “Oh, well, Betty didn’t answer that very well. She’s broken.” It’s like, no, actually, that’s awesome because she only doesn’t know it if you haven’t given her access to something that’s exists, or you just assumed that, that subject matter was published. About this topic because you work in it every day. But in fact, it’s never been published. That’s why she couldn’t find anything useful to respond with. And so she identifies these gaps. She identifies outdated information. When we connected to an association’s website and they have 100,000 pages published and they didn’t even know they had 100,000 pages published, and how much of it is old information that’s giving out. So this is, this gap analysis work that, that Betty just automatically does when she can’t answer something or doesn’t answer something accurately, it’s not because she’s making stuff up, it’s because there isn’t a content source that is up-to-date, current, or, or useful.

(30:39):

Um, and so, uh, this again reduces another governance risk that associations have, which we, we’ve started calling interpretation risk. If I ring ISA and I get Andrea on the phone, she’s gonna give me her interpretation of a technical standard or, or a document or a policy based on her expertise, her experience. If I call back an hour later and I get someone else on the team who’s been there for three weeks and thinks that they should just know the answer and they tell me the answer, I’m gonna accept that. But that’s based on their interpretation.

(31:10):

Mm-hmm.

Rob Barnes (31:11):

Betty doesn’t have interpretation risk because she’s trained on the same set of standards, the same documentation, the same content, and she is designed to give similar responses, accurate, effective, cited responses. So all of a sudden she’s reducing this interpretation risk, which again, then speaks to trust.

Bill Sheehan (31:28):

It’s wonderful. And I think, you know, from, Andrea, from your standpoint, I, it just reminds you of how, you know, this, how powerful Mimo, uh, can really be to just continually re-in – um, um, what’s the word I’m looking for? You know, just c – reconfirm the value of the organization. You have so much content that is trusted, right? Right. And then, and, and, and I think because if you go out to chat or some of these other bots, and you get this information back, can you really, really trust it? But when it’s coming from ISA, absolutely. You guys are setting the standards, right, for that. And I’m just wondering if you’re, if you’re starting to say, I don’t know if you’ve done any member surveys or anything, but I’m, I bet you that satisfaction is going through the roof with this.

Andrea Holovach (32:16):

Well, I, I mean, do I have a satisfaction score? No. Has our membership grown by 50% in the last five years? Yes. And I don’t know many associations that are putting up with this kinds of number.

Bill Sheehan (32:29):

What do you, what do you attribute that to? I mean, that’s a, that’s a year’s growth.

Andrea Holovach (32:33):

We’ve been focused on providing value and meeting our members where they are. And, and I, I wish I would say we would do surveys, and I know there’s so many people that write articles and books and –

Bill Sheehan (32:43):

Yeah.

Andrea Holovach (32:44):

I just look at real life. I look at my own life. I look at my college kid who doesn’t wanna use ChatGPT because they’ve been taught that that’s not trusted. And when I hear college kids go, “I won’t use it because I know it’s not factual. I know it’s hallucinating. I know it’s making things up.” It makes me feel so good as an association professional because I can fill that gap. I could tell you, we’re trusted.

Bill Sheehan (33:13):

Absolutely. And I think that’s where every association is u- uniquely positioned, right? To cut a g – and all the noise out there and all the other competitors out there for, you know, for trying to steal the attention of, from your members and the like. I, I think that to me is, is, is so. I mean, that, that to me is, is, is the proof that this is working. And I, getting back to surveys, I think surveys are gonna go away. You know why? Once you get the answers back, that’s changed. The whole marketplace has changed, right? You know, if you do these surveys and, uh, that you get it back even in a week, I mean, it’s, it’s gotta be. I think what you’re doing now is basically showing, you know, and, and providing member satisfaction in real time, right? Because I could be very happy at the beginning of the year, but if I’m not getting what I need at the end of the year, come renewal time, that’s what I remember.

(34:01):

I don’t remember those positive engagements I had earlier on. It was that last engagement I had with the association, be it a magazine, be it a newsletter, be it an, an, an experience with Mimo. That’s what I’ll remember before I already know. So it’d be interesting to see. I mean, that is an impressive statistic, Andrew, where you’re growing membership over five years by 50%. I got. The board’s gotta be ecstatic.

Rob Barnes (34:23):

So here’s the thing I, here’s, here, here’s what I would say about ISA, Bill, from a, from a perspective, you know, and I’ll say that’s all, all I do is talk to association executives and, and, and board members. And it’s indicative of not just ISA, um, but everyone that’s been successful with, in partnership with us. Um, it takes courage. So there’s two things that I would call out courage and curiosity. And if I then reflect on that experience with ISA and what’s happened with Mimo in retrospect, um, and then look at all of the 150-odd other customers that are working with us right now, I see these similar things, right? In the executives that work, the, the staff, and also the boards, is this level of curiosity and courage that it takes to try new things. Um, and that’s, that’s what’s proving to be an underpinning culture, if you like, that is driving the kind of successes that, that Andrea is sharing with us today.

Bill Sheehan (35:26):

Andrea, if I could ask, I know we just have a few minutes left, but if I can ask, was there any, um, what were some of the concerns? I mean, were, if you were getting them from the board, were there concerns from the board or the staff or everybody?

Andrea Holovach (35:37):

To be honest, we were, we were terrified. I, I mean, I love. I, I describe this as it felt like we were on the edge of a cliff. Um, it still continues with, as we, um, investigate new opportunities with, with Betty. You really do feel like you’re standing on a cliff. And it’s like, if I step forward, am I going to fall? Uh, I, I, I look and say, I often say, “I don’t wanna make a decision that somebody 20 years from now goes, uh, what were they thinking?” But I also don’t wanna fail to make a decision right now that someone 20 years from now says, “What were they thinking?” Yeah. And, and it took having trusted partners. We probably threw every question at them we could come up with. Won’t it do this? Won’t it do that? When, when you are a standards developing organization, that peer review, that, that accredited process of developing standards is so culturally critical to who we are.

(36:37):

We had a lot of questions. What’s happening with our IP? Are you really gonna be able to do this correctly? Um, and we took that leap. We were very curious. We were also very scared. Um, and it’s, it’s been worth it. And I think there are so many new opportunities with taking that trusted content from within your AI, from within your large language model, and licensing that, that associations haven’t even dreamed of the business opportunities that exist to share our content in ways that we can keep it safe.

Bill Sheehan (37:21):

A- Andrea, let me ask you too, for those association executives listening, um, what. Did you put some level of parameters if the answer is yes to this concern or fear, we move forward? If the answer is no to this, let’s fix it before. I mean, were there any of those type of, you know, if there were milestones put in place to say, “Hey, it’s answering these questions.” And I’m sure as soon as you got that one answered, there was another question that came up. Never though about asking that one. What about that one? Did you have that type of roadmap in place?

Andrea Holovach (37:52):

No. I, I honestly think if the answer had been no to anything we asked, we would’ve, we would’ve exited. I, I don’t want this lockdown just to members. I want it to be this way. I need it to answer less for these people, more for these people. I, I want it to put more precedence on this newer content, but it still needed to have the older content.

Bill Sheehan (38:14):

The difference between associations and any other type of organization, corporation, higher ed, K through 12, you, associations really only get a chance to screw up once. The corporations can screw up, fix it, and, and that, that your, your, their customers are still gonna like them. But when an association that represents an industry and provides standards and accreditation, you only get a chance to s – you know, to screw up once. And so I think, you know, what, what you’re saying, Andrea, is, yeah, if the answer is no, we gotta move on. Or if the answer is no, why is it no? Can we fix it? You know, those type of things, get those answers. But that’s the thing. I think a lot of people who are not involved in the association world or never worked in it, you don’t get a chance to screw up, right?

(38:51):

A- and that’s why, you know, it takes, it takes a long time for associations to make a decision. A lot of them will say, yes, I’ll sign today, but I gotta go through these, uh, this protocol and these procedures and the like, but.

Andrea Holovach (39:04):

I would challenge, if there’s one area where you can screw up, it is AI. It, it, it’s just our members, right? Our members work in automation. This is their lives.

Bill Sheehan (39:15):

Yep.

Andrea Holovach (39:15):

Um, it is coming on so fast and so furious right now that, that I do think we get a little bit more grace than, than maybe we do in any other way, right? You, you can’t really mess up, um, online commerce. You don’t get another chance at that. Um, but I, I think there’s an element of our members appreciated that we were willing to, to step into this. We were willing to try. Um, and therefore, we got a lot of grace as well.

Bill Sheehan (39:43):

Yeah. And I think, I, I know, I, that’s a, that’s an excellent point. Thanks for that distinction because, you know, a lot of times, particularly in the pharmaceutical side, finance side, these, you know, uh, that AI, you have to be able to trust what that information is coming back, right? And, and, and I think if it is wrong one or, you know, one or two times, that’s a problem, and, and you gotta do it. But I think with AI getting into it, yeah, you are gonna make mistakes. Things are, are not gonna be there. You, you’re gonna find out, you know, maybe we do need to do a little bit more, you know, vetting or, or confirming this information’s true. But I, I think, I think the fear, and, and correct me if I’m wrong, Andrea, but I think don’t let fear or concern, I don’t know what, if there’s a distinction there, stop you from engaging with AI, right?

(40:25):

Would you agree with that, Rob? I mean, you, you see it all the time.

Rob Barnes (40:29):

Yeah, we, I mean, I, and I speak to ev- everyone these days, and I, I always say AI is gonna happen to you or for you. And in the association market, you cannot allow, you cannot, um, sit back and let it happen to you. And what, what I mean by that is that your members, your constituents, your stakeholders are go – they’re using AI every day. If they’re not using your AI, what are they doing? And where are they getting their information from? And how is that impacting the way that they do their work? And how does that impact the standards within your industry or your profession? So associations have to provide an alternative, and it doesn’t have to be Betty. And if it’s not Betty, that’s fine. But it’s gotta be something that can be trusted, that can be coached to do the job that you need it to do in order to maintain the trust, the protection of the intellectual property, all of the core businesses of an association.

(41:15):

So you can either let it happen to you or for you. And right now, we want it to happen for associations because they’re the best placed organizations in the world to benefit from this technology.

Bill Sheehan (41:27):

A- Andrea, and, and a final w – you know, closing, what, what would you, what piece of advice would you give associations, regardless of what department they’re in, you know, when you’re considering implementing AI, you know, what, what piece of advice would you give them that, so they’re not so afraid you’re gonna fall off that cliff?

Andrea Holovach (41:45):

Um, find a good partner, um, and, and approach it as an organization. Uh, those were two really important and impactful things for us, right? Our AI journey was not focused inwardly. Um, we still battle that every day. What AI tools can we use for the staff to do our jobs better? We started our AI journey focused on our members and focused on our content and focused on our strategy for the organization. Um, but it would be find a partner that you trust, find a partner that you trust, and do this holistically. Any organization I talk to that it’s, “Oh, we’re just the membership department and we’re doing this, or we’re just the publications department, and we’re doing this.” Uh, I’m very blunt. I think you’ll fail. Um, this has gotta be a strategic play for the entire organization, and everyone’s needs have got to come in from the word go.

(42:41):

Um, and that’s, that’s been a secret to our success.

Bill Sheehan (42:44):

I think that, that’s great advice. And, and Rob, closing, closing words. Where, where do you see AI going? I mean, where, where do you see, um, you know, in the next. I, I hate to say five years. In the next 12, 18 months, what, what are you seeing?

Andrea Holovach (42:55):

In the next six months?

Bill Sheehan (42:56):

In the next four and a half hours.

Rob Barnes (42:59):

Yeah, yeah. Yeah, that, that’s, that’s the trick. I, you know, I think, you know, we, we, um. Where it’s going is crazy to think in the big, in the big, wide world. Um, where it’s going for associations, uh, we can be a little bit more sort of focused. I think that there is now the capabilities to have, um, AI agents so specifically trained, um, to support the human endeavor, to support the humans that are doing particular jobs, and having those, having that level of specificity, um, where an agent that’s trained to do standards development is connected to another agent that’s trained specifically to do exam development. And when one needs help with the other, they can just talk to each other, and they’re both, but they’re both based, they’re both grounded in the same body of knowledge. Um, and to support the human in the loop.

(43:53):

And so, regardless of where the technology goes for, for us as a company and for one focused on associations, the technology will always be built to support the human in the loop. Um, but make the human intervention part, the human component so focused and so impactful. And in fact, to be fair, so enjoyable that that’s what they joined the association to do in the first place. Yeah. Let the machines do the other stuff, um, which is incredible. And you can build these kind of experiences yourselves now using, like, Claude Cowork or Codex, um, to, to do these amazing things. And then the other big piece I think Andrea already talked about for us as well is, uh, we are going to change the way associations are able to license their intellectual property to third parties in the market, um, through Betty’s API. I

Bill Sheehan (44:44):

Was just talking earlier this morning, uh, to an association that, that business model is gonna change a little bit for associations. And you talk about non-dues revenue and the licensing of this content. Finally, associations can be, you know, compensated for the wonderful tech, uh, information they put out as opposed to giving it away for free or trying to make it ev – So I think that that, that is going to be the, the new, I think, the new, new revenue model for, for associations. But I, I will say in closing, I, I do think, I still believe, and I have believed this for a while, that now with, with the advent of these technologies, particularly AI, that associations are more uniquely positioned than ever to rise above the clutter and become that single source of truth for their industry. A trusted, accurate, and complete information. And I think ISA, you’re demonstrating that by growing membership over five.

(45:31):

It’s gotta be the governance person who is ever doing that for your organization. That person is responsible for that growth, uh, for that. But, um, I can’t thank you both enough for your time today. You’ve been listening to Learning by Association, a podcast where we delve into the ever-evolving world of associations and the challenges they face in navigating the currents of change. This episode was produced by D2L, a global learning innovation company helping organizations reshape the future of education and work. To learn more about our solutions, please visit www.d2l.com. And don’t forget to subscribe so you can stay up to date with new episodes. Thanks for joining us, and we’ll see you next time. Thank

Resources Discussed in the Episode

• Mimo: A Large-Language Model Educated on ISA Content
• International Society of Automation (ISA)
• Betty:ai