Vanity, Vanity, Thy Name is Adoption Metrics
In the modern SaaS economy, adoption metrics abound. Sure – they measure something that VC investors care about, and sometimes something that product recommenders and even decision-makers want to track. But does adoption speak to business impact?
One thing for sure: when it comes to business impact, adoption metrics are pure vanity. A business doesn’t measure return on investment by asking how much time its employees are spending as “users.” Horror stories abound of products that suck up time due to their own internal inefficiencies, sending employees on wild goose chases to figure out what to put in that so-called “required field,” or how to coax a shiny new SaaS product into spitting out a coherent report on what it did for you — or, more likely, what you did for it. At its worst, a focus on adoption invites corruption, as the SaaS vendor needs to make a claim that their goodness is spreading throughout your organization and the buying committee needs to justify, and feel good about, their purchase.
Join Chris and Corey as they talk with Mike Genstil, co-founder and CEO of VisualizeROI, and analyze the practices and dilemmas of determining adoption, the difference between theoretical value and harvestable value, what a QPR has to do with renewal, and the role of a VP of Value.
Announcer (00:25):
In the modern SAS economy, adoption metrics abound, sure they measure something that VC investors care about and sometimes something that product recommenders and even decision makers want to track. But does adoption speak to business impact? One thing’s for sure. When it comes to business impact, adoption metrics are pure vanity. A business doesn’t measure return on investment by asking how much time its employees are spending as users.
Horror stories abound of products that suck up time due to their own internal inefficiencies. Sending employees on wild goose chases to figure out what to put in that so-called required field, or how to coax a shiny new SAS product into spitting out a coherent report and what it did for you, or more likely, what you did for it. At its worst, a focus on adoption invites corruption. As a SAS vendor needs to make a claim that their goodness is spreading throughout your organization, and the buying committee needs to justify and feel good about their purchase.
Join Chris and Corey, as they talk with Mike Gentsil, co-founder and CEO of VisualizeROI, and analyze the practices and dilemmas of determining adoption. The difference between theoretical value and harvestable value, what a QPR has to do with renewal, and the role of a VP of Value. Join us for this episode of the Market Dominance Guys.
Corey Frank (02:07):
Well, here we are altogether for another episode of the Market Dominance Guys with Chris Beall, the Sage of Sales, and Corey Frank. Chris, as you know, we don’t have guests on too often, but we seem to be saying that more and more often that we don’t have guests on often. But we just keep running into so many interesting folks, smart folks, that we want to get on the air. Some of the information, some of our conversations that we’ve had are just too powerful to have just in our little own Zoom world.
Chris, we want to certainly welcome our newest, oldest friend here, Mike Gentsil, CEO of VisualizeROI. Is that the name of the company? Is that the tool? Is that a little bit of both, Mike?
Mike Gentsil (02:44):
VisualizeROI is the application that our company sells. Yes, that’s the name of how we market ourselves.
Corey Frank (02:49):
Well, fantastic. Well, welcome to the Market Dominance Guys. Chris, how about you tell our audience a little bit about how you came across Mike. Especially in our topic today, which I think is so captivating, which is customer success and customer adoption. And some of the flypaper, and stickiness that we all as sales leaders are trying to get at, and some of these vanity metrics, as we talked about that people chase. But yeah, let’s talk a little bit about how you tripped over Mike, and why he’s such a pertinent guest for us here at the Market Dominance Guys.
Chris Beall (03:21):
Sure. Absolutely, Corey and welcome, Mike. This is going to be fun. Mike and I ran into each other a few years ago, right? Three years ago, something like that?
Mike Gentsil (03:28):
Yeah.
Chris Beall (03:29):
And started working together with us as a client of VisualizeROI, to figure out how we can take our test drives and turn them into ROI-centric case studies. And we do hundreds and hundreds of test drives every year. I’ve been frustrated through the years with our lack of, shall we say, sophistication and compelling presentation of the value and the return on investment for our customers. Now, their investment in the test drive, of course, is just three or four hours of their people’s time. Nobody pays for our test drives, but still you want to be able to show business impact.
And our conversation has evolved over the years to be much more about this big question of the business impact of what you buy in business, and measuring it, and making it abundantly clear. Both from the vendor’s perspective, they’d love to be seen as having big impact, but primarily from the customers’ perspective. We were just having a discussion the other day about this. Mike and his team were taking what we call our attribution report, which is a lame name for what did you get out of all that ConnectAndSell you bought, right? How much pipeline value did you generate?
How much directly, how much indirectly, and how much kind of, maybe, sort of? We shipped off some data to them. His team put together something that was just awesome that allows for an interactive QBR. We started talking about what is the role of the QBR? What does it all mean with regard to renewal? Mike said something to me right as we were wrapping up, which was, he said it kind of hesitantly. He says, “I’m not a hundred percent fond of the adoption metrics that people use,” and I just jumped out of my skin and I said, “I hate them. I think they’re corrupting. I think they’re terrible. I think that they’re misleading.”
“I think they’re gameable. I think they’re for venture capitalists to care about something that they should be more careful about. It encourages people who build SAS companies to lie about their business.” I gave him an example. There is a company that will remain unnamed in our space that we’re in the same account with. The people at the account said, “These guys at this company, they measure adoption. If you send one email using their tool, that’s adoption.” We don’t see it that way. You guys talk to us about business impact. I said, “Yeah, but we fail to actually quantify it for you.” I said to Mike, “You hate it, I hate it. Let’s talk about it,” so here he is.
Corey Frank (05:56):
That is a setup. I think the first question then, Mike, with that is I remember working at a drugstore when I was in grade school and I had to stack the Sunday papers. Remember those coupon sections that came in the Sunday papers, and on the bottom of every coupon, they always said no cash value. Or they said cash value one 100th of a cent. And it seems to me, Chris, what you’re setting up Mike on is these vanity metrics where one man’s trash is another man’s treasure and certainly vice versa.
But a lot of these adoption metrics, if you really look closely from a venture perspective or a valuation perspective, I can’t pay my employees with coupons. I can’t pay them with adoption rates, and so who cares about that? So let’s talk a little bit about what are some of these other adoption metrics that you’ve seen, certainly in your years, that have that cash value of one 100th of a cent, or maybe nearly nothing to the rest of us business owners?
Mike Gentsil (06:56):
Yeah. I think the impetus for this discussion is when you think about as a vendor, getting a renewal from a customer. So often what you’ll hear from your contact is, “Well, we did an internal survey and we learned that the adoption was okay or it was great and other tools or other services have more adoption.” And you’re like okay, well, that’s interesting but the value of the adoption is what? Let’s say that you give your employees free crackers and they love the free crackers. And you do a survey, hey, what’s the value of the free crackers?
Well, everybody loves the free crackers. What’s it worth? Should I continue to spend money on the free crackers, or if I have to make a choice, should I spend money on something like an automatic dialer that actually gets connections with people where I grow my business? I think CFOs want to spend money on services and solutions that create value and ROI, that’s how they’re wired to think. But the metric that they’re given by the business owners of these solutions typically is just an adoption metric because there’s nothing else that they are trained or capable of providing to the CFO.
So I think that kicked off the discussion around how do we bridge the gap here? Because the seller wants to communicate the ROI. The buyer wants to understand it. But in the meantime, the only thing people are looking at is adoption. I think it spans all of the main services that people are spending money on from a procurement perspective in B2B.
Corey Frank (08:27):
So there’s a lot of noise out there of what really is determinant of true ROI. Very few of them, it seems actually have a dollar associated with it. Instead, it takes a little bit of extrapolation to get to the actual real value of what the impact is to that business.
Mike Gentsil (08:45):
Exactly. The good news is it’s relatively easy to measure adoption. Then the second piece of good news that Chris and I have been discussing is if you roll up sleeves a little bit, you can extract and extrapolate some kind of value calculation, whether you’re subscribing to an invoice processing service, or an automatic dialer, or a service that helps you reduce fuel costs. If you actually do the math and make some basic assumptions, you can get to a value estimate.
In that case, you’re going to make everyone happy. The CFO’s going to be happy, the business owner is going to be happy, and then the vendor is able to quantify the ROI and communicate that to you. The work can be done. We just need to roll up our sleeves as buyers and sellers and do it.
Corey Frank (09:31):
Chris, when Mike struck that nerve on adoption or so, what were some of the vapid metrics that you’ve seen over the years that prompted the visceral reaction that you gave?
Chris Beall (09:46):
Well, most of it’s been adoption. That’s the one. The fact is, it’s not considered that but it’s considered essential. With hundreds of millions or billions of dollars being invested based on these adoption numbers, which seem completely, well, not wholly uncorrelated. After all, if you get no adoption, nobody uses your product, you’re probably going to fail spectacularly, right? In much the same way in our business if you were to buy ConnectAndSell as a service, and then nobody pushes the button and talks to anybody.
It really doesn’t matter how great those conversations would have been and how much business they would have led to. This is the quantity, quality thing. At quantity zero, the quality is always the same. It’s zero by default. It’s not like it’s a totally dumb metric. It’s a highly gameable metric. Gameable metrics suffer from inflation on one side. Whoever is going to make the most money at the margin by gaming the metric is going to game it in an inflationary kind of way.
And then discounting on the other side. So then CFOs and other people who, with flinty eyes and green eyeshades will look at it and say, “Yeah, yeah, yeah, yeah, yeah.” And so you get a runaway process of discounting versus gaming. And eventually, you just end up with is goo, which has no intellectual integrity to it and really no predictive power. What’s kind of funny though, is on the other side, if you look at any estimate of ROI that’s rational. Say you’re measuring something, you’re measuring a real something and at the end you get dollars.
Dollars are invested, and dollars have come out, whether in the form of savings, or in the form of gross profit contribution. The two ways that dollars move around. Could also be in the form of risk, which is the trickiest of the dollars to measure, this big industry around doing something about that one. Every once in a while, they get shocked and surprised by something like, I don’t know, say a global pandemic, or a hurricane that’s a little fiercer than normal. For the most part, it’s cost savings or it’s this other side of people call it revenue, but it’s really gross profit contribution.
If you get a measurement going, the beauty is, say you love it now. You love what you’re getting from Mike’s company. I’m using VisualizeROI, everybody’s happy, and we feel like we’re getting something good. When we measure, we get a five. And the next time we measure it, we get a seven. It’s the same measurement. We probably can rely on that being a 40% improvement over the five.
By the way, if the five were five percent, and seven were seven percent, most salespeople would report that as a twp percent improvement, which shows that they could use a little math education, as could many people. But it’s a 40% improvement over a baseline that was established as being above threshold for investment. We can rely on real measurements of dollars in and dollars out to some degree in the absolute, but really, really strongly relative to a baseline that we’ve established. We cannot do this with measurements like adoption.
Chris Beall (13:13):
Where the marginal increase tends to be, shall we say frothy?
Mike Gentsil (13:27):
Corey, let me give you maybe a different example because probably everybody on this call recruits candidates. Part of your candidate recruitment you probably have an HR platform. You might have Google Sheets. You might have some, there’s a number of other platforms. Now, if you are the VP of HR, and you’re subscribing to one of these recruitment platforms, some people call it applicant tracking software, for example, and we have customers that sell this kind of software.
The VP of HR might say, “Hey, our applicant tracking software system was great last year. We had 2000 candidates submit their resumes into our software and we hired a hundred people.” Great. She could say, or he could say, “We love the software, but guess what? What if it so turns out that those hundred candidates are on average, lower quality than another set of a hundred candidates they could have hired?” And what if it took them five months to hire those candidates, versus what could have been three months with a superior platform?
They might think this is a classic vanity metric. We used it a lot. We submitted a whole bunch of candidates. We actually hired people. But what you really care about, as the VP of HR, and the CEO, and the executive team is hiring high quality candidates that can add value on day one, and hire them quickly at the right price. That’s a valuable system. You’re not overpaying the market, you’re not overpaying recruiters, et cetera.
If you were the vendor of that applicant tracking software, you would want to communicate, “Yeah, you used it a lot, but the value you got was substantial,” and you would quantify all those pieces, and we’d back into how you could do that. I can give you a couple more examples where you want we could…
Corey Frank (14:58):
Yeah, no, that value communication, right, I think is key. And even what you were alluding to earlier, Chris, on discounting, I think Mike is appropriate, right? Because again, I’m a big, dumb farm animal. I’m a sales rep, and I’ve been a sales rep for a long time. Sometimes I have a tendency to discount when I don’t need to discount. Mike, you’re my boss, you’re my VP of Sales. You think, “Gosh, Corey does a pretty good job selling,” but am I really, because I’m giving away the farm when I don’t need to. How would I be able to kind of track that?
I maybe think I ran into some pain and I have to discount, but I think that’s also a metric that maybe is a little elusive for companies, that would be wonderful to be able to determine if I run into resistance is proportionate to discounting, true?
Mike Gentsil (15:45):
Absolutely. Corey, I think you’re referring to a pre-sales process. I think it would also apply in a post-sales case. Discounting is a problem, both in pre-sales and in post-sales. In pre-sales, it’s a problem because that rep, who is high velocity, likes to close sales, likes to hit the number at the end of the quarter, is very quick to give discounts. And truly the VP would be happy to take the deal at 80% of the price. It helps him get over his number versus if we had simply quantified value and quantified pain, we probably could have gotten 95% to 100% of the price.
On the renewal side, we have the same challenge. What you run into there is in an average or above average case, the customer will say, ” Yeah, the adoption was pretty good. Bobby and Susie love your product, and we want to keep it, but we’ve got some bad news. The bad news is we’re in a recession. The CFO’s looking at everything very carefully and we have been asked for across the board, 30% cuts on everything.” I’m like, “Okay, well, that is bad news. I understand. I’m glad you’re happy with the product.”
Now, what would I ideally be able to do in that case? I’d say, “Well, I appreciate that CFO’s perspective. What would be great, because I’m not sure I can get that discount across the board. We can’t tell our investors that we’ve lowered our average revenue per customer. Why don’t we take to your CFO, the value that you’ve realized from the solution and perhaps you guys should buy more of it next year. Maybe you could redirect some span from some of those other solutions that are getting less ROI. Why don’t you cut two of those a hundred percent and redirect that over here, because there’s real ROI.”
That’s the discussion you want to have. If you’ve set that up over the course of the year, through your quarterly business reviews where you’ve associated adoption with value, you’re in a strong position to do that. In the best case, you’re actually paying that CFO somehow quarter over quarter. “Hey, by the way you used our solution 222 times and it generated $5 million in value.” And he’s like, “Who are these folks?” And then you’ll raise the awareness. Then you’re in a much better position at that last minute, and you wouldn’t have to do the renewal.
Corey Frank (17:44):
To your point, Mike, where does this live? Chris, where would you and Mike see, ideally, is this a new role, a VP of Customer Success, a VP of Value? Because it seems like this discussion traverses so many different roles in an organization. Is it worthy of its own little responsibility? Is it an enablement? Let’s talk a little bit about that.
Mike Gentsil (18:07):
I’ll give you a quick perspective. What we see across our customers is there is a growing number of customers that do have a title VP of Value. That VP of Value can report into the chief revenue officer. They can report into the head of solution engineering, in some cases. They can also report into the CFO, in some cases. The more important point I think is not where they report, but the person that quantifies that value inserts those value estimates at every step in the customer life cycle.
There is a version of value calculations for marketing people, who are trying to entice people to become a lead. There’s a version for your inside sales team that’s trying to get you to take a meeting and you’re quantifying value as part of that. There’s a version for the sales person, and then there’s a version for the customer success person. I think the VP of Customer Success, and the VP of Sales, and the VP of Marketing, they should be able to quantify value for their motions.
They should extract that quantification from that function. That function again, could live in its own silo or they could report to any of those folks theoretically. But each customer facing functions should be able to extract those calculations and use them to their purposes properly.
Chris Beall (19:18):
Wow, VP of Value. I got to go and apply for that job. That sounds like fun.
Mike Gentsil (19:23):
Not an easy job. You better love Excel.
Chris Beall (19:26):
Well, I do have a certain fondness for the occasional spreadsheet, as Cory knows. So yeah, it’s interesting. In our company, we do something a little funny, which is our VP of Customer Success is actually evaluated based on the customers getting the maximum amount of business impact, which we tend to measure in terms of meetings that are set. Now, people will argue and say, ” Oh, the meeting set might not have happened, blah, blah, blah,” but I guarantee you over time, it’s linearly related to value.
And that’s the main thing is that you need a linear relationship between whatever you’re measuring and whatever you’re getting out, so that should it trend, the trending actually will represent a linear increase or decrease in the value that’s being achieved. James Thompson, our VP of Customer Success, is held directly accountable for the value that our customers get per dollar that they spend with us. His job is to minimize the dollars that they spend for the same unit of value.
You could look at it two ways, but the easiest thing to hit is always costs. He’s always looking for opportunities with his team to keep folks from using too much ConnectAndSell. You might’ve experienced this, Corey when you were our customer and I was acting in that role of VP of Customer Success. When you were a customer at StormWind and I was telling you to use less.
Corey Frank (20:44):
Here’s a guy that didn’t want me to spend. “Spend less, Corey. Spend less with me.”
Chris Beall (20:48):
Well, it’s not altruistic. I think it’s like, look, we put a keel on a sailboat, not in order to make it go faster, but so that it can go the direction that you want and it doesn’t ever tip over and leave you upside down turtle in the water, right? It actually slows the boat down, ticks down in the water, but try sailing in rough seas without a keel on your boat, and you get a little bit nervous. I believe that this attachment to maximizing the customer value per unit spend, that is their ROI with us and focusing on it because we have the inside track.
The thing that vendors, I think, need to realize is look asking your customer to do this is asking them to do something really hard. If they provide you with a little bit of data, you can provide them with a lot of insight, as long as you both agree what you’re trying to do, which is to get them to spend the least with you to get the maximum business impact. That’s what I hate about this whole business of adoption because it leads to exactly the opposite.
I want the marginal adopted user, think of it this way. If I have a SAS solution and for 10 core users, it provides for every $100 spent, it provides a $1,000 of value per year. Then for the next 50 users for every $100 spent, it provides $200 of value. Then for the next 1,000 users for every $100 spent, it provides no value whatsoever. I run out of the users that are really the high-impact users fairly early, but I’m under pressure to extend the usage beyond that group.
That’s what I’m referring to as the corrupting influence of the adoption metric on customer success. It runs counter to the mission to help the customer be successful for the least amount of money that they need to spend. Then you can find their budget. And by the way, they’re likely to reward you next year in a funny way. They’ll discount less. The discounting of price is irrelevant compared to the discounting of value, as skepticism will cost you more than transactional discounting over time, every time.
If you can dispel skepticism by being upfront about the value that’s being created and transparent about it, transparency is a big movement in business. Lay it out and say, “Here’s what we’re doing.” And by the way, there’s a big difference between theoretical value and harvestable value like at ConnectAndSell. I can tell you, I can save you money, right? After all, you can have a smaller team. Well, what if you don’t shrink the team, then you didn’t harvest that value. Are you likely to shrink the team within the timeframe that we’re talking about for harvesting the value?
If not, it’s illegitimate of me to talk to you about the cost savings. I need to talk to you about the opportunity, about getting more on the top line. A little bit more painful, but I got to go there. Maybe in the next budget cycle, you won’t grow the team as much.
Mike Gentsil (23:35):
Chris, that was a fantastic example of that distribution or histogram, if you will, of value by user. A good example, perhaps for folks listening, is you think about a service like LinkedIn, where if you’ve got a team of a hundred sales reps, sure. They’re all going to want access to LinkedIn. Now of those hundred sales reps, 10 of them, for example, are going to use that so effectively, they’re going to find the best contacts. They’re going to generate half a million dollars in pipeline per month because they’re able to use LinkedIn so well.
The bottom 10%, they’re going to use it, and they may or may not pay for their subscription at all, because they’re not as connected already. First degree connections, secondary connections, they’re not as skilled. So as a VP of Sales Operations, subscribing to LinkedIn across those hundred users, if push comes to shove, you have to become a little bit tough and very analytical around, well, maybe these guys got a premium subscription, maybe these guys get nothing at all, or that’s where it gets very tricky.
Then the customers, to Chris’s point, the customer success rep for LinkedIn managing that account, should be ensuring that the high value users really know all the great features, so they’re getting more, and more, and more value. Maybe even be able to charge them more for more features. Whereas, they get the people at the end of the histogram, at least to a point where they can prove to themselves and the CFO they’re getting value out of it. That’d be very sophisticated customer success. I have to believe that’s where we’re going as an industry.
Corey Frank (24:55):
Well, I think so too. And that’s a great way to kind of end this part one session with Mike from VisualizeROI. And I think maybe in our part two, Mike, we can expand on this that you and Chris were talking about this. As sales guys, like me, the goal is to create a compelling narrative that sparks creativity and inspires that prospect to make a buying decision, a purchase. But often, I’m going to focus on what my product can do for the prospect’s business, but I don’t spend enough time demonstrating how it will actually make an impact.
I think in our part two, we’d really like to hear you and Chris expand on this evolution from moving from traditional, how do I justifiably earn an ROI and how I process that to more value-based messaging, which I think what you and Chris are talking about. And how that value based messaging can take ROI and go beyond just merely budget conversations. I think that’s a good place to stop session one here, and to thank Mike for his time on this.
