Video G2 Crowd Wants to Crowdsource Enterprise Software Reviews (Video) 26
Matt: So Robin, Matt Gorniak here. So G2 Crowd actually, we want to be the trusted source for business application reviews. And basically some folks have called us the Yelp for business software. And this is also kind of disrupting the traditional analysts’ model in business applications.
Robin: What is that? What do you mean the traditional analysts’ model?
Matt: So currently how most Fortune 1000 and Fortune 5000 companies learn about business applications is that they will buy reports such as Gartner or Forrester, the traditional IT app analysts. And we kind of believe that model is broken.
Robin: Okay, what is broken about it?
Matt: So for starters, I mean how we came about by the way with this service is we compared how we buy business applications with our consumer lives, and so I think fundamentally what’s challenging is when – in the 1980s where the first business apps came out, it made sense to have analysts to model where people came in and gave you their opinions.
But the challenge is basically, Robin, that it’s just not based on personal experience. It is based on opinion. And when you compare it to consumer lives now with Yelp, go into a restaurant or having a personal experience, when I was on a trip last year and I was able to book hotels with TripAdvisor. So based on other peer reviews, and frankly actually it was very interesting, because I went to an island, and next there was a bunch of brand hotels, and tragically some of them were not updated. And there was no way I would have known that through traditional methods, right, of calling a travel agent who probably hasn’t been there, or calling the hotel.
So went in, looked at peer reviews and was able to pick the perfect hotel. So that is kind of the analogy to business applications. But here’s the thing, the repercussions are much, much bigger, right, in terms of picking the applications, ERP system, CRM system. If you pick the right one, it is wonderful; if you make a mistake, it is a lot of money, lot of time, and frankly something that’s hard to make a change, right.
Robin: Now let me just be the skeptic. Now I have done a lot of software reviews in my life, including a lot of enterprise software, and I did my best to be not only objective, and to test the stuff, but to bang it, and to use it, and misuse it and figure out how it could fail. And one could be made to fail by one of my wife’s secretary friends or somebody, and another one was pretty bullet proof, I would tend to recommend the bullet proof one. Now that is an analyst review, old style.
Matt: Right.
Robin: Now how are we going to get peer reviews that are going to measure up to that or beat that?
Matt: Right. Now that is a really good point. And actually what we are finding in our community is that, it is all about the numbers, right, crowdsourcing is basically the idea that we gather – actually for one CRM vendor, actually for a CRM category, we just launched on February 14, our public beta by the way. We have over 700 ratings and reviews. So the answer, I guess, Robin is in the numbers. We are finding that there are some very “hardcore” users already in there, administrators, who will voice their opinion and essentially taking that application for a very deep spin. And then you have also casual users. But in those numbers will have that answer for you.
Robin: Now, I notice that, and this is because you are new, for instance, I see a lot of stuff on, oh I don’t know, let’s take an example, MySQL.
Matt: Okay.
Robin: MySQL, you know Oracle is coming apart. I don’t see a single mention in there of Maria SQL.
Matt: So a couple of things are at play here; I mean first of all, we have seen a tendency by users just to want to share, right, so folks need, they want a platform that is trusted, where they can share this type of review information with their peers. And so what we’ve done is basically, we have built this platform utilizing LinkedIn, so every user that is in there uses LinkedIn as a log in so basically we can eliminate competitors, we can eliminate employees to do reviews. The point is it is a trusted site. So that gets folks in there.
The other aspect is to contribute. The other aspect is we also have gamification built in. Gamification means that apart from some folks just want to share, others want to share how much they know, and third, they want to also to get thank-yous from the community. So one is a point system where you can win iPad minis, and another system we have is a reward system where you get paid, in this case, for a CRM system, not the SQL example, but currently $20 for 50 minutes’ worth of work, if you do a very detailed review. So we are trying to kind of get the community on multiple dynamics in there, but the most important thing is, it is a trusted site, everyone knows these are real people, real users, administrators, and so we bring this all together, we will get quality reviews of whatever product you have.
Robin: So you think that the – you are new, so the lack of even mention of MariaDB as an example, is just because it is new, that you feel they will be there eventually?
Matt: Right. Partly, it is also, we are getting users’ submissions of products, the site has that built in that are not included. So yeah, the starting point would be either we or users passionate about it will say well that’s not included, let’s put that into the site, and then the other mechanisms I was mentioning kick in to drive others to do it. But that is the starting point, right, if it is not in there, either we put in because it is part of our core categories, or users will submit themselves, or vendors frankly as well.
We are getting vendors submitting it because one of the other features we have in the site, is a vendor comparison. So our goal is to help buyers select products and make better decisions, and one of the features is a product comparison that is completely crowd based, that is very much unbiased and so if you don’t have your product in there, it can’t participate. So that is a three- legged stool.
Robin: Okay. You mentioned Yelp. Now I live near Tampa, Florida.
Matt: Okay.
Robin: And there has been a lot of controversy locally about Yelp. It started with beauty parlors where the owner of one beauty parlor may have had friends and relatives saying this other beauty parlor nearby was no darn good. And they got into like a hate war on Yelp. Now this can happen again. How do you keep, you say unbiased reviews from the crowd, how do we know? How do we know?
Matt: Really good point. And that was actually a key – I mean this is one of our key considerations as well to make sure it is quality data, it is unbiased, it is not – that’s one example. Another one maybe employees rating down “products” which make sense if they are slightly biased. So with that in mind, we do require contributors to log in through LinkedIn. And that is fairly important because: a) This way the site knows who you are, even by the way, you may not attribute your profile to your review, the site knows that, there’s actually mechanisms behind the scenes that filter for employees and competitors, so we actively do that. Our terms and conditions of our website actually ask our users not to do it, but besides asking we have mechanisms and also review teams going through that. And so far, it has been pretty good at making sure the site doesn’t have a problem with that. So there are mechanisms in there to maintain that.
Robin: So you don’t actually have your own login. You use LinkedIn logins?
Matt: Correct. Well, it is technical.
Robin: So if somebody wanted to go bogus with you, they would have to create a LinkedIn identity.
Matt: Right. So just to be specific, we use LinkedIn to help log in to the site, it is their API that we use for the login. But exactly. So, you’d have to create a separate profile, and here’s the thing, Robin, in our site, what’s really important, we believe, is for buyers to understand who’s behind that, so when you see a review from Robin on MariaSQL and Robin’s profile has nothing, then there’s another component which you are probably familiar with from other sites, which is the community votes, and so you can also then vote on contributions, which gives you points, which allows you to win, but the point is someone that has a poor review with a profile that is nonexistent, we’ve already seen that doesn’t do very well at all. And frankly, we don’t see much of that at all, because it is a high bar to get in, and then the community kind of takes care of it. So we think the bar is fairly high to – probably also not worth it to game the system that way.
Robin: I heard of a site called Slashdot that does that.
Matt: Okay. No comments.
Robin: I am not sure where this is going, I mean you are on Slashdot right now. So obvious. Now comes the other question. How do you make your money? Where is your money come from?
Matt: Yeah, so the monetization approach essentially for us is to provide quality data to the buyers. And so in essence, we don’t take money from vendors or parties that would influence the standings, so in essence it is offering up; so in this case, our first offer is a report, we’ve chosen to start with a CRM space. So you can go on now and buy a CRM report, a buyer’s guide, if you will, that gives you a PowerPoint and XL spreadsheet that has all the data from all the reviews summarized, that can have you save time, save money, and avoid risk. So instead of going through the site, here’s a tool set, this is still public beta, we are just announcing this to the world. So everything has got to go according to plan for us, I mean, you are part of the process, obviously getting the message across.
Robin: Yes. No I am not knocking. I am just asking because the big thing is it does come right back to the trust issue. We at Slashdot get accused all the time, I get accused that this is an advertisement, that you bribed for, which is obviously not true, (alright thank you for the envelope, kidding! No!) if something catches one of the editors’ eyes, we just say, hey let’s check that out, maybe some of our readers would be interested in it, and a lot of our readers are corporate IT guys and CIOs and whatnot, and they are looking for good reasons to choose this one over that one, so CRM is your starting place. What other areas, where else do you think you are going to be big?
Matt: So, it is a very good point. I mean, starting with CRM, marketing automation, then you kind of have to go on the other side, social marketing, big data, I mean we are going to hit all the big categories that are essentially not only big decisions but also they are very much being disrupted right now. And that is what we feel the data, quality data is the least right now to make decisions on. And our goal is to cover all the categories that we have on our site over time as well, with reports and data.
And back to your question Robin, as well, I do believe this is a no-brainer, that’s how we go to market with that right now, because back to the CRM example, we have over 700 ratings and reviews from real users. You go in, you look at some of their backgrounds, there are 64 questions, identity is public, you can see editorial, you can see feature reviews, so package it all up and “selling the quality data.” We believe that is what the market wants right now. And so we are very hopeful with that.
So I LinkedIn myself to a review... really? (Score:3)
Re: (Score:3)
Well, I wish him well (Score:2)
But I hope he's researched the idea rather more thoroughly than the summary suggests. /. 'experts' when you're already on the launchpad.
It's a bit late to consult the
Probably more a slashvert than anything else, then.
Hard to see the added value over existing sources of information...especially since giving an non-AC, unvarnished review via LinkedIn of how you screwed-up your latest sf.com, SAP, whatever implementation is hardly going to endear you to your bosess, peers, support team, vendor/support provider
Re: (Score:2)
Corporate PR nightmare (Score:4, Insightful)
It won't work mainly because the people they're asking to review the software are in positions where good relations with the vendor are important. Their employer won't allow them to bad-mouth their vendors because that can impact contract terms the next time the employer has to negotiate licenses. And the requirement for a LinkedIn profile means there's no way to post a review without the vendor knowing exactly which customer has admins who think their offering is crud.
Remember, when customers are saying that there's major problems with a product, to the vendor the problem isn't that there's major problems with the product, it's that customers are saying their are. So the vendor will proceed to fix the problem: they'll go after the customers who're saying bad things to make them shut up.
Re: (Score:2)
There's some truth to this.
Really good administrators may not have exposure to a sufficiently diverse number of products to make observations that are usable. Doesn't mean they're bad administrators, rather, that the depth in administrative software often also disciplines a different workflow than they might be used to, or care for.
If this is Yelp for enterprise software, fine. It'll be just as good and bad as Yelp is, in describing various products and services. For that, it might have some use. But Yelp a
Re: (Score:2)
With enterprise software you could easily have a situation where the IT group is happy and the end users aren't happy. The IT group may not have much pull over end users. Also people move from job to job. I can easily picture this working.
Besides the Streisand effect applies. If an employee gets disciplined for bad mouthing a vendor and goes public that could actually be a news item with millions of hits.
Astroturing (Score:5, Insightful)
How on earth is this any less susceptible to astroturfing? Linked-in like every other form of social media has large numbers of fake accounts that are available as shill accounts. Has anyone thought this through at any appreciable level? Most enterprise management software has requirements that you either go through a sales team to get an evaluation or often times has license restrictions that prevent you from freely discussing and or bench marking it without prior approval. The only way to get approval is to have results that are for all intents and purposes written by the company themselves.
Even if you don't have to worry about license issues for reviews you still have to worry about the logistical issues of implementing enterprise management software tools. These tools often involve dedicated architects, consultants that get paid hundreds of dollars per hours for months on end, dedicated SQL servers and months of exclusive staff time in order to implement correctly. Implementing a tool like this correctly easily runs into the tens of thousands of dollars on the small side and multi-million dollar budgets are common. Dedicated servers are required that can easily cost tens of thousands dollars just in hardware, never mind license costs for things like SQL server licensing and maintenance. The idea that someone who doesn't have a clue could possibly provide a meaningful review of an enterprise management product is as absurd as expecting a freshly minted college grad to work as an enterprise architect.
The idea that you can install an enterprise management tool and run a review like you can the latest version of a game shows me that the people proposing this have absolutely no experience actually doing this. This is why white papers are written, because people sink these costs into real world projects and write up about their experiences and lessons learned. You want a review of a product, find the white paper, or surf the forums for vendors product. I'm sorry, but this has got to be the dumbest idea I have ever seen submitted to Slashdot.
Re: (Score:2)
How on earth is this any less susceptible to astroturfing? Linked-in like every other form of social media has large numbers of fake accounts that are available as shill accounts
I think it is pretty easy to tell shill accounts from real accounts. It is also easy for a computer. There are cross correlations that human accounts will have and shill accounts won't. My biological sister is on my linkedin account. My linked in account has diverse connections and comments.... This sort of depth doesn't exis
Re: (Score:2)
Shill accounts are not only created by computers though [infoworld.com]. High quality shill accounts are created and managed by human beings who do so as their professional job. Some accounts are quite obvious, however other accounts are created and professionally maintained for years.
You can buy cheap shills and you can pay for high quality shills, it all depends on what your needs are. Professional companies or industry lobbying groups will maintain and develop shill accounts for years or even decades to increase their v
Re: (Score:2)
The system won't work against that kind of concerted effort. But I'd assume that kind of concerted effort is rare.
Usenet (Score:1)
I remember a place which offered trusted, crowd-sourced reviews of enterprise products by "people who use or administer enterprise software as part of their jobs" ... it was called Usenet.
The relentless advertisers invaded Usenet, they have invaded Slashdot (this topic offered as Exhibit A), they invaded TripAdvisor and they will invade this new website (if anybody is insane enough to invest in starting it up).
Now excuse me while I go back to my rocking chair and yell at the kids on my lawn.
I know what I'd like to pan... (Score:2)
Deltek time and expense tracking software...
But it takes time and effort to write reviews, with more time required to write an unfavorable review (same holds true for peer review of articles, which is one reason why I declined the opportunity to review a journal article today...) A really good system would have some means to evaluate and weight well written reviews. (That's a characteristic that's probably missing from Slashdot's scoring criteria.)
Larger employers forbid this (Score:2)
Other than that, great idea!
Re: (Score:2)
And people leave all the time.
I don't see the value in this... (Score:2)
Enterprise Software is a different animal, so to speak. The software itself does what it's supposed to do but your success with it boils down to a few key areas:
1) Choosing the right ERP solution that fits your needs. I can't tell you how many square-peg-round-hole solutions there are out there.
2) Choosing a good implementer. ERP products are nearly infinitely configurable and every configuration requires analysis and decision making. Some of the decisions you make can be difficult or nearly impossible to u
LinkedIn login (Score:2)
This was getting interesting until I red: "Logins require a LinkedIn identity"
No, thanks.