I'm Tommy, I built ReviewMeta - a site that detects "fake" reviews on Amazon. AMA!

Hello Tommy: how does ReviewMeta differentiate itself from FakeSpot? I use that site regularly for Amazon purchases and it serves me well.

I subscribe to Planet Money, but I haven’t heard your podcast yet. If I had, I might have already checked out your site. Thanks and good luck.

There's a few key differences - the main one is with transparency. Fakespot shares practically nothing about their analysis. I discovered them while I was still in the process of building ReviewMeta and decided that I wanted ReviewMeta to show everything I possibly could about our analysis.

Both sites are estimates - and that's why I think it's important for visitors to sorta check over our work and make sure that we're on the right track.

We've got a bunch of extra tools and tricks on our site that the common visitor might be unaware of: https://reviewmeta.com/blog/reviewmetas-hidden-features-for-power-users/

Here's an article I wrote a few years back going into more detail about the differences between the two sites: https://reviewmeta.com/blog/fakespot-vs-reviewmeta-for-analyzing-reviews/

What would be a better dirty play for an ecommerce seller: increase its own rating with 5 star fake reviews or sabotage its competitors' rating with 1 star fake reviews?

Ah, glad you asked. I always tell people to imagine you are an Amazon seller (with a lack of morals) and you have a budget to "buy" 100 fake reviews on Amazon. What is a better investment? To boost your own product with 100 5-star reviews? or to give each of your (dozen?) competitors a few 1-star reviews? Keep in mind that every one of those competitors is going to have a massive incentive to challenge those 1-star reviews and likely complain to Amazon until they get removed. They won't have the same incentive with your 100 fake 5-star reviews.

That's not to say that fake 1-star reviews don't exist. There's a lot of niche categories on Amazon that only have a few sellers, or when two top dogs in the field are duking it out. It definitely happens, but I think it's much less common than you'd imagine.

The more common cause for bogus 1-star reviews (in my opinion) are review brigades. You see it happen a lot to political books - a bunch of people who disagree with the person flood Amazon to bash the book without even reading it. Here's a bit more on brigades: https://reviewmeta.com/blog/review-brigades/

Yeah, I saw that happen to a book on health where two profit-motivated doctors 1-starred a book by a health- and truth-motivated doctor. Pissed me off. I haven’t been back to see of other “wellness” shills have left even more 1-star reviews.

Tell me about it. The health and wellness industry can be absolutely toxic!

I don’t disagree with the fact that it’s much more common for sellers to fake their own reviews rather than sabotage others.

However, I have had fake negative reviews written on my products before and it is nearly impossible to get Amazon to remove them.

I know you won't like to hear this, but in my experience, a lot of sellers on Amazon claim that all their 1-star reviews are fake...

Have you ever noticed that some products listed on Amazon have a high review volume and rating, only to scroll down and see that the reviews given are for a completely different and unrelated product?

I'm wondering how these 3rd party sellers are able to manipulate their product listing with popular products to make theirs seem popular and highly purchased. I've witnessed about 4 different cases of this on various items.

Yup. This is called Review Hijacking and we have an warning we display when detected: https://reviewmeta.com/blog/amazon-review-hijacking/

Is there usually a hard cut-over between the old product and the new one? Can you simply apply a date filter to eliminate the other-product reviews, or is it more complicated than that?

More complicated than that. You can group a bunch of different products together, all with different dates.

What algorithms do you use to accomplish this? Do you detect suspicious patterns in the text of a review, or does the algorithm detect suspicious behaviors of users? Both?

Yes and yes. We have 15 tests that look for "unnatural" patterns in the review data. We look at things like the dates that the reviews were posted, whether they are from verified or unverified purchasers, if all the reviews are repeating the same language, if the reviewers have similar reviewing habits, etc.

We then look to see if the ratings are vastly different between the reviews we think are "unnatural" and those that are not. So, for example, if a product gets 50% of its reviews in one day, and those reviews give it 5 stars on average, but all other reviews give it 4.3 stars on average, we know something's up. This helps us throttle back our suspicion-meter in case there's a natural spike in reviews (eg. could be a holiday item).

We show our work in as much detail as possible on every report page. It's a lot of information at first, but if you're a data nerd like myself, it's fun to go through it. Here's an example report to look at: https://reviewmeta.com/amazon/B07S9YBZMK

What unlikely products have fake reviews?

Hmm, a few thoughts come to mind.

First, I would say that pretty much every "natural" mosquito repeller product I've seen is propped up entirely by fake reviews. Mainly because they simply don't work.

Second, I've noticed that a lot of self-published e-books are also completely full of fake reviews. That's a category that makes sense but I guess I didn't expect it at first.

Are you sure they're fake though, you haven't gotten to my question yet but I pointed out that it flags several reviews as fake for not being a verified purchase when the kindle library won't add the tag to an account that 'rented' the book.

E: I should add that my own reviews were flagged for that reason which is why I brought it up.

flags several reviews as fake for not being a verified purchase

There's a lot more going on inside the algo; this is a misleading oversimplification. We never flag a review simply because it is unverified. As I have answered in other questions, the verified purchaser badge is only considered in 1 of the 15 tests.

Furthermore, we're looking at the total number of unverified purchase reviews for each product - are all the reviews coming from unverified purchases? Even if a lot are, we look to see if the unverified purchasers are rating the product more positively than the verified purchasers. If there's no statistically significant discrepancy in rating then it's not as much of a flag.

Also, as I've mentioned in many other answers, we're never labeling reviews as "fake". We use the term "unnatural" and our standards are likely different from yours. If an author gives away a bunch of copies to his loyal readers, and they all give it a positive review, that's going to look unnatural in our eyes. If you have the same group of people following the same group of all authors and all giving them positive reviews left and right, that also might leave some unnatural patterns in our eyes.

I just tested a handful of products on ReviewMeta. On one of the products I tested, it said that it removed 28% of the reviews (because it thought they might be fake/un-natural reviews) but the rating stayed the same. The Amazon rating was 4.7 and the adjusted rating was still 4.7. So it appears that the adjusted ReviewMeta rating isn't getting dinged because of the fake reviews. What's going on with products like this? Are there instances where ReviewMeta's adjusted rating is actually BETTER than the Amazon rating? Thanks.

Great question!

Yes, this happens A LOT. I was surprised by it at first, but there's two reasons why this happens:

First: the Average Rating that is given by Amazon is already a weighted average. So it could be the case that the average rating is already adjusted by Amazon to begin with. Here's their official language on it:

Amazon calculates a product’s star rating using a machine learned model instead of of raw data average. The machine learned model takes into account factors including : the age of a review, helpfulness votes by customer and whether the reviews are from verified purchases.

Pro tip: if you click the "Show rating distribution" on the RM report, you can see the change in raw star-rating distribution.

Second: Just because a product has fake reviews does not mean that the product is garbage. The Amazon marketplace can be extremely competitive and difficult to break into, so sometimes sellers resort to "seeding" their new products with fake reviews until the honest ones can come in and take over. Sellers know that it's not a long-term strategy to prop a garbage product up with fake reviews because eventually the honest reviews will take over. They know that the only feasible long term strategy is to create a quality product that will continue to get positive reviews on it's own.

That said, there are still sellers who are NOT in it for the long-term and just looking for that quick scam. So sometimes you will see the adjusted rating dropping significantly. Check out https://reviewmeta.com/best-worst to see some examples of when the adjusted rating drops.

So if I'm understanding correctly, ReviewMeta is just removing (ignoring) the reviews it determines to be fake... and then bases its adjusted rating on all of the remaining reviews deemed to be real/natural. Is that correct?

So in theory, a product's adjusted rating could actually be HIGHER than its Amazon rating (in a case where competitors may have sabotaged the product by posting a bunch of fake 1-star reviews). Is that right?

Thanks, Tommy!

Yes, absolutely. It's not often that the score goes up, but it does happen sometimes.

This may be outside of your domain but what could Amazon do to give high quality, new products the attention they deserve without requiring them to resort to Astroturfing? I'm a reviewer snob and spend inordinate amount of time analyzing these reviews. Anecdotally the difficulty of finding really great new products on Amazon is becoming near impossible, especially for consumer electronics.

This is a great question. I think Amazon could offer a lot more for new brands. They already have the "Vine" program (basically their in-house free-product-for-review system), however I hear it's crazy expensive and not everyone can participate.

I wrote this a year ago and I think that some of the principles could be applied to newer products. If Amazon had an official tester program, new brands should be able to pay a fee to get their products tested: https://reviewmeta.com/blog/what-amazon-reviews-should-look-like/

Any advice with dealing with a company that's astroturfing reviews on Google? A company I unfortunately have to deal with in my personal life has sent out mass emails where if you write 5 star reviews on google they're going to have a raffle to win prizes which amount to hundreds of dollars in cash. I've reported them to google via telephone but I'm not sure what else to do.

They're currently sitting at a 3.6 star cumulative with most reviews being 5 stars from the past two months. They're a terrible company and I hate standing back and watching this.

Man, that sounds frustrating. My friend Jason Brown over at https://reviewfraud.org/ specializes in Google reviews, so maybe he can help you.

Is there anyway to get the results of your analysis while I browse amazon, or do I have to read the full report on the website each time?

We have browser extensions for Chrome, Edge and Firefox: https://reviewmeta.com/blog/extensions/

These will show you the adjusted rating and PASS/FAIL/WARN color in the extension icon itself. I'm planning on overhauling the extension in the coming months to show you some of the RM details directly on the Amazon page itself!

You can use ReviewMeta by copying and pasting any Amazon product URL into the search bar at ReviewMeta.com.

and

I'm planning on overhauling the extension in the coming months

Is that why you chose not to plug the extension in the post details?

Anyway, if you do overhaul, be careful when designing how RM inserts itself. Look at FakeSpot for clues on how NOT to do it (way too intrusive!) and ask for UX suggestions (even here on reddit) before you start.

Of course! I'm going to take my time with the update. We've got a 4.6/5 rating on Chrome that I don't want to screw up.

How many of those ratings are fake? 🤔

Was just thinking that. Who’s watching the watchers lol

BRB, making ReviewMetaMeta.com real quick.

Don’t forget the ios app so we can share links to the app

Ah of course, the apps! https://reviewmeta.com/blog/reviewmeta-phone-apps/

Often when something becomes too popular, it is UTTERLY RUINED (haha) by its own popularity. See reddit.com ;) Or supplementreviews.com

The ReviewMeta site itself is of course shielded because it has no user-generated content, but what do you think will happen to the way unscrupulous sellers try to game the system?

Ha! I feel like people literally say that about EVERYTHING. "This [city/country/game/company/brand/website] has gone downhill. It used to be cool 10 years ago!"

When I was building ReviewMeta, I was thinking about what would happen if sellers would try to "game" ReviewMeta (since we are so transparent about our algorithm). The answer is that yes, it would be possible to "game", but it would take a lot more coordination and planning of the fake reviews to make sure they fly under the radar of all the tests.

Also, consider that probably less than 1% of Amazon shoppers use ReviewMeta (which is still A LOT of traffic). Is it still worth it for sellers to "game" ReviewMeta?

PS - For those of you that don't know about SupplementReviews - that was a site I started and ran for about 12 years. I think it was going well until I stepped away last year, and then the new owner basically killed it. I also think that whole industry (fitness supplements) is just toxic.

If it's profitable and gamable what would stop them from doing it? You mention that the fact that ONLY 1% of Amazon shoppers use it as the defensive wall, what if all of a sudden 10% does? that would probably make it profitable to game.

The first part of the answer above:

The answer is that yes, it would be possible to "game", but it would take a lot more coordination and planning of the fake reviews to make sure they fly under the radar of all the tests.

Bro, I didn't know ReviewMeta and SupplementReviews were from the same creator. Cool stuff. I was a big fan of SR and valued the reviews my brand had gotten there in the past.

Thanks!

Hi Tommy. Thanks for your work!

Which product category has the most fake reviews on Amazon?

I get asked this a lot, and my usual answer is "cheap electronic gizmos", but really anything that gets a lot of search volume, is easy to manufacture and has high margins - so the most competitive categories on Amazon. I came from a background in fitness supplement reviews, and must admit that the entire industry is what made me skeptical of reviews in the first place.

Let's do a little wet dreaming this morning...

1. How could you improve ReviewMeta if you had access to ALL the data Amazon has? I mean literally everything, including exact time reviews are submitted, full user profile info, full purchase history, even IP addresses.
2. What are some data that would be especially useful in spotting fake unnatural reviews?

Oh man, if I just had access to ALL the data from Amazon. We could do thinks like have a live meter showing the current rate of reviews flowing in, what % of those reviews look unnatural, which products received the most reviews today, which received the most unnatural reviews today, etc.

I think that the IP addresses, shipping addresses, credit cards and event dates (when the product was ordered, shipped, received, reviewed) would help me come up with additional tests with which would help the analysis.

What would you most like to tell us that no one ever asks about?

People can be MEAN! Before I updated the TOS and made it clear we can publish harassment, I would get a few sellers a month sending us very hateful and threatening messages. I even had one guy call my parents house. I have no idea how he got that phone number, but it was kinda creepy.

What is the average amount (a ball park) of fake reviews you found on Amazon since you started? Is it bad?

We estimate anywhere from about 7% to 11%, but our data might be slightly skewed because we only analyze products that our visitors check on ReviewMeta. That said, we still have hundreds of millions of reviews in our DB, so we shouldn't be too far off.

Amazon's PR team always say "less than 1%" which my response is always:

1. if you know it's 1%, why not delete them?
2. with 1 billion reviews on Amazon, 1% fake still means 10 MILLION fake reviews on their own platform by their own admission.

Hey Tommy. Your website is great —very helpful in theory but can be misleading in practice. I'm the author of a self published book on Amazon and have 73 5* reviews that admittedly surpassed my greatest expectations in their positivity. But my book reviews completely fail your tests even though not 1 review is solicited, nor 1 reviewer known by me.

As a result my reviews are measured as fake (70% removed) even though they're all genuine. Is there anything I can do about this? Or what precautions can you take to not to "penalise" the true reviews.

One thing that's important to keep in mind is that not everyone has the same notion of what constitutes an "fake" review. We don't actually use the term "fake" - we use the term "unnatural" which encompasses much more than just "fakes".

So things that may be standard industry practice in the self-published author field might not be acceptable on ReviewMeta. For example, giving away free copies for a review or reviewing friend's/fellow writer's books.

What will it take to clean up Amazon and the fake review eco-system?

Transparency! I think Amazon (and all the large review platforms) need to dramatically increase their transparency. I'm actually working on an argument for why Amazon (and all major review platforms) should be required to open up their review data to the public. Will be posting that next week.

Last year, I mocked up what I think Amazon reviews should look like: https://reviewmeta.com/blog/what-amazon-reviews-should-look-like/

Why does Amazon let this happen? It seems pretty obvious when something is fake. Typically the sentences are all disjointed, with bad wording and grammar. "Is great flashlight. Use with waking dog bright! Great value!"

I think Amazon is doing more behind the scenes than people realize. Though I agree that there are often times I see things where I just have to shake my head in disbelief.

However, I have to mention that not all Amazon shoppers are native English speakers. Just because someone doesn't sound like English is their first language does NOT immediately mean it's a fake review. Yes it can be a red flag if EVERY review for a product is like this, but just one review with grammar issues doesn't immediately mean it's fake.

Why are businesses allowed to have reviews about a completely different product on a product page? Often times this boosts the rating and I see it as deceptive.

Ah yes, Review Hijacking. They aren't "allowed" to, but they do it anyway. It has to do with the open marketplace nature of Amazon. It's been well over a year since we first reported on this and I can't believe Amazon hasn't fixed it yet: https://reviewmeta.com/blog/amazon-review-hijacking/

How much does it cost to maintain a website of ReviewMeta's size? Do you rely on it to make a living?

It's thousands a month to run, but we sell advertising, and now the traffic is to the point where it's finally covering all the costs and putting a little extra in my pocket on top. I had a fair amount of success with other projects online before ReviewMeta, and made some good (aka lucky) investments, so I'm not really relying on ReviewMeta's income to pay the bills. It didn't start out as a way to "get rich", more just a project I thought would be fun to hack together.

Hey buddy. You could maybe include an option to overlay an amazon affiliate code on products. (This would be a low effort way to allow users to support you more directly)

Can't. As I answered somewhere else, Amazon kicked us off the affiliate program very quickly.

Hi! Thank you so much for your work, I love it! I work on a data science and research engineering team. We gave a weekly journal club where we read an academic paper and discuss it as a team. I was thinking it would be fun to read something related to your work (e.g. algorithms or detection methods for fake reviews) - can you recommend anything?

Thanks for the kind words!

I don't know if anything I've written meets the standard of "Academic paper" for the journal club. There's a lot of stuff on the blog, but I think the top three posts that would spark a good debate would be:

https://reviewmeta.com/blog/what-amazon-reviews-should-look-like/

https://reviewmeta.com/blog/how-accurate-is-reviewmeta-com/

https://reviewmeta.com/blog/analysis-of-7-million-amazon-reviews-customers-who-receive-free-or-discounted-item-much-more-likely-to-write-positive-review/

This might fall outside of the scope of your project, but one thing I find particularly annoying is when Amazon seems to arbitrarily combine different products/versions/sizes/etc. into a single ASIN. Many of those reviews them seem to no longer apply, or only apply to a specific marketplace seller who is no longer listed, or other issues that make them as worthless as fake reviews. Have you looked into this problem at all?

What about the "Amazon's Choice" badge? This isn't a review, but it always seems completely fake and meaningless, trying to mislead people into thinking a product has actually been reviewed personally by Amazon.

The arbitrary grouping of products (called Review Hijacking) isn't done by Amazon, but by sellers. We have a warning in place when it's detected: https://reviewmeta.com/blog/amazon-review-hijacking/

I also agree about the "Amazon's Choice" badge. Congress recently sent Jeff Bezos a letter asking about the badge. We did a blog post about it a few months ago: https://reviewmeta.com/blog/amazons-choice-badge-given-to-product-with-1000s-of-incentivized-hijacked-and-deleted-reviews/

Hi Tommy, thanks for your site, I use it a lot.

How often have you had to adjust your algorithms since you originally went live? Have you had to make additional changes as scammers became savvy to how your site works?

Thanks! Glad you are using it!

I've had to tweak some things here and there, added several features (including one that helps detect review hijacking), but nothing too major. Here's a few changes we've made:
https://reviewmeta.com/blog/0-unnatural-reviews-august-2019-algo-updates-explained/

https://reviewmeta.com/blog/amazon-review-hijacking/

As far as scammers "gaming" RM, here's an answer https://np.reddit.com/r/IAmA/comments/dsh6ou/im_tommy_i_built_reviewmeta_a_site_that_detects/f6pga5w?utm_source=share&utm_medium=web2x

I think we can all agree that Amazon could do A LOT more to filter out worthless reviews, especially since they have access to much more internal data. Do you suspect that Amazon simply doesn't care THAT much about unnatural reviews, or rather that their calculations show that cracking down harder would affect sales too much?

They are definitely doing something. I think they do a lot more than the average person realizes because the average person doesn't get to see what's going on behind the scene.

Our data shows that they've deleted millions of reviews, and we don't even get to see how many reviews never saw the light of day!

That said, there are times when you look at the reviews and you just have to smack yourself on the forehead because you have no idea how they made it through their system.

I recently bought an item that came with a card asking for me to leave a good review on their product in return for a free Headphone stand. I then emailed my review and order number to an outlook email and received the item.

How would your algorithm detect this, if my review makes no mention of being compensated (which is against TOS and will get you banned from leaving further reviews.)?

It might not detect this. What we do is an estimate, and it would be impossible for anyone to catch every single inauthentic review.

However, if everyone that reviewed their headphones ALSO reviewed their headphone stand, that would certainly get flagged in our system. There's a lot of other ways your review might get flagged, but it could definitely slip through the cracks.

Hi Tommy. ReviewMeta looks really helpful to people who shop on Amazon, which is pretty much everyone. Great work!

My question is... Is it necessary to read through the entire review analysis provided on ReviewMeta? Or can I basically just look at the "adjusted rating" at the top of the page?

Thanks!

I think that everyone uses ReviewMeta differently. I'll share with you how I personally use it when I shop on Amazon:

First, I check the adjusted rating and analysis results (PASS/FAIL/WARN). If it's a PASS and there's 0% reviews removed, I pretty much stop using ReviewMeta for that product.

If there's some reviews removed, I scroll through the tests and see which ones failed, trying to get a sense of whether or not I want to make a gamble on the product.

There's some categories where pretty much EVERY PRODUCT gets a fail. I was looking to buy a projector (a cheap one, under $200) and pretty much every product I looked at failed. So this is an instance where I was looking to see which ones failed worse.

Shitty. I just bought some 4+ star $80 projector with over 6000 reviews. I guess I'll check out your site.

Return it! I always encourage returning products on Amazon. Send Amazon and the seller a financial message that you won't be swindled!

I might just do that! I'm going to set it up first to see if it will work for what I want it to do but if not I'll certainly send it back. Here it is by the way:

https://reviewmeta.com/amazon/B07174LM85

dang, FAIL with 4.4 to 3.8 stars. That's pretty strong.

It's definitely important to at least click on the extension and look at the report. For instance, go to this link HERE. It shows that the adjusted rating is still 5 stars, but if you click on the report, it shows you that the listing was previously used for a completely different product. That's about the biggest red flag there is, and would immediately get me to stop looking at that product

Oh yeah, always go to the report. You don't necessarily have to read every test though.

I see that ReviewMeta uses an "Adjusted Rating" system, rather than assigning a letter grade (A - F) like FakeSpot does. Why did you decide to use the system you did?

I think that the most important thing to know is what the honest reviews are saying. Not whether there are fake reviews present, or some completely arbitrary letter grade. We calculate the adjusted rating based ONLY off the existing reviews - we never "punish" a product for having "fake" reviews. So, for example, if a product ONLY has 5-star reviews, the adjusted rating is either going to be 5 stars or "insufficient reviews".

I was enjoying the free products 'in exchange for an unbiased review' which was obviously just all 5* reviews since you would not get much free stuff if you trashed everything. So good that this practice came to an end. However, as most people know, the companies now contact users off amazon, have the user buy and then reimburse once the review is made. Obviously they get a lot of 5* because they would likely stop responding if someone put a 2* review and then asked for their money.

I noticed that a lot of reviews that seem 'fake' now , at least to me, are the ones where the user puts some photos and writes a long descriptive overly positive review. I think these are related to the payback scheme outlined above. The real ones are generally less professional looking and more anecdotal. However, there are some people that go all out and are still legitimate, including photos and long winded descriptions. What is your thoughts on this and is there a way to determine which are legit and which are bogus among those lengthy reviews?

I've seen similar patterns as well. I think you can tell the products that have the paid reviewers because EVERY SINGLE REVIEW is like the ones you described above.

I've answered a similar question in terms of the underground facebook incentivized review detection: https://np.reddit.com/r/IAmA/comments/dsh6ou/im_tommy_i_built_reviewmeta_a_site_that_detects/f6pdqbz?utm_source=share&utm_medium=web2x

How much knowledge do you need to create a website such as yours?

I had spent about 10 years developing websites on my own prior to launching ReviewMeta, but I would by no means consider myself an expert in the field.

One thing I always tell people is that I didn't study Computer Science - I studied Construction Management. It has almost nothing to do with what I'm doing now, but that fact hasn't held me back at all. There's a gazillion tutorials to learn any programming language online for free, so if you want to start a project, I encourage you do just dive in and start playing around with things.

That said, ReviewMeta took way longer and was much harder to build than I originally thought. I came up with the idea in maybe October, 2015, and didn't launch until maybe May, 2016, and didn't even work out most of the bugs until 2017. It was a LONG process. I had a few contractors to help with some of the stuff but I did most of the coding myself.

Edit: wrong year

As a professional web developer, I can speak to this a bit, just looking through the site. All the below being said, I don't want to discount how impressive this site is! I've done projects like this and they're (1) not easy and (2) very fulfilling when you get them up and running. I learned all my web skills through personal projects like this, but not typically on this kind of scale. Diving in to a personal project that you feel passionate about and looking up tutorials really is an excellent way to learn.

tl;dr: The site is a pretty decent project for a beginner if you do it in small sections. The front end will be more simple than the back end. The most difficult part is the algorithms for actually analyzing the data accurately.

When it comes to the front-end of the site itself, it's not too difficult:

• There's not a lot of animations, nor really complex designs. It doesn't appear that any professional designer was involved, which will typically make the coding more difficult, but will give you more polish on the final product. The general UI of the site just uses Bootstrap. Their system is quite good, and has some built in dynamic site stuff that allows for both mobile and desktop viewing.
• There's embedded YouTube videos, which you get by basically just copy and pasting embed code generated by YouTube.
• He's integrated the Disqus comment system on some of the pages, also not too difficult to put in.
• He's integrated with Google Analytics as well, not too difficult.
• Looks like there's a bit of cookie management, in the form of rm-first-time-modal-welcome. This manages the pop-up that occurs when you first visit the site, and is saved when you press "I understand and agree". Seems like it may also have something to do with the slide-up modal for watching the YouTube video. Pretty easy to set that sort of thing up.
• Also looks like there's maybe some code in there to enable push notifications, but I haven't seen where that actually gets triggered. Setting up push notifications can be a little tricky for a beginner.

Basically, the front-end of the site is something that is a pretty realistic project for a beginner. That being said, if you're really starting out, it's going to take you a few months and won't be easy.

The back-end of the site is the more difficult thing, depending on what part of it we're talking about. I'll split it up:

• Web scraping: When you paste in the url, the url is likely accessed by a web scraper to pull the reviews for a given product. This is tedious to set up, but definitely not too difficult of a task even for a beginner with basic HTML and JS knowledge.
• Hosting: I'm not seeing any sort of "Powered by xyz" or anything at the bottom of the page, so there's no quick clues here on how the hosting is set up. It's likely AWS at some point, because a lot of resources are being requested from d1kmhjlvxp8o6o.cloudfront.net, which is an AWS service. I'm going to guess that either he's using a service that's backed by AWS, or he's using Amazon Lightsail, or he's gone through the trouble to set it all up through a load balancer and everything manually. Not that difficult, but it's not an easy thing to get set up properly for a beginner. If you were to try this, I'd go with Lightsail. Seems to be pretty straightforward to use, even for a beginner. If you want really beginner stuff, go with something like Wix or Squarespace.
• Database: This is definitely a more difficult part. From what's been said, it seems that the scraped reviews are stored for analysis. With the sheer volume of reviews, this is a decent task. Might be Postgres, might be DynamoDB, not too sure. But it is a good amount of data to deal with. To begin with, not too difficult to set up, but the difficulty with databases comes as the product scales and as use cases vary. Knowing how to index the data properly and access it efficiently can be tricky. Setting up, not too hard for a beginner, maintaining as the user base increases, definitely a more difficult thing for a beginner.
• Algorithms: This is probably the most complex part of the entire site. Figuring out how to accurately analyze the data that you scrape and present useful, truthful, accurate data to the user is no easy task. This is where you'll (likely) use a combination of database tables and back-end code, but you can do it all through just the back-end code. If you're using SQL, you can create tables that join data in different ways (I'm not an expert in this) to basically perform some pre-code data transformations. Then in the code, you do your final analysis of everything and send the final data to the front-end for display. Due to the complexity of the algorithms, this is likely going to be a more intermediate level task than a beginner task, although less complex algorithms can definitely be used.

Hopefully this helps! Also, I'd be interested to know how accurate this is u/ReviewMeta :)

Not bad!

I did hire a professional designer when I first built the site, but yes, it's built on Bootstrap which is awesome and so easy to work with.

The web scraping might have been the hardest part. It's not hard to scrape a couple of URLs, but when you're doing millions of pages and trying not to get blocked, it's a lot harder.

The site is built on MySQL and PHP. I had wanted to learn some newer technologies to build the site but decided it was easier to just go with what I knew.

Please stay away from the 'three wolves and a full moon' shirt reviews! How do you rate the 'Haribo' gummies?

How do you rate the 'Haribo' gummies?

Depends which flavor...

how much money are you raking in from the affiliate links?

None. Amazon kicked us off the affiliate program after like a week or something.

Hi Tommy, how do you plan to generate revenue? Are you in discussions with Amazon to license your tech directly? If so what do they think about the problem and your solution?

We sell advertising on ReviewMeta, so that's how we generate revenue. I'm not in talks with Amazon to license the tech directly. I'd be happy to help them, but they don't seem receptive to my help.

As far as what Amazon thinks - in my opinion, they are a very reactive company. If you take a look at the incentivized review problem from 2016, it wasn't until there was massive public outcry that they actually solved the problem. This is a pattern with them. Sellers exploit loophole, public gets mad, Amazon reacts.

I don't think I'm aligned with their values and goals. First, they are a publicly traded, for-profit company, looking to increase profits and share price. From what I've seen, they are just trying to get you to click the "add to cart" button faster.

My goals and values are around bringing more information and transparency to the process. That will slow down the process of the consumers hitting the "add to cart" button. So I don't think they'd want my help.

I wrote this a few years back, thought it would be an interesting read for you: https://reviewmeta.com/blog/what-amazon-reviews-should-look-like/

Alexa skills are free to download, so they will never have verified purchases, does that skew the results?

Great question! The "Unverified Purchasers" test is just one of 15 tests that we run on the data. Reviews that are verified obviously do better in that test, but the badge doesn't give them any benefit at all in the other 14 tests.

I wonder if you could do this for sites like Airbnb or VRBO?

I had a situation over the summer where I had rented an Airbnb for a week that, in retrospect, absolutely had to have been populated with fake reviews by the owner, his friends, or whatever scam organization owed the property. There were people who had supposedly stayed at the property just the prior week who were extolling its cleanliness. When we got there there was grease dripping out of a vent hood on to the stove and then down onto the floor and that was merely the most egregious of issues with the cleanliness of the property.

I've touched down on why we don't support other platforms in a few other questions - mainly because it's a massive undertaking from a programming standpoint, and other platforms don't share key data we would need.

As far as AirBnb goes, I think their review system is much tighter than Amazon's. You have to have actually stayed in a place to leave a review. Yes, you could have friends pay you through the platform and leave fake reviews, but it's much more of a process to get those reviews through. Not to say there aren't fakes - just in my opinion, I don't think the problem is nearly as bad.

Thanks for this excellent site, will be using it from now on.

Is there any sites you can use this on, that you didn't expect it to be used on?

Just every Amazon site - .com, .ca, .co.uk, .fr, .it, .co.jp, .cn, .co.mx, etc. Here's why we only support Amazon: https://np.reddit.com/r/IAmA/comments/dsh6ou/im_tommy_i_built_reviewmeta_a_site_that_detects/f6pgz0l?utm_source=share&utm_medium=web2x

Hi Tommy,

You seem to use medians/averages to detect outliers. Considering the millions of products available on Amazon, a whole lot is not bought and a lot more is not reviewed.

How does your algorithm proceed when it's the first review on a product?

That's a great question. With one review it's almost impossible to run the analysis. There's actually a disclaimer that shows up on our report if you analyze a product with just one product that says:

With only 1 review for this product, it's difficult for us to analyze review trends. We'll try our best, but keep in mind that there isn't a lot of data for us to go off of.

With two reviews, it's still hard to perform the analysis. Our tool works best when there's 20+ reviews.

How is ReviewMeta different from FakeSpot?

FakeSpot had a tendency to flag reviews as “low quality” or fake with poor accuracy, and negatively impacting adjusted review scores to an extent that it did more harm than good. What makes ReviewMeta better?

https://reviewmeta.com/blog/fakespot-vs-reviewmeta-for-analyzing-reviews/

https://www.reddit.com/r/IAmA/comments/dsh6ou/im_tommy_i_built_reviewmeta_a_site_that_detects/f6pbx0e?utm_source=share&utm_medium=web2x

https://www.reddit.com/r/IAmA/comments/dsh6ou/im_tommy_i_built_reviewmeta_a_site_that_detects/f6pi8dk?utm_source=share&utm_medium=web2x

https://www.reddit.com/r/IAmA/comments/dsh6ou/im_tommy_i_built_reviewmeta_a_site_that_detects/f6pixrf?utm_source=share&utm_medium=web2x

Lately I've been seeing products whose reviews are for completely different products.

Clearly the seller just swaps out the picture and description and price, and Amazon allows that.

Does your algorithm detect those real precious reviews which are for unrelated products?

Yup, this is called Review Hijacking and we display a warning when detected: https://reviewmeta.com/blog/amazon-review-hijacking/

Has Amazon itself ever reached out to you directly about ReviewMeta? Do you know how they feel about your site?

Someone from Amazon's PR team and I had a chat a few years back. It was about a recent article that was written by a journalist who misquoted me. We were both annoyed about the misquote and they were just sharing tips on how to better work with journalists in the future. Other than that, nothing!

I understand how your service works, respect such a noble cause, and am very impressed by its performance. However, I can't help but feel like the goal itself is "turtles all the way down," so to speak. A conclusion reached from a series of assumptions relying on one another. One can reasonably assume that shill Amazon reviewers use repeated phrases - that's just common sense. But do you know that such behaviors are indicative? If so, how?

I'm not sure I understand the question. There are 15 tests - no single test is perfect on it's own, and there's a lot more that goes into each test than meets the eye.

Any way this could be ported to a chrome or Firefox extension that flags posts live as you're browsing?

Way ahead of you! https://reviewmeta.com/blog/extensions/

This may be a pretty obvious one, but can you put in words why fake reviews are bad/damaging to customers and their shopping experience? I think some people would just say "don't read the reviews" but obviously a lot of people rely on reviews to make purchase decisions. Could just comment generally on how fake reviews are used/abused and how that impacts consumers?

I think that the ones hurt the most are the honest sellers. The ones that play by the rules and lose all their business to the fakers.

Love your site and extension!

What could Amazon do to its review system that would make ReviewMeta obsolete in a year?

Who is asking? Is this Amazon???

I think there's a lot that Amazon could do, but people will always be skeptical of the self-policed platform and look for third-party analysis to double-check. So I don't know that Amazon could make us obsolete (aside from deleting all the reviews entirely).

Here's our suggestions on what Amazon reviews should look like: https://reviewmeta.com/blog/what-amazon-reviews-should-look-like/

I don’t know if this is something you can determine, but one of my biggest issues with Amazon are all the fake knockoff products being sold as real. I don’t know if the reviews are accurate but many seem to state that the quality isn’t up to par with what you can buy at a store or they receive a completely different product in the same or similar packaging. A large amount of health and beauty products have this issue. Is there any way to determine fake products as well as fake reviews?

Totally agree. Unfortunately, I don't see ReviewMeta being in a position to determine this automatically.

I’ve noticed the new trend with fake reviews are private Facebook groups, where the seller organizers reviewers to purchase their product themselves, then later receiving a reimbursement via PayPal. This leaves no money trail that amazon could track, and gives the reviewer a “Verified Purchaser” tag. Is your website able to detect this type of fake review?

Yup, this is definitely something we see as well. There are 15 tests on ReviewMeta, and the "Verified Purchaser" badge only comes into account on one of them (called the "Unverified Purchases" test). So the badge doesn't give the reviews any extra protection on the other 14 tests.

We can still pick up the patterns in other tests. For example, our "Overlapping Review History" test will look at the product review history for all the reviewers of a product, and see if there's a lot of overlap. Basically it checks to see if the reviewers are all reviewing the same products. This happens to be the case for a lot of facebook groups - the cluster of reviewers participating in the group will all end up having similar review histories.

It looks like the main function of ReviewMeta is to copy & paste in an Amazon product URL to see the analysis of its reviews. Don't get me wrong... that's an awesome function... but it seems like it would be cool if there was a SEARCH function that allows me to actually search for a product I'm looking to buy. This way, I could actually START my shopping at ReviewMeta & then go to Amazon to buy the product I want that has good, real reviews (rather than starting my shopping on Amazon & then using ReviewMeta to check it before buying it).

Do others agree that a SEARCH function would be useful?

I agree, a search function would be nice, but it's much harder to build than you'd think. Amazon has a large team of well-paid engineers all dedicated to nothing but the search engine. I don't have the luxury of that resource. It would be really hard for me to make a decent search engine that was able to find the best results for 100's of millions of products, many of which may be out of stock, unavailable or discontinued.

We do have free browser extensions available: https://reviewmeta.com/blog/extensions/ - and we're working on some enhancements that will streamline the process even more.

Are there plans for other web stores?

I was watching a youtuber buying random stuff from wish, he pointed out alot of the reviews had 5 stars but the user hadn't even tried the product yet. I've heard buying from wish can be a gamble, and something tells me fake reviews are apart of the shadyness

I've never really looked at "wish" until now!

We actually get asked this a lot (about supporting other platforms). The answer is that it's a massive undertaking to program in support for other platforms, and Amazon is taking up all of my time at the moment.

Also, there's some technical reasons that we can't analyze reviews on a few other platforms that I've looked at. Walmart, for example, does not show any unique identifier for the reviewer. They just show the "name" of the review (eg. "fred"). There could be a million reviewers named "fred", and I'm sure these reviews could change their display name as well. This makes it impossible for us to collect a reviewer's history which is a huge part of our analysis.

Another reasons why these large review platforms need to become more transparent and open their review data up to the public!

Edit: Heard of Wish, never really looked at the reviews on the platform.

Why do you suppose Amazon, which sells AI via AWS, doesn't use any to combat the fake reviews? Are they that uncontaminated with integrity?

I'm sure they DO use AI to combat fake reviews - we just never see what is going on behind the scenes. Our data shows that Amazon has deleted millions of reviews, and that doesn't include any that might have never seen the light of day in the first place.

What’s the worst fake review you’ve seen?

Earlier this year, there was this flood of millions of unverified 5-star reviews on Amazon. A product would literally have exactly 1,000 reviews written in 2 days, all unverified, all 5-star, all just a few words. It was crazy to see that getting through the filter.

Here's a post about what we saw at the time. Amazon seems to have closed this loophole up though: https://reviewmeta.com/blog/amazon-flooded-with-millions-of-fake-reviews-in-2019/

Awesome contribution. I’m curious to know - what happens to vendors once it’s proven out they are pushing fake reviews?

Do you mean if they are called out on ReviewMeta? Nothing. We just show our findings and that's all.

If Amazon catches them, they could either have some/all reviews deleted or be banned (temporarily or permanently) from selling on Amazon. Some say that sellers can sign right back up under a different name though, but others say it's impossible to get back on.

Any plans for a chrome extension?

https://chrome.google.com/webstore/detail/reviewmetacom-review-anal/fjifglfkcaipnmhngbigdebkoikioend

Do you store scanned reviews (content, not just review ID) on ReviewMeta servers for future analysis? Have you analyzed any reviews that Amazon deleted and what did you find if so? For example were they especially egregious?

Yes, we store the review data on our servers. Yes, we look at deleted reviews. Here's an analysis we did of deleted reviews on Amazon a year or so ago:

https://reviewmeta.com/blog/how-accurate-is-reviewmeta-com/

Does ReviewMeta use Big Data technology? If so, I would like to know what should I learn to understand it, since they say the future (and present) will be built on Big Data.

The analysis is custom built in PHP and MySQL. There's billions of rows of data in our DB. Does that count as Big Data technology?

Everything I do has been self-taught. I'd probably learn Python and R if you're looking into a career in the field of data science, but I'm not the best person to be giving career advice.

I do a lot of shopping on the Amazon app now (Android). Copy pasting a link sounds cumbersome. Any workarounds?

Yup! There's an app. When you're on the Amazon Shopping app, just click the three little dots -> share -> ReviewMeta, and we pull up the report!

https://reviewmeta.com/blog/reviewmeta-phone-apps/

Thank you for creating this. I've just run my book Kama Pootra (excellent white elephant and christmas gift btw) and you give it 4.5 stars for 175 reviews.

https://reviewmeta.com/amazon/1402237146

What types of products get the most fake reviews? Electronics, books, toys etc?

Thanks!

Asked and answered: https://www.reddit.com/r/IAmA/comments/dsh6ou/im_tommy_i_built_reviewmeta_a_site_that_detects/f6pa5dd?utm_source=share&utm_medium=web2x