_oscilloscope

I'm not asking you to trust anyone. We calculate two ratings for each article, a journalist rating and a user rating. If you don't trust users, don't look at the user rating. If you don't trust journalists, don't look at the journalist rating. If you don't trust either one, don't use our site.

That said, we are doing our best to constrain how people can leave reviews. Here are a few of our methods:

Constraining behavior

• Force reviewers to pick specific reasons to help focus reviews
• Not allow reviews to be left directly on outlets, only on articles

Reviewer accountability

• Site not anonymous
• Make it against our Terms of Service to have more than one account
• Trust Ladder (The more a user verifies themselves, the more weight given to their reviews)
• Upvotes/Downvotes on reviews effect weight of user reviews (Yes we know there is potential for abuse here, we’re working on it)

Aggregating Information

• Article ratings are only calculated after a minimum number of reviews
• Outlet/Author ratings are only calculated after a minimum number of their articles have calculated ratings

Analysis tools

• Nothing new here, Sentiment Analysis, Machine Learning, and other tools have been proposed by many others. The difference is they want it to be the main solution, and we want to use it to supplement our solution.

So we're dealing with fake reviews in a few ways. First, users will not be anonymous on the platform. We will be implementing a “trust ladder”, where the more information you’ve verified about yourself the more weight will be given to your reviews. In addition, it will be a violation of our Terms of Service to have more than one account on our site. If we catch you with multiple accounts, you will be banned from the site.

The verification of journalists is manual right now. They must either respond to an email from us using an email address associated with their most recent outlet, or respond to a direct message from us on a verified Twitter account. In the future we'll also be doing video calls for cases we're not sure about.

So one thing that we're doing to try to combat this within our review process is we make people choose a specific reason why they think there is a problem with an article. We only display the top three reasons with an article's rating, so in this way it's harder for a person to try to trash an article just because they don't like it. In addition, if a person it constantly reviewing certain types of articles as good or bad, that will eventually be reflected in their user rating.

Our review process is a little different than on other review sites. We make people pick a specific reason for what is wrong with an article. Now it's true that they could pick a reason at random, but if they do their review will most likely stick out like a sore thumb. The next (less different) part of that is that we will be allowing people to upvote/downvote reviews.

How does that make us different than Reddit or any other upvote/downvote site? Well votes factor into a users internal user rating, making it so in the future their rating won't hold as much weight.

Yes of course we are aware of the potential for abusing this, voting will only be able to effect a user rating a certain amount, and we're still working out the kinks.

We won't ever be able to completely stop people from rating based on their opinion. However, part of why we started this was to increase the public knowledge of media literacy. As part of that, with every review that people leave they have to select a specific reason why they left their review. The list includes most logical fallacies, biases, and common mistakes. Soon we're going to be including short descriptions of each of the reasons to help people pick the best one. If the reason that they select is at odds with the review they left, it will negatively effect their user rating. The goal here isn't to be the end-all-be-all of truth, it's to re-engage consumers with journalists and try to make the new ecosystem less hostile.