Last year I did an AMA about a website I started called Tribeworthy with the idea of creating a rating and review platform for news, with the goal of improving trust and understanding between journalists and news consumers.

The original AMA

When we did the original AMA, there seemed to be a feeling of cautious interest. There were lots of questions, many making good points. I think many saw us as a flash in the pan, others saw us as naive. Well we’re still here for better or worse, and a lot has changed.

A few things that have happened since then:

  • We took down our browser extension, and went private again.
  • We’ve done our best to listen to feedback, and have made many changes.
  • We renamed from Tribeworthy to Credder.
  • We relaunched the site as a closed beta, only letting journalists on through invitation only.
  • We were featured on TechCrunch.
  • We are relaunching our site to the public again at the end of May.

One of the major changes is that we now have two ratings per article. A journalist rating, and a user rating. The journalist rating is calculated from reviews left by journalists, and the user rating is calculated from reviews left by users. When we did the original AMA, we were still a little early in our development cycle. We have since completely restructured and built out a lot more underlying infrastructure.

So now we are reopening the site as a public beta, and we are currently allowing users early access by using the invitation code TCNEWS.

You can check out the website here:

My name is Austin Walter, ask me anything!


Further Proof:

Comments: 824 • Responses: 22  • Date: 

FatherPrax1841 karma

How are you differentiating people who report articles as false when they simply disagree with them?

Glares1430 karma

We need a review site for news review sites to differentiate which ones are better.

_oscilloscope96 karma

It's review sites all the way down.

semtex94452 karma

Yeah, funny how the OP mentioned Musk, who floated this idea when the media was reporting on how he called a minor celebrity a pedophile and how he had no proof.

_oscilloscope136 karma

Honestly just didn't think anyone would remember my name from the last AMA.

_oscilloscope203 karma

Our review process is a little different than on other review sites. We make people pick a specific reason for what is wrong with an article. Now it's true that they could pick a reason at random, but if they do their review will most likely stick out like a sore thumb. The next (less different) part of that is that we will be allowing people to upvote/downvote reviews.

How does that make us different than Reddit or any other upvote/downvote site? Well votes factor into a users internal user rating, making it so in the future their rating won't hold as much weight.

Yes of course we are aware of the potential for abusing this, voting will only be able to effect a user rating a certain amount, and we're still working out the kinks.

GreyICE3464 karma

How does that make us different than Reddit or any other upvote/downvote site? Well votes factor into a users internal user rating, making it so in the future their rating won't hold as much weight.

You've recreated Digg. Good job.

_oscilloscope37 karma

You've uncovered my secret goal.

UnabashedRust110 karma

What if instead of making people dependent on an app to tell us if things are credible, we just teach people critical thinking skills?

Edit: If you want to make an app, make an app to teach people critical thinking skills and allow them to practice those skills.

_oscilloscope127 karma

Actually we make people pick specific reasons in their reviews in part to help teach people more about critical thinking skills. The list of reasons includes logical fallacies, different types of bias, and more. We are eventually going to be including examples and explanations with each reason that can be chosen. We want people to be able to think critically on their own.

SRRC424 karma

I'm sure you get asked this all the time, but why should we feel like we can trust the public more than shitty news sites?

_oscilloscope458 karma

I'm not asking you to trust anyone. We calculate two ratings for each article, a journalist rating and a user rating. If you don't trust users, don't look at the user rating. If you don't trust journalists, don't look at the journalist rating. If you don't trust either one, don't use our site.

That said, we are doing our best to constrain how people can leave reviews. Here are a few of our methods:

Constraining behavior

  • Force reviewers to pick specific reasons to help focus reviews
  • Not allow reviews to be left directly on outlets, only on articles

Reviewer accountability

  • Site not anonymous
  • Make it against our Terms of Service to have more than one account
  • Trust Ladder (The more a user verifies themselves, the more weight given to their reviews)
  • Upvotes/Downvotes on reviews effect weight of user reviews (Yes we know there is potential for abuse here, we’re working on it)

Aggregating Information

  • Article ratings are only calculated after a minimum number of reviews
  • Outlet/Author ratings are only calculated after a minimum number of their articles have calculated ratings

Analysis tools

  • Nothing new here, Sentiment Analysis, Machine Learning, and other tools have been proposed by many others. The difference is they want it to be the main solution, and we want to use it to supplement our solution.

VenetianGreen144 karma

I'm skeptical, but I like the explanations that you've given in here so I've decided to give it a try.

Question: how are stories sorted on the homepage? There don't seem to be sorting options, will you be implementing things like 'most popular today/this week' or 'rising' etc? How many users are currently active this early on?

_oscilloscope74 karma

Right now articles are sorted on the homepage based off a combination of newness and activity, but we're going to be rewriting how it works in a few weeks. We're still deciding if we want to provide sorting options, but they will probably be included in the iOS app at the very least.

Number of active users has been low, but that's because we were in private beta. The users who have been active though are very active.

BimSwoii38 karma

I would use it mostly for when I read an article and want to look up that specifc article on your site

_oscilloscope50 karma

That's actually one of the main uses cases we are predicting. When we have time we're going to be pouring a lot more resources into search.

daweitopost299 karma

How do you expect to deal with fake reviews, also do you verify journalists or could a user pose as one?

_oscilloscope399 karma

So we're dealing with fake reviews in a few ways. First, users will not be anonymous on the platform. We will be implementing a “trust ladder”, where the more information you’ve verified about yourself the more weight will be given to your reviews. In addition, it will be a violation of our Terms of Service to have more than one account on our site. If we catch you with multiple accounts, you will be banned from the site.

The verification of journalists is manual right now. They must either respond to an email from us using an email address associated with their most recent outlet, or respond to a direct message from us on a verified Twitter account. In the future we'll also be doing video calls for cases we're not sure about.

Bjoorden104 karma

I love this idea, but I must ask something: do you have any countermeasure to prevent review bombs like those that happen on Steam?

_oscilloscope105 karma

We are addressing that in a few ways. First, new accounts will not have an effect on ratings until they have verified more information about themselves. Next, if we detect that a person is trying to manipulate ratings with multiple accounts, they will be banned.

The next measure is to temporarily pause reviewing on specific articles or outlets if we detect unusual activity. Since we only allow an article to be posted once to the platform, and reviews are attached to them forever we can afford to take actions like this occasionally.

As well, we don't allow people to review news outlets or authors directly, only through articles. So it would have to be a well planned review bomb to have a negative effect on an outlet or author.

ghostoutlaw67 karma

Will there be a function to go back and edit the value of reviews historically?

Example: Headline - "President caught embezzling money to Cayman Island bank accounts"

The reviews all indicate this is incredible journalism, 100% factual and the president is guilty of this crime.

2 Years later, article comes out indicating there was 0 evidence and a total fabrication.

This needs to be evaluated on it's merits. Was the original article a brilliant fabrication and forgery that even the most skilled reviewer would never catch? Or was there a giant circle jerk going on that gave this initial article credibility that it shouldn't have been merited.

It's okay to be wrong, but fabrications and praise need be adjusted accordingly after the fact. It's a huge problem with judicial precedence (different issue) that's really hard to address and your system would be abusable by those who give credit to highly praised articles and use their high scores to discredit news that just differs with their viewpoints.

I'd be curious if/how you plan on addressing this?

_oscilloscope35 karma

So this is a situation that we are still heavily thinking about. One way we're thinking about dealing with it is by having article pages be long lasting. By that I mean that there would only be one article page for each article, if it was reposted or had multiple urls they'd still all be group into the same article page. Then we'd allow people to leave reviews on an article forever, or at least a very long time. This way, users who didn't leave a review on it in the past could leave reviews on it in the future if something was revealed.

As for editing reviews, we're still determining if we would want to allow that and under what circumstances and for how long.

AllWhiteInk44 karma

Any connection with logosnews?

Just recently was an AMA on the same topic.

Or is it just coincidence?

_oscilloscope38 karma

Complete coincidence.

chickaboomba43 karma

Will users be able to create an anonymous profile after having their real identity verified by the platform? I ask because of this: If a user is not allowed to be anonymous (instead of platform verified with anonymous public-facing user name), one thing that won't happen is journalists being honest about the work of their fellow journalists. There is no way a journalist is going to point out lazy fact verification, bias, etc. in a piece by another writer employed by the same company - because it could get them fired. How will your platform encourage journalists to not be biased in their reviews if they have to be identified?

_oscilloscope27 karma

We are looking into building the option for journalists to anonymously leave reviews.

KJ6BWB19 karma

What if you only dislike part of an article? Like an article gets one important thing wrong but gets four important things right. it seems like your review model might be a little simplistic for something like that.

_oscilloscope11 karma

We actually get asked this a lot. So we're changing our review process. Users will first leave a 1-5 rating on how much they trust an article, then pick a specific problem or positive reason for how they feel about an article.

mooncow-pie11 karma

How about a highlight feature, so that they can highlight a part of the article they didn't think was accurate? And then you could stack the review highlights on top of each other to see what all reviewers have highlighted.

or... maybe not. I could see how that would clutter things quickly.

_oscilloscope9 karma

Yeah we thought about that but didn't see a way to implement it where it would actually get used or be intuitive. We also noticed that a lot of other sites that went down that route struggled to grow.

humourlessOH2 karma

Thought about using the Blockchain? For building trust, it works well for cryptocurrencies... Except for the whales

_oscilloscope4 karma

We thought about it, but decided it wasn't right for us. At least for now.

MrchntMariner861 karma

What is the deal with your r/eggwhitereplication subreddit?

Browsing your history, it was a...peculiar topic to come across.

_oscilloscope3 karma

I'm also interested in biotech and read about it in my spare time.

sheparooo-4 karma

Do you actually expect me to file out a form to read news?

_oscilloscope2 karma

Only for the next couple weeks.