I'm Dr Tim van Gelder, working on the University of Melbourne (Australia) based SWARM Project, which is a 4.5 year program of research into how human reasoning can be improved. We are funded by IARPA (Intelligence Advanced Research Projects Activity) to develop a new type of platform for collaborative intelligence analysis, combining crowdsourcing with new structured analytical techniques.

Learn more:





My proof: https://www.facebook.com/unimelb/photos/a.10150252311080195.481207.21591025194/10160745885370195/

EDIT: 10.25PM here - Thanks everyone - signing off now. Will look in again tomorrow and answer more questions.

Comments: 127 • Responses: 37  • Date: 

moration9 karma

Can you give a concrete example of some human reasoning you’re trying to improve? Otherwise this sounds like a bunch of research speak to get funding.

tvg3 karma

Sure - remember a few months back when there was chemical weapons attack in Syria? Was this done by the Syrian government forces? Or was it a "false flag" operation, designed to bring US forces to harm the Syrian government? How do we know? That is a classic intelligence problem.

lintujen_sukulainen6 karma

Iraq's chemical weapons were a classic intelligence problem. How will swarm help if only the results that fit an agenda get acted upon?

tvg7 karma

We're trying to limit our ambitions to improving analytical reasoning at the moment :)

zyxzevn1 karma

With any possible false flag, one should first look critically at the physics of the event. This excludes all manipulations of people pushing a certain agenda. How does your system incorporate basic physics? And how does your system identify narratives / agendas?

tvg2 karma

Our system can incorporate basic physics - up to a point - in a way that is very analogous to the way a Stack Exchange site might deal with questions involving physics. As for agendas, the relevant analogy is Wikipedia - a combination of rules/guidelines, development of a suitable culture, and having moderators. Wikipedia does pretty well on this though of course dealing with agenda-driven contributions will always be a challenge.

moration0 karma

I thought you were talking about human intelligence.

How would your system make any headway with the example you have here?

tvg2 karma

Yes, there is always that ambiguity between intelligence (IQ) and intelligence in the sense of what intelligence analysts do. We focus on the latter. With regard to Syria chemical weapons, anyone who draws a conclusion here, one way or the other, does so at least partly on the basis of evidence and arguments. Our work focuses on how to improve this reasoning.

Feminition7 karma

Is it possible during these exercises that the result may not be the most logical reasoning due to the dominant over the subordinate factor?

tvg2 karma

Yes that is always possible, but the various aspects of the design are meant to counteract that. For example, users are all automatically assigned pseudonyms such as Gecko29 to avoid higher-status individuals having too much influence.

thimble-kitten6 karma

What is your perspective on how this form of analysis might help or hinder safety when applied in a national security context? How do you ensure information remains appropriately compartmentalised / confidential if you cast a broader net than relying on small groups of analysts?

tvg6 karma

Re first question - very interesting. Our goal is to improve analytical reasoning. This can potentially help national security greatly. Think about the invasion of Iraq - arguably based on faulty analytical reasoning (and not just because of politicization). Better intelligence analysis might have averted that massively tragic episode, and all the damage it has done to the national security of the US and allied countries. And if countries like the US can make blunders based on bad intelligence, that's going to be true also for its enemies. So we might all be safer if we share any methods and technologies for improved intelligence.

tvg6 karma

Re second question - this is certainly a technical challenge. It is faced all the time already in organisations where information has to be shared, but different people have different levels of access. SWARM isn't dealing with that problem yet because the platform is only being tested on fictional scenarios with no sensitive or classified information.

HH-Rob2 karma

Is there any literature you could recommend on faulty analytical reasoning with regards to the Iraq invasion?

tvg2 karma

Sure - check out Robert Jervis, Why Intelligence Fails: Lessons from the Iranian Revolutio and the Iraq War.

thenewapollo6 karma

Hey, I'm a first year undergraduate at unimelb and I registered to participate in the SWARM challenge. Exciting!

What do you think the applications for crowdsourced intelligence could be?

tvg5 karma

Good to have you on board :) Of course the near-term, intended application is to intelligence analysis. This happens in a great many organisations, both government and private, and not just in the well;-known security agencies like the CIA. But "open source" and even crowdsourced open intelligence is also a growing field, and some suitable version of SWARM might be useful there.

Sonny_Crockett1235 karma

Do you see any ethical issues working on this project, given the history of rampant human rights abuses by US Intelligence agencies, such as disclosures that the CIA relies sometimes exclusively on SIGINT to target and carry out illegal assassinations which have killed thousands of people, very few of whom have been the intended targets (who themselves have never been charged with any crimes)? Or their sordid history of kidnapping, arbitrary detention, torture, development and use of chemical and biological weapons, mind control experiments, recruitment and collaboration with Nazi scientists, etc.?

tvg2 karma

Yes there are ethical issues to consider. I share your concern about many things that have happened in the past and may be happening today. At the same time, things are complicated - the security agencies do an enormous amount of really good work helping make the world a better, safer place. Our work is pure research which we hope will help not only improve intelligence but also might help improve ethical decision making.

anf87015 karma

Do you foresee crowdsourcing analysis applied to all forms of research in future, or do you think it lends itself more to some types of research rather than others?

tvg6 karma

Interesting question... crowdsourcing of one kind or another will be relevant to very many kinds of research - Michael Nielsen's book Reinventing Discovery is great on this. But "all" is a tough standard to meet. My guess is that there will always be some forms of research that are not suited to crowdsourcing. An interesting example is writing novels - not research exactly, but not too far away. Crowdsourcing hasn't been very good for that. BTW the SWARM Project is not exactly crowdsourcing our own research. Rather we're researching into crowdsourcing (or rather, groupsourcing - see an earlier reply) can improve analytical reasoning. I think what we are seeing is that organisations are increasingly realising that tough analytical problems need collaboration among increasingly large groups. The problem is how that collaboration works, and what kind of technological support it needs. SWARM is one kind of approach to that.

nowAttention4 karma

How do you maintain civility when collaborators are working on contentious issues?

tvg2 karma

That's a really interesting challenge. It is not so different of course to maintaining civility on Twitter or even Reddit! There will be platform design elements that can help here, but technology alone can't solve the problem. One direction we'll be exploring is the use of a reputation system. I've had previous experience with that developing a platform called YourView.

PureBredMongrel4 karma

Will the platform be used in a wider context such as group project work in the business or development domain, uni students for group assignments or any type of collaborative group work?

tvg2 karma

Yes, we're seriously looking into that possibility. Currently it supports analytical reasoning generally, and as you point out, there are many possible applications.

Pojiku4 karma

Do you believe that this project could successfully filter out biases towards popular, but ineffective ideas?

tvg5 karma

Well, yes, we hope so. At the heart of the SWARM platform is an approach we call "Contending Analyses." The basic idea is that in a large enough group, multiple alternative analyses can be raised and evaluated on their merits simultaneously. So the idea is to avoid having all attention focused on particular (popular) ideas.

ffsppl3 karma

Was there particular criteria for gaining access to participate in the SWARM project? Just asking because I'm a general member of the public, did the survey, and was accepted. I'm looking forward to contributing though :)

tvg3 karma

You signed up for the 2018 Challenge as "General public"? We don't set any eligibility criteria at all. (For Professional Analysts the only relevant criterion is experience being employed as an analyst.) We find that the kind of people who are interested enough to sign up are well-qualified to participate. So hope you find it a valuable experience :)

nowAttention3 karma

Have you experimented with collective bayesian predictions?

tvg3 karma

We have definitely experimented with groupsourcing bayesian reasoning. We've found that teams on SWARM can do a remarkably good job answering difficult bayesian problems of the kind that you'd normally need a Bayes net to solve. In fact sometimes the groups actually use Bayes nets as part of their reasoning - which underscores the point I made in a previous reply, that good analytical reasoning is about applying the right tool for the job, and a large team in an appropriate collaborative environment can figure out what the right tool is and how to use it.

Disastrophication3 karma

Are you familiar with Vernor Vinge's 2006 scifi novel Rainbow's End? Set in an ever hi paced technologically enhanced society the security forces themselves, not just the pretty much automated intel feeds, swarm to the threat. Your project reminded me of this. Aside from that, what are your feelings about the role of intuition alongside logical reasoning? Not just inductive reasoning but full on subconscious gut feelings guiding last minute decision making. Is it real?

tvg3 karma

Haven't heard of Rainbow's End - thanks for the tip. Intuition and logic are thoroughly intertwined. As soon as you step aware from pure formal logic, human reasoning can't function without intuition. Its not either-or, but about how to get the best out of logic and intuition working together. And there are definitely times when intuition should have the upper hand, as in rapid decision making experts under pressure - see Gary Klein's writings on this.

nowAttention3 karma

Do you apply any particular knowledge about reasoning from philosophy, psychology or the social sciences?

tvg6 karma

We draw on all three areas. One particular topic we draw on is a theory about expertise in human reasoning. One view is that good reasoning is about applying the rules of logic and probability theory. There's of course some truth in that, but an alternative view is that humans build up a diverse toolkit of reasoning skills. The SWARM platform is designed to support a team to draw upon the range of skills the members of the crowd bring to bear, finding the most appropriate tool combination for any given problem.

daynomate2 karma

Have you considered how useful this system might be for training AI?

tvg1 karma

No - interesting idea - will have to give that some thought.

joyshri-lobo2 karma

I signed in with SWARM yesterday. This particular episode is almost into the 3rd week. Do you think I can proceed from here?

tvg1 karma

Get in touch with [email protected]

MarkHFX2 karma

I am doing the onboarding survey atm. I am finding that many of the options do not apply to me. Asking what my training is rather than strengths or weaknesses. Should I continue the survey based solely on my natural abilities to think three dimensionally, or leave this to those educated in analytics?

tvg1 karma

Make your best guesses to complete the survey. You'll find that the on-platform work has a very different flavour.

mikeHals1 karma

Does the Swarm project interact with IARPA's other projects, such as Good Judgement?

tvg1 karma

No direct interaction, but there are relationships. Good Judgement was a "performer" team in the IARPA ACE program, which focused on forecasting. They got good results, and CREATE (the program we're in) can be seen as a natural successor, focusing on the reasoning behind forecasts (and other types of judgements).

blindtoreason1 karma

Could this research help improve on the models of democracy that are currently in play around the world?

tvg2 karma

Improving democracy is a tough problem, and we're of course not trying to confront that directly. But I'm a fan of "deliberative democracy" in its various guises, and crowdsourced analytical reasoning might well have a role to play in some forms of deliberative democracy. So I wouldn't rule it out - much further down the track.

inderjalli1 karma

There's studies that show that if you show a person evidence that contradicts their beliefs, they dig in and resist that evidence even harder. How are you tackling this phenomenon?

tvg2 karma

You're right - called the backfire effect. Interestingly, phenomena like backfire, and confirmation bias, might be problems at an individual level, but they might be something that can be harnessed at a group level. In other words, it might help a group to have members that stick to their positions really doggedly. So the idea is you may not need to overcome/eliminate backfire and other effects; it might be better to figure out how to take (better) advantage of them.

HeWhoQuestions1 karma

That's a cool thought. Will SWARM automatically detect logical fallacies in it's users input? It's usually the case that if a fallacious argument is made and not obvious to everyone, it will still have its (undeserved) effect even if someone tackles the fallacy shortly after.

It seems like a relatively simple task to filter, say, ad hominem attacks out before they do any damage.

tvg1 karma

We're not trying to do something as fancy like automatically detect logical fallacies. Actually that would be extremely difficult. We're building an environment in which the group as a whole can - all going well - detect and avoid logical fallacies adequately well.

yvo841 karma

Why were you inspired by bees?

tvg6 karma

Bees have remarkable collective intelligence. Particularly in the way they select a new home. See this article: https://pursuit.unimelb.edu.au/articles/crowdsourcing-security-intelligence for more about swarms and SWARM! (Also, as it happens, I keep bees)

Xavad1 karma

Bees are also genetically hierarchical. Do you take this assumption to be applied to humans?

tvg1 karma

Certainly bees are genetically disposed to occupy different roles. The so-called "Queen" for example is a kind of slave to worker bees, who collectively rule the colony. Does this apply to humans? That is way outside my are of expertise, but I take it to be an empirical question as to whether there are genetic influences on the roles different people take in society. One for the evolutionary biologists!

Feminition1 karma

Do we use the probability equations to confirm analysis? I would be useless using the numerical equations to solve problems.

tvg1 karma

Yes, when we give groups probabilistic reasoning puzzles, we use Bayes nets (based on underlying equations) to check answers. Most people have very limited knowledge of probability math, but we're interested in what groups can do when there is a mix of expertise in the groups. People who aren't strong in math can still play useful roles.

Feminition1 karma

How can someone be useful if they don’t have strength in mathematics?
This could be helpful 🙂

tvg1 karma

A good report presents reasoning clearly. Often "math geeks" aren't very good at explaining their reasoning. They need help from others who can work with them to find a way to express the technical stuff in more widely comprehensible terms.

ChezRoxwell1 karma

Intelligence huh? Too bad you can’t fix liberals!! Seriously though do you support president Donald Trump and can you rank his intelligence on the Mensa scale somewhere between Einstein and Steven Hawking?

tvg1 karma

No comment on Liberals and/or Trump, but it is important to realise that intelligence (IQ) is different to rationality (RQ). Keith Stanovich has argued this point extensively - see his book The Rationality Quotient.

warlord911 karma

What's the single biggest rule when trying to find the meaning of what is reasonable?

tvg1 karma

I think many experts would say that if there was a single biggest rule, it would have to be Bayes' Theorem, which tells us how to modify our existing beliefs in the light of new evidence. But I'd say there are many important rules - or rules of thumb - rather than just one.

mentat_md1 karma

Could you recommend any resources as an introduction to Bayesian Networks that would be accessible for someone who didn't study math at the University (step by step explanations, commented examples)?

tvg1 karma

Yes, a good place to start is the videos on Bayesian thinking by Julia Galef on Youtube https://www.youtube.com/results?search_query=julia+galef+bayes

nowAttention1 karma

Is the scope of the project specifically about security intelligence? Or does it range further than this?

tvg1 karma

The project is funded by US intelligence, but it is pure, unclassified research and all our results will be openly available. The analytical reasoning we're trying to improve applies not just in security intelligence but very broadly to any kind of analytical work. We're lucky that the intelligence folks are willing to fund this kind of non-specific, widely applicable research.

waaltergarcia1 karma

Congrats on your job, it's awesome what you are doing!

I have some questions: On which regions of the brain are you guys focusing on? What kind of stimulation are you using? How can you be sure there are no "over-function" effects such as seizures? Why not to focus on other regions of the brain?

Thank you!

tvg1 karma

We're not looking inside the head, so to speak. For us the brain is a black box; we're interested in human behavior, particularly in groups. That said, years ago I was involved in a small fMRI project looking at what lights up in the brain when people are doing critical thinking tasks. Our very preliminary findings suggested that this overlapped with areas of the brain involved in social interaction. The extremely speculative suggestion was that critical thinking evolved in order to help us know whether/how much to trust what other people are telling us.

Dante4720 karma


tvg1 karma

Not at all. Basically we're interested in how, if you put people in groups, and support appropriate interactions, they can collectively be much smarter than they would individually, when dealing with the kinds of problems that intelligence analysts have to tackle.

sf-keto0 karma

Considering the disaster of your first couple of SWARM pilots, what are you doing to improve your cumbersome platform, stop the free riding & actually improve the training so that participants follow the process? The 2 I participated in definitely made me feel you guys are on the WRONG TRACK.

tvg1 karma

Sorry to hear that your experience wasn't great. Generally however teams in our beta tests are producing good (sometimes great) analyses. What seems like free-riding might instead be a natural phenomenon when you get large (largish) groups tackling a problem together.

sf-keto0 karma

Of course you’re going to say it’s great & needs no improvement. So biased & irrational - you can’t even offer what improvements you’ll be making to that horrible clunky wiki thing of yours. It needs a lot of work & you know it. You guys need a more objective & open point of view.

tvg3 karma

I'll bet our list of ways the platform could be improved is a lot longer than yours :)