Hey Reddit, I’m Scott Patterson. I'm an author and staff reporter for the Wall Street Journal, covering the government's regulation of the financial industry. I also write extensively on advanced technology used by sophisticated firms to trade on the stock market—and sometimes manipulate it—such as high-frequency trading, high-powered algorithms, artificial intelligence and dark pools.

My new book is Dark Pools: High-Speed Traders, A.I. Bandits, and the Threat to the Global Financial System

In this book I show how the stock market has come to rely on computerized trading over the past 20 years, how many suspect it is rigged against average investors, and what I think is coming next.

If you want a quick preview, Zero Hedge did a 3 part excerpt

During an interview about The Quants Jon Stewart and The Daily Show helped me come up with the idea for writing Dark Pools, so who knows Reddit, maybe we can come up with my next book idea today!

I'm excited and will be here to chat for as long as I can--hopefully no Black Swans hit and force me to go back to work in the middle.


Ask away!

Comments: 573 • Responses: 44  • Date: 

nyseed67 karma

When you interviewed Mark Cuban about the problems with high-frequency trading you asked him “What’s the solution?”

How would you respond to the same question?

scott_patterson91 karma

The regulators need to get on top of what's going on and fast. The public is losing trust in the markets and right now our regulators can't give them any assurances that they're wrong. Right now we just don't know and our regulators don't either -- that's why the market is dark.

underwaterbear39 karma

Aren't the regulators just people from the financial houses taking a break? SEC is all about arbitration and not prison time, no? Fox guards the henhouse, etc.

scott_patterson62 karma

I think there are some good well-intentioned people at the SEC and other regulators. But they're totally outgunned. And the financial incentives just aren't there. If you have the technical ability to understand much of what drives our markets, you're more than likely going to cash in for a multi-million dollar paycheck on Wall Street than work in the SEC. Sure, it's always been like that. But because the market is so technical, so computer-driven now, that our regulators need to have that technological literacy or they can't surveille the market.

Bottom line: The SEC needs to hire more quants and fewer lawyers.

davidmanheim28 karma

"The SEC needs to hire more quants and fewer lawyers."

Only if they want to manage what may be an unmanageable problem. The truth will always be that if a regulator understand the market, they could easily multiply their salary by a factor of at least 2, and typically more like 10, by going to industry. The tide won't go back, shout as Canute may. (I worked at an I-Bank for a while, and the CFO, when asked about the auditors understanding of our complex transactions, was quoted as saying something along the lines of "I wish our auditors understood our business well enough to audit it - because then I could hire them for 10 times what they are being paid. We desperately need those people.")

Regulatory capture and the inability to regulate to to political pressure is the biggest issue facing any such body - and not to change subjects, but this is a campaign finance and income inequality issue.

f289 karma

"I wish our auditors understood our business well enough to audit it - because then I could hire them for 10 times what they are being paid. We desperately need those people."

Asking out of ignorance; What skill set in particular is he referring to here? What do those people need to do to understand the business?

edit: Thanks for the enlightening responses, Gents.

scott_patterson10 karma

They need to understand the complex, computer-driven financial products and strategies that run Wall Street today. And too often they don't. This causes many people on Wall Street to look down on the regulators and even skoff at them. A bad situation for everyone.

davidmanheim10 karma

Is there a possibility of knowing what is going on if the markets remain fully automated, and computerized trading can go on at speeds that make the data processing to understand them so difficult? How can regulators understand a market with arbitrarily complex, constantly changing automatic program running on them?

Do changes need to be made first?

scott_patterson26 karma

They simply need to get the computer fire power to monitor the market, and it exists. They just haven't done it yet. There's a proposal to build a so-called Conoslidated Audit Trail that could help solve some of these problems. I wrote about it for WSJ last year: http://online.wsj.com/article/SB10001424053111904491704576574883908453622.html

[deleted]6 karma

I'm suspicious about this concept have having a government computer monitor an automated market. As long as it's a bunch of computers operating based on set rules, won't there be room to invent new exploits that take advantage of those rules without being detected?

scott_patterson20 karma

Are you suspicious of cops using radar to catch speeders on the highway? Because right now the SEC doesn't have that radar.

[deleted]5 karma

Well no, sorry for not being clear. I'm not saying I'm suspicious of the government, but I'm suspicious of the idea. Essentially my worry is that if the government computer is only searching for certain things, then someone might figure out what that computer is searching for and be able to enact manipulation that doesn't set off any alarms.

I'd be all in favor of the government having the sort of infrastructure you suggest, assuming it would be helpful to prevent manipulation. I just suspect that it wouldn't be sufficient, and would instead lead to an arms race of "who has more clever computer programmers."

But to answer you're question, yes, I'm suspicious of cops.

scott_patterson7 karma

Well, to be honest I am too...

But they do serve a role, right? And my concern is that the regulators of our market simply don't have that radar detector, and firms know that and are able to exploit it.

As far as HFT firms figuring out what the computer is looking for and looking for loopholes, that is certainly a risk. But I think that any system like this would be an AI pattern recognition program that would constantly be looking for new patterns. It wouldn't be cookie cutter. I'm sure IBM would love the business...

TominatorXX2 karma

What about "front running"?

It used to be crime. Now it's called "co-location" and is, apparently, perfectly legal when big banks do it. If a little trader front runs, he gets crushed by regulators. I've had a trader tell me this. That it's illegal for him; legal for the big boys.

scott_patterson3 karma

Not so sure about this. Most firms today are colocated to it's not as if anyone has a significant advantage. I think colocation is a red herring.

One area people should look into more--and that I cover in my book Dark Pools--is the use of order types. There are many many order types today (limit orders are child's play) that give firms unique ways to thread their orders through the exchang matching engines. We've reported that the SEC is looking into this: http://online.wsj.com/article/SB10001424052702304636404577297840134760650.html

davidmanheim2 karma

.Even with all of the data, and having every order fully tracked, the complexity issue is still a problem - it is easier to create a complex program than to understand one - and the HFT traders will be spending arbitrarily large sums on creating such programs. How can regulators attempt to understand a system if the competitive advantage of many of the trading systems is that they are opaque to outside observers?

(I won't even get into issues with only tracking completed orders, and having unmonitored dark pools, derivative replication of equities, and side payments that cannot be included since they occur off the exchanges, in private party transactions.)

scott_patterson9 karma

You're right, it's very hard. But they have to make an effort. Another potential solution could be alterations that cut down on the level of orders, such as mandatory holding periods of one second. I'm not sure that will work, though.

nyseed8 karma

Honestly, how confident are you that this could actually happen? Can the public help? Or is reform just a faint hope?

scott_patterson19 karma

I'm not confident, but I'm hopeful. I'm hopeful that my book might help trigger enough outrage that they're forced to do it. But it's a hard fight because the industry has all the money and the lobbyists. Regulation is a dirty word on Wall Street.

[deleted]43 karma


scott_patterson44 karma

I'm not surprised that the NYSE went for this because they've been getting taken to the cleaners by the firms that are allowed to do this. So it doesn't really change the overall structure of the market that much. I can say that I was surprised that the SEC approved it. It means more fragmentation--and we already have about 70 venues to trade stocks today. Isn't that enough? Or I mean far far too much?

[deleted]3 karma


davidmanheim11 karma

I would think that simply changing the execution of trades to occur at the end of every minute would eliminate the false price problem. This would destroy HFT completely, but would also create new transparency in the markets.

scott_patterson20 karma

Maybe. This is the long-running debate over the different benefits of continuous trading and auctions in which price discovery happens periodically (a bunch of traders put in orders and the exchange spits out an average). The trouble with auctions is that it can lead to reduced liquidity--market makers are going to be reluctant to participate because they will have more trouble controlling their risk. Continuous trading lets them get out when they need to, so they're more willing to play ball. Of course, that's the problem isn't it? Is the risk worth the reward?

davidmanheim17 karma

I'm not fully up to date on modern economic mechanism design, but I'm aware of the issues. Continuous markets are not, as far as I am aware, cannot be even theoretically incentive compatible or strategy proof - something that even having an auction every second would create.

Continuity of market prices is an illusion, based on economic assumptions that are demonstrably false. The increased liquidity has helped exactly one set of participants in the market - large firms that collect the fees, and do the automated trading. Economic policies that enrich these firms at the expense of retails investors and the physical economy seems like a strange thing to favor.

And if a firm needs sub-second liquidity in an asset in order to hedge thir exposure, it's because they are taking large risks that cannot be hedged in the market directly with derivatives. A firm with $1b in equity that has offsetting +/- $1 trillion in liabilities and assets is probably dynamically hedging themselves - liquidity masks this instability until a crash happens. Aren't these firms bad for economic stability? Wouldn't the economy be better off without them?

scott_patterson2 karma

One issue is that the auction mechanism isn't seen as the optimal system by HFT firms, for obvious reasons. So you'd need to mandate the one-second holding period, because the market isn't going choose it, and I don't see the SEC ever doing anything like that. At the end of the day it's an interesting theory but ain't going to happen. The SEC in many ways has encouraged the modern-day high-speed market structure.

scott_patterson9 karma

Dark pools are in many ways a symptom of the disease. The alternative is to clean up the exchanges. Big firms are moving their orders away from the "lit" markets because they're concerned about gaming (ie front running) on the exchanges by high-speed firms. Sadly, the exchanges have played into this game because they're competing with one another to attract the flow of the high-speed firms. To do so, they offer them advantages. One source for my book who worked at an exchange told me that they provided HFTs "guaranteed economics." If found the phrase very telling--and disturbing.

DrGrinch35 karma

Do you think the markets still serve their original purpose, to allow people to invest in the longterm wellbeing/health of an organization, or do you think it's strictly now about fast and easy money for the already wealthy and powerful?

If you lean towards the latter, how do you think we can correct it?

Would a completely non-automated secondary market be feasible?

scott_patterson38 karma

That is and should be the purpose of the market--to allow companies to raise money so that they can expand (and hire people) and let people get a chunk of their growth and profits. But the market has turned into a casino. And big shock--the computer revolution thas has taken over the market grew out of so-called SOES bandits, day traders riding stocks for short term gains. Check out Dark Pools for the whole story--no one has told it before. It's totally fascinating! (Sorry, had to get in a random book plug....)

zeno11 karma

Every player whether speculating or investing brings liquidity to the market, which is a good thing. To call it a "casino" seems a simplification of the various motives of the players in the market.

scott_patterson10 karma

The market has always has been casino-like. But it's increasingly so today. The liquidity is driven by HFT firms and quant hedge funds who are doing nothing more than betting on patterns--just like a blackjack player would in Las Vegas. These guys have always been around--they are just more dominant today (at least that how it seems to me).

rskurtzer21 karma

Scott, I just read the excerpt and it looks like a fascinating subject - I live in NYC and a few of my friends are software engineers who help create the tools used in high frequency trading among other things.

My question is, do you think the stock market is a "safe" place for regular individual investors to be in the long term?

scott_patterson27 karma

This is complicated. Because public pension funds are no longer available to most employees in America, we have little choice but to default to the stock market for a least part of our retirment savings. The 401K plan has basically been forced upon Americans as the only way to plan for our retirements. So what choice do we have? Some of our money needs to go into the stock market. But the Flash Crash of May 6, 2010, when the market tanked 10% in a matter of minutes, was a serious warning signal that the market isn't stable, it's dangerously unstable. And it's getting worse.

MyPoopsAreBallads38 karma


scott_patterson41 karma

Well, we don't have much choice to we. But I'm trying to say that the market has changed in many ways, dramatically. This isn't the same market that we've had over the past 80 years.

[deleted]21 karma


scott_patterson26 karma

I'm concered that the average investor in a mutual fund isn't getting a fair shake because the guy doing the trading FOR THE MUTUAL FUND can't compete with the HFTs. Those guys are getting taken to the cleaners and investors are left holding the bag. Even traders for giant hedge funds tell me that they can't do their jobs anymore because the market is so toxic and fragmented and complex.

[deleted]9 karma


scott_patterson11 karma

I'd rather have a system in which the mutual funds can trade on the public markets safely and fairly rather than sell their order flow to the likes of Knight.

Oh-Wee-Oh-Wee-Oh18 karma

Maybe your next book can be about the LIBOR scandal.

What's your take on this? Do you ultimately think it will come out that every bank that contributes rate information was involved? Will we finally start seeing these assholes go to jail?

scott_patterson19 karma

I mean, I thought writing a book about market microstructure was taking a leap...but a book about LIBOR?

Not that it's not a gripping story. I'm not an excerpt on the subject but this is just another example of how the global financial system often operates in the dark. I'm trying to shine a light on some of those dark practices.

Boozdeuvash17 karma

How can we regulate against the risk of people (bankers, politicians,...) being assholes? I feel it's the major issue nowadays...

I mean, you can always try to prevent discrete misconduct on a case by case basis, but wouldnt it be more efficient to eliminate or mitigate the root cause?

dontera8 karma

This is essentially my question as well - is there any hope for a fair system that disincentivizes greed.

scott_patterson28 karma

Greed and fear run the market. But we need strong regulators to keep it in check. But the dramatic technological evolution in the markets in the past decade has left the regulators in the dust. I think they hoped that with the robots in charge they could just sit back and relax--but the robots are at the end of the day doing the bidding of humans. Often very greedy ones.

Stormmando16 karma

How has being a NYT bestseller changed your life? Did you imagine that your work would be so wildly successful?

scott_patterson81 karma

Good question Mom.

wickedlittle14 karma

So how awesome is Jon Stewart in person?

scott_patterson36 karma

Pretty awesome, and smart. His brother Larry Leibowitz, who I know and is a top executive at the New York Stock Exchange, is also a really smart guy--and funny too.

[deleted]17 karma


scott_patterson44 karma

He actually read it! Unlike most of the people who interviewed me. He's even more skeptical of what's going on on Wall Street than me. And I'm pretty damn skeptical.

looseleafer12 karma

While I wholly agree on the problem of HFT when it comes to flash crashes - and am in favor of volatility regulation as an attempt to control panic - I see it as a losing battle. Technology is allowing people to realize, and act upon, negative news all at once.

I feel that everyone (mainstream economics) is beginning to accept the fact that the market is NOT truly random. I am a huge proponent of behavioral economics - hence my position as a boutique trader. The advance of technology has greatly increased investor sentiment as a major influence of the markets, rather than simply fundamentals.

Much of HFT (what you described as latency arbitrage) I don't have a problem with. Many people see it as a problem because as of now, it's a certain bunch making the money off of this. The firms are not misleading the ordinary investors in day-to-day situations. They've simply gotten on top of said technological advances.

My problem is with the regulators themselves. For the passed few years - first America, and now much of Europe - has relied upon satisfying short term demand of investors rather than actually fixing the problem. In my opinion much of this is a result of the nature of political systems and the desires of those to get re-elected. The same goes for a large amount of wealth managers - that sector is based upon beating the average, so it doesn't matter if the average sucks - as long as the unedcated clients with money are happy to be doing better than everyone else.

So personally, I don't see the firms who are profiting for themselves and their client - through (as of now) legal means, as a problem. They aren't misleading anybody.

Am I crazy? Should I read your book to get proven wrong? Thanks a bunch for providing easily one of the most interesting AMAs from my point of view.

scott_patterson20 karma

Great question(s). Fund managers are completly focused on beating the averages, and end up tracking it themselves. So why not just invest in a cheap index fund? But I also think that the fascinating world (to me) of market microstructure is increasingly playing a macro role in investor returns and investor confidence. People don't trust it anymore and that's very worrisome. And the regulators can't provide assurances that their lack of confidence is justified. It's a dark market.

brentnoose11 karma

In a recent article, you quoted Sen. Ted Kaufman as saying, "Human beings were no longer doing the trading, computers were. They developed these algorithms. It ran automatically. It grew and grew. There is no way to know what is going on. No one knows what is happening in these exchanges when this trading is going on. We have a very dangerous situation."

Are the individuals initiating this high-frequency trading aware of what the potential for damage could be, or do they simply not care?

scott_patterson14 karma

I don't think they think about the damage. In fact I believe many of them believe that they're helping the market. Any maybe they are--some of them. My concern is that this one species of trader has become so dominant that the present a systemic threat to the market. Kind of like a pool (a dark one of course) in which the only living creatures left are the sharks. Then what do the sharks eat?

dev_jim11 karma


scott_patterson33 karma

First, thanks for the compliment on the Island chapters--I loved writing about it. Josh Levine (founder of Island) is a fascinating person. Second, I think my book is the best book on the market about modern microstructure that everyday people can read and learn from. So we plainly disagree! It's a controversial topic and if everyone agreed with me it would have been a really boring book.

YayNewZealand8 karma

I think i heard in my computer architecture course that a lot of effort goes into making sure the computers running the programs to deal with trading are as close as possible to each other to reduce data transfer times by some stupidly small amount of time. How true is this and is that effort being done because of a lack of the ability to increase times in processors and software?

scott_patterson12 karma

This is absolutely true and is in fact how the market largely works today. The technical term is "colocation"--trading firms "colocate" with the exchange's matching engine (where buy and sell orders match up). And guess what--I could care less about this. For a couple reasons. One is, it's not going to change, it's the march of progress, etc. And it's become so dominant that everyone does it. Your mutual fund trades through a colocated server at a data center in New Jersey. That just the way it is. There are better things to get concerned about.

kasey_junk8 karma

The May 6th Flash Crash seems to be a major point of concern for the "average investor". But given that it corrected itself so quickly, weren't the only one's hurt in this those who did not have long term positions (ie the HFT/Day traders)?

When people say that crashes like this make the public lose trust in the markets, isn't that just as much an indictment of financial reporting than of HFT trading strategies?

scott_patterson18 karma

Many HFT firms actually made a large amount of money on the Flash Crash.

But that's beside the point. The Flash Crash revealed a major flaw in the market structure, and one that hasn't gone away. The new liquidity providers--HFT firms--cut and run when the market gets too chaotic. So when the market needs them most, they aren't there.

kasey_junk6 karma

If many HFT firms made a large amount of money, isn't it true that many lost?

Also, this doesn't get to my major question. How does a flash crash that self corrects quickly hurt the "average investor" by which I assume you mean long term investors.

scott_patterson12 karma

I hope that we are not going to have to hope that the next flash crash will self correct. If that event had happened later in the day, it might not have corrected. And there's nothing to guarantee that another flash crash, or worse, might happen again. Investors are justifiably concerned about the stability of the market.

[deleted]5 karma

I read this review of your book, bought it anyways, but now I can't help but think this guys was 100% right. There seems to be a lot of "ends justifies the means" logic. The review was posted on Amazon about 2 years ago, but as I read the book I kept referring back to this guys review. I think he nailed it.

he first problem with Patterson's book is that it's wrong at its core. Quant traders weren't guilty of causing the credit crisis. Some of them were victimized by it (when Lehman went bust, it took with it a bunch of money belonging to some very good, honest, and hardworking quant traders that were Lehman's prime brokerage clients). It's foolish to claim that market neutral trading, CTAs, and high frequency traders were somehow responsible for investment banks' over-leveraged, toxic balance sheets. The responsibility for this falls squarely on the shoulders of banks' managers, and perhaps also on the shoulders of free-market disciples who believe, despite all the evidence throughout history to the contrary, that regulation of human behavior is bad. The crime in this is that it dramatically changes the focus from the real source of the problem that nearly buckled our economic system--namely unchecked greed, incompetent or impotent risk managers, screwed up incentive structures, and misguided regulation--to a group of traders that people are naturally inclined to hate anyway. If Patterson's disingenuous take on the credit crisis is widely adopted, it will make for a very convenient scapegoat enabling greedy, ego-hungry Goldman Sachs execs once again to make the very same kinds of bets that (at least nearly) brought them down to begin with. Did these execs use statistics to justify their position? Sure. But to make it sound like quants are somehow responsible for the stupidity or greed of their bosses who didn't (want to?) understand the weaknesses of a model is moronic.

Another fundamental problem with this book was the arbitrariness of Patterson's use of the label "quant." Whenever it was convenient (when it sounded evil), he labeled or insinuated the activity as being quant. But math is used pretty much everywhere in finance, and it always has been. Patterson: - Treats the computation of a price-to-book ratio (P/B) as "value investing" but taking the difference in two interest rates (X minus Y) as a "quant carry trade". Why is subtraction "quant" and division "value"? Patterson also ignores the fact that the bulk of carry trading is done by discretionary traders, such as those in the global macro space. - Confuses financial engineers, derivatives experts employed by the sell-side investment banks to create products like Principal Guaranteed Notes, Collateralized Debt Obligations, and compute VaR with buy-side quant trading outfits that are simply speculating their own, or their clients', capital in the markets alongside everyone else. - Calls the belief that investors are rational a "quant theory," which is stupid. It's a basic tenet of economics and not a premise of quant trading. - Treats the efficient market hypothesis as central to quants. By definition, quant traders believe the market is at least somewhat inefficient. - Refers to capital structure arbitrage and distressed debt trading, respectively, as a though they are quant strategies. They're not. Cap structure arb is at the intersection of legal and accounting expertise. Deciding to buy a bunch of toxic assets from a company to which you already have lots of exposure (ETrade) is not a quant trade either. - Equates the move by banks to take huge risk off their balance sheets through tricky accounting practices with quants. - Somehow treats Jerome Kerveil's very plain vanilla long equity futures trade as a "complex derivatives trade," which (for the author) puts it under the heading of quant. This was a fully discretionary trade that moved markets down by 8-9% as it was unwound.* Saving the worst for last...Patterson writes: "The quants were killing Bear Stearns." This is so foolish that it should make anyone with half a brain question his integrity. Because two funds with quant trading activities withdrew their funds' capital from a brokerage house rumored to be on the brink of failing, they are somehow quants killing a bank? Are the quants who trusted Lehman (and had their money evaporate as a result) called martyrs for the cause of our financial system because they kept their capital there too long? Is it a quant model that is responsible for the manager of a fund deciding it was a matter of common sense and fiduciary responsibility to move his cash to a safer haven? What kind of nonsense is Patterson trying to peddle here? This kind of arbitrary labeling is helpful for his rhetoric, but it's also garbage. In reality, quants are no better or worse citizens of humanity than George Soros (who was responsible for breaking the Bank of England in 1992 and maybe for bringing Asian economies to the brink of collapse in 1997) or Warren Buffett.

Will you respond? Seems like the book is somewhat politically motivated.

scott_patterson5 karma

I never responded to Amazon reviews and I still don't. It's pointless to get sucked into these debates.

[deleted]4 karma

Ok, kinda figured as much....

Even though I could just reword this and ask you the same questions on this forum...which you do apparently "get sucked into." If by "sucked in" you mean make a reddit account and announcing that you will be answering questions pertaining to the book... unless of course they are questions from a person that might actually have a clue what you are writing about.

I did enjoy a fair amount of the book until it became more clear that it was a political statement.

scott_patterson3 karma

I think there is a political element, but it's more of an analysis of the quantitative revolution on markets. One part of that revolution, or a subset, is the efficient market hypothesis that has its base at the University of Chicago and specifically in the classroom of Gene Fama. I think that the free market ideology and the deregulatory wave that it inspired has hurt our markets and our economy. So sure, you can call that political.

throwaway2468104 karma

How do you think currency manipulation should be enforced? As I understand it (from the guys that build the clocks), international currency markets operate on a per-second basis, leaving a lot of time for manipulation in the sub-second realm. There is a lot of time to make fractions of a dollar in each second there remains a discrepancy in currency prices in different exchanges.

scott_patterson14 karma

I don't know much about currency markets but I've heard some chilling stories. Let's say I'm suspicious. And for some reason the US government didn't mandate that currency derivatives would be covered by Dodd Frank. Hm.

kiwi_tree4 karma

Mmh, do you think a lot of people cheat on the stock markets, or that overall it's quite well regulated ? And when you say "manipulate", is what they do legal or not ?

scott_patterson23 karma

There is right now what some people might call legalized manipulation. It's built into the system. Because these firms can react so quickly--trade times measured in a millionth of a second (or even a trillionth)--these high-speed firms can trade in ways that many people might think is totally crossing over into manipulation. For example, they can see a price of a stock move in one part of the market using high-speed data feeds and trade somewhere else that hasn't seen the price move yet. They call it "latency arbitrage." Of course they also call it making the market more efficient.

jazerac4 karma

Do you believe in secret societies/organizations that run things behind the scenes, something like the Illuminati for example.

scott_patterson18 karma

No. But I do think that there are insiders who help one another out. You scratch my back...the cynics say that's the way it's always been on Wall Street. That doesn't mean I have to like it or put up with it, especially I'm in a position to draw attention to it. Whether anyone is listening who can do anything about it remains to be seen.

JohnWad4 karma

Hey Scott. Really enjoy your work. Keep it up.

scott_patterson8 karma

Well thanks! I'm trying--and keep tuned to the pages of WSJ. There's more coming and it will be quite shocking (I think).

[deleted]3 karma


scott_patterson11 karma

Good question. The reaction is mixed but I think most of them are as you say pissed. They like operating in the shadows, they like being unregulated, and I'm saying we need to regulate this market because it's out of control.

dztmn3 karma

will we ever see a world in which trading algos cuts out human trading, and would this potentially make markets more efficient/less behavioural?

scott_patterson7 karma

We're almost there already. I'm not sure if anyone knows the actually number but most likely more than 90% of all trading on the market is conducted by algorithms. And no, I don't think totally cutting out people would be a good thing. We need people at the switch. They say the market is run by two factors: fear and greed. Robots don't have that.

_alexkane_3 karma

Do dark pools have to report their trades back to the exchanges at some point?

scott_patterson8 karma

Yes, after a trade happens they report it to the consolidated tape. What's dark are the buy and sell orders, so outsiders don't have a good sense of supply and demand. This can serve a useful purpose--it can allow big firms to execute large trades without getting picked off. But when taken too far it can basically suck the life out of the "lit" markets--information about supply and demand that helps price discovery.

questifer3 karma


scott_patterson8 karma

Either the stock market is going to reform itself from within or it will be imposed by regulators. There are some interesting ideas out there. I recently wrote an article for WSJ about a firm that's trying to put together an anti-HFT trading platform for mutual funds who are sick of getting picked off on exchanges -- you can read it here. http://online.wsj.com/article/SB10001424052702304065704577422583959529066.html

spenchey2 karma

Loved both your books, great info for the retail investor. But whats on the verge of being the next sweeping change that takes over wall street?

scott_patterson3 karma

Trading from Outer Space? I'm just trying to get caught up on what happened in the past 6 months...

I hope a sweeping change is that the regulators get a handle on the market and clean it up. Who knows maybe they will?

PeachesMcPie2 karma

Hi Scott, thanks for taking the time. Just to play devil's advocate here; Couldn't you make the argument that books like yours are somewhat sensationalist and basically contribute to the lack of faith in the marketplace and you are mostly capitalizing on the fact that fear sells books? To be honest, I think that most retail investors are disenfranchised by the market because their savings were down 40% at one point after the credit crisis, not because block orders are getting sniffed out by algos. If some 'expert' on CNBC is screaming that mom and pop should be worried about their savings because of robots, thats probably going to get a lot of people worked up about a problem that doesn't really affect them (except maybe flash crash, which seems to be basically what a lot of this discussion leverages).

For example, it seems that all the people over at Themis Trading who are so noisy (along with dinosaurs in mutual funds that aren't really adding any value to begin with), are just trying to protect their turf.

Besides the exception of the self-correcting flash crash, in the last 10 years markets have become more liquid, beta exposure through ETFs has become far cheaper than paying fees to a mutual fund, and its cutting out a lot of middlemen, many of whom provide no additional value or stability. The problems you address in terms of lack of liquidity for institutions due to market fragmentation seems to mostly be a burden for unsophisticated players that probably won't be around for much longer anyway due to the natural evolution of the marketplace. And hey, maybe they still do manage 20 billion dollars today and can afford to lobby, but the point is that to some extent we need the old to die off to make room for the new (maybe some fund managers who don't just collect assets and fees)?

scott_patterson3 karma

Fair question. When I started reporting on HFT about 3/4 years ago I didn't have much of a concern about it. But the more I've learned, the most concerned I gotten. I've been inside some of the largest high-frequency firms in the world, spoken with designers of strategies, top exchange officials, creators of the systems that run the exchanges. And I'm hear to tell you that there's something amiss in our markets. A technology arms race has taken over the market and I find it hard to believe that a firm gaining a 10 microsecond edge over another is going to add to the efficiencies. How long is that race going to go on? I concede that efficiencies have been added (though on some points that's debatable) but that doesnt mean that we can't take a long hard look at our market today and look for cracks, and try to do something about it.

curiousfinguy111 karma

Alright, I'm going to be a jerk and ask this.

There's a passage in The Quants where you are explaining the Law of Large numbers and you write

Ten flips of a coin could produce seven heads and three tails, 70 percent heads, 30 percent tails. But 10,000 flips of a coin will always produce a ratio much closer to 50-50.

This is the negation of the Law of Large Numbers. In fact, you italicized the part that it makes it wrong. 10,000 flips of a coin could most certainly wind up being further from the expected outcome than 10 flips.

Maybe this was just an uncaught error, but the question my friends and I all ask is how can we trust someone who appears not to understand one of the most basic results of statistics to explain a complicated financial system to us?

scott_patterson3 karma

This is the Law of Large Numbers. It's simply incorrect to say that 10,000 flips of a coin would be further from the expected outcome (50-50) than 10 flips.

cawcawparrot0 karma

I'm really sorry, but I work in the high frequency space, and you're totally off the mark. Your answers are wrong, and you speak in generalities and hyperbole.

I'm not sure if it's due to ignorance or because you'd rather shout that the sky is falling to sell more books. Nothing personal, and I don't fault you either way.

scott_patterson3 karma

Usually when I get a criticism like this I ask people to please be specific. What is wrong?

This is a big complex subject with many angles. I don't work in the HFT space, I'm a reporter who is writing about HFT and its impact on markets. I've spoken with many people on all sides, from the Bible thumbing believers that HFT is how Jesus would trade to the most jaded traders who think it has ruined the market forever. I'm not on either extreme but I do strongly believe that the system has evolved in a way that our regulators can't monitor it anymore, it poses systemic risks, and investors are at times getting hurt by some practices.

What's wrong?