Iam Scott Patterson, staff reporter for Wall Street Journal and New York Times bestselling author of “The Quants” and “Dark Pools”. I write about the government’s regulation of Wall Street, and the technology used to manipulate it. AMA

When you interviewed Mark Cuban about the problems with high-frequency trading you asked him “What’s the solution?”

How would you respond to the same question?

The regulators need to get on top of what's going on and fast. The public is losing trust in the markets and right now our regulators can't give them any assurances that they're wrong. Right now we just don't know and our regulators don't either -- that's why the market is dark.

Aren't the regulators just people from the financial houses taking a break? SEC is all about arbitration and not prison time, no? Fox guards the henhouse, etc.

I think there are some good well-intentioned people at the SEC and other regulators. But they're totally outgunned. And the financial incentives just aren't there. If you have the technical ability to understand much of what drives our markets, you're more than likely going to cash in for a multi-million dollar paycheck on Wall Street than work in the SEC. Sure, it's always been like that. But because the market is so technical, so computer-driven now, that our regulators need to have that technological literacy or they can't surveille the market.

Bottom line: The SEC needs to hire more quants and fewer lawyers.

"The SEC needs to hire more quants and fewer lawyers."

Only if they want to manage what may be an unmanageable problem. The truth will always be that if a regulator understand the market, they could easily multiply their salary by a factor of at least 2, and typically more like 10, by going to industry. The tide won't go back, shout as Canute may. (I worked at an I-Bank for a while, and the CFO, when asked about the auditors understanding of our complex transactions, was quoted as saying something along the lines of "I wish our auditors understood our business well enough to audit it - because then I could hire them for 10 times what they are being paid. We desperately need those people.")

Regulatory capture and the inability to regulate to to political pressure is the biggest issue facing any such body - and not to change subjects, but this is a campaign finance and income inequality issue.

"I wish our auditors understood our business well enough to audit it - because then I could hire them for 10 times what they are being paid. We desperately need those people."

Asking out of ignorance; What skill set in particular is he referring to here? What do those people need to do to understand the business?

edit: Thanks for the enlightening responses, Gents.

They need to understand the complex, computer-driven financial products and strategies that run Wall Street today. And too often they don't. This causes many people on Wall Street to look down on the regulators and even skoff at them. A bad situation for everyone.

Is there a possibility of knowing what is going on if the markets remain fully automated, and computerized trading can go on at speeds that make the data processing to understand them so difficult? How can regulators understand a market with arbitrarily complex, constantly changing automatic program running on them?

Do changes need to be made first?

They simply need to get the computer fire power to monitor the market, and it exists. They just haven't done it yet. There's a proposal to build a so-called Conoslidated Audit Trail that could help solve some of these problems. I wrote about it for WSJ last year: http://online.wsj.com/article/SB10001424053111904491704576574883908453622.html

I'm suspicious about this concept have having a government computer monitor an automated market. As long as it's a bunch of computers operating based on set rules, won't there be room to invent new exploits that take advantage of those rules without being detected?

Are you suspicious of cops using radar to catch speeders on the highway? Because right now the SEC doesn't have that radar.

Well no, sorry for not being clear. I'm not saying I'm suspicious of the government, but I'm suspicious of the idea. Essentially my worry is that if the government computer is only searching for certain things, then someone might figure out what that computer is searching for and be able to enact manipulation that doesn't set off any alarms.

I'd be all in favor of the government having the sort of infrastructure you suggest, assuming it would be helpful to prevent manipulation. I just suspect that it wouldn't be sufficient, and would instead lead to an arms race of "who has more clever computer programmers."

But to answer you're question, yes, I'm suspicious of cops.

Well, to be honest I am too...

But they do serve a role, right? And my concern is that the regulators of our market simply don't have that radar detector, and firms know that and are able to exploit it.

As far as HFT firms figuring out what the computer is looking for and looking for loopholes, that is certainly a risk. But I think that any system like this would be an AI pattern recognition program that would constantly be looking for new patterns. It wouldn't be cookie cutter. I'm sure IBM would love the business...

What about "front running"?

It used to be crime. Now it's called "co-location" and is, apparently, perfectly legal when big banks do it. If a little trader front runs, he gets crushed by regulators. I've had a trader tell me this. That it's illegal for him; legal for the big boys.

Not so sure about this. Most firms today are colocated to it's not as if anyone has a significant advantage. I think colocation is a red herring.

One area people should look into more--and that I cover in my book Dark Pools--is the use of order types. There are many many order types today (limit orders are child's play) that give firms unique ways to thread their orders through the exchang matching engines. We've reported that the SEC is looking into this: http://online.wsj.com/article/SB10001424052702304636404577297840134760650.html

.Even with all of the data, and having every order fully tracked, the complexity issue is still a problem - it is easier to create a complex program than to understand one - and the HFT traders will be spending arbitrarily large sums on creating such programs. How can regulators attempt to understand a system if the competitive advantage of many of the trading systems is that they are opaque to outside observers?

(I won't even get into issues with only tracking completed orders, and having unmonitored dark pools, derivative replication of equities, and side payments that cannot be included since they occur off the exchanges, in private party transactions.)

You're right, it's very hard. But they have to make an effort. Another potential solution could be alterations that cut down on the level of orders, such as mandatory holding periods of one second. I'm not sure that will work, though.

Honestly, how confident are you that this could actually happen? Can the public help? Or is reform just a faint hope?

I'm not confident, but I'm hopeful. I'm hopeful that my book might help trigger enough outrage that they're forced to do it. But it's a hard fight because the industry has all the money and the lobbyists. Regulation is a dirty word on Wall Street.

[deleted]

I'm not surprised that the NYSE went for this because they've been getting taken to the cleaners by the firms that are allowed to do this. So it doesn't really change the overall structure of the market that much. I can say that I was surprised that the SEC approved it. It means more fragmentation--and we already have about 70 venues to trade stocks today. Isn't that enough? Or I mean far far too much?

[deleted]

I would think that simply changing the execution of trades to occur at the end of every minute would eliminate the false price problem. This would destroy HFT completely, but would also create new transparency in the markets.

Maybe. This is the long-running debate over the different benefits of continuous trading and auctions in which price discovery happens periodically (a bunch of traders put in orders and the exchange spits out an average). The trouble with auctions is that it can lead to reduced liquidity--market makers are going to be reluctant to participate because they will have more trouble controlling their risk. Continuous trading lets them get out when they need to, so they're more willing to play ball. Of course, that's the problem isn't it? Is the risk worth the reward?

I'm not fully up to date on modern economic mechanism design, but I'm aware of the issues. Continuous markets are not, as far as I am aware, cannot be even theoretically incentive compatible or strategy proof - something that even having an auction every second would create.

Continuity of market prices is an illusion, based on economic assumptions that are demonstrably false. The increased liquidity has helped exactly one set of participants in the market - large firms that collect the fees, and do the automated trading. Economic policies that enrich these firms at the expense of retails investors and the physical economy seems like a strange thing to favor.

And if a firm needs sub-second liquidity in an asset in order to hedge thir exposure, it's because they are taking large risks that cannot be hedged in the market directly with derivatives. A firm with $1b in equity that has offsetting +/- $1 trillion in liabilities and assets is probably dynamically hedging themselves - liquidity masks this instability until a crash happens. Aren't these firms bad for economic stability? Wouldn't the economy be better off without them?

One issue is that the auction mechanism isn't seen as the optimal system by HFT firms, for obvious reasons. So you'd need to mandate the one-second holding period, because the market isn't going choose it, and I don't see the SEC ever doing anything like that. At the end of the day it's an interesting theory but ain't going to happen. The SEC in many ways has encouraged the modern-day high-speed market structure.

Dark pools are in many ways a symptom of the disease. The alternative is to clean up the exchanges. Big firms are moving their orders away from the "lit" markets because they're concerned about gaming (ie front running) on the exchanges by high-speed firms. Sadly, the exchanges have played into this game because they're competing with one another to attract the flow of the high-speed firms. To do so, they offer them advantages. One source for my book who worked at an exchange told me that they provided HFTs "guaranteed economics." If found the phrase very telling--and disturbing.

Do you think the markets still serve their original purpose, to allow people to invest in the longterm wellbeing/health of an organization, or do you think it's strictly now about fast and easy money for the already wealthy and powerful?

If you lean towards the latter, how do you think we can correct it?

Would a completely non-automated secondary market be feasible?

That is and should be the purpose of the market--to allow companies to raise money so that they can expand (and hire people) and let people get a chunk of their growth and profits. But the market has turned into a casino. And big shock--the computer revolution thas has taken over the market grew out of so-called SOES bandits, day traders riding stocks for short term gains. Check out Dark Pools for the whole story--no one has told it before. It's totally fascinating! (Sorry, had to get in a random book plug....)

Every player whether speculating or investing brings liquidity to the market, which is a good thing. To call it a "casino" seems a simplification of the various motives of the players in the market.

The market has always has been casino-like. But it's increasingly so today. The liquidity is driven by HFT firms and quant hedge funds who are doing nothing more than betting on patterns--just like a blackjack player would in Las Vegas. These guys have always been around--they are just more dominant today (at least that how it seems to me).

Scott, I just read the excerpt and it looks like a fascinating subject - I live in NYC and a few of my friends are software engineers who help create the tools used in high frequency trading among other things.

My question is, do you think the stock market is a "safe" place for regular individual investors to be in the long term?

This is complicated. Because public pension funds are no longer available to most employees in America, we have little choice but to default to the stock market for a least part of our retirment savings. The 401K plan has basically been forced upon Americans as the only way to plan for our retirements. So what choice do we have? Some of our money needs to go into the stock market. But the Flash Crash of May 6, 2010, when the market tanked 10% in a matter of minutes, was a serious warning signal that the market isn't stable, it's dangerously unstable. And it's getting worse.

[deleted]

Well, we don't have much choice to we. But I'm trying to say that the market has changed in many ways, dramatically. This isn't the same market that we've had over the past 80 years.

[deleted]

I'm concered that the average investor in a mutual fund isn't getting a fair shake because the guy doing the trading FOR THE MUTUAL FUND can't compete with the HFTs. Those guys are getting taken to the cleaners and investors are left holding the bag. Even traders for giant hedge funds tell me that they can't do their jobs anymore because the market is so toxic and fragmented and complex.

[deleted]

I'd rather have a system in which the mutual funds can trade on the public markets safely and fairly rather than sell their order flow to the likes of Knight.

Maybe your next book can be about the LIBOR scandal.

What's your take on this? Do you ultimately think it will come out that every bank that contributes rate information was involved? Will we finally start seeing these assholes go to jail?

I mean, I thought writing a book about market microstructure was taking a leap...but a book about LIBOR?

Not that it's not a gripping story. I'm not an excerpt on the subject but this is just another example of how the global financial system often operates in the dark. I'm trying to shine a light on some of those dark practices.

How can we regulate against the risk of people (bankers, politicians,...) being assholes? I feel it's the major issue nowadays...

I mean, you can always try to prevent discrete misconduct on a case by case basis, but wouldnt it be more efficient to eliminate or mitigate the root cause?

This is essentially my question as well - is there any hope for a fair system that disincentivizes greed.

Greed and fear run the market. But we need strong regulators to keep it in check. But the dramatic technological evolution in the markets in the past decade has left the regulators in the dust. I think they hoped that with the robots in charge they could just sit back and relax--but the robots are at the end of the day doing the bidding of humans. Often very greedy ones.

How has being a NYT bestseller changed your life? Did you imagine that your work would be so wildly successful?

Good question Mom.

So how awesome is Jon Stewart in person?

Pretty awesome, and smart. His brother Larry Leibowitz, who I know and is a top executive at the New York Stock Exchange, is also a really smart guy--and funny too.

[deleted]

He actually read it! Unlike most of the people who interviewed me. He's even more skeptical of what's going on on Wall Street than me. And I'm pretty damn skeptical.

While I wholly agree on the problem of HFT when it comes to flash crashes - and am in favor of volatility regulation as an attempt to control panic - I see it as a losing battle. Technology is allowing people to realize, and act upon, negative news all at once.

I feel that everyone (mainstream economics) is beginning to accept the fact that the market is NOT truly random. I am a huge proponent of behavioral economics - hence my position as a boutique trader. The advance of technology has greatly increased investor sentiment as a major influence of the markets, rather than simply fundamentals.

Much of HFT (what you described as latency arbitrage) I don't have a problem with. Many people see it as a problem because as of now, it's a certain bunch making the money off of this. The firms are not misleading the ordinary investors in day-to-day situations. They've simply gotten on top of said technological advances.

My problem is with the regulators themselves. For the passed few years - first America, and now much of Europe - has relied upon satisfying short term demand of investors rather than actually fixing the problem. In my opinion much of this is a result of the nature of political systems and the desires of those to get re-elected. The same goes for a large amount of wealth managers - that sector is based upon beating the average, so it doesn't matter if the average sucks - as long as the unedcated clients with money are happy to be doing better than everyone else.

So personally, I don't see the firms who are profiting for themselves and their client - through (as of now) legal means, as a problem. They aren't misleading anybody.

Am I crazy? Should I read your book to get proven wrong? Thanks a bunch for providing easily one of the most interesting AMAs from my point of view.

Great question(s). Fund managers are completly focused on beating the averages, and end up tracking it themselves. So why not just invest in a cheap index fund? But I also think that the fascinating world (to me) of market microstructure is increasingly playing a macro role in investor returns and investor confidence. People don't trust it anymore and that's very worrisome. And the regulators can't provide assurances that their lack of confidence is justified. It's a dark market.

In a recent article, you quoted Sen. Ted Kaufman as saying, "Human beings were no longer doing the trading, computers were. They developed these algorithms. It ran automatically. It grew and grew. There is no way to know what is going on. No one knows what is happening in these exchanges when this trading is going on. We have a very dangerous situation."

Are the individuals initiating this high-frequency trading aware of what the potential for damage could be, or do they simply not care?

I don't think they think about the damage. In fact I believe many of them believe that they're helping the market. Any maybe they are--some of them. My concern is that this one species of trader has become so dominant that the present a systemic threat to the market. Kind of like a pool (a dark one of course) in which the only living creatures left are the sharks. Then what do the sharks eat?

[deleted]

First, thanks for the compliment on the Island chapters--I loved writing about it. Josh Levine (founder of Island) is a fascinating person. Second, I think my book is the best book on the market about modern microstructure that everyday people can read and learn from. So we plainly disagree! It's a controversial topic and if everyone agreed with me it would have been a really boring book.

I think i heard in my computer architecture course that a lot of effort goes into making sure the computers running the programs to deal with trading are as close as possible to each other to reduce data transfer times by some stupidly small amount of time. How true is this and is that effort being done because of a lack of the ability to increase times in processors and software?

This is absolutely true and is in fact how the market largely works today. The technical term is "colocation"--trading firms "colocate" with the exchange's matching engine (where buy and sell orders match up). And guess what--I could care less about this. For a couple reasons. One is, it's not going to change, it's the march of progress, etc. And it's become so dominant that everyone does it. Your mutual fund trades through a colocated server at a data center in New Jersey. That just the way it is. There are better things to get concerned about.

The May 6th Flash Crash seems to be a major point of concern for the "average investor". But given that it corrected itself so quickly, weren't the only one's hurt in this those who did not have long term positions (ie the HFT/Day traders)?

When people say that crashes like this make the public lose trust in the markets, isn't that just as much an indictment of financial reporting than of HFT trading strategies?

Many HFT firms actually made a large amount of money on the Flash Crash.

But that's beside the point. The Flash Crash revealed a major flaw in the market structure, and one that hasn't gone away. The new liquidity providers--HFT firms--cut and run when the market gets too chaotic. So when the market needs them most, they aren't there.

If many HFT firms made a large amount of money, isn't it true that many lost?

Also, this doesn't get to my major question. How does a flash crash that self corrects quickly hurt the "average investor" by which I assume you mean long term investors.

I hope that we are not going to have to hope that the next flash crash will self correct. If that event had happened later in the day, it might not have corrected. And there's nothing to guarantee that another flash crash, or worse, might happen again. Investors are justifiably concerned about the stability of the market.

I read this review of your book, bought it anyways, but now I can't help but think this guys was 100% right. There seems to be a lot of "ends justifies the means" logic. The review was posted on Amazon about 2 years ago, but as I read the book I kept referring back to this guys review. I think he nailed it.

he first problem with Patterson's book is that it's wrong at its core. Quant traders weren't guilty of causing the credit crisis. Some of them were victimized by it (when Lehman went bust, it took with it a bunch of money belonging to some very good, honest, and hardworking quant traders that were Lehman's prime brokerage clients). It's foolish to claim that market neutral trading, CTAs, and high frequency traders were somehow responsible for investment banks' over-leveraged, toxic balance sheets. The responsibility for this falls squarely on the shoulders of banks' managers, and perhaps also on the shoulders of free-market disciples who believe, despite all the evidence throughout history to the contrary, that regulation of human behavior is bad. The crime in this is that it dramatically changes the focus from the real source of the problem that nearly buckled our economic system--namely unchecked greed, incompetent or impotent risk managers, screwed up incentive structures, and misguided regulation--to a group of traders that people are naturally inclined to hate anyway. If Patterson's disingenuous take on the credit crisis is widely adopted, it will make for a very convenient scapegoat enabling greedy, ego-hungry Goldman Sachs execs once again to make the very same kinds of bets that (at least nearly) brought them down to begin with. Did these execs use statistics to justify their position? Sure. But to make it sound like quants are somehow responsible for the stupidity or greed of their bosses who didn't (want to?) understand the weaknesses of a model is moronic.

Another fundamental problem with this book was the arbitrariness of Patterson's use of the label "quant." Whenever it was convenient (when it sounded evil), he labeled or insinuated the activity as being quant. But math is used pretty much everywhere in finance, and it always has been. Patterson: - Treats the computation of a price-to-book ratio (P/B) as "value investing" but taking the difference in two interest rates (X minus Y) as a "quant carry trade". Why is subtraction "quant" and division "value"? Patterson also ignores the fact that the bulk of carry trading is done by discretionary traders, such as those in the global macro space. - Confuses financial engineers, derivatives experts employed by the sell-side investment banks to create products like Principal Guaranteed Notes, Collateralized Debt Obligations, and compute VaR with buy-side quant trading outfits that are simply speculating their own, or their clients', capital in the markets alongside everyone else. - Calls the belief that investors are rational a "quant theory," which is stupid. It's a basic tenet of economics and not a premise of quant trading. - Treats the efficient market hypothesis as central to quants. By definition, quant traders believe the market is at least somewhat inefficient. - Refers to capital structure arbitrage and distressed debt trading, respectively, as a though they are quant strategies. They're not. Cap structure arb is at the intersection of legal and accounting expertise. Deciding to buy a bunch of toxic assets from a company to which you already have lots of exposure (ETrade) is not a quant trade either. - Equates the move by banks to take huge risk off their balance sheets through tricky accounting practices with quants. - Somehow treats Jerome Kerveil's very plain vanilla long equity futures trade as a "complex derivatives trade," which (for the author) puts it under the heading of quant. This was a fully discretionary trade that moved markets down by 8-9% as it was unwound.* Saving the worst for last...Patterson writes: "The quants were killing Bear Stearns." This is so foolish that it should make anyone with half a brain question his integrity. Because two funds with quant trading activities withdrew their funds' capital from a brokerage house rumored to be on the brink of failing, they are somehow quants killing a bank? Are the quants who trusted Lehman (and had their money evaporate as a result) called martyrs for the cause of our financial system because they kept their capital there too long? Is it a quant model that is responsible for the manager of a fund deciding it was a matter of common sense and fiduciary responsibility to move his cash to a safer haven? What kind of nonsense is Patterson trying to peddle here? This kind of arbitrary labeling is helpful for his rhetoric, but it's also garbage. In reality, quants are no better or worse citizens of humanity than George Soros (who was responsible for breaking the Bank of England in 1992 and maybe for bringing Asian economies to the brink of collapse in 1997) or Warren Buffett.

Will you respond? Seems like the book is somewhat politically motivated.

I never responded to Amazon reviews and I still don't. It's pointless to get sucked into these debates.

Ok, kinda figured as much....

Even though I could just reword this and ask you the same questions on this forum...which you do apparently "get sucked into." If by "sucked in" you mean make a reddit account and announcing that you will be answering questions pertaining to the book... unless of course they are questions from a person that might actually have a clue what you are writing about.

I did enjoy a fair amount of the book until it became more clear that it was a political statement.

I think there is a political element, but it's more of an analysis of the quantitative revolution on markets. One part of that revolution, or a subset, is the efficient market hypothesis that has its base at the University of Chicago and specifically in the classroom of Gene Fama. I think that the free market ideology and the deregulatory wave that it inspired has hurt our markets and our economy. So sure, you can call that political.

How do you think currency manipulation should be enforced? As I understand it (from the guys that build the clocks), international currency markets operate on a per-second basis, leaving a lot of time for manipulation in the sub-second realm. There is a lot of time to make fractions of a dollar in each second there remains a discrepancy in currency prices in different exchanges.

I don't know much about currency markets but I've heard some chilling stories. Let's say I'm suspicious. And for some reason the US government didn't mandate that currency derivatives would be covered by Dodd Frank. Hm.

Mmh, do you think a lot of people cheat on the stock markets, or that overall it's quite well regulated ? And when you say "manipulate", is what they do legal or not ?

There is right now what some people might call legalized manipulation. It's built into the system. Because these firms can react so quickly--trade times measured in a millionth of a second (or even a trillionth)--these high-speed firms can trade in ways that many people might think is totally crossing over into manipulation. For example, they can see a price of a stock move in one part of the market using high-speed data feeds and trade somewhere else that hasn't seen the price move yet. They call it "latency arbitrage." Of course they also call it making the market more efficient.

Do you believe in secret societies/organizations that run things behind the scenes, something like the Illuminati for example.

No. But I do think that there are insiders who help one another out. You scratch my back...the cynics say that's the way it's always been on Wall Street. That doesn't mean I have to like it or put up with it, especially I'm in a position to draw attention to it. Whether anyone is listening who can do anything about it remains to be seen.

Hey Scott. Really enjoy your work. Keep it up.

Well thanks! I'm trying--and keep tuned to the pages of WSJ. There's more coming and it will be quite shocking (I think).

[deleted]

Good question. The reaction is mixed but I think most of them are as you say pissed. They like operating in the shadows, they like being unregulated, and I'm saying we need to regulate this market because it's out of control.

will we ever see a world in which trading algos cuts out human trading, and would this potentially make markets more efficient/less behavioural?

We're almost there already. I'm not sure if anyone knows the actually number but most likely more than 90% of all trading on the market is conducted by algorithms. And no, I don't think totally cutting out people would be a good thing. We need people at the switch. They say the market is run by two factors: fear and greed. Robots don't have that.

Do dark pools have to report their trades back to the exchanges at some point?

Yes, after a trade happens they report it to the consolidated tape. What's dark are the buy and sell orders, so outsiders don't have a good sense of supply and demand. This can serve a useful purpose--it can allow big firms to execute large trades without getting picked off. But when taken too far it can basically suck the life out of the "lit" markets--information about supply and demand that helps price discovery.

[deleted]

Either the stock market is going to reform itself from within or it will be imposed by regulators. There are some interesting ideas out there. I recently wrote an article for WSJ about a firm that's trying to put together an anti-HFT trading platform for mutual funds who are sick of getting picked off on exchanges -- you can read it here. http://online.wsj.com/article/SB10001424052702304065704577422583959529066.html

Loved both your books, great info for the retail investor. But whats on the verge of being the next sweeping change that takes over wall street?

Trading from Outer Space? I'm just trying to get caught up on what happened in the past 6 months...

I hope a sweeping change is that the regulators get a handle on the market and clean it up. Who knows maybe they will?

Hi Scott, thanks for taking the time. Just to play devil's advocate here; Couldn't you make the argument that books like yours are somewhat sensationalist and basically contribute to the lack of faith in the marketplace and you are mostly capitalizing on the fact that fear sells books? To be honest, I think that most retail investors are disenfranchised by the market because their savings were down 40% at one point after the credit crisis, not because block orders are getting sniffed out by algos. If some 'expert' on CNBC is screaming that mom and pop should be worried about their savings because of robots, thats probably going to get a lot of people worked up about a problem that doesn't really affect them (except maybe flash crash, which seems to be basically what a lot of this discussion leverages).

For example, it seems that all the people over at Themis Trading who are so noisy (along with dinosaurs in mutual funds that aren't really adding any value to begin with), are just trying to protect their turf.

Besides the exception of the self-correcting flash crash, in the last 10 years markets have become more liquid, beta exposure through ETFs has become far cheaper than paying fees to a mutual fund, and its cutting out a lot of middlemen, many of whom provide no additional value or stability. The problems you address in terms of lack of liquidity for institutions due to market fragmentation seems to mostly be a burden for unsophisticated players that probably won't be around for much longer anyway due to the natural evolution of the marketplace. And hey, maybe they still do manage 20 billion dollars today and can afford to lobby, but the point is that to some extent we need the old to die off to make room for the new (maybe some fund managers who don't just collect assets and fees)?

Fair question. When I started reporting on HFT about 3/4 years ago I didn't have much of a concern about it. But the more I've learned, the most concerned I gotten. I've been inside some of the largest high-frequency firms in the world, spoken with designers of strategies, top exchange officials, creators of the systems that run the exchanges. And I'm hear to tell you that there's something amiss in our markets. A technology arms race has taken over the market and I find it hard to believe that a firm gaining a 10 microsecond edge over another is going to add to the efficiencies. How long is that race going to go on? I concede that efficiencies have been added (though on some points that's debatable) but that doesnt mean that we can't take a long hard look at our market today and look for cracks, and try to do something about it.

Alright, I'm going to be a jerk and ask this.

There's a passage in The Quants where you are explaining the Law of Large numbers and you write

Ten flips of a coin could produce seven heads and three tails, 70 percent heads, 30 percent tails. But 10,000 flips of a coin will always produce a ratio much closer to 50-50.

This is the negation of the Law of Large Numbers. In fact, you italicized the part that it makes it wrong. 10,000 flips of a coin could most certainly wind up being further from the expected outcome than 10 flips.

Maybe this was just an uncaught error, but the question my friends and I all ask is how can we trust someone who appears not to understand one of the most basic results of statistics to explain a complicated financial system to us?

This is the Law of Large Numbers. It's simply incorrect to say that 10,000 flips of a coin would be further from the expected outcome (50-50) than 10 flips.

I'm really sorry, but I work in the high frequency space, and you're totally off the mark. Your answers are wrong, and you speak in generalities and hyperbole.

I'm not sure if it's due to ignorance or because you'd rather shout that the sky is falling to sell more books. Nothing personal, and I don't fault you either way.

Usually when I get a criticism like this I ask people to please be specific. What is wrong?

This is a big complex subject with many angles. I don't work in the HFT space, I'm a reporter who is writing about HFT and its impact on markets. I've spoken with many people on all sides, from the Bible thumbing believers that HFT is how Jesus would trade to the most jaded traders who think it has ruined the market forever. I'm not on either extreme but I do strongly believe that the system has evolved in a way that our regulators can't monitor it anymore, it poses systemic risks, and investors are at times getting hurt by some practices.

What's wrong?