For more than a decade, a training program known as 911 call analysis and its methods have spread across the country and burrowed deep into the justice system. By analyzing speech patterns, tone, pauses, word choice, and even grammar, practitioners believe they can identify “guilty indicators” and reveal a killer.

The problem: a consensus among researchers has found that 911 call analysis is scientifically baseless. The experts I talked to said using it in real cases is very dangerous. Still, prosecutors continue to leverage the method against unwitting defendants across the country, we found, sometimes disguising it in court because they know it doesn’t have a reliable scientific foundation.

In reporting this series, I found that those responsible for ensuring honest police work and fair trials — from police training boards to the judiciary — have instead helped 911 call analysis metastasize. It became clear that almost no one had bothered to ask even basic questions about the program.

Here’s the story I wrote about a young mother in Illinois who was sent to prison for allegedly killing her baby after a detective analyzed her 911 call and then testified about it during her trial. For instance, she gave information in an inappropriate order. Some answers were too short. She equivocated. She repeated herself several times with “attempts to convince” the dispatcher of her son’s breathing problems. She was more focused on herself than her son: I need my baby, she said, instead of I need help for my baby. Here’s a graphic that shows how it all works. The program’s chief architect, Tracy Harpster, is a former cop from Ohio with little homicide investigation experience. The FBI helped his program go mainstream. When I talked to him last summer, Harpster defended 911 call analysis and noted that he has also helped defense attorneys argue for suspects’ innocence. He makes as much as $3,500 — typically taxpayer funded — for each training session. 

Here are the stories I wrote:

If you want to follow my reporting, text STORY to 917-905-1223 and ProPublica will text you whenever I publish something new in this series. Or sign up for emails here.  

Comments: 571 • Responses: 16  • Date: 

evilleppy87886 karma

If lie detector tests are inadmissible in court, how in the hell would they be able to use this? At least with a lie detector you have the subject in a more controlled environment, but false positives are still a huge problem. The way someone said something sounds like a lie!? That's ripe for misinterpretation and abuse.

propublica_541 karma

Great question — one that bugged me early in the reporting too. There are a couple reasons 1. prosecutors sidestep the hearings meant to determine the admissibility by disguising 911 call analysis as anything but "science"; 2. it's relatively new and unknown, unlike lie detectors, so it's less easy to identify for judges and defense lawyers; 3. it can seem very much like regular, lay testimony instead of expert opinion so some judges have let it in. In emails, prosecutors have laid out these sort of loopholes for getting 911 call analysis into trial via testimony from students trained in it

tenodera589 karma

Do you think this makes 911 even more dangerous for neurodiverse people?

I'm autistic, and I spend a lot of time worrying about phrasing things just right so I get my point across. Cops terrify me, because they have expectations for eye contact and facial expressions that I do not always meet.

I think a lot about the stories of autistic people killed by police. I'm "high functioning" but that means nothing sometimes.

propublica_536 karma

Hey great question. This was a huge concern from the linguists and pychologists I consulted with. They said practitioners have to be extra careful when applying any academic research that purports there are "right" and "wrong" ways to speak. Especially true in a high stakes setting like a murder investigation/trial. The original 911 call analysis study I saw no mention of autistic folks being factored in. Since we've published, I've heard from many neurodiverse folks who said just that: the criteria used to identify a "guilty" caller looks a lot like the way I just speak.

DepartmentofNothing331 karma

What, if any, structural incentives are leading to use of this technique? And how successful or unsuccessful is it in court thus far, given how ridiculous it seems?

Love ProPublica, keep up the good work!

propublica_336 karma

Good question. The structural incentives are baked into the pitch: the training will let 911 operators know if they are talking to a murderer, give detectives a new way to identify suspects, and arm prosecutors with evidence they can exploit at trial. Students who take the class then bring what they've learned to the real world, apply it to a case and, often, tell Harpster (the founder) about how they used it. Those testimonials are then used as more marketing. It's a feedback loop.

Police leaders and district attorneys will listen to their employees' positive reviews and invite Harpster back to speak again. One thing I learned in the reporting is that those reviews are really powerful. That's why conferences host him too: people really like him and the training.

The court question is a tricky one. I don't have enough data to say whether it's more often successful or not in court. That said, we found several cases where a student of Harpster's — usually a detective or dispatcher — testified to their analysis of someone's 911 call and then that someone was convicted. Some judges, like that one I cited in Nevada, wouldn't allow the testimony. But it's often slipping in, largely because of the way it's been disguised as lay opinion, as one expert put it. Sometimes, and this is rare, it's getting in as actual expert testimony. (See the Riley Spitler example from the story.)

ShutYourDumbUglyFace257 karma

I guess my question is, how familiar are lawyers with this phenomenon and the junk science behind it, such that they would be able to provide a sufficient defense? Do juries tend to believe prosecution experts more than defense experts? And what should any of us do if we find ourselves targeted in this way?

propublica_262 karma

Hey great questions. They are not at all familiar about it, which was super surprising to me. Even in the counties where I knew police had taken the training. A lot of them have reached out since and told me they'll now be on the lookout. Some defense attorneys have learned about it in the court room for the first time – they didn't know a detective or dispatcher was going to testify about "guilty indicators" because the prosecutors didn't offer them as experts.

On the juries question, I'm not sure. I don't have enough data to say they who they put stock into and who they don't. The NAS report I discuss briefly in the story gets into how judges seldom restrict experts offered by prosecutors, which I think may play a part. Riley Spitler —the teenager who was convicted of murdering his brother before that was overturned — believed the detective who testified about 911 call analysis had much more authority in the eyes of the jury than he did. "I was just a kid," he said.

Atascosa120 karma

Something I think a lot about since the pandemic began is just how exhausting news like this can be. I’ve seen public health, politics, and STEM take some heavy hits during the pandemic, to say nothing of shared morals and mental health.

In this case, I find myself wondering about your effort to unearth the truth, and the mental energy and gymnastics required to stay focused, happy, and positive. We all benefit from the work people like you undertake, but seldom spare much thought for the wear and tear it potentially exerts upon each of you. And so, I’m curious - how do you process all that you have found/learned? Or rather, how has what you’ve learned of this program leading up to this moment affected you, and what do you usually do about it?

propublica_102 karma

Thanks for this. Very considerate. I'm happy as a clam with the work, which feels worthwhile most of the time. Not to say there aren't those listless days. And given the subject matter, it can be exhausting and depressing. I like to take walks and cook! ProPublica has a great culture and supports journalists taking personal time when we need it for all the reasons you raised. I also learned a while ago that however hard reporting these stories might be, it is/was much harder for the sources who lived it. So that perspective helps too, I think.

AnotherHiggins107 karma

On a scale from 1 to David Duke, how racist has the implementation been, and why is that number so high?

propublica_171 karma

I wasn't able to collect enough to data to say whether or not this was disproportionally impacting certain groups. BUT it was the primary concern of almost every expert I spoke with. The idea that you can prescribe what a 911 caller should say when reporting an emergency — after listening to 100 calls, mostly from white callers and mostly from callers from Ohio — is absurd, they said. Dialects, geography, race, education, all of these things factor into the way we speak. A 911 caller from the Midwest may very well have a much different way of communicating than someone from Mississippi

Salt_Savings_655888 karma

There's been a lot of junk science used by the cops. Blood spatter. Hair Analysis. Now this. How does this junk science keep getting through all the systems which are designed to prevent it?

propublica_119 karma

Hey. This is a good question and one we tried to address in the stories best we could. The guardrails that are in place — training boards, state supreme courts, local agencies that host instructors — didn't really do any type of vetting. There wasn't a scientific review or, from what I saw, basic questions about the program. For example, the architect of the program told all these agencies that 30 percent of people who call 911 to report a death are actually the murderer. I found no evidence to back that up. But I didn't see anyone ever question it.

Tron_Little60 karma

To what degree do you think the proliferation of this technique is the product of good salesmanship? It seems like the guy responsible for the training stands to make some good money from its implementation. Is he just a good salesman? Or do you think there is culpability/willful ignorance on the other side of the equation, where police see this as an opportunity to increase their chances of getting a guilty verdict when their 'gut' tells them someone is guilty but they can't find a way to prove it?

propublica_75 karma

Hey this is a really good question but I don't have a good answer. Based on my reporting, I would say both. Harpster is indeed a good salesman and his pitch certainly has the appearance of bona fides. He mentions his affiliation with the FBI, all the places where he's spoken and published, and the 1,500+ cases he's personally consulted on. He gets paid up to $3,500 for the single day course, double that for the basic + advanced course.

There is also clearly just an appetite for what he's selling. In his pitches to departments, Harpster includes endorsement after endorsement from satisfied customers. So if a training board, police department, DA's office, etc. sees all that, it's not surprising that they might think his program has value in helping solve a murder. They said just that in their emails amongst one another.

As we reported in the story, it's also been legitimized by institutions up and down the justice system. For instance, the Ohio State Supreme Court approved the course for prosecutors' continuing education credits and, from what I saw, did so without scrutinizing any of the research/claims behind it. More legitimacy.

SecretSkwirrel38 karma

Is there any way to research agencies or experts who may have attended the “call analysis” trainings?

propublica_60 karma

I learned there is no list anywhere, so you have to send records requests to individual departments to see who has hosted or attended the trainings. That said, we are working on some potential follow-up stories I hope will help pinpoint a little bit better

kwit-bsn35 karma

I read your article a cpl wks ago, great piece! Do you know how many states are using that bullshit training seminar program now?

propublica_44 karma

I don't. I found cases in 26 states and the training had occurred in roughly the same — but that was just from what I'd found in the records, which is of course not exhaustive. Fair to say widespread. We're getting more tips and working on some follow-up stories to hopefully pinpoint a little better.

leshake17 karma

I thought the concept of a human lie detector test is not admissible. How many states allow this to happen?

propublica_29 karma

That's a very good point. I found that 911 call analysis surfaced in cases in at least 26 states. "Surfaced" there means everything from it was used by police during their investigation all the way up to a jury heard it.

I found out that there are a couple reasons why it's made it that far: 1. prosecutors sidestep the hearings meant to determine the admissibility by disguising 911 call analysis as anything but "science"; 2. it's relatively new and unknown, unlike lie detectors, so it's less easy to identify for judges and defense lawyers; 3. it can seem very much like regular, lay testimony instead of expert opinion so some judges have let it in. In emails, prosecutors have laid out these sort of loopholes for getting 911 call analysis into trial via testimony from students trained in it

RulerOfHotTopic16 karma

What made you want to research this ? I think it's really interesting but I want to know what about it made you wanna learn more

propublica_35 karma

Thanks for this. I first heard about it when I was in Louisiana for USA Today, poking around a District Attorney accused of botching a murder investigation. Someone brought up the "COPS Scale" and when I saw what that meant, I was a little incredulous. I sort of filed it away as something to look into: were more departments using this? what was the research behind it? was it actually making its way into court? if so, how? It raised a million questions and once I started reporting on it, even more questions.

unicatprincess13 karma

Is this more prevalent in poorer areas of the country than richer, or rather, country than in the coasts? I guess my question is: does region influence in any way in the use of this non-science? Is there a difference that has been noted in this aspect?

propublica_18 karma

Good question with an unsatisfying answer: I don't know. I didn't collect enough data to analyze in that way or make any definitive statements about geography, income levels, etc. However, the experts I spoke with were really concerned about disproportionate impact

PeanutSalsa12 karma

How is it determined to be legal to use the call recording? Is it legal for anyone who receives or makes a call to record it and use it against the other person even if they don't consent, or do there have to be special parameters in place?

propublica_15 karma

Good question! I've never seen the legality of the actual recording raised before. 911 tapes are common pieces of evidence in criminal cases, so I'm not sure about any special parameters that need to be in place. The question our stories focus on, though, is how exactly police and prosecutors "use" those recordings.

SmokyDragonDish3 karma

How closely do you work with the Innocence Project?

propublica_12 karma

Good question. As journalists we don't work with advocacy groups in that sense. But they were certainly a valuable resource in the reporting and one of my first calls after I learned that people had been convicted after 911 call analysis was used against them. They had — and still have — lots of good information. I was pretty new to this space so interviews with experts from a diverse pool helps me get my bearings

Abstractteapot2 karma

Is there a chance it'll be wrong? I try to imagine a situation where something bads happened and I need to call for help and I feel like I'd get stuck in repetitive cycles of trying to repeat what i think is important.

Obviously, I don't know if that'll be the case but I imagine I would.

How do you test for that? Or do you link it up to their normal speech patterns you might observe in police interviews etc..

propublica_19 karma

To your first question, yes indeed. Check our second story for examples where that sort of scenario plays out. I'll paste the anecdote about Kathy Carpenter here.

On a cold, clear night in February 2014, Kathy Carpenter sped from a secluded house in the Rocky Mountains and toward the police station in downtown Aspen. She clutched the wheel with one hand and a cellphone with the other. “OK my, my, my friend had a — I found my friend in the closet and she’s dead,” Carpenter told a 911 dispatcher between wails.
Her friend Nancy Pfister, a ski resort heiress and philanthropist, had been bludgeoned to death. Local police asked the Colorado Bureau of Investigation to help find out who did it. Kirby Lewis, agent in charge with CBI and one of Harpster’s earliest students, stepped in to analyze Carpenter’s call.
This is what he noted in a report: Carpenter said “help me”; she interrupted herself; she didn’t immediately answer when the dispatcher asked for the address. She provided “extraneous information” about Pfister’s dog. When the dispatcher asked if a defibrillator was in the house, Carpenter paused before saying, “Is there what?”
Lewis found 39 guilty indicators and zero indicators of innocence. Carpenter was arrested eight days later. Newspapers and television stations published the 56-year-old’s mugshot.
She spent three months in jail before someone else confessed to the crime.
Even when people weren’t convicted, some have faced irreparable harm after others decided they chose the wrong words on the phone. Carpenter recently told me the ordeal ruined her life. She lost her job as a bank teller, along with all of her savings and her home. Her car was repossessed. She was diagnosed with post-traumatic stress disorder. She had to move in with her mother across the state and now disguises herself in public. People still call her a murderer, she said. “I just want to go into solitude and just hide.”