Police didn't care that Linsey Isaacs' car was the wrong color, and wasn't damaged. They still arrested her because of Flock data.

257 points•HotGarbage•5 days ago•145 comments•

145 comments

freddealmeida5 days ago
Flock did not put the woman in jail. The police and DA put her in Jail. This is not a failure of the technology (do they have confidence levels?). This is a failure of how police are outsourcing their reasoning and decision making to an AI camera service that seems to fail at the tail.
CrendKing5 days ago
But Flock gives police overconfidence to commit actions instead of having due diligence. It is like the best student in your classroom tells you 2 + 2 = 5, and you naturally feel compelled to believe him, even there is clear evidence against the claim.

This is also comparable to self-driving car: Waymo's system has achieve the level of "less error than human on average" long time ago, but they are still only doing slow expansion. Why? Because when AI is involved, human naturally expect them to be almost 100% correct, not just barely better than human, otherwise why bother use AI (and all the troubles to legalize for AI) in the first place? So in this case, I guess the police just blindly trust the AI being 99.99% correct.

nrmitchi5 days ago
Flock isn’t an “AI” though? It doesn’t put together a case and say “yes, this is the criminal”. It’s not a crime-solving AI.

It’s a camera system, and data can be queried by police. They have to search for something. So the cops searched for a vehicle matching the description in the general area, seemed to take the first one, and called it a day.

That’s not an AI-uncanny-valley. That’s the equivalent of (pre-flock/cameras) looking at vehicle registration list and assuming the one at the top of the list must be it.

The automation-trust is a legitimate concern but it’s not applicable here.

jaredklewis4 days ago
I don't feel we are asking too much of police and lawyers here: yes, you can use technology and tools (such as Flock) to help you do your job more efficiently, but that doesn't mean you can turn your brain off.

Perhaps more facts will come out, but if we take the article at face value, the cops are at this woman's house, looking at her car and can see with their own eyes that the car right in front of them has obviously not been in an accident. All that is required is for these cops to rub two brain cells together and they will reach the conclusion that Flock misread a license plate.

It's also worth emphasizing that this wasn't for a parking ticket. The extremely low level of care with which these public servants are conducting a manslaughter investigation is somewhat alarming.

vconnor5 days ago
> It is like the best student in your classroom tells you 2 + 2 = 5, and you naturally feel compelled to believe him, even there is clear evidence against the claim.

Yes, but it is still your fault. “I am dumb and I was deceived” is no valid defense, least of all for a police officer.

testplzignore5 days ago
Software developers approving AI PRs without reading them: This is the way the world works now. Get with the times.

Police approving AI PRs without reading them: This is the way the world works now. Get with the times.

boxed5 days ago
Cops regularly frame people and get caught only because of cameras that the cops are too stupid/incompetent to disable or avoid. This goes both ways.
latexr5 days ago
While you are technically correct (which we all know from Futurama is “the best kind”), that is neither helpful nor consolation to the affected, and I wish these kinds of comments didn’t dominate the top spot in HN discussions. They’re the “guns don’t kill people, people kill people” of tech.

Yes, the system is broken even before we get to Flock (or whatever tech) but Flock makes it worse. If a bad tool is badly used to cause harm, it’s fair to criticise its existence. You can’t fix police abuse with a snap of the finger, so you need to start but taking away the tools which facilitate it.

If it were your ass which had been thrown in solitary, would you be pleading with your lawyer to “go easy on Flock. Sure, they allowed a major disruption and trauma into my life, but they’re just a tool”?

ArchD5 days ago
Then the right fix is to fix the system, e.g. imposing accountability. Removing tools is just a band aid. Why is the tool bad? A tool does what it's designed to do. You can use it properly or improperly. There should be no conflation of regular civilians with supposedly trained government officers.
Buttons8404 days ago
The people who say there are problems with Flock are perceived as excusing the human police officers.

The people who say there are problems with the human police officers are perceived as excusing Flock.

Who is responsible for the area inside a square, the width or the height? It is not an additive relationship.

atoav5 days ago
The technology is not a neutral thing here. Automation bias¹ exists and is especially problematic in this context.

And if your country happens to have a police force that is historically bad at its job, giving them that power seems like a really stupid idea.

¹: https://en.wikipedia.org/wiki/Automation_bias

newtonsmethod5 days ago
I very much agree with what you say on automation bias.

Certain technology just has too much potential to be misused / people repeatedly prove they can't use it properly. It also comes with risks of making authoritarian takeover easier.

However, here, it seems they used Flock a bit like a regular ANPR. The actions they took independent of Flock show there are better stories to use to criticise this technology.

neya5 days ago
Which makes Flock cameras even worse right? It provides false confidence to law enforcement to make decisions that can change someone's life for worse. I don't understand why people hate the NSA spying on them and yet support Flock cameras.
SpicyLemonZest4 days ago
The question is whether Flock does provide false confidence to law enforcement, or whether it provides a media hook to cover the ways in which bad police departments routinely misbehave.

A lot of people coming to this story seem to have the idea that it's unthinkable for someone to spend 13 days in jail when they have clear evidence they're innocent, so there must be some novel explanation for how this could happen. But as any criminal defense lawyer will tell you it's very thinkable. She was ordered to report to jail on a Friday afternoon, the standard stratagem cops use to ensure people they don't like spend at least a weekend behind bars. She was denied bail, even though she made no move to flee in the months where she knew the police thought she did it, because the courts in many places view mandatory pretrial detention as a punishment for presumptively guilty people.

p-e-w5 days ago
Indeed. It’s insane to see these arguments implying the contrary. The institution committing the violence is the one responsible for the outcome.

If a random person walks into a police station and accuses a random stranger of murder, and the police put that stranger in jail based on that alone, the main problem is not the accuser.

jimdavenport5 days ago
But in this case, it wasn't a random person, it was a police-vetted system. Of course the authorities are to blame, but Flock hold a lot of responsibility, too.
Cider99865 days ago
There was a senate hearing a few days ago where this woman testified. Also Benn Jordan and Chad Marlow from the EFF were htere. I was happy hearing them put into words the problems on the national stage.

https://www.youtube.com/watch?v=klE498xFjOY

Zak5 days ago
Interestingly, conservative Republican sheriff Ross Teeple also spoke against the idea of networked/data-storing ALPRs and Flock in particular at that hearing.
UberFly5 days ago
Plenty of individuals regardless of politics resent the loss of privacy and the growing thumb of big brother.
Cider99865 days ago
They assembled the avengers.

But seriously, privacy must not become a sided issue. Attacks on privacy are bipartisan, the defense of it would be weakened if it was not bipartisan. Civil liberties protected by the constitution should not be controversial.

zelphirkalt5 days ago
Man, this just feels like proving RMS right all over again, as usual, when it comes to our rights in the digital world.
JumpCrisscross5 days ago
What is the age-education surface for Flock support?
nomilk5 days ago
Had the 'evidence' been a random dashcam rather than Flock cam, this wouldn't even be a story beyond severe police incompetence.

Flock didn't fail to inspect the car for damage and not find any, flock didn't take 13 days to conduct a cursory review of evidence, flock didn't fail to request cell tower triangulation data and rule out her being in the location of the accident...

I get that people feel their privacy's invaded by being filmed, but this story doesn't appear to have any more to do with Flock than what the officers had for breakfast or the brand of air freshener she had in her car at the time.

selfhoster13125 days ago
A random dashcam does not serve the police someone's identity on a platter. It had to be a stupid "smart" object for such an inexcusable offense to be committed.

With dumb camera footage, the cops looks at the plates, visual indicators, etc. That does not make them more competent or less prone to abuse. But here, Flock's software literally told them this woman was the one they were looking for, despite no evidence supporting this.

This is 100% both on the police and on Flock. Some technologies don't deserve to exist. Facial recognition and ALPR is one of those, because the error rates are too high and the stakes are too high.

nomilk5 days ago
> A random dashcam does not serve the police someone's identity on a platter.

Nor does Flock - Flock merely provides a photo and timestamp, which should have been scrutised like any other evidence.

The Flock data showed:

- Car of same make/model nearby at similar time to the accident

- Possible colour match (eyewitnesses gave varying reports on the colour of the perpetrating vehicle). But a match could have been ruled out in minutes had police inspected paint transfer onto the struck vehicle

- A number plate non-match (first three characters were reported to 911 and did not match)

- That's all!

So police didn't do their job (scutinise the evidence), and possibly worse, they may have fabricated evidence (since police claimed Isaacs' vehicle was damaged, but it was in factory-new condition) - which may explain why they're shifting blame to Flock!

petre5 days ago
It just enables incompetence. If they were forced to do actual police work they would have probably used their policeman brains.
verisimi5 days ago
While using one's brain has never been 'the thing', it's positively unfashionable nowadays.
wredcoll5 days ago
Except of course the entire reason the police picked her in specific is because of information flock gave them.

Could they have, you know, double checked it? Yes, obviously. But they didn't.

I agree that this story is more about the incredible incompetence of the police involved than flock attempting to record everyone and everything, but flock giving the police an excuse to stop looking and arrest an innocent is by no means irrelevant.

nomilk5 days ago
Had a random person submitted a pic taken on their phone of a vehicle in the same location the Flock cam did, police would ask:

- Does anything connect this vehicle to the accident?

- Did it look damaged; as though it had been in a crash?

- Did it look like it was fleeing or driving erratically?

- Was there anything suspicious about the vehicle or the driver?

Police would have deemed the photo irrelevant and moved on. But that didn't happen in this case.

Their failure to do so doesn't have to do with source of the image (Flock, shop CCTV, dashcam), but with their inexplicable lack of rigour.

sixothree4 days ago
It's pretty clear that flock is presented as reliable evidence to the people acting on it.
scarecrowbob5 days ago
Yeah, if things were different, people would react differently.
delichon5 days ago
> Regardless, the true takeaway of Isaacs’ experience is that ALPR technology, whether it’s from Flock or any of the other competitors in this space providing the same functionality (Axon, etc), is dangerous not only because it is so easily abused by police (or even by hackers) to invade the privacy and liberty of citizens, but also because it enables a lazy police department to outsource basic critical thinking to machines and then take action based on a single data point to ruin a person’s life.

The accusation that some machine made any decision here goes beyond the reporting in the article. It seems concocted to raise AI alarm. All we have is that it supplied accurate data that police then abused. That isn't in the vicinity of "outsource[d] basic critical thinking to machines".

I want such general government surveillance banned, but the machines in this story had no more volition than a breathalyzer.

preg_match5 days ago
The machines matter, because we know how human behavior works. We know that if you give humans an automated tool that works most of the time, they get lazy, to the point where they make more mistakes than if they had no automation at all.
nrmitchi5 days ago
Flock isn’t an automated tool though; you don’t give it a crime, and it gives back an automated package saying “this is who the criminal is”.

As far as I know, it lets law enforcement query for things against the data. Can be specific (like a license plate) or more general (like a dark colored mid-sized SUV around <location> at <time>).

To get where this story is at, cops here must have done the latter, and taken the first name off the top of the pile and patted themselves on the back, and then waited 7 months, and went and arrested her. Unless there’s an explanation for a 7 month delay, it seems designed to allow defensive evidence to spoil.

There isn’t an uncanny-valley of automation here. There is just lazy, grossly-incompetent, bad-faith, and malicious cops/investigators.

sixothree4 days ago
I think what we're seeing is that flock is presented to police as a reliable technology that can be trusted for this sort of thing. If a random person had snapped a picture they would have questioned it.

But they didn't question the quality of the evidence. And there is a reason for that.

wredcoll5 days ago
This should be emphasized more. Regardless of how technology could be used, we (should be) are responsible for how it is likely to be used.

Guns can be used for all sorts of things. A handgun in a house is likely to be used for murder/suicide.

LocalH4 days ago
Piles of evidence that humans tend to treat AI as infallible, just like the police did here. They didn't use this as a point for further investigation, they saw the match and said "let's go get her" without so much as looking at the damaged vehicle for paint transfer, which should be remedial-level police instruction.

Modern AI was and is a mistake. It's only going to get worst before it gets better.

Unfortunately, our current government in the US is hard AI-pilled (oh, sorry, "super-intelligence-pilled"). They use it every day to present propaganda to the public via social media.

Ban AI use in all branches of government, and in policing. At least until there are ethical standards put in place that are very heavily punished when violated.

Also, state and federal level politicians should be prohibited from using social media in anything other than a purely personal basis without mention of political office or without speaking on the topic of politics. Want to post cool pictures of your family? Go for it. Speaking to your constituency? Must be strongly vetted for ethical violations, and punishable by immediate removal from office and a 10-year blacklist.

Walf5 days ago
Volition has nothing to do with it. A breathalyser doesn't point to a person and say this is the likely drunk, so that comparison is fatuous. These identification systems do point the finger.
samudrijan5 days ago
Police abusing the law? Never.
teravor5 days ago
the problem here appears to be that police officers are generally of average or below average intelligence. combined with the fact that they face no consequences for incompetence, you have the perfect mixture for nonsense like this happening.

I expect the solution they will settle on is to form a department for interacting with technology like this, who understand its outputs are not edicts.

you can't really expect any meaningful police reforms (such as liability and insurance) because anyone who is in any position to enact them is perfectly fine with dumb muscle that holds a monopoly on violence and is deferential to them. in fact the public disliking the police is an ideal outcome for those decision makers as it gives them more leverage over the police.

LocalH4 days ago
The problem is that they're treating the output of the AI algorithm as 100% fact. Countless examples of Flock matching incorrectly or police otherwise using the match found by Flock as "here's who to go arrest" and not "here's a starting point for further investigation".

Partial matches, correct matches but no investigation into things like paint transfer, it's happened way too many times (hint: one is too many times for things that can incarcerate people).

It's like they're speedrunning unworthiness to hold the monopoly on violence. This is how you get riots and revolutions. Sadly, today's Western world is complicit and complacent. Nobody who is pro-surveillance state will change their mind until it affects them or someone close to them.

Read the full thread on Hacker News →

Related stories