The Dark Side of Smart Policing: Flock’s AI-Powered Watchlist Raises Alarms

The rise of AI-powered policing tools has been touted as a solution for law enforcement agencies seeking to stay ahead of crime. However, beneath this surface lies a more insidious reality – one that threatens to erode civil liberties and undermine the very fabric of justice.

Flock Safety’s AI search tool has been at the center of dozens of police-stalking cases. This technology allows officers to cordon off an area and run continuous, automated searches for individuals fitting a written description. While this may sound innocuous, it is actually a system that prioritizes surveillance over transparency.

The tool’s AI model decides which descriptions of a person an officer may search for – without the department using it being aware of these decisions. Experts say no technology can perform this job reliably. Flock Safety itself warns officers that results may be inaccurate or incomplete, but places the risk of any inaccuracies squarely on the officer running the search.

Flock’s system scores written descriptions against categories of sensitive content, including race and religion. However, it only implements a speed bump when searching for “political, social, and cultural expression,” which is an area courts have protected most closely under the First Amendment. This discrepancy raises concerns about bias in the technology.

The use of Flock’s system also raises red flags. It has been deployed in dozens of police-stalking cases, with officers using its AI-powered watchlist to search for individuals without adequate oversight or accountability. When WIRED examined the code governing a search, they found that much of what governs it happens on Flock’s servers – out of reach for anyone outside the company.

Data flows in both directions when the software flags a search. It sends Flock the language used by the officer, the category and confidence score assigned by the model, and whatever action the officer took next. However, Flock’s statement on these findings is opaque – refusing to describe how its AI model was built or what instructions it follows.

The implications are far-reaching and disturbing. As we continue down this path of relying on AI-powered surveillance tools, we risk creating a society where our every move is monitored, judged, and potentially punished without due process. This is not just about the technology itself – but also the values we’re willing to uphold in the name of “public safety.”

Flock Safety’s response to these criticisms has been underwhelming. The company defends its tools as built to help police find relevant information while maintaining clear safeguards around how the technology can be used. However, it does not provide any specifics about what these safeguards are or who decides what constitutes a “relevant” search.

As we watch this unfolding drama, it’s essential to recall that policing is not just about enforcing laws – but also upholding our collective values as a society. We must demand more transparency from companies like Flock Safety and hold them accountable for their role in shaping our surveillance state.

The future of AI-powered policing hangs in the balance. Will we choose to prioritize security over liberty, or will we find a way to harness this technology for the greater good? The answer lies not just with policymakers – but also with us as citizens who must remain vigilant and demand that our government uphold its duty to protect our rights.

The stakes are high, and it’s time to take a closer look at what’s really at play here. As we move forward, one thing is clear: the line between public safety and public surveillance has grown perilously thin. It’s up to us to ensure that we don’t sacrifice our civil liberties on the altar of convenience.