AI is rocking the world of policing — and the consequences are still unclear.
British police are poised to go live with a predictive artificial intelligence system that will help officers assess the risk of suspects re-offending.
SEE ALSO: Mark this as the moment when United trolling jumped the shark
It’s not Minority Report (yet) but certainly sounds scary. Just like the evil AIs in the movies, this tool has an acronym: HART, which stands for Harm Assessment Risk Tool, and it’s going live in Durham after a long trial.
The system, which classifies suspects at a low, medium, or high risk of committing a future offence, was tested in 2013 using data that Durham police gathered from 2008 to 2012.
Its results are mixed.
Forecasts that a suspect was low risk turned out to be accurate 98 percent of the time, while forecasts that they were high risk were accurate 88 percent of the time.
That’s because the tool was designed to be very, very cautious and is likely to assign someone as medium or high risk to avoid releasing suspects who may commit a crime.
A self-learning system
According to Sheena Urwin, head of criminal justice at Durham Constabulary, during the testing HART didn’t impact officers’ decisions and, when live, it will “support officers’ decision making” rather than define it.
Urwin also explained to the BBC that suspects with no offending history would be less likely to be classed as high risk — unless they were arrested for serious crimes.
Police could use HART to decide whether to keep a suspect in custody for more time, release them on bail before charge or whether to remand them in custody.
However, privacy and advocacy groups have expressed fears that the algorithm could replicate and amplify inherent biases around race, class, or gender.
“This can be hard to detect, particularly in self-learning systems, which carry greater risks,” Jim Killock, Executive Director of Open Rights Group, told Mashable.
“While this process is reported to be ‘advisory’, there could be a tendency for officers to trust the machine on the assumption that it is neutral.”
Racial bias?
Whenever the data is systematically biased, outcomes can be discriminatory because learning models bring into the foreground assumptions that have been tacitly made by humans.
The Durham system includes data such as postcode and gender which go beyond a suspect’s offending history.
Even though the system is very accurate, let’s say 88 percent of the time, a “subset of the population can still have a much higher chance of being misclassified,” Frederike Kaltheuner, policy officer for Privacy International, told Mashable.
For instance, if minorities are more likely to be put in the wrong basket, a system that is accurate on paper can still be racially biased.
“It’s important to stress that accuracy and fairness are not necessarily the same thing,” Kaltheuner said.
Last year, an investigation by U.S. news site ProPublica shone a light on the alleged racial bias of an algorithm used by law enforcement to forecast the likelihood of a repeated offense.
Among other things, the algorithm was making overly negative predictions about black versus white suspects. The firm behind the system denied the allegations.
For example, ProPublica reports the cases of Brisha Borden, a black 18-year-old teenager who stole a child’s bicycle and scooter, and Vernon Prater, a white 41-year-old who was picked up for shoplifting $86.35 worth of tools.
