When children are charged with crimes, the court system usually takes some steps to prevent their acts from defining them in later years. Children who are 16 and under, for instance, are generally sent to Family Court, where records are not public.
Yet including their photos in a facial recognition database runs the risk that an imperfect algorithm identifies them as possible suspects in later crimes, civil rights advocates said. A mistaken match could lead investigators to focus on the wrong person from the outset, they said.
“It’s very disturbing to know that no matter what I’m doing at that moment, someone might be scanning my picture to try to find someone who committed a crime,” said Bailey, a 17-year-old Brooklyn girl who had admitted guilt in Family Court to a group attack that happened when she was 14. She said she was present at the attack but did not participate.
Bailey, who asked that she be identified only by her last name because she did not want her juvenile arrest to be public, has not been arrested again and is now a student at John Jay College of Criminal Justice.
Recent studies indicate that people of color, as well as children and women, have a greater risk of misidentification than their counterparts, said Joy Buolamwini, the founder of the Algorithmic Justice League and graduate researcher at the M.I.T. Media Lab, who has examined how human biases are built into artificial intelligence.
The racial disparities in the juvenile justice system are stark: In New York, black and Latino juveniles were charged with crimes at far higher rates than whites in 2017, the most recent year for which numbers were available. Black juveniles outnumbered white juveniles more than 15 to 1.
“If the facial recognition algorithm has a negative bias toward a black population, that will get magnified more toward children,” Dr. Ricanek said, adding that in terms of diminished accuracy, “you’re now putting yourself in unknown territory.”