Copy article

Facial recognition oversight is not about policy. It is about proving control before harm

ended 04. March 2026

New York City’s tech office has put something rare on the record: a plan to treat facial recognition and biometric systems like public-risk infrastructure, not just ‘innovation’. Testimony this week describes disclosure of algorithmic tools that affect rights and benefits, pre-deployment assessments, and a new Office of Algorithmic Accountability.

The hard part is not writing principles. It is making them operational when an agency buys a vendor tool, plugs it into a workflow, and starts making decisions at speed. Biometric systems are especially unforgiving: misidentification is not a bug, it is a person being stopped, denied, or flagged.

A useful accountability standard is dull and specific. What data was used? What thresholds were chosen and by whom? What are the known error rates across groups? What is the escalation path when the tool is wrong? And can an agency prove, after the fact, which model version and policy settings drove a decision?

If the answers live in a slide deck, the city is running compliance theatre. If they live in logs, audits, and procurement clauses that survive staff turnover, the approach becomes a template other governments can copy.

We'd like your views:

  • What should a ‘pre-deployment assessment’ include for facial recognition in public services?
  • Should biometric tools be banned for some use cases even if accuracy improves?
  • Who should own liability when an agency relies on a third-party biometric system?
  • What is the minimum viable audit trail for public-facing AI decisions?
  • How should the public be able to challenge an AI-assisted decision in practice?

3 responses from the Newspage community

Copy all

Star Quote
Copy

Rohit Parmar-Mistry
Founder at Pattrn Data
Facial recognition governance is not a policy problem, it is an evidence problem. Anyone can publish principles. The question is whether an agency can prove what the system did, on what data, with what thresholds, and who signed it off.

A proper pre-deployment assessment should be dull and forensic: purpose and legal basis, data provenance, accuracy and error rates by subgroup, human review and escalation, retention, and an abuse case (what happens when it flags the wrong person). Then lock it into procurement clauses, logging, and change control so the audit trail survives staff turnover and vendor updates.

In our AI audits, the failure mode is predictable: answers live in slide decks, not in logs. If the public cannot challenge an AI assisted decision with a clear route to evidence, you do not have oversight, you have theatre.
Copy

Editor
Mitali Deypurkaystha
AI Governance Strategist & CMO at AISensa
Home Office testing shows facial recognition can be up to 100 times more likely to wrongly flag minority faces, a stark statistic that exposes the stakes of sloppy AI oversight. The same research found false positives of just 0.04% for white subjects but 4–5.5% for Asian and Black people, with Black women facing the highest rate at nearly 10%. New York’s push for algorithmic accountability is exactly the kind of scrutiny our government should demand, because when biometric systems fail, the consequences are human. Wrongful stops, arrests and surveillance aimed at people for things they are born with and cannot change. If authorities cannot prove what data trained a model, who signed it off and what thresholds it used, then oversight becomes theatre, and the public becomes a frightening test case.
Copy

Patricia McGirr
Founder at Repossession Rescue Network
Facial recognition is not a technology question. It is a power question. Who gets flagged, stopped, or denied because a machine said so. If governments want legitimacy, pre deployment tests must answer four blunt questions: Is it lawful. Is it accurate across groups. Who signs off the thresholds. And who carries the liability when it gets it wrong. Some uses should never be normalised. Real time street surveillance and automated eligibility decisions for public services cross a democratic red line. Vendors will always promise accuracy. The public sector must demand proof, audit rights, and full decision logs before a single system goes live. If an agency cannot show exactly which model, which settings, and which human approved a decision, then accountability does not exist. And if a citizen cannot challenge that decision quickly with a real person, the system has no place in public service.