Copy article

Government "unbiased AI" procurement rules will fail if vendors are allowed to self-grade

ended 27. March 2026

A new US procurement memo about "unbiased" AI reads like a victory for common sense. Systems bought by government should be truthful, neutral, and willing to admit uncertainty.

The problem is that a principle is not a control. Large language models do not verify facts when they speak. They generate plausible text. They can look neutral while quietly amplifying whatever is most common in their training data. And they can drift with every model update, fine-tune, or policy tweak.

If the procurement model lets suppliers mark their own homework, the state gets a compliance story rather than a safety mechanism. The incentives point towards selective disclosure, glossy testing, and exemptions for existing contracts. That creates a two-tier reality: strict rules for new buyers, a free pass for legacy deployments, and very little accountability where the risk is already live.

The contrarian view: public trust in government AI will be won by independent testing, reproducible logs, and real consequences for failure, not by aspirational wording.

We'd like your views:

  • What evidence should vendors be required to provide: independent bias testing, audit logs, training data disclosures, or all of the above?
  • Should existing AI contracts be forced onto the same standard, even if it slows deployment?
  • What does "truthful" mean for a model that outputs analysis rather than simple facts?
  • Who should be liable when an AI tool is repurposed beyond its documented use case?
  • What is the minimum viable kill switch for government systems that rely on LLMs?

2 responses from the Newspage community

Copy all

Star Quote
Copy

Rohit Parmar-Mistry
Founder at Pattrn Data
‘Unbiased AI’ is a slogan, not a control. If procurement relies on vendors self-grading, you will get compliance theatre: selective benchmarks, pretty reports, and very little signal about how the system behaves in the wild.

Government buyers should demand evidence you can reproduce: independent bias and robustness testing, versioned model cards, decision logs you can audit, and a clear change-control trail for every update, fine-tune, and policy tweak. If the supplier will not let an external party re-run tests on the same artefact, you are buying a promise, not a product.

The awkward bit is legacy contracts. The highest risk systems are often the ones already deployed, so exemptions make the rules performative. Start with a minimum bar (logging, incident reporting, rollback, and a kill switch), then ratchet. One question procurement teams should ask up front: who carries liability when the tool is repurposed beyond its documented use?
Copy

Colette Mason
AI Ethics Consultant at Clever Clogs AI
AI is still the Wild West, and for the 'man in the street' the hype machine leads them like lemmings straight over the cliff into to a sea of disappointment.

Self-certification is a joke. There are simply too many moving parts, all in perpetual motion. An AI battle plan never survives contact with the enemy, and users are the enemy, wanting to do sensible things, illogical "100% hands-off" things, and everything in between.

No vendor can test for real-life use cases in sufficient detail even if they wanted to. Commercially, it's not in their interest to even try. Rely on the glossy brochure and "AI can make mistakes" defence.

All AI systems need a strong framework, intense training, rigorously enforced guardrails, and ongoing self-policing with a good dose of "human-in-the-loop" keeping things safe.

A company marking its own homework isn't a safety mechanism. It's a liability and the penalty is paid by the staff, the state, and the tax-paying citizens.