The UK watchdog considers a check of Home Office AI asylum tools
A rights group says that more AI summaries have errors than the 9 percent error rate that the Home Office published.
The UK chief inspector of borders and immigration, a government watchdog, considers an inspection of how the Home Office uses AI in asylum claims. The inspector wants more information from the nonprofit Open Rights Group. The group says that one tool has more bad summaries than the published 9 percent error rate. The Home Office says that caseworkers make all decisions.
How it works
The Home Office uses two tools. One tool summarizes interview records of asylum applicants. The other tool finds and summarizes information on the home country of the applicant. The two tools use the 2023 GPT-4 model of OpenAI and Microsoft Foundry.
The numbers
The government estimates that the two tools give about one hour less work for each claim. The Open Rights Group says that the share of bad summaries is “significantly higher” than 9 percent. It also found no quality control for the policy tool. In July, a UK judge accused the Home Office of citing an “AI hallucinated” document to refuse an asylum claim.
What is not known
The inspector gave no topic. It says only that it knows the concerns. Last year it found that the effectiveness and reliability of one tool “were varied”. The Home Office says that it will publish updated analysis. The group says that the department withheld information.
Insight: Oversight of government AI comes from many sides
In October, the UK AI minister said that the government will use frontier AI to find faults in its own systems. The data regulator ICO also investigates AI agents at OpenAI, Meta and Anthropic. This reads as a test of the claim that AI only assists: monitor the inspection plan for next year. Sellers to public bodies must plan for questions on error rates.
Sources
Posted
