US AI safety unit has $10 million and no permanent leader

CAISSI is a unit of the Commerce Department of the US government that tests the safety of new AI. Two persons who were there before tell that it is to report informally to an office of the White House. The unit has no permanent leader and does not publish its research. Its budget is $10 million each year.

What changed

In April, the unit hired a director and fired him after some days. His replacement resigned after some months. In August, Congress wanted to know the cause that the reports stopped.

The numbers

The budget is about one eighth of the budget of the UK counterpart. Sayash Kapoor of the University of California, Berkeley, tells that no standard procedure tests how safely AI agents work in use. AI agents are software that does tasks without a person.

What is not known

The White House did not tell more about the unit. It tells that the president set up a Super Intelligence Force. Critics think that the findings can change. Others think that the move can give the unit more money.

Insight: Companies select who tests them

In September, the president told AI companies to do “tremendous self-policing”. The US did not tell how it will check this. Dario Amodei and Sam Altman promote test firms of a third party, but the labs can limit their access. This reads as a system where each lab selects its tester, and the labs get control. A safety claim from a lab is not proof from a third party.