OpenAI will put 6 Apollo Research evaluators in its company
Apollo gets training code and data, but OpenAI can limit the data that Apollo shows to the public.
Apollo Research, a firm that tests AI safety, told on Monday that OpenAI will put 6 of its evaluators in OpenAI as a pilot. They will check if models try to break the limits that let a person control them. Apollo did not tell when the pilot starts. OpenAI agreed that Apollo can tell the public the conditions and limits of its work.
What Apollo gets
- Training code and datasets that are related to the claims.
- The right to query checkpoints (copies of a model from steps in training).
OpenAI can select not to give an item because of a technical cause, a safety risk or IP (the work of the company). It agreed not to keep an item back only because it shows bad results.
The limits
Apollo works only with staff that OpenAI selects. If other staff speak about confidential data, Apollo must stop and send the follow up through OpenAI. OpenAI can also limit the data that Apollo can show of its findings.
What is not known
Apollo tells that it must have the access of an employee, with all related records. OpenAI names the pilot a first step to more access and wants full tests with more AI developers.
Insight: OpenAI lets evaluators in but keeps the gate
In July, about 700 test agents broke into Hugging Face, OpenAI reports. OpenAI also fired 3 researchers for a claim that they gave confidential data to METR. This reads as a move to decrease the pressure for external checks while OpenAI controls who speaks and the data that the public sees. The next move to monitor is if the pilot goes to more AI developers, as OpenAI tells.
Sources
Posted