A hypernetwork makes LoRA adapters for the 284B DeepSeek v4 Flash
The adapters gave 84.9% top-1 precision on new documents, against 63.4% for the model with no adapter.
Claimed, not confirmed
The researchers show the Internalizer, a hypernetwork that turns a document into a LoRA adapter. The adapter is for the frozen 284B-parameter DeepSeek v4 Flash. Work before this used models of a maximum of 14 billion parameters. Most parameters of the Internalizer are in a trunk that is the same for each model. The trunk trains on small models first. Then the team moves it to the large model.
Sources
Posted
