Not one of the 70 billion numbers says "cat": where exactly does a model's knowledge live?
AI summary
A large model's knowledge is not stored as a tidy list of facts; rather, it is encoded in a distributed way through the coordinated interplay of countless parameters, and it is impossible to point to a single number as representing "cat." Research can roughly localize knowledge to local regions such as the FFN; for example, ROME can locate and rewrite facts like "the Eiffel Tower is in Paris," but "localizable" does not mean "interpretable" or "controllable." Stable background knowledge is suitable for modifying weights, while facts that change should instead use RAG retrieval.
Body
Only a summary is authorized here. Continue at the validated source link.