Dan Shipper 回应 Yudkowsky：可解释性不止于识别概念
Of course that’s your contention. You just got finished reading about polysemanticity, so naturally you think interpretability means untangling superposition. Y
Dan Shipper 回应 Eliezer Yudkowsky 关于机制可解释性的争论，指出识别模型中的概念（如某个神经元对应埃菲尔铁塔）并不等于理解模型在计划什么。他认为从多义性、叠加态到单神经元识别，只是理解心智运作的最初起点，知道模型识别法国不代表能解释超级智能的对齐问题。
查看原帖Okay, so now I'm looking at there and it's "polysemanticity and superposition convinced me interpretability was hard, but with big networks there's single neurons after all" and do. do you have any concept of what utterly. fucking. primitive. playgrounds you are still playing around in. "We thought we might have a hard time finding single neurons that…
展开完整原文
Okay, so now I'm looking at there and it's "polysemanticity and superposition convinced me interpretability was hard, but with big networks there's single neurons after all" and do. do you have any concept of what utterly. fucking. primitive. playgrounds you are still playing around in. "We thought we might have a hard time finding single neurons that light up for the Eiffel Tower and would have to look at *multiple* neurons, but now maybe we can find single neurons again!" Do you have any fucking clue of how close this is to the absolute first beginning of understanding how a mind is thinking? Do you think mechanistic interpretability at the level of "do alignment" is about noticing that particular kinds of concepts fired?
正文
当前显示原文，中文全文译文尚未就绪。
Of course that’s your contention. You just got finished reading about polysemanticity, so naturally you think interpretability means untangling superposition. You’re gonna believe that until next semester, when you find a neuron that lights up for the Eiffel Tower. Then you’re gonna spend the summer explaining how we can align a superintelligence because we know when it’s thinking about France. And next year you’ll realize that knowing it recognizes the Eiffel Tower doesn’t tell you what the fuck it’s planning.