猴子视觉神经元语言转译方法

Marc Andreessen 🇺🇸 (@pmarca) 转发了 Surya Ganguli (@SuryaGanguli) 的帖子： Our new paper lead by @vedanglad w/@AToliasLab "Letting the neural code speak..." https://t.co/B6H1Vvcmso We show how to get *monkey* visual neurons to TELL us in *human* language what images make them fire. We do this is an automated verifiable way at scale! How? 1) Build a digital twin of monkey visual areas that can accurately map visual inputs to neural activity. 2) Perform in-silico experiments on this twin to find many complex images that make a model neuron fire. 3) Use a vision-language model to describe these complex images. 4) Verification: use a language-conditioned diffusion model to generate new images, and check they make the monkey digital twin neurons fire a lot. To our knowledge, for the first time, we have a way to convert monkey vision to human language, and from *human* language to sample infinitely many images that make any given *monkey* visual neuron fire, all in an algorithmic fashion. For more exciting details, see @vedanglad's excellent thread! > **引用原帖 Vedang Lad (@vedanglad):** > How well can you describe the feature selectivity of a vision neuron … with words? Interpretability has long borrowed from neuroscience — and maybe it can give back too! 🧵 https://t.co/cSN6TfnJcI > https://x.com/vedanglad/status/2066956817100677236