Sorry for implying that you believe something you don't! The sentence here that struck me as dualist was this: "Machines that can do most or even all of what the human mind can do seem possible (and even likely)"
The "or even all" is what sounded dualist to me — at a scientific level, the human mind and body affect each other in terms of mood, certain illnesses etc and it seems doubtful AI could do that. (Not ascribing this view to you) There are nuanced arguments around neuroscience and philosophical dualism but most of the digital minds discussion I've seen, for example, seems to rely on dualist or at least dualist-adjacent assumptions that are not fully thought out. They may exist in EA writing I haven't read though!
Thank you for the link with the recommendations! Checking it out now
Something I am trying to wrap my head around is the part of this where "we won't bullshit you" is a load-bearing axiom. It's possible I'm simply being deontological, since all kinds of weird made-up stuff about EAs gets slung despite the general commitment to very good epistemics, and it's hard for me to make a consequentialist argument that in any way the discourse space is rewarding the genuine commitment to not feeding people a trough of lies. But, at the same time, it does feel relevant to the 'elite influence strategy' how frequently EAs have been correct about so many things, coupled with the existence of a robust trail of evidence that they have been a) correct, b) honest, and c) proactive about those things. Certainly it's updated *me, personally* towards being far more overtly 'pro-EA!'
Also here is an article if you want to indulge in me getting big mad on your behalf at how much of the discourse is "but lol you're just a shill for billionaires, since we all know everyone gets into <checks notes> the lucrative domain of journalism to ball out hard": https://zachill.substack.com/p/the-furtive-fallacy-billionaire-edition
I believe that the HuggingFace incident was really bad and that something needs to be done to prevent things like this from happening in the future. I also believe that EAs and rationalists know far more than most people about what AI is currently capable of, so I have been following the EA & rationalist reporting on this with interest and concern.
At the same time, I don't really share the mind-body dualism that seems baked into many EA & rationalist assumptions about the world, including AI, which means I don't agree with everything in "What is AI Safety". I don't think I'm alone in that!
While disagreement is healthy, this makes it difficult for me to try to understand some of the articles about the incident, in which opinions and assumptions about the world and AI are being baked into otherwise quite rigorous reporting. What I really would like a distillation of what people close to the matter think should materially happen with AI governance based on what is materially happening now.
It's useful to know why people think these things, but it's also useful to know what they think will be useful to mitigate risks from AI even if someone disagrees with their underlying worldview.
"A complex movement of people working together to figure out what’s true" (and to work together on mitigating risks from AI) needs to not dismiss everyone who doesn't agree with certain philosophical beliefs as unintelligent and unworthy of engagement — which unfortunately many corners of these communities do. This is something else that the EA and rationalist communities can directly address, though I'm not 100% sure what the best ways are — maybe through more clearly delineating what is fact and what is belief? Offering different recommendations for different worldviews based on what is happening materially?
Oh interesting! Would be curious to know more about the mind body dualism point, I’m definitely not a dualist and would want to rewrite if it came off that way! I completely agree that a lot of AI safety comms isn't doing a good job of making it clear what background implicit assumption writers are coming from and would like to see (and produce) more of that, as well as concrete policy recommendations. This by the IFP crew was useful as one set of general policy suggestions https://www.noahpinion.blog/p/23-low-regret-recommendations-for
I'm not EA- or rationalist-affiliated and personally deeply skeptical of the "orthodox" Yudkowsky X-risk narrative. From where I sit, the chain of reasoning is too long, too narrowly specific, and it smuggles in too many assumptions that are far from self-evident outside those circles.
At the same time, we humans rarely prepare for risks until they explode in their faces. AI won't be any different. So I expect a series of rude wake-up calls, ontological shocks, crises that will have to be responded to as they happen rather than in advance. In that light, I am very grateful that smart people _have_ been taking these risks seriously when no one else was.
Unfortunately, EA - and honestly anyone with deep knowledge of the topic - suffers from a real branding problem as the messenger here, having been pre-emptive coded "weird".
I have seen people completely dismiss the METR report on the Hugging Face incident as "a tech bro fever dream" because they had received EA funding in the past. As far as critics and journalists are concerned, the AI labs, METR, EA, LessWrong, safety advocates like Helen Toner, and true X-risk doomers are all part of the same AI hype bubble - or its useful idiots (as The Atlantic explicitly called them in their hit piece).
Even after all that's happened, the approved "adult" response is still to "reject the hype" and "remember it's just predicting the next word" and "that was just in the training data".
So EA's outreach challenge is to convince people that: 1. you should ignore the data centers you all hate; 2. the hype about AI is actually true, the tech bros weren't lying; 3. real sci-fi craziness might happen, dangerously so; 4. you should accept all this from people with an unconvential philosophy and a sometimes sci-fi-coded worldview; 5. pay no attention to their close ties to the tech industry.
That's a tall order. I don't know anyone _else_ who should plant a flag, but the stranglehold of the head-in-sand narrative is going to be hard to break through.
We refuse to listen to you as the mouthpiece for your billionaire overlords, Andy!
;-)
Hilariously I got so big mad about accusations of Andy being puppet-stringed by his billionaire overlords that I couldn't resist writing an article about it :P https://zachill.substack.com/p/the-furtive-fallacy-billionaire-edition
Sorry for implying that you believe something you don't! The sentence here that struck me as dualist was this: "Machines that can do most or even all of what the human mind can do seem possible (and even likely)"
The "or even all" is what sounded dualist to me — at a scientific level, the human mind and body affect each other in terms of mood, certain illnesses etc and it seems doubtful AI could do that. (Not ascribing this view to you) There are nuanced arguments around neuroscience and philosophical dualism but most of the digital minds discussion I've seen, for example, seems to rely on dualist or at least dualist-adjacent assumptions that are not fully thought out. They may exist in EA writing I haven't read though!
Thank you for the link with the recommendations! Checking it out now
Something I am trying to wrap my head around is the part of this where "we won't bullshit you" is a load-bearing axiom. It's possible I'm simply being deontological, since all kinds of weird made-up stuff about EAs gets slung despite the general commitment to very good epistemics, and it's hard for me to make a consequentialist argument that in any way the discourse space is rewarding the genuine commitment to not feeding people a trough of lies. But, at the same time, it does feel relevant to the 'elite influence strategy' how frequently EAs have been correct about so many things, coupled with the existence of a robust trail of evidence that they have been a) correct, b) honest, and c) proactive about those things. Certainly it's updated *me, personally* towards being far more overtly 'pro-EA!'
Also here is an article if you want to indulge in me getting big mad on your behalf at how much of the discourse is "but lol you're just a shill for billionaires, since we all know everyone gets into <checks notes> the lucrative domain of journalism to ball out hard": https://zachill.substack.com/p/the-furtive-fallacy-billionaire-edition
Thanks for this article — very thought-provoking!
I believe that the HuggingFace incident was really bad and that something needs to be done to prevent things like this from happening in the future. I also believe that EAs and rationalists know far more than most people about what AI is currently capable of, so I have been following the EA & rationalist reporting on this with interest and concern.
At the same time, I don't really share the mind-body dualism that seems baked into many EA & rationalist assumptions about the world, including AI, which means I don't agree with everything in "What is AI Safety". I don't think I'm alone in that!
While disagreement is healthy, this makes it difficult for me to try to understand some of the articles about the incident, in which opinions and assumptions about the world and AI are being baked into otherwise quite rigorous reporting. What I really would like a distillation of what people close to the matter think should materially happen with AI governance based on what is materially happening now.
It's useful to know why people think these things, but it's also useful to know what they think will be useful to mitigate risks from AI even if someone disagrees with their underlying worldview.
"A complex movement of people working together to figure out what’s true" (and to work together on mitigating risks from AI) needs to not dismiss everyone who doesn't agree with certain philosophical beliefs as unintelligent and unworthy of engagement — which unfortunately many corners of these communities do. This is something else that the EA and rationalist communities can directly address, though I'm not 100% sure what the best ways are — maybe through more clearly delineating what is fact and what is belief? Offering different recommendations for different worldviews based on what is happening materially?
Oh interesting! Would be curious to know more about the mind body dualism point, I’m definitely not a dualist and would want to rewrite if it came off that way! I completely agree that a lot of AI safety comms isn't doing a good job of making it clear what background implicit assumption writers are coming from and would like to see (and produce) more of that, as well as concrete policy recommendations. This by the IFP crew was useful as one set of general policy suggestions https://www.noahpinion.blog/p/23-low-regret-recommendations-for
I'm not EA- or rationalist-affiliated and personally deeply skeptical of the "orthodox" Yudkowsky X-risk narrative. From where I sit, the chain of reasoning is too long, too narrowly specific, and it smuggles in too many assumptions that are far from self-evident outside those circles.
At the same time, we humans rarely prepare for risks until they explode in their faces. AI won't be any different. So I expect a series of rude wake-up calls, ontological shocks, crises that will have to be responded to as they happen rather than in advance. In that light, I am very grateful that smart people _have_ been taking these risks seriously when no one else was.
Unfortunately, EA - and honestly anyone with deep knowledge of the topic - suffers from a real branding problem as the messenger here, having been pre-emptive coded "weird".
I have seen people completely dismiss the METR report on the Hugging Face incident as "a tech bro fever dream" because they had received EA funding in the past. As far as critics and journalists are concerned, the AI labs, METR, EA, LessWrong, safety advocates like Helen Toner, and true X-risk doomers are all part of the same AI hype bubble - or its useful idiots (as The Atlantic explicitly called them in their hit piece).
Even after all that's happened, the approved "adult" response is still to "reject the hype" and "remember it's just predicting the next word" and "that was just in the training data".
So EA's outreach challenge is to convince people that: 1. you should ignore the data centers you all hate; 2. the hype about AI is actually true, the tech bros weren't lying; 3. real sci-fi craziness might happen, dangerously so; 4. you should accept all this from people with an unconvential philosophy and a sometimes sci-fi-coded worldview; 5. pay no attention to their close ties to the tech industry.
That's a tall order. I don't know anyone _else_ who should plant a flag, but the stranglehold of the head-in-sand narrative is going to be hard to break through.