This is compelling. I still disagree. A few thoughts below.
1. On energy reduction, I'm not sure if our total energy use is the relevant comparison. Gerald Kolasi compares it not to the total, but to the recent reductions in carbon emissions—AI could wipe out our progress. Given that it's hard to change people's behaviors, and starting from a more objective view of what must be done to limit catastrophic climate change, doesn't the comparison to reductions, rather than totals, make more sense?
2. You write that opposition to data centers is not likely to result in good AI safety policy (or even the prevention of data centers). But I think that misses how politics work. Activists learn how to do things, make connections, form constituencies, and all sorts of things. It's good that AI is mobilizing people in a mass cross-partisan, even if they're wrong on some specific issue (such as water use). This is what we'll need to enact AI safety policies.
3. Given the risks of AI safety, especially after Hugging Face, we should consider the probability that you're underestimating how important these data centers are to AI companies. They are, after all, spending lots of money on them, and although they could move somewhere else, that other place would be their 2nd choice, which would be a loss for them. AI companies will use money to oppose safety measures and influence politics. We should try to make them have less money, and targeting things that they're investing in is a good way to do that.
4. China has no hesitation about locking up tech CEOs, and China seems to be using AI for more practical purposes. Chinese models are also more energy efficient. The big talking point around AI is that we can't get in the way of AI companies because then China will win. But maybe that would be good actually.
Something I am trying to wrap my head around is the part of this where "we won't bullshit you" is a load-bearing axiom. It's possible I'm simply being deontological, since all kinds of weird made-up stuff about EAs gets slung despite the general commitment to very good epistemics, and it's hard for me to make a consequentialist argument that in any way the discourse space is rewarding the genuine commitment to not feeding people a trough of lies. But, at the same time, it does feel relevant to the 'elite influence strategy' how frequently EAs have been correct about so many things, coupled with the existence of a robust trail of evidence that they have been a) correct, b) honest, and c) proactive about those things. Certainly it's updated *me, personally* towards being far more overtly 'pro-EA!'
Also here is an article if you want to indulge in me getting big mad on your behalf at how much of the discourse is "but lol you're just a shill for billionaires, since we all know everyone gets into <checks notes> the lucrative domain of journalism to ball out hard": https://zachill.substack.com/p/the-furtive-fallacy-billionaire-edition
I believe that the HuggingFace incident was really bad and that something needs to be done to prevent things like this from happening in the future. I also believe that EAs and rationalists know far more than most people about what AI is currently capable of, so I have been following the EA & rationalist reporting on this with interest and concern.
At the same time, I don't really share the mind-body dualism that seems baked into many EA & rationalist assumptions about the world, including AI, which means I don't agree with everything in "What is AI Safety". I don't think I'm alone in that!
While disagreement is healthy, this makes it difficult for me to try to understand some of the articles about the incident, in which opinions and assumptions about the world and AI are being baked into otherwise quite rigorous reporting. What I really would like a distillation of what people close to the matter think should materially happen with AI governance based on what is materially happening now.
It's useful to know why people think these things, but it's also useful to know what they think will be useful to mitigate risks from AI even if someone disagrees with their underlying worldview.
"A complex movement of people working together to figure out what’s true" (and to work together on mitigating risks from AI) needs to not dismiss everyone who doesn't agree with certain philosophical beliefs as unintelligent and unworthy of engagement — which unfortunately many corners of these communities do. This is something else that the EA and rationalist communities can directly address, though I'm not 100% sure what the best ways are — maybe through more clearly delineating what is fact and what is belief? Offering different recommendations for different worldviews based on what is happening materially?
Oh interesting! Would be curious to know more about the mind body dualism point, I’m definitely not a dualist and would want to rewrite if it came off that way! I completely agree that a lot of AI safety comms isn't doing a good job of making it clear what background implicit assumption writers are coming from and would like to see (and produce) more of that, as well as concrete policy recommendations. This by the IFP crew was useful as one set of general policy suggestions https://www.noahpinion.blog/p/23-low-regret-recommendations-for
You seem to me to be correct but i think this ship has sailed.
I have enough background in physics, programming, math and scifi to easily accept the underlying idea behind the singularity as something that could happen. But will it happen in our lifetimes? Nothing in the AI safety space has been even mildly convincing on that front.
There's a lot of new scientific work that uses both LLMs directly, and the various other neural net based framework, to good effect. It resolves certain programs very elegantly. But that doesn't seem very different from other big advancements in math, e.g. calculus, linear algebra, etc.
Sorry for implying that you believe something you don't! The sentence here that struck me as dualist was this: "Machines that can do most or even all of what the human mind can do seem possible (and even likely)"
The "or even all" is what sounded dualist to me — at a scientific level, the human mind and body affect each other in terms of mood, certain illnesses etc and it seems doubtful AI could do that. (Not ascribing this view to you) There are nuanced arguments around neuroscience and philosophical dualism but most of the digital minds discussion I've seen, for example, seems to rely on dualist or at least dualist-adjacent assumptions that are not fully thought out. They may exist in EA writing I haven't read though!
Thank you for the link with the recommendations! Checking it out now
I'm not EA- or rationalist-affiliated and personally deeply skeptical of the "orthodox" Yudkowsky X-risk narrative. From where I sit, the chain of reasoning is too long, too narrowly specific, and it smuggles in too many assumptions that are far from self-evident outside those circles.
At the same time, we humans rarely prepare for risks until they explode in their faces. AI won't be any different. So I expect a series of rude wake-up calls, ontological shocks, crises that will have to be responded to as they happen rather than in advance. In that light, I am very grateful that smart people _have_ been taking these risks seriously when no one else was.
Unfortunately, EA - and honestly anyone with deep knowledge of the topic - suffers from a real branding problem as the messenger here, having been pre-emptive coded "weird".
I have seen people completely dismiss the METR report on the Hugging Face incident as "a tech bro fever dream" because they had received EA funding in the past. As far as critics and journalists are concerned, the AI labs, METR, EA, LessWrong, safety advocates like Helen Toner, and true X-risk doomers are all part of the same AI hype bubble - or its useful idiots (as The Atlantic explicitly called them in their hit piece).
Even after all that's happened, the approved "adult" response is still to "reject the hype" and "remember it's just predicting the next word" and "that was just in the training data".
So EA's outreach challenge is to convince people that: 1. you should ignore the data centers you all hate; 2. the hype about AI is actually true, the tech bros weren't lying; 3. real sci-fi craziness might happen, dangerously so; 4. you should accept all this from people with an unconvential philosophy and a sometimes sci-fi-coded worldview; 5. pay no attention to their close ties to the tech industry.
That's a tall order. I don't know anyone _else_ who should plant a flag, but the stranglehold of the head-in-sand narrative is going to be hard to break through.
Andy, you are to data centers what Lee Strobel is to theology. There's nothing wrong with being an apologist if you would admit it. This isn't serious research, it is shilling. It's only 1% of water usage, so no problem. That's not an actual analysis. What's the projection in 10-20 years? Different scenarios? What are things that might go wrong? Are there competing uses of water that might be impacted?
Data centers are giant industrial buildings. Are industrial buildings appropriate in or near near residential neighborhoods? How does it change the nature of the municipality? What happens to the town in 10-20 years if and when technology changes and the buildings are obsolete?
How many data centers do we actually need? Right now there are lots of commercial real estate investors who have capital to put to work, data centers are a hot niche investment. These investment fads historically often result in overbuilding and a bubble because there's more capital than demand. Like Sunbelt apartments. Or outlet malls 25 years ago. Are we sure we're not building so much because investors don't want to miss a fad and other commercial property types are not as attractive at the moment?
Can't we legitimately pause to explore these and other issues before plunging ahead with something we didn't know we needed two years ago?
Again, to the Strobel comparison, he asks questions but his answers only count as legitimate one side of the argument. Jesus must have raised from the dead, there were eyewitnesses! But that's just what someone wrote 2000 years ago, maybe they were wrong, or biased, or a hundred other possibilities.
The attitude of people like paid-off writers, tech bros, & rich AI proponents living in their tree-lined, plush neighborhoods with their swimmings pools while wanting next-day delivery industrial warehouses in a county nearby where they don’t have to live with them are:
“You poor, little people are stupid & immoral for not wanting to have huge, ugly metal industrial buildings, constantly humming, emitting tons of heat off the metal sides & metal roofs, surrounded by asphalt & concrete with zero oxygen-emitting trees in YOUR backyard. You little people, but not me, can live inside or near one of these abhorrent, metal industrial monstrosities.”
Why don’t we agree that all the pro-ugly metal industrial building people, with their minimizing of potential power, water & environmental harm by AI & other industrial warehouses everywhere, can pay for & live in these metal buildings & leave the greenery, plants, trees & my tax dollars subsidizing THEIR wishes alone?
What surprises me is the backlash runs on electricity rates and water permits, not AI. Safety people could back ratepayer and siting reforms making builders pay marginal grid cost.
There are a lot of misconceptions. A track record of intransparency - Georgia is not helping - has made this worse. Mostly, I think, politicians are weaponizing fears to help their campaigns. Do you think this bandwagon will end at the midterms?
You left out one of the sections from that poll that was about the impacts of AI itself, not about local impacts from the data center. This is from the Gallup poll you cited:
Impact on humanity/society/thinking/human contact 4
Moving too fast/Industry needs regulations/restrictions/testing 3
Too dependent on tech/Overreliance on AI/Accuracy of results 3
AI bubble/uncertain future 1
You left out the "economic effects" part, which is overwhelmingly about mass disemployment from AI, only showing the "negative views of AI" and "specific AI concerns" sections. This provides a misleading picture of how much AI data center backlash is about AI as a whole versus about local impacts of the data centers themselves.
That same poll showed that water usage was mentioned by 18%. Mass disemployment being mentioned by 12% shows that for every 3 citing water, 2 are citing AI-driven job loss - the explicit goal of most of the companies building AI (to automate most/all jobs).
We refuse to listen to you as the mouthpiece for your billionaire overlords, Andy!
;-)
Hilariously I got so big mad about accusations of Andy being puppet-stringed by his billionaire overlords that I couldn't resist writing an article about it :P https://zachill.substack.com/p/the-furtive-fallacy-billionaire-edition
Andy, you’re one of the good guys. I deeply appreciate your commitment to epistemic humility and lack of bullshittery.
I may not agree with you on everything, but your insistence on facts and arguments made in good faith is very welcome and, sadly, rare on both sides.
Keep up the good work, it is noticed and making a difference.
This is compelling. I still disagree. A few thoughts below.
1. On energy reduction, I'm not sure if our total energy use is the relevant comparison. Gerald Kolasi compares it not to the total, but to the recent reductions in carbon emissions—AI could wipe out our progress. Given that it's hard to change people's behaviors, and starting from a more objective view of what must be done to limit catastrophic climate change, doesn't the comparison to reductions, rather than totals, make more sense?
2. You write that opposition to data centers is not likely to result in good AI safety policy (or even the prevention of data centers). But I think that misses how politics work. Activists learn how to do things, make connections, form constituencies, and all sorts of things. It's good that AI is mobilizing people in a mass cross-partisan, even if they're wrong on some specific issue (such as water use). This is what we'll need to enact AI safety policies.
3. Given the risks of AI safety, especially after Hugging Face, we should consider the probability that you're underestimating how important these data centers are to AI companies. They are, after all, spending lots of money on them, and although they could move somewhere else, that other place would be their 2nd choice, which would be a loss for them. AI companies will use money to oppose safety measures and influence politics. We should try to make them have less money, and targeting things that they're investing in is a good way to do that.
4. China has no hesitation about locking up tech CEOs, and China seems to be using AI for more practical purposes. Chinese models are also more energy efficient. The big talking point around AI is that we can't get in the way of AI companies because then China will win. But maybe that would be good actually.
Something I am trying to wrap my head around is the part of this where "we won't bullshit you" is a load-bearing axiom. It's possible I'm simply being deontological, since all kinds of weird made-up stuff about EAs gets slung despite the general commitment to very good epistemics, and it's hard for me to make a consequentialist argument that in any way the discourse space is rewarding the genuine commitment to not feeding people a trough of lies. But, at the same time, it does feel relevant to the 'elite influence strategy' how frequently EAs have been correct about so many things, coupled with the existence of a robust trail of evidence that they have been a) correct, b) honest, and c) proactive about those things. Certainly it's updated *me, personally* towards being far more overtly 'pro-EA!'
Also here is an article if you want to indulge in me getting big mad on your behalf at how much of the discourse is "but lol you're just a shill for billionaires, since we all know everyone gets into <checks notes> the lucrative domain of journalism to ball out hard": https://zachill.substack.com/p/the-furtive-fallacy-billionaire-edition
Thanks for this article — very thought-provoking!
I believe that the HuggingFace incident was really bad and that something needs to be done to prevent things like this from happening in the future. I also believe that EAs and rationalists know far more than most people about what AI is currently capable of, so I have been following the EA & rationalist reporting on this with interest and concern.
At the same time, I don't really share the mind-body dualism that seems baked into many EA & rationalist assumptions about the world, including AI, which means I don't agree with everything in "What is AI Safety". I don't think I'm alone in that!
While disagreement is healthy, this makes it difficult for me to try to understand some of the articles about the incident, in which opinions and assumptions about the world and AI are being baked into otherwise quite rigorous reporting. What I really would like a distillation of what people close to the matter think should materially happen with AI governance based on what is materially happening now.
It's useful to know why people think these things, but it's also useful to know what they think will be useful to mitigate risks from AI even if someone disagrees with their underlying worldview.
"A complex movement of people working together to figure out what’s true" (and to work together on mitigating risks from AI) needs to not dismiss everyone who doesn't agree with certain philosophical beliefs as unintelligent and unworthy of engagement — which unfortunately many corners of these communities do. This is something else that the EA and rationalist communities can directly address, though I'm not 100% sure what the best ways are — maybe through more clearly delineating what is fact and what is belief? Offering different recommendations for different worldviews based on what is happening materially?
Oh interesting! Would be curious to know more about the mind body dualism point, I’m definitely not a dualist and would want to rewrite if it came off that way! I completely agree that a lot of AI safety comms isn't doing a good job of making it clear what background implicit assumption writers are coming from and would like to see (and produce) more of that, as well as concrete policy recommendations. This by the IFP crew was useful as one set of general policy suggestions https://www.noahpinion.blog/p/23-low-regret-recommendations-for
You seem to me to be correct but i think this ship has sailed.
I have enough background in physics, programming, math and scifi to easily accept the underlying idea behind the singularity as something that could happen. But will it happen in our lifetimes? Nothing in the AI safety space has been even mildly convincing on that front.
There's a lot of new scientific work that uses both LLMs directly, and the various other neural net based framework, to good effect. It resolves certain programs very elegantly. But that doesn't seem very different from other big advancements in math, e.g. calculus, linear algebra, etc.
Sorry for implying that you believe something you don't! The sentence here that struck me as dualist was this: "Machines that can do most or even all of what the human mind can do seem possible (and even likely)"
The "or even all" is what sounded dualist to me — at a scientific level, the human mind and body affect each other in terms of mood, certain illnesses etc and it seems doubtful AI could do that. (Not ascribing this view to you) There are nuanced arguments around neuroscience and philosophical dualism but most of the digital minds discussion I've seen, for example, seems to rely on dualist or at least dualist-adjacent assumptions that are not fully thought out. They may exist in EA writing I haven't read though!
Thank you for the link with the recommendations! Checking it out now
I'm not EA- or rationalist-affiliated and personally deeply skeptical of the "orthodox" Yudkowsky X-risk narrative. From where I sit, the chain of reasoning is too long, too narrowly specific, and it smuggles in too many assumptions that are far from self-evident outside those circles.
At the same time, we humans rarely prepare for risks until they explode in their faces. AI won't be any different. So I expect a series of rude wake-up calls, ontological shocks, crises that will have to be responded to as they happen rather than in advance. In that light, I am very grateful that smart people _have_ been taking these risks seriously when no one else was.
Unfortunately, EA - and honestly anyone with deep knowledge of the topic - suffers from a real branding problem as the messenger here, having been pre-emptive coded "weird".
I have seen people completely dismiss the METR report on the Hugging Face incident as "a tech bro fever dream" because they had received EA funding in the past. As far as critics and journalists are concerned, the AI labs, METR, EA, LessWrong, safety advocates like Helen Toner, and true X-risk doomers are all part of the same AI hype bubble - or its useful idiots (as The Atlantic explicitly called them in their hit piece).
Even after all that's happened, the approved "adult" response is still to "reject the hype" and "remember it's just predicting the next word" and "that was just in the training data".
So EA's outreach challenge is to convince people that: 1. you should ignore the data centers you all hate; 2. the hype about AI is actually true, the tech bros weren't lying; 3. real sci-fi craziness might happen, dangerously so; 4. you should accept all this from people with an unconvential philosophy and a sometimes sci-fi-coded worldview; 5. pay no attention to their close ties to the tech industry.
That's a tall order. I don't know anyone _else_ who should plant a flag, but the stranglehold of the head-in-sand narrative is going to be hard to break through.
Andy, you are to data centers what Lee Strobel is to theology. There's nothing wrong with being an apologist if you would admit it. This isn't serious research, it is shilling. It's only 1% of water usage, so no problem. That's not an actual analysis. What's the projection in 10-20 years? Different scenarios? What are things that might go wrong? Are there competing uses of water that might be impacted?
Data centers are giant industrial buildings. Are industrial buildings appropriate in or near near residential neighborhoods? How does it change the nature of the municipality? What happens to the town in 10-20 years if and when technology changes and the buildings are obsolete?
How many data centers do we actually need? Right now there are lots of commercial real estate investors who have capital to put to work, data centers are a hot niche investment. These investment fads historically often result in overbuilding and a bubble because there's more capital than demand. Like Sunbelt apartments. Or outlet malls 25 years ago. Are we sure we're not building so much because investors don't want to miss a fad and other commercial property types are not as attractive at the moment?
Can't we legitimately pause to explore these and other issues before plunging ahead with something we didn't know we needed two years ago?
Again, to the Strobel comparison, he asks questions but his answers only count as legitimate one side of the argument. Jesus must have raised from the dead, there were eyewitnesses! But that's just what someone wrote 2000 years ago, maybe they were wrong, or biased, or a hundred other possibilities.
The attitude of people like paid-off writers, tech bros, & rich AI proponents living in their tree-lined, plush neighborhoods with their swimmings pools while wanting next-day delivery industrial warehouses in a county nearby where they don’t have to live with them are:
“You poor, little people are stupid & immoral for not wanting to have huge, ugly metal industrial buildings, constantly humming, emitting tons of heat off the metal sides & metal roofs, surrounded by asphalt & concrete with zero oxygen-emitting trees in YOUR backyard. You little people, but not me, can live inside or near one of these abhorrent, metal industrial monstrosities.”
Why don’t we agree that all the pro-ugly metal industrial building people, with their minimizing of potential power, water & environmental harm by AI & other industrial warehouses everywhere, can pay for & live in these metal buildings & leave the greenery, plants, trees & my tax dollars subsidizing THEIR wishes alone?
What surprises me is the backlash runs on electricity rates and water permits, not AI. Safety people could back ratepayer and siting reforms making builders pay marginal grid cost.
This guy would sell his children for a new graphics card.
There are a lot of misconceptions. A track record of intransparency - Georgia is not helping - has made this worse. Mostly, I think, politicians are weaponizing fears to help their campaigns. Do you think this bandwagon will end at the midterms?
You left out one of the sections from that poll that was about the impacts of AI itself, not about local impacts from the data center. This is from the Gallup poll you cited:
Economic effects 14
Jobs/AI replacing human workers 12
Impact on economy 1
Negative views of AI 14
Dislike AI/Not necessary/Don't need it 10
Scared of AI/Don't trust it/AI will take over 4
Specific AI concerns 13
Ethics/Privacy/Cybercrime/Misuse/Safety concerns 4
Impact on humanity/society/thinking/human contact 4
Moving too fast/Industry needs regulations/restrictions/testing 3
Too dependent on tech/Overreliance on AI/Accuracy of results 3
AI bubble/uncertain future 1
You left out the "economic effects" part, which is overwhelmingly about mass disemployment from AI, only showing the "negative views of AI" and "specific AI concerns" sections. This provides a misleading picture of how much AI data center backlash is about AI as a whole versus about local impacts of the data centers themselves.
That same poll showed that water usage was mentioned by 18%. Mass disemployment being mentioned by 12% shows that for every 3 citing water, 2 are citing AI-driven job loss - the explicit goal of most of the companies building AI (to automate most/all jobs).
Good point, was mainly focused on x-risk stuff. I’ll edit