I think the 'bottle' part of the statement hits hard too. Normally we measure water in litres, but by framing it as a wasted bottle of water, people are prompted to imagine the wasted plastic too. A bottle of water sounds far more polluting than 5 seconds of running a shower.
There's also an even more disingenuous take where data center water use is compared to "all the bottled water in a year", which sounds like a lot until you realize that drinking water is a tiny fraction of all water use, and most drinking water isn't bottled - especially outside of specific socio-economic groups in specific developed nations.
>the amount of energy required to write two emails using Llama-3-70B is enough to give you a full charge on your iPhone 16 Pro.
It's appalling that a supposed AI expert made this mistake. Llama-3-70B can be run on consumer-level hardware, which would let Rem test its energy usage himself. Alternatively, he could have read up on people that run their own LLMs locally, which would also help him understand just how off his estimate is.
What is even funnier is that people act as the water is a non-renewable resource. As if when a bottle of water is used, it is gone forever. Like everyone skipped the water cycle lessons in grade school.
The users of water in Los Angeles pay at least 20x more for their water than the farmers in the imperial and central valley do, and use a tiny fraction of what the farms do (mostly nuts, fruits and vegetables, with a little rice). And economically (although not politically) as part of California’s GDP the AI industry is much more important than the ag industry.
Holy jeepers, you are both so darn right. The book Cadillac Desert is really interesting on this. (We live in Tucson, and it is an issue for data centers *here* - but as Buzen notes, agriculture is almost infinitely worse.)
I'm curious - have you ever done a comparison with pot farming in terms of water usage?
AI is giving me estimates like "An acre of outdoor cannabis typically requires 1 to 4 acre-feet of water per year (about 325,000 to 1.3 million gallons)" and "A single standard pot joint uses about 1.5 to 2.5 gallons of water to produce."
That's in line with what I've read in terms of use. I wonder how much of the anti-datacenter is poorly (or un-) disguised anti-corporation / anti-industry ideology.
Dwarkesh Patel has started a series of deep-five videos on his channel, where he has experts explain the math and economics of AI. The first one - about inference in data centers - is really eye-opening (it was to me, anyway). Basically, any back-of-a-napkin math and naive estimates are going to be orders of magntiude off, even with commonsense extrapolations from local models and GPUs.
Surprisingly, AI companies and infrastructure providers aren't exactly dumb, and OpenAI did not just decide to give away 100-1,000x more compute for free when they launched GPT-4.
ChatGPT consumes approximately the volume of a standard bottle of water for a single 100 word email, according to a peer-reviewed paper in Communications of the ACM by researchers at the University of California: https://dl.acm.org/doi/10.1145/3724499
This paper says "Additionally, GPT-3 needs to “drink” (that is, consume) a 500ml bottle of water for roughly 10–50 medium-length responses, depending on when and where it is deployed." This is 10-50 responses, not 1, almost all of that water is offsite in the power plants generating it, and half of that is water evaporated off lakes dammed by hydroelectric plants
What is the point of these kinds of studies being done in the first place? Are they trying to preserve water by convincing people to stop using AI? If so, they need to consider that AI is used globally and inference is done all over, not at your local data center. Another big mistake is adding a fixed energy cost per query on training, since the training is only done once per model release, and the cost per query goes down the more requests are made.
If I’m in Phoenix, then limiting my use of AI, even if the queries were handled locally, isn’t going to drain Lake Mead (although, the power used in the central Arizona data centers - where most of them are in the state - likely comes from the Palo Verde Nuclear power plant, which uses reclaimed sewer water from Phoenix metro area, not ground or river water ) or solar generation whose only water use is to clean panels after a dust storm.
Mostly people want to know if the monk's knowledge Renaissance is going to be gasoline on the AI fighter?
I like how you were trying to figure things out. I wonder how the monks are going to affect you? I was hoping because you are more social than I am, you could figure out a way to help get these ideas out because they were going to change every aspect of everything in our lives. I have a tendency to talk about what's wrong with science, but they are so down that rabbit hole that that really doesn't work!
I'm continually impressed by how far you're willing to run to not just disprove, but fully model the mistakes of your adversaries.
If only the world would reward that effort with some public understanding...
Thank you for the analysis.
I think the 'bottle' part of the statement hits hard too. Normally we measure water in litres, but by framing it as a wasted bottle of water, people are prompted to imagine the wasted plastic too. A bottle of water sounds far more polluting than 5 seconds of running a shower.
There's also an even more disingenuous take where data center water use is compared to "all the bottled water in a year", which sounds like a lot until you realize that drinking water is a tiny fraction of all water use, and most drinking water isn't bottled - especially outside of specific socio-economic groups in specific developed nations.
>the amount of energy required to write two emails using Llama-3-70B is enough to give you a full charge on your iPhone 16 Pro.
It's appalling that a supposed AI expert made this mistake. Llama-3-70B can be run on consumer-level hardware, which would let Rem test its energy usage himself. Alternatively, he could have read up on people that run their own LLMs locally, which would also help him understand just how off his estimate is.
What is even funnier is that people act as the water is a non-renewable resource. As if when a bottle of water is used, it is gone forever. Like everyone skipped the water cycle lessons in grade school.
The users of water in Los Angeles pay at least 20x more for their water than the farmers in the imperial and central valley do, and use a tiny fraction of what the farms do (mostly nuts, fruits and vegetables, with a little rice). And economically (although not politically) as part of California’s GDP the AI industry is much more important than the ag industry.
Holy jeepers, you are both so darn right. The book Cadillac Desert is really interesting on this. (We live in Tucson, and it is an issue for data centers *here* - but as Buzen notes, agriculture is almost infinitely worse.)
Thanks for these details. Here’s another person who posted about it in a similar way as you, in Oct. 2024. https://www.seangoedecke.com/water-impact-of-ai/
I'm curious - have you ever done a comparison with pot farming in terms of water usage?
AI is giving me estimates like "An acre of outdoor cannabis typically requires 1 to 4 acre-feet of water per year (about 325,000 to 1.3 million gallons)" and "A single standard pot joint uses about 1.5 to 2.5 gallons of water to produce."
That's in line with what I've read in terms of use. I wonder how much of the anti-datacenter is poorly (or un-) disguised anti-corporation / anti-industry ideology.
Dwarkesh Patel has started a series of deep-five videos on his channel, where he has experts explain the math and economics of AI. The first one - about inference in data centers - is really eye-opening (it was to me, anyway). Basically, any back-of-a-napkin math and naive estimates are going to be orders of magntiude off, even with commonsense extrapolations from local models and GPUs.
Surprisingly, AI companies and infrastructure providers aren't exactly dumb, and OpenAI did not just decide to give away 100-1,000x more compute for free when they launched GPT-4.
ChatGPT consumes approximately the volume of a standard bottle of water for a single 100 word email, according to a peer-reviewed paper in Communications of the ACM by researchers at the University of California: https://dl.acm.org/doi/10.1145/3724499
This paper says "Additionally, GPT-3 needs to “drink” (that is, consume) a 500ml bottle of water for roughly 10–50 medium-length responses, depending on when and where it is deployed." This is 10-50 responses, not 1, almost all of that water is offsite in the power plants generating it, and half of that is water evaporated off lakes dammed by hydroelectric plants
What is the point of these kinds of studies being done in the first place? Are they trying to preserve water by convincing people to stop using AI? If so, they need to consider that AI is used globally and inference is done all over, not at your local data center. Another big mistake is adding a fixed energy cost per query on training, since the training is only done once per model release, and the cost per query goes down the more requests are made.
If I’m in Phoenix, then limiting my use of AI, even if the queries were handled locally, isn’t going to drain Lake Mead (although, the power used in the central Arizona data centers - where most of them are in the state - likely comes from the Palo Verde Nuclear power plant, which uses reclaimed sewer water from Phoenix metro area, not ground or river water ) or solar generation whose only water use is to clean panels after a dust storm.
Mostly people want to know if the monk's knowledge Renaissance is going to be gasoline on the AI fighter?
I like how you were trying to figure things out. I wonder how the monks are going to affect you? I was hoping because you are more social than I am, you could figure out a way to help get these ideas out because they were going to change every aspect of everything in our lives. I have a tendency to talk about what's wrong with science, but they are so down that rabbit hole that that really doesn't work!