I guess the risk of human extinction over the next 10 years is like 10^-7. A typical mammal species lasts 1 M years (which suggests an extinction risk of 10^-5 over 10 years), and I think humans are much more resilient. Mammals usually go extinct due to competition from other species or climate change, and I believe both of these are way less likely to drive humans extinct. Species living in larger areas are also less likely to go extinct, and humans live all across the globe.
I am open to updating to a much higher risk than suggested by the above priors, but I would need much stronger evidence. For example, a catastrophe caused by AI killing 1 M people in 1 year, or a detailed quantitative model outputting a high risk of extinction with inputs informed as much as possible by empirical data.
Would've been useful to put both of these in the same unit. So we've got about 100-120W for a human or 8200-16500kcal for a GPU. But of course we'll need several GPUs to run one LLM, then again the LLM works faster than a human. Also, solar cells produce energy more efficiently than agriculture. By the time we reach human level AI, GPUs might be much more efficient or maybe it'll run on something else entirely. So it's not really that simple. (Also doesn't really say anything about risk.)
Thanks for sharing, Andy!
I guess the risk of human extinction over the next 10 years is like 10^-7. A typical mammal species lasts 1 M years (which suggests an extinction risk of 10^-5 over 10 years), and I think humans are much more resilient. Mammals usually go extinct due to competition from other species or climate change, and I believe both of these are way less likely to drive humans extinct. Species living in larger areas are also less likely to go extinct, and humans live all across the globe.
I am open to updating to a much higher risk than suggested by the above priors, but I would need much stronger evidence. For example, a catastrophe caused by AI killing 1 M people in 1 year, or a detailed quantitative model outputting a high risk of extinction with inputs informed as much as possible by empirical data.
The magic is that the human body operates on 2000-2500 calories of energy whereas GPUs require between 400-800W.
Would've been useful to put both of these in the same unit. So we've got about 100-120W for a human or 8200-16500kcal for a GPU. But of course we'll need several GPUs to run one LLM, then again the LLM works faster than a human. Also, solar cells produce energy more efficiently than agriculture. By the time we reach human level AI, GPUs might be much more efficient or maybe it'll run on something else entirely. So it's not really that simple. (Also doesn't really say anything about risk.)