So, first, that’s just a reduction. But set that aside, and let’s talk big picture here.
My GPU can use something like 400 watts.
A human is about 100 watts constant power consumption.
So even setting aside all other costs of a human and only paying attention to direct energy costs, if an LLM running on my GPU can do something in under a quarter the time I can, then it’s more energy-efficient.
I won’t say that that’s true for all things, but there are definitely things that Stable Diffusion or the like can do today in a whole lot less than a quarter the time it would take me.
That said, the LLM isn’t running an array of bonus functions like breathing and wondering why you said that stupid thing to your Aunt’s cousin 15 years ago and keeping tabs on your ambient noise for possible phone calls from that nice boy who promised to call you back.
So, first, that’s just a reduction. But set that aside, and let’s talk big picture here.
My GPU can use something like 400 watts.
A human is about 100 watts constant power consumption.
So even setting aside all other costs of a human and only paying attention to direct energy costs, if an LLM running on my GPU can do something in under a quarter the time I can, then it’s more energy-efficient.
I won’t say that that’s true for all things, but there are definitely things that Stable Diffusion or the like can do today in a whole lot less than a quarter the time it would take me.
That said, the LLM isn’t running an array of bonus functions like breathing and wondering why you said that stupid thing to your Aunt’s cousin 15 years ago and keeping tabs on your ambient noise for possible phone calls from that nice boy who promised to call you back.
Chat GPT can output an article in a much shorter time than it’d take me to write one but people would probably like mine more