https://pluralistic.net/2024/01/27/here-comes-the-sun-king/#sign-here
'Rooftop solar is the future, but it's also a scam. It didn't have to be, but
America decided that the best way to roll out distributed, resilient, clean and
renewable energy was to let Wall Street run the show. They turned it into a
scam, and now it's in terrible trouble. which means we are in terrible trouble.
There's a (superficially) good case for turning markets loose on the problem of
financing the rollout of an entirely new kind of energy provision across a
large and heterogeneous nation. As capitalism's champions (and apologists) have
observed since the days of Adam Smith and David Ricardo, markets harness
together the work of thousands or even millions of strangers in pursuit of a
common goal, without all those people having to agree on a single approach or
plan of action. Merely dangle the incentive of profit before the market's
teeming participants and they will align themselves towards it, like iron
filings all snapping into formation towards a magnet.
But markets have a problem: they are prone to "reward hacking." This is a term
from AI research: tell your AI that you want it to do something, and it will
find the fastest and most efficient way of doing it, even if that method is one
that actually destroys the reason you were pursuing the goal in the first
place.
<
https://learn.microsoft.com/en-us/security/engineering/failure-modes-in-machine-learning>
For example: if you use an AI to come up with a Roomba that doesn't bang into
furniture, you might tell that Roomba to avoid collisions. However, the Roomba
is only designed to register collisions with its front-facing sensor. Turn the
Roomba loose and it will quickly hit on the tactic of racing around the room in
reverse, banging into all your furniture repeatedly, while never registering a
single collision:
<
https://www.schneier.com/blog/archives/2021/04/when-ais-start-hacking.html>
This is sometimes called the "alignment problem." High-speed, probabilistic
systems that can't be fully predicted in advance can
very quickly run off the
rails. It's an idea that pre-dates AI, of course – think of the Sorcerer's
Apprentice. But AI produces these perverse outcomes at scale…and so does
capitalism.'
Cheers,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
