<
https://pluralistic.net/2024/10/30/a-neck-in-a-noose/#is-also-a-human-in-the-loop>
'AI's ability to make – or assist with – important decisions is fraught: on the
one hand, AI can
often classify things very well, at a speed and scale that
outstrips the ability of any reasonably resourced group of humans. On the other
hand, AI is sometimes very wrong, in ways that can be terribly harmful.
Bureaucracies and the AI pitchmen who hope to sell them algorithms are very
excited about the cost-savings they could realize if algorithms could be turned
loose on thorny, labor-intensive processes. Some of these are relatively
low-stakes and make for an easy call: Brewster Kahle recently told me about the
Internet Archive's project to scan a ton of journals on microfiche they bought
as a library discard. It's pretty easy to have a high-res scanner auto-detect
the positions of each page on the fiche and to run the text through OCR, but a
human would still need to go through all those pages, marking the first and
last page of each journal and identifying the table of contents and indexing it
to the scanned pages. This is something AI apparently does
very well, and
instead of scrolling through endless pages, the Archive's human operator now
just checks whether the first/last/index pages the AI identified are the right
ones. A project that could have taken years is being tackled with never-seen
swiftness.
The operator checking those fiche indices is something AI people like to call a
"human in the loop" – a human operator who assesses each judgment made by the
AI and overrides it should the AI have made a mistake. "Humans in the loop"
present a tantalizing solution to algorithmic misfires, bias, and unexpected
errors, and so "we'll put a human in the loop" is the cure-all response to any
objection to putting an imperfect AI in charge of a high-stakes application.
But it's not just AIs that are imperfect. Humans are
wildly imperfect, and
one thing they turn out to be
very bad at is supervising AIs. In a 2022 paper
for
Computer Law & Security Review, the mathematician and public policy
expert Ben Green investigates the empirical limits on human oversight of
algorithms:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3921216
Green situates public sector algorithms as the latest salvo in an age-old
battle in public enforcement. Bureaucracies have two conflicting,
irreconcilable imperatives: on the one hand, they want to be fair, and treat
everyone the same. On the other hand, they want to exercise discretion, and
take account of individual circumstances when administering justice. There's no
way to do both of these things at the same time, obviously.'
Cheers,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
