https://www.danmcquillan.org/chatgpt.html
"Large language models (LLMs) like the GPT family learn the statistical
structure of language by optimising their ability to predict missing words in
sentences (as in 'The cat sat on the [BLANK]'). Despite the impressive
technical ju-jitsu of transformer models and the billions of parameters they
learn, it's still a computational guessing game. ChatGPT is, in technical
terms, a 'bullshit generator'. If a generated sentence makes sense to you, the
reader, it means the mathematical model has made sufficiently good guess to
pass your sense-making filter. The language model has no idea what it's talking
about because it has no idea about anything at all. It's more of a bullshitter
than the most egregious egoist you'll ever meet, producing baseless assertions
with unfailing confidence because that's what it's designed to do. It's a bonus
for the parent corporation when journalists and academics respond by generating
acres of breathless coverage, which works as PR even when expressing concerns
about the end of human creativity.
Unsuspecting users who've been conditioned on Siri and Alexa assume that the
smooth talking ChatGPT is somehow tapping into reliable sources of knowledge,
but it can only draw on the (admittedly vast) proportion of the internet it
ingested at training time. Try asking Google's BERT model about Covid or
ChatGPT about the latest Russian attacks on Ukraine. Ironically, these models
are unable to cite their own sources, even in instances where it's obvious
they're plagiarising their training data. The nature of ChatGPT as a bullshit
generator makes it harmful, and it becomes more harmful the more optimised it
becomes. If it produces plausible articles or computer code it means the
inevitable hallucinations are becoming harder to spot. If a language model
suckers us into trusting it then it has succeeded in becoming the industry's
holy grail of 'trustworthy AI'; the problem is, trusting any form of machine
learning is what leads to a single mother having their front door kicked open
by social security officials because a predictive algorithm has fingered them
as a probable fraudster, alongside many other instances of algorithmic
violence."
Via David.
Cheers,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
