https://blog.katanaquant.com/p/your-llm-doesnt-write-correct-code
"One of the simplest tests you can run on a database:
Doing a primary key lookup on 100 rows.
SQLite takes 0.09 ms. An LLM-generated Rust rewrite takes 1,815.43 ms.
It’s not a misplaced comma! The rewrite is 20,171 times slower on one of the
most basic database operations.
The thing is though: The code compiles. It passes all its tests. It reads and
writes the correct SQLite file format. Its README claims MVCC concurrent
writers, file compatibility, and a drop-in C API. On first glance it reads like
a working database engine.
But it is not!
LLMs optimize for plausibility over correctness. In this case, plausible is
about 20,000 times slower than correct."
Via Christoph S.
Cheers,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
