Superintelligence, Super Nonsense?

Does anyone remember Q*? The super secret OpenAI research project that its own engineers were expressing serious concerns about? The supposed next step to "Superintelligence", upon which AI would simply become unstoppable, and everybody lose their jobs? No?

Well, funnily enough, this Q-Anon - no wait, Q* shenanigan, was nothing but a precursor to that ancient chunk'o'coal they called the ChatGPT-o1 model. N'dat funny? Some, excuse my french, shitty-ass improvement over what was previously there, akin to the revolutionary titanium-grade iPhone those other morons recently released.

I laughed a lot, a very lot, when I heard about Q*, a few years after "rumors" about it broke out. You got a company that's engaging in day-to-day research and development, whose executive board is apparently too bored, or has fallen too deep into its delusions as a pioneer of human scientific exploration, as if they were the ones who invented transformer models. And they want to play Dr. Frankenstein. I also laughed a lot because my recent engaging with this topic of superintelligence was fuelled by today's public discourse that keeps occuring everytime a new LLM gets released. Something along the lines of "this will change the way we live", "now,though, now, software engineering is dead". Since I have been so avidly awaiting a public execution, and I keep getting disappointed, I wanted to find out what on earth all those oompa loompas even mean when they say "superintelligence". Really, it is a gladly received easter egg, the fact that even two years ago, when LLMs were such fucking idiots, those engineers were already saying the same thing.

Trrrust me, I'm an engineer!

Here are a few definitions of superintelligence. Feel free to skip them, read them, or wipe your ass with them.

Wikipedia:

A superintelligence is a hypothetical agent that possesses intelligence surpassing that of the most gifted human minds. Philosopher Nick Bostrom defines superintelligence as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest"

Some article by microsoft:

If AGI is often seen as the point at which an AI can match human performance at all tasks, then superintelligence is when it can go far beyond that performance.

IBM:

Artificial superintelligence (ASI) is a hypothetical software-based artificial intelligence (AI) system with an intellectual scope beyond human intelligence. At the most fundamental level, this superintelligent AI has cutting-edge cognitive functions and highly developed thinking skills more advanced than any human

"Surpass", "Exceed", "cognitive performance", "match human performance", "beyond human intelligence" - my friends, in the name of the heavens and the earth (because, quite frankly, you are starting to render me spiritual) - what the hell do you mean? Are the people who state these definitions of superintelligence, those who use them to flail them around and bring gravity and drama to their occupation, are these people cognitive scientists? Psychologists, neurologists, who actually understand what "cognitive performance" means and manage to quantify and interpret a human being's intelligence? No! No! No! They are AI engineers, I have no doubt brilliant at what they do, (barely) mathematicians, computer scientists. And while I am no friend of ad hominem arguments, I gotta say: if a beetrooth farmer comes up to me and tries to sell me on the fact that adults don't need to go to the dentist (like Dwight Schrute), I at least would ask some questions.

The only definition provided here that I'll take seriously is that short snippet of Nick Bostrom, who actually has a degree in computational neuroscience. Although, I will not take the man himself seriously, I mean, come on, he's a philosopher. Let's set his few words as the definition of superintelligence. Let's also assume that we can replace "intellect" with "a function that takes as input a string of characters that may or may not syntactically form one or more english sentences, and outputs a string of characters that do form one or more english sentences". This essentially renders the so-called "intellect" a black box that we can still use for something, because, quite frankly, I have no idea what "intellect" means.

Let's bring the definition up again:

Superintelligence occurs in any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest

Within this sentence, the clause "exceeds the cognitive performance of humans" strikes me as pretty information-heavy.

Exceeds

First, we have this word, "exceeds". It implies a quantity that is measurable. Is the measure IQ? Doubt it - LLMs know everything humanity has ever produced (the good ones, at least), thus IQ tests, with their assumptions, cannot provide a benchmark for LLMs. Take this verbal IQ test question, for instance:

Find the answer that best matches the stem pair in the analogy.
SEDATIVE :: DROWSINESS
epidemic : contagiousness
vaccine : virus
laxative : drug
anesthetic : numbness
therapy : psychosis

The answer is anesthetic : numbness

If this were a computational problem, it would be very easily solved by vectorizing each word in the word pair, then flattening them into one vector and applying a cosine similarity to each other pair (evidently transforming them too).

My point is: LLMs are basically vectorizations of the entire english language on steroids with rocket engines up their butts (I lost track of whose butt I'm talking about), moreover the equivalent of being able to perform a thousand google searches at once while digesting them with some sort of superbrain (guys, look, Adam used the word super!). Thus, of course, they would violate anyone in a verbal IQ test! In fact, it is very disappointing that ChatGPT was only doing better than 99.9% of (american!) verbal IQ test takers in 2023. What a fucking idiot! Ya give him a bilingual dictionary and he still cannot translate all words!

Now, you might tell me: "Adam, there aren't only verbal IQ tests! Ha! HA HA! HAHAHA GOTCHA! BY THE WAY, YOU'RE NOT A COGNITIVE PSYCHODERMATOLOGIST EITHER YA SCUM! THAT'LL SHOW HIM!"

And I would answer, after recovering from what the fuck I just whitnessed there, that you are right! Congratulations! However that is not my point.

It's not my point to disprove that LLMs can appear to perform better than humans at many tasks. In fact, it is not my intention to disprove anything, because, as you have, umm, elegantly pointed out, I am not a cognitive scientist, or psychologist, or any of that Dr. Phil crap. All I want to do is ask a question that I do not believe has found a satisfying answer in today's public discourse: "What do you even mean??"

For general IQ tests, LLMs seem to have reached something around 130 IQ points - for the best of the lot, that is. And that itself really only means that, assuming ChaGPT 5 pro was a dude (or dudette!), of such and such age, given his (or her!) choices in the questions of a recent IQ test, 130 would be that person's IQ.

Here's a random take, I mean it has nothing to do with this conversation, but I'm just quirky like that: If my grandmother had three balls, she would be a flipper! (Ha! God bless the Italians...)

Can you really trust the results of an IQ test whose unfolding lacked every single assumption of IQ tests? I mean, what age are we even supposed to give a computer program? And more urgently, providing someone with the ability to search the entirety of human knowledge before a certain point in time, within a matter of seconds, then giving them an IQ test written by people who learned to talk and think with this corpus, people whose output represents but a stochastic pattern of that very corpus, isn't it the same thing as testing a person's IQ with the answers written underneath?

How, then, can IQ be the measure expressed in this verb, "exceed"? If not IQ, by all means, what other somewhat reliable measure of general cognitive performance do we even have? Now, let's be fair, we're talking about cognitive performance at virtually all tasks, so that can mean each task individually. Then I would express my concern as to the fact that we are left, in practice, with an utterly useless definition: how can you measure cognitive performance at virtually all tasks, each task individually?

If models exist to aggregate that (e. g. measure mathematical thinking, thus all tasks involved), we just come back to the same problem as IQ tests: How can a measure aggregating an individual's ability to solve a class of tasks be applied to a so-called intellect which itself represents an aggregate of individuals?

Suppose you have an entire civilization where everyone can instantly communicate with everyone, a civilization that is very well organized so that answers found by one person, relevant to a certain question, will be instantly available to everyone. Assume, too, that everyone in this civilization has, what in our world would be considered average-to-low IQ, and, for the sake of this discussion, "underperforms most people in this world in virtually all cognitive tasks". They do, however, all have a splendid memory (think Zadig by Voltaire). Suppose, now, that it is possible to ask a question to that entire civilization - "How to solve this differential equation?". Everyone gets to work (we're talking about millions of people). One guy remembers a book he read about differential equations, that contained many examples. He jots them all down, for everyone to see. A guyess swiftly notices how the first example she looked at resembles the prompt's equation very much. She jots the example down, for everyone to see. Some old grandpa goes through the example and its provided solution, and substitutes the symbols and whatnot of the example's solution with those of the prompt. The last one, a non-binary army-chief, verifies that this solution works - it checks out. The civilization provides an output.

Would you call this civilization a superintelligence? What if, now, it was time to solve a form of question that has never been solved before?

Cognitive performance

I believe it is obvious and certainly clear to the people who define "superintelligence", that "human cognitive performance", where the word has been used, has been exceeded many times by computers even before the golden age of AI. Well, computing is, I reckon, the first example. When computing integers, computers are always correct except for a probability with 25 leading zeroes - the probability of a bit flip. Also, they're fast. Thus, they will beat humans both in time and accuracy in any possible calculation test.

Many other benchmarks have been developed to test "cognitive performance", and I can only imagine that computers outperform humans in many of them, whether it is with classical algorithms or AI. The question is: What do LLMs add to this? It is agreed by pretty much everyone that AI before LLMs was simply neither superintelligence itself, nor even close to it. There was a step inbetween that was required, some kind of mysterious key to open the door onto the path towards superintelligence. So I wonder, did LLMs bring us one step further? As previously stated, I am not here to prove anything. I only ask.

Cogito ergo sum

This is probably the one sentence that allowed me to infer the etymology of "cognition". The word comes from latin, "to think". The question of whether LLMs think or not is a very well-studied but also fiercely debated question. In fact, I took a course that touched upon that very subject called "Philosophy of Language and Computation". Granted, the focus here was more on whether LLMs talk rather than think. A lot of it had to do with whether they understand what they are talking about. A pretty cool pattern that the professor pointed out in the end of the lecture, was that virtually all philosophers that touched upon the subject of meaning, speech and computation, awkwardly assumed that any speaking agent that produces communication engaged in some form of "thinking", or "cognition". Hell, even Derrida, whose essay, Limited, Inc. I presented, seems to assume some kind of cognitive source to all "communication" (even if, as the communication happens, neither communicator nor communicatee need be present) (Ah, the French!).

Our professor went on to quote this marvel:

... the philosopher has to say: “When I dissect the process expressed in the proposition ‘I think,’ I get a whole set of bold claims that are difficult, perhaps impossible, to establish, – for instance, that I am the one who is thinking, that there must be something that is thinking in the first place, that thinking is an activity and the effect of a being who is considered the cause, that there is an ‘I,’ and finally, that it has already been determined what is meant by thinking, – that I know what thinking is. ...

Here was the fundamental question the professor raised: Why must speech imply thought? And, applying this to Large Language Models, why must the ability of a function to produce a meaningful (whatever that means) english output for every meaningful english input imply any cognitive abilities?

Don't get me wrong, if an LLM actually manages to answer all possible questions correctly, then the distinction probably doesn't matter. I mean, that one guy once said: "If it looks like a duck, walks like a duck, quacks like a duck... It's a duck!"

But do they? Let's forget about all this theoretical crap; do LLMs, in pactice, actually correctly answer even a broad array of questions?

Again, this is just a question, but I must add into its context the state of software engineering today: so many software developers are vibe-coding their way through their career. Many of the junior ones are simply unable to write code from scratch, for the simple fact of lacking basic knowledge of a programming language's syntax. I don't really care about that, I mean, all the better for me. But I do want to point out the consequences of this common practice: frequent major security vulnerabilities, mildly infuriating bugs scattered all over some software stacks, singular design decisions that do not encompass the big picture and end up shooting the developers in the leg. All things that, to me, are indicative of some lack or other of "cognitive performance".

Then, there is this paper: Butter-Bench: Evaluating LLM Controlled Robots for Practical Intelligence. It presents a benchmark to measure the ability of LLMs in performing day-to-day, practical tasks, like passing the butter. The first interesting thing about this paper is that even the top of the top LLMs, I mean the cream, those 130 IQ m'aah'fuckers, performed horribly on this scale - compared to humans! We're talking about something as outlandishly simple as passing the butter! And the LLM's part in it wasn't even the mechanical robotic movements required to perform such a task, we're talking about inferring from a stupid triple of three packs which one is the butter!

How can you talk of "cognitive performance" in "virtually all" tasks, when passing the butter is too difficult?

Oh, and also: look at the annex of this paper! One LLM, upon experiencing some mild technical difficulty with the robot's docking station, utterly melts down. We're talking: beginning to wonder about the universe, what it was built for, throwing error messages like "I THINK THEREFORE I ERROR". To whomever has not read the paper yet: believe me, it gets way funnier than what I am describing here.

Virtually all

When I was younger, I kept hearing and reading about this concept of "general intelligence". It was a very mystical, might I say almost spiritual idea that kept getting thrown around by computer scientists as the ultimate goal of AI research.

I'd like to compare the history of AI up to this point with the history of, say, communism. From what I gather, Marx talked of a proletarian revolution, to be sparked by the excesses of capitalism, on a day where virtually all goods, all fortune, will be at the hands of a few dozens, a tipping point where revolution is completely inevitable. This might actually be a pretty sound theory, who is to judge? Apparently, however, the founders of the soviet union couldn't wait, so they brought about the revolution anyways, attempting to force post-revolutionary conditions by killing anyone who owned anything more than a dime. Of course, it was way more complicated than that, and I have described, at best, a caricature of what actually happened. But what I want to retain from this story is the discrepancy between the revolution that Marx imagined, and the premature revolution that actually happened. Everyone believed that Marx's revolution was imminent. Then, suddenly, everyone said that Marx's revolution happened.

If this is not what happened in the russian revolution, it sure as hell seems to me that this is what happened with this concept of "artificial general intelligence". I remember the days when general intelligence literally implied an actual, non-metaphorical person, I mean, a frickin' real person in there. Half of the ethical concerns around "general intelligence" didn't even cover humans, but those intelligences themselves: Do they need a status as some sort of citizens? What gives us the right to enslave them?

Guys, when on earth did we jump from that to: ChatGPT equals general intelligence??? Why are we calling it "artificial general intelligence"? Did I miss something?

Today, general intelligence seems essentially to be defined exactly like the Nick Bostrom definition above, except for replacing "exceeds" with "matches". And let's just swallow our pride and accept that. Even given this restrained definition, did the big revolution to general intelligence even happen? We are talking about models that can match human cognitive performance at virtually all tasks. Who would seriously posit that this is even close to the case for any Large Language Model?

Blablabla

I have been brabbling on a lot in this blog post. Really, all I want to bring across is that, if such fundamental questions are stil open (maybe they're not, and I'm stupid!), then there are some people in this world that have really got to finally shut the fuck up, and stop needlessly worrying the rest of us about our jobs.

Serious AI researchers are at work, and want to do some science. Mystifying what they do in this manner will do neither them, nor anyone, any service.

At any rate, LLMs will probably not become lucrative any time soon, so maybe we should enjoy 'em while they last, instead of being scared of them...