Known Unknowns

What we know about what we don't know about knowledge creation.

Knowledge creation is a thing that happens. Perhaps this is so obvious it need not even be said, but if we listen to some commentary on how knowledge is “produced” one would think that literal creativity was not part of the picture. Many seem to think, with the advent of LLMs and “evolutionary algorithms”, that it is not actual creation but a form of extrapolation, pattern matching or some other already well understood regression-type mathematical formula that produces knowledge. Given some existing library of information, or given a criteria for optimisation then the application of a kind of predictive process (“next token prediction” say) can lead to novel solutions.

However, insofar as LLMs or evolutionary algorithms generate novelty (which is to say something never before generated in this universe) - are they truly creative?

Large language models and deep learning neural networks such as the transformer architecture aggregate vast amounts of information or at the very least are trained on vast amounts of information in order to find within that corpus of already created human knowledge - whether by a process of “search and find” or “recombination” responses to prompts (when it comes to, for example, chatbots such as ChatGPT or Grok). And we judge the usefulness of such chatbots to the extent they give sensible and accurate answers to questions asked.

The process going on in the mind of a child (or anyone else for that matter!) when creating knowledge (which is to say learning) is starkly different. People are not trained on vast libraries of information and yet they manage to generate complex ideas (often false, sure, though sometimes containing deep truth) by guessing and checking against reality to create knowledge. The novelty they create can be of a completely different kind: arrived at through rule breaking. It can be the case that whatever a person has already learned, they reject utterly and step outside and beyond whatever knowledge is encoded in their minds and on the brains. Disobedience, considered broadly is a kind of creativity: it is rejection and rule breaking. What is the criteria for “good” or “best” or “excellent” or “beautiful” or “fun”? Whatever it may be said to be now, a child can say: no, doesn’t sound good or fun to me! I want to do something different. I will invent a game I find fun that has rules utterly different to other games I have been taught to play. They are not relying upon any library of information and then recombining that information to generate via some process of extrapolation.

For centuries it was simply received wisdom, known to the point of “common sense” and it was said “proved beyond doubt” that the truth just was gravity was a force of the kind described by Isaac Newton. As the problems mounted that the theory could not explain (such as irregularities in the orbit of Mercury) attempts were made to salvage the theory (for example by postulating other planets that were causing gravitational irregularities and upsetting the predictions of the theory). Known to be a force, perhaps slightly changing the formula for calculating the strength of that force could work? Some tried. That, perhaps, is the kind of thing an LLM would do.

But Einstein did something many thought - if not literally then metaphorically - unthinkable. He rejected the very notion that gravity was a force. He thought of space and time themselves rather than being an inert background like a stage on which players and sets moved and changed, as being itself a player that had an active role to play in the cosmos. Space and time were a unified whole: spacetime that could bend and stretch guiding planets around stars. In this view spacetime guided mass and energy how to move while mass and energy guided spacetime how to bend. Roughly speaking and roughly paraphrased from the legendary physicist and general relativist John Wheeler.

Einstein literally created that theory of spacetime, of gravity: General Relativity. He did not extrapolate from data or observe his theory of gravity “in nature” - he did not extract the theory from libraries or simply repurpose existing ideas. He invented something new. And yes he expressed it in language (like German and English that already existed) and mathematics that has been used for other things already. But he added much that was new by solving problems that were outstanding and which go beyond the scope of this present piece. All of that is a demonstration that Einstein’s creation constituted objective knowledge. Knowledge of the real physical world. That spark of creation within his mind is at present a mystery for us to explain and we call it creativity and it is the same thing we all do all the time when learning anything at all or discovering anything at all (those are actually the same process).

A child is genuinely inventing ex-nihilo - just as Einstein did. But how does this work? How could we program an artificial system to do the same? We don’t know. Is consciousness necessary? Are qualia? We don’t know. But if knowledge creation did not at least include some element of genuine creation-from-nothing (in other words something coming into being that did not exist in any form) beforehand then in principle that knowledge could be predicted by simply looking at the state of the way things are now. But if that were possible then, for example, we could predict the content of future scientific theories. Which means when we made the prediction we would then have that scientific theory. This is a strict contradiction. We assume a scientific theory which will be created in the future can be predicted now. But if that is true and it can be predicted now, it need never be created in the future.

The truth is we cannot predict future scientific theories. We cannot predict knowledge creation in general. And that includes, for example, predicting what kinds of biological species might be found on some island (or indeed some planet) yet to be explored. In this biological case one reason is that the mutations that occur allowing for selection to occur are as close to genuinely random as one can imagine. On Earth, DNA which carries the genetic code, can be affected by solar radiation or even a cosmic ray from the other side of the universe because some star in a distant galaxy exploded many billions of years ago. That is as unpredictable an event as it is possible to conceive of in this world: a star we cannot see now because it exploded billions of years ago in just such a way that a high energy photon (or other particle) raced towards the Earth causing an event in our atmosphere which generated more particles that interacted with the DNA in the germline of some species that causes a mutation that changes its offspring. Again: in an entirely unpredictable way and leading to the creation of offspring with different traits. If that process goes on often enough a completely new species can arise. (Of course it need not be cosmic or solar rays causing this: other things like chemicals or just errors in copying DNA can cause mutations too).

The creation of knowledge therefore happens in two domains: our minds in the form of explanations, and in biology in the form of organisms utilising genes. But in neither case can we simulate the creation of explanations or of evolution by natural selection. If we understood either we could write the program that simulated that process. But hang on, can’t we simulate evolution?

So called “evolutionary algorithms” are always aiming for a particular goal: find the optimal shape of a wing, or antenna design or efficiency in robot walking. But in those cases there is a literal intelligent designer involved: the programmer. They set the goal and once the goal is achieved there are no further improvements or changes made. This is utterly unlike actual evolution by natural selection where there is no goal (evolution is blind) and besides when some “local maxima” is reached - say for example a bird can fly the fastest in a particular niche - it is not like environments, or niches do not change and mutations occur meaning that change in the species and birds and their wings won’t change too.

We cannot program “evolution by natural selection” that is like this even if we can use so-called “evolutionary algorithms” to try out a bunch of designs far more rapidly by trial and error to achieve an aim set by the programmer. This algorithms are certainly a form of evolution - they can lead to “optimisation” for a given task - but it is not natural selection. Again: evolution of the natural selection kind is blind and structures can arise in species and persist for a long time that are rather damaging to the species.

And as to “games” or other kinds of pedagogical simulations of evolution by natural selection they lack an important feature of genuine evolution: universality. DNA is able to generate an infinite repertoire of organisms from single celled bacteria or archaea through to trees, dinosaurs, pot plants, platypuses and people. But those simulated creatures? They are drawn from a very restrictive finite repertoire: a closed set defined by the limitations of either the hardware and software of a computer or the imaginations of the programmer. Think something like Conway’s Game of Life: indeed simple rules generate amazingly complex structures that, given the rules, are unpredictable. But repetition occurs after some time and anyone watching the “evolution” occurring in any simulated system soon comes to realise “this is all there is” and beyond that point no further novelty is encountered. That there is such a point always in these simulated games of evolution: play parks where alien species can evolve. “Thrive” is one such computer game available (https://store.steampowered.com/app/1779200/Thrive/). And whilst the repertoire of creatures is indeed vast and surprising as one finds when playing such games - it’s always surprising only for a time. This difference between a computer simulated world of evolving species and the real world which is always open ended and throws up even to this day new species barely even imagined by biologists before (whether from the ocean depths, or deep in some forest or far underground) is a neon sign above our best theory in biology that reads “We still don’t understand it well”. Open questions in neo-Darwinism remain. How exactly is new biological knowledge created?

But that mystery is, in fact, not as important in one sense as the mystery about how to generate new explanatory knowledge. In both cases we can say many true things, but there is much left to know and the reason the “unknowns” around explanatory knowledge generation are deeper is because in the long run it will be memetic evolution, not genetic, that determines the structure of the cosmos and the multiverse. It is ideas - explanatory knowledge - that get passed from generation to generation transforming the useless matter throughout galaxy after galaxy into unpredictable but wondrous structures not imaginable now because we lack the knowledge now.

But one day we will have those explanations. And when we do we will create entities like us - AGI - and other technology to augment both them and us to make progress ever more rapidly because we understand the world ever more deeply. A process that never ends for every solution reveals to us more problems that are more interesting and more fun to solve including the problem of what criteria to use to decide between more and less fun problems.

None of this is to say there is before us there do not exist many unknown unknowns when it comes to biology and epistemology. I expect there are and those unknown unknowns far outnumber all the known unknowns we can state. Indeed, they always will because that is a necessary truth entailed by always being at the beginning of infinity.

Credit: To David Deutsch and his book “The Beginning of Infinity” for the inspiration.