Lecture notes for ISE483/SSIE583 - Evolutionary Systems and Biologically Inspired Computing. Spring 2023. Systems Science and Industrial Engineering Department, Thomas J. Watson School of Engineering and Applied Science, Binghamton University. Also available in adobe acrobat pdf format
Von Neumann thought of his logical model of self-reproduction as an answer to the observation that, unlike machines, biological organisms have the ability to self-replicate while seemingly increasing their complexity without limit. Mechanical artefacts are instead produced via more complicated factories (as opposed to self-production) and can only degenerate in their complexity. He was searching for a threshold of complexity beyond which machines self-reproduce (with no outside control) while possibly increasing their complexity.
Von Neumann concluded that this threshold entails a memory-stored description Φ(X) that can be interpreted by a universal constructor automaton A to produce any automaton X; if a description of A, Φ(A), is fed to A itself, then a new copy of A is obtained. However, to avoid a logical paradox of self-reference, the description, which cannot describe itself, must be both copied (uninterpreted role) and translated (interpreted role) into the described automaton. This way, in addition to the universal constructor, an automaton B capable of copying any description, Φ(X), is included in the self-replication scheme. A third automaton C is also included to perform all the manipulation of descriptions necessary—a sort of operating system. To sum it up, the self-replicating system contains the set of automata (A + B + C) and a description Φ(A + B + C); the description is fed to B which copies it three times (assuming destruction of the original); one of these copies is then fed to A which produces another automaton (A + B + C); the second copy is then handled separately to the new automaton which together with this description is also able to self-reproduce; the third copy is kept so that the self-reproducing capability may be maintained (it is also assumed that A destroys utilized descriptions). See Figure 1(i) for a visual representation.
Notice that the description, or program, is used in two different ways: it is both translated and copied. In the first role, it controls the construction of an automaton by causing a sequence of activities in the machine—Von Neumann named it the active role of information. In the second role, the description is simply copied without reference to its meaning—the passive role of information. In other words, the interpreted description controls construction, and the uninterpreted description is copied separately, passing along its stored information (memory) to the next generation. This parallels the horizontal and vertical transmission of genetic information in biological organisms, which is all the more remarkable since Von Neumann proposed this scheme before the structure of the DNA molecule was uncovered by Watson and Crick—though after the Avery-MacLeod-McCarty  experiment which identified DNA has the carrier of genetic information.
The notion of description-based self-reproduction implies a language. A description must be cast on some symbol system while it must also ultimately be implemented by some physical structure (or axiomatic/logical system if considering an exclusively formal treatment). When A interprets a description to construct some automaton, a semantic code is utilized to map instructions into construction commands to be performed. When B copies a description, only its syntactic aspects are replicated. Now, the language of this semantic code presupposes a set of primitives (e.g. parts and processes) for which the instructions are said to "stand for". Descriptions are not universal insofar as they refer to these building blocks which cannot be changed without altering the significance of the descriptions. The building blocks ultimately produce the dynamics, behavior, and/or functionality of the overall system. In Biology, we can think of the genetic code as instantiating such a language. Genes are descriptions that encode specific parts: amino acids chains. In a computational setting parts are typically logical operations, but they must ultimately be material building blocks (in hardware) where instructions in the language are decoded/translated to specific physical actions. Von Neumann  (posthumously aided by Arthur Burks) produced a completely formal specification of a universal constructor using a 29-state cellular automaton. On a formal setting, decoded instructions map to other formal instructions in an underlying formal axiomatic system, which stands for physical causality. Implementations of this automaton appeared only fairly recently [e.g. Pesavento, 1995, see Sipper, 1998]. Von Neumann famously said that "by formalizing the problem" perhaps he was "throwing away the baby with the bath water." Clearly he was interested in the material aspects that allow organisms(and machines) to that evolve.
Perhaps the most important consequence of separate descriptions in Von Neumann's self-reproduction scheme (and Turing's Tape) is its opening the possibility for open-ended evolution [Rocha, 1998; McMullin, 2000]. As Von Neumann  discussed, and shown in Figure 1(ii), if the description of the self-reproducing automata is changed (mutated), in a way as to not affect the basic functioning of (A + B + C) then, the new automaton (A + B + C)` will be slightly different from its parent. Von Neumann used a new automaton D to be included in the self-replicating organism, whose function does not disturb the basic performance of (A + B + C); if there is a mutation in the D part of the description, say D`, then the system (A + B + C + D) + Φ(A + B + C + D`) will produce (A + B + C + D`) + Φ(A + B + C + D`). Von Neumann [1966, page 86] further proposed that non-trivial self-reproduction should include this "ability to undergo inheritable mutations as well as the ability to make another organism like the original", to distinguish it from "naive" self-reproduction like growing crystals.
Notice that changes in (A + B + C + D) are not heritable, only changes in the description, Φ(A + B + C + D), are inherited by the automaton's offspring and are thus relevant for evolution. This ability to transmit mutations (vertically) is precisely at the core of the principle of natural selection of modern Darwinism. Through variation (mutation) populations of different organisms are produced; the statistical bias these mutations impose on reproduction rates of organisms will create survival differentials (fitness) on the population which define natural selection. In principle, if the language of description is rich enough, an endless variety of organisms can be evolved: open-ended evolution.
The evolvability of a self-reproducing system is dependent on the parts used by the semantic code. If the parts are very simple, then the descriptions will have to be very complicated, whereas if the parts possess rich dynamic properties, the descriptions can be simpler since they will take for granted a lot of the dynamics that otherwise would have to be specified. In the genetic system, genes do not have to specify the functional characteristics of the proteins produced, but simply the string of amino acids that will produce that functionality "for free" [Moreno et al, 1994]. Furthermore, there is a trade-off between programmability and evolvability [Conrad, 1983, 1990] which grants some self-reproducing systems no evolutionary potential whatsoever. When descriptions require high programmability they will be very sensitive to damage. Low programmability grants self-reproducing systems the ability to change without destroying their own organization, though it also reduces the space of possible evolvable configurations [Rocha, 2001].
Turing and Von Neumann were the first to correctly formalize the required inheritance mechanism behind neo-Darwinian evolution by Natural Selection. This understanding of the most fundamental design principle of life, puts Turing and Von Neumann on the Parthenon of great thinkers in Biology, alongside Darwin and Mendel. The dovetailing of computational thinking and biology, inherent in the cybernetics movement of Turing, Von Neumann, Shannon, Wiener and others, emphasizes how (material) control of symbolic information is the hallmark of both computation and biocomplexity.
Avery, Oswald T.; Colin M. MacLeod, Maclyn McCarty . "Studies on the Chemical Nature of the Substance Inducing Transformation of Pneumococcal Types: Induction of Transformation by a Desoxyribonucleic Acid Fraction Isolated from Pneumococcus Type III". Journal of Experimental Medicine 79 (2): 137-158.
Brenner, S. . "Turing centenary: Life's code script." Nature 482 (7386): 461-461.
Conrad. M. , Adapatability. Plenum Press.
Conrad, M. , "The geometry of evolutions". BioSystems 24: 61-81.
McMullin, B. . "John von Neumann and the Evolutionary Growth of Complexity: Looking Backwards, Looking Forwards". Artificial Life 6(4):347-361.
Moreno, A., A. Etxeberria, and J. Umerez , "Universality Without Matter?". In Artificial Life IV, R. Brooks and P. Maes (Eds). MIT Press. pp 406-410
Pesavento, U.  An implementation of von Neumann's self-reproducing machine. Artificial Life 2(4):337-354.
Rocha, L.M. ."Selected Self-Organization and the Semiotics of Evolutionary Systems". In: Evolutionary Systems: The Biological and Epistemological Perspectives on Selection and Self- Organization, . S. Salthe, G. Van de Vijver, and M. Delpos (eds.). Kluwer, pp. 341-358.
Rocha, L.M. . "Evolution with material symbol systems". Biosystems. 60: 95-121
Sipper, M. . "Fifty Years of Research on Self-replication: an Overview". Artificial Life, 4 (3): 237-257.
von Neumann, John . The Theory of Self-Reproducing Automata. Arthur Burks (Ed.) University of Illinois Press.
Watson Jd, Crick Fh . "Molecular structure of nucleic acids; a structure for deoxyribose nucleic acid". Nature 171 (4356): 737-8.
Last Modified: May 2nd, 2022