Inventing Fundamental
New Computing Technologies

Fundamental research is not necessarily impractical nor is it abstract and far-off. Powerful examples of fundamental research that were practical and had immediate benefits were the inventions of the networked personal computer, dynamic-object oriented programming, the graphical user-interface and the Internet. These have generated many trillions of dollars of GWP, changed the lives of several billion people and created many new businesses that have been built on the fundamental inventions.

An excruciating example of an area that needs more than incremental improvements is programming, both in the large and in the small. Code is too: large, complex, costly, buggy, insecure, segregated, and inexpressive. We have plans to attempt a qualitative reinvention of programming and to start one of the subprojects this year: to make a practical working mathematical model of a complete personal computer system.

Science uses models - usually made in mathematics - to both represent theories and with which to think. These models are most useful if they are powerful enough to capture the phenomena and small enough to be comprehensible. Quite a bit of abstraction and invention in mathematics has been done to deal with both of these criteria. Maxwell's Equations, the formulations of General Relativity and of Quantum Mechanics have all required new invention in mathematics in order to be represented well and compactly.

This has been done less often in computing, where the artifacts produced are most often made to be functional rather than understandable or good models. A notable early exception was McCarthy's rendering of LISP in LISP which exhibited a powerful programming language. In a half page of description, McCarthy created a model in such a way as to be both understandable and an object with which to think fruitfully. Smalltalk at PARC in the seventies created quite a lot of function and expressivity with a small amount of code; the entire system was about 10,000 lines of attractive code.

The ante has been raised in the 21st century, but most coding today is still done using techniques from the late sixties, with the usual result of millions of lines of complicated, problematic code. On the other hand, the Squeak system, which was derived from PARC Smalltalk, includes its own operating system, GUI, development tools, graphics, sound, Internet sockets, and many applications including Etoys and Croquet, yet is only about 230,000 lines of code. The executables are about 2.8MB and only half of this code is generally used.

This opens the possibility of doing a design much better than Squeak's, both fundamentally and at the user-level, to create a model of an entire personal computer system that is extremely compact (under 20,000 lines of code) and yet is quite practical enough to serve as a system for succeeding versions of the $100 laptop and for embedded systems of the future. This will also be a stepping stone to a complete qualitative reinvention of programming which will be overlapped with this project a few years hence. Our computing models will be partly drawn from our work on Etoys, Croquet, and a new bootstrapping method by Ian Piumarta.

Even a cursory exposition of how we plan to do this is outside the scope of this summary, yet something does need to be said here. The simplest slogan that applies is “Math Wins!” That is, both many of the ideas in classical mathematics and new techniques that are essentially mathematical in their nature bring forth great expressiveness in very compact and understandable forms.

For example, a classical idea is the notion of an algebra - an abstraction of operations and types that includes many concrete kinds of things and relations. This has already been used in both high-level functional and object-oriented programming to great effect, but has not yet been used really strongly in the larger design of a system. We can see a glimpse of how this might work to our advantage by looking at the object structure in the Squeak Etoys system. Every object is essentially the same kind of object. The differences are small and mostly parametric: objects have different costumes, have a few different specific behaviors for their special roles, etc. But the entire system was designed to emphasize the similarities and diminish the differences. This leads to an enormous contraction of the number of meanings that have to be specified.

Another example of a powerful abstraction is “typical element programming,” which draws its inspiration from the physical sciences. The idea is that great progress in compact description can be made if you only have to specify the elementary particles and fields, or in larger aggregates, a typical cell. This also lies at the roots of dynamic object programming but has been lost over the last 25 years. Part of the key to success is that not just the “particle” side of things has to have a good model, but also the “field” side (which is often almost neglected in so-called “object-oriented” programming which is usually anything but).

The “field” side has to do with messaging and events, the interstitial side of interactions (which is almost invisible because the concreteness of the objects takes too much of the center stage). In fact, the objects and the messaging system form duals of each other and have to be kept in balance for the most powerful and compact modeling. Once the “centralist mind-set” is abandoned in favor of looking at complex systems as typical elements and dynamic relationships, a new kind of mathematics arises that allows an astonishing range of phenomena to be handled that seemed previously to be very different cases.

Another key idea here is to separate meaning from tactics. E.g. the meaning of sorting is much simpler and more compact than the dozens of most useful sorting algorithms, each one of which uses different strategies and tactics to achieve the same goal. If the “meanings” of a program could be given in a way that the system could run them as programs, then a very large part of the difficulties of program design would be solved in a very compact fashion. The resulting “meaning code” would constitute a debuggable, runnable specification that allows practical testing. If we can then annotate the meanings with optimizations and keep them separate, then we have also created a much more controllable practical system. This is critical since many optimizations are accomplished by violating (hopefully safely) module boundaries; it is disastrous to incorporate optimizations into the main body of code. The separation allows the optimizations to be checked against the meanings.

If the meanings can be elevated in important cases to policies then we can think of constitutional programming as a center for large systems design. To go back to the physical analogies, we can think of the compact laws of physics as a constitution for the entire universe. An example in computing is to think about the role of TCP/IP in the governing of the Internet as the Internet's Constitution. The careful design of a policy that was both small enough to be understood and correctly implemented and yet powerfully comprehensive in what it was able to cover, resulted in the Internet which has been able to scale by 12 or more orders of magnitude without breaking. Another way to phrase these ideas is that we want to learn how to think and design and build systems in terms of “covering heuristics” that will make things robust even if specific tactics are not yet formulated.

We have a number of further strategies for doing this very compact reimplementation, but will only mention one more here. There are a number of systems that embody special knowledge about their domains, including desktop media, computer-aided design, structural engineering, and classical mathematics (e.g. Mathematica). This has worked less well with trying to make a system that “understands” (embodies) how to do programming. Higher level languages encourage particular styles, but unsophisticated or stubborn programmers often defeat the best styles (“one can write COBOL in any language”). “4-Gen” systems are usually too inflexible to be usable outside the narrowest of domains. However, one can well imagine a system that does have a “sense of system” and can participate actively in new structuring of itself. We don't intend that it be intelligent so much as constitutional in the above sense. It should be able to detect various kinds of weaknesses, propose useful kinds of extensions, etc. Parts of this are possible today, other parts will emerge from the reimplementation effort itself and will be the building blocks of a real reinvention of programming.