Simplicity vs Complexity

http://www.ted.com/ta...

"Simplicity: We know it when we see it -- but what is it, exactly?"

Emergence - Knowing the parts does not mean you know the whole, or how it will act with the environment
Simplicity - Reliable/Predictable, Cheap, High performance per cost (high value), can work as "Building Blocks" (12:49)

_________________________

We can make "simple" but flexible models that nurture emergence to get AI. We don't have to create intelligent models, just models that have the potential to be intelligent.

I would love to hear some reactions to this talk from this group's members.

To me, Whitesides seems to be saying two very contradictory things. At first he claims that complexity is at the same time irreducible/ever-changing/evolving (novelty always outpaces predictability), then he asks towards an effort to "simplify" and thus reduce this very same ever more intractable complexity stream.

Am I missing something?

I am also a little miffed by his declaration "nobody knows how life works". What we know is far more valuable, we know how life doesn't work. We know the parameters within which life and everything else works. We know that life isn't special, that it doesn't break any standard model strictures, that it requires nothing new. We know that no temporal line divides pre-life from life by any qualitative measurement. This is not trivial knowledge.

If one wanted to, one could make the same overwrought statement about physics. One could say that we know nothing about physics simply because the equation E=mC^2 isn't the universe. An abstraction isn't supposed to be the thing it abstracts. If it was, it would be of no use. The test of an abstraction is whether it allows accurate calculation. We know that E=mC^2 is accurate. But to expect that the equation would mean a perfect knowledge of everything in the universe is absurd and is yet more evidence that we humans have a hard time understanding and remaining aware of the difference between abstraction (which is a recipe) and knowledge (which is a calculation process that follows these recipes). You can have the perfect recipe (always produces an accurate result) and still not be able to practically cook up a perfect prediction of a future state.

Be careful. That you can't predict doesn't necessarily mean your recipe is wrong. There are other tests for the validity of a recipe. And there are real limits to what can be predicted and what these predictions cost, that have nothing to do with the construction parameters of recipes. Languages are more robust and flexible than the systems they abstract. That, in fact, is the central requirement of a language.

The fact that we can not ever perfectly predict the near term future of a physical system in no way proves that our abstractions are incorrect. It simply means that a perfect calculation demands a larger calculation system than the system that is being predicted. If the subject of your prediction attempt is small enough and your available calculation resources are big enough, you can have perfect knowledge of that subject.

I have not ever seen but would be very interested in seeing, a graph that exposes the relationship between the amount of computing power we have at any given time and the scale of the system that can as a result be perfectly calculated. I am sure this calculation is a non-linear curve that reflects into the future, physical limits in the local region of the universe available for acquisition, ultimately approaching an asymptotic limit. Where are we on that curve?

But lets get back to the simplicity subject. This is one of these TED talks where it is hard to tell whether we are audience to a profound set of ideas or a profound presentation.

When I hear the word simplicity, I go back to Monica Andersen's concept of "salience". The thing is that simple , by itself, is easy. A crystal is simple. I am pretty sure that Whitesides concept of simplicity is closer to Andersen's salience. And salience refers to the capacity of an abstraction describe of a system such that only those aspects of the system that matter are expressed. Whitesides quotes Einstien; "Make everything as simple as possible, but not simpler." In this sense, simplicity is synonymous with compression.

But there is something surprisingly unintuitive about simplicity, it only happens as a result of complexity. Ronald Reagan was famous for extolling the virtues of "common sense". I fear that Whitesides wades into the same rhetorically murky waters when he asks for simplicity without talking to its cost.

When one asks for simplicity, they are asking that more effort be put towards some endeavor. I think it was Blaise Pascal who said, "I would have written a shorter letter, but I didn't have the time." The act of reduction, if it is done with the requirement that what matters is saved and what isn't, isn't is costly. Think of Maxwell's Daemon, but instead of sorting fast and slow moving molecules, it has to be able to detect the relative importance of a part of a description and the dependency chain that may be effected if it is included or excluded. That is a much higher order activity than measuring the speed of a molecule.

To simplify an abstraction, one of two processes must be engaged. Either the decider daemon has to have a pre-existing template with which to compare each situation to (a knowledge base), or it has to perform a validity test at each decision point and either move to the next deletion point or reject, go back to a saved version, and start over. Both processes are costly and carry their own situational advantages and disadvantages.

When something is simple, it is the result of or product of a complex system and this only as a result of a long unbroken chain of evolutionary process. It's simplicity is a trick of vantage. It is only by hiding the complexity of the machinery that produced it and upon which it is built that it appears simple. Like the shadow of a hand dancing across a mud wall, what appears simple only happens as a result of great complexity.

Simplicity is so obvious an attribute to extoll. I wish Whitesides had discussed instead the costs of achieving simplicity and methods one might use to produce it more reliably and efficiently, and why exactly simplicity, or salience matters.

Randall Reetz