Thursday, August 07, 2008

Black Swans, Open Systems and Fractal Geometry

Nassim Nicholas Taleb's book "Black Swan" is an expansion of the ideas that he put forward in his book "Fooled by Randomness- The Scandal of Prediction" . As regular readers will know, I think that the philosophical ideas that Taleb puts forward have profound political implications. In particular ideology as a Grand Theory- that is a systematised explanation of everything- falls to pieces in the face of the uncertainty that is the basis of empiricism.


Intuitively I have always distrusted grand theories, and as an undergrad and as a research student I proceeded from the basis of a partial theorist- in other words that most human activities, especially socially interactive disciplines like politics, are not closed systems, isolated from their context, but open ones. Marxists, by making a statement such as "All history is the history of class struggle" show that they believe that history proceeds from the basis of an ideological driver and thus is, by definition, fundamentally a closed system.


Liberal philosophers, such as Mill, Hayek and Popper, are far more interested in the problems of uncertainty. Their empiricism implies limits to human theorising and therefore Liberalism is by definition a partial theory, and by the implied acceptance of human activities being within an open system also accept that their ideas incorporate structurally the idea of uncertainty.


Taleb by describing the fundamental uncertainties of how humans perceive and interact, has mirrored the work of his friend and collaborator Benoit Mandelbrot, whose work on fractal geometry has pointed out mathematically the fundamental uncertainties at the very root of maths. The "fuzziness" of fractals is mirrored across the entire structure of the Universe- including the Big Bang, where it becomes clear that that the precise beginning of the Universe breaks down into the fractal fuzziness of quantum mechanics.


Even more critically, Taleb talks about the high impact of what the US military inelegantly call "unknown unknowns". These "Black Swan events", are often so transformational that they require a complete alteration to are pre-existing world view, which again carries a lightly inelegant moniker: "the paradigm shift". In fact, humans shift their perceptions to try to rationalise the random.


So what, you may say, has this piece of esoterica got to do with political ideology?


The fact is that our politicians do not understand why they are being dishonest with their electorates. They do not understand that any given policy has little direct effect that can be predicted, but may have a large number of unpredicted effects- not all positive. Of course when politicians talk about unpopular or nasty things they call it "making hard choices", little understanding that the effects were, from their point of view, more or less random- no choice involved.


Even more fundamentally, the question of closed political ideology forces political leaders to respond to events in inflexible, pre-determined and usually wrong ways.

The only intellectually honest response is to focus not on political policies and planning, but on the political mechanism itself. The question then becomes not one of desired policy outcomes, but rather the high themes of political process. Simple goals, based on the knowledge that politics is indeed an open system. It is only by working on establishing open societies that we can establish a political structure that is robust enough to maintain the Liberal idea of freedom.

That requires political leaders to admit what they do not know.


It probably requires the reversal of the increasing separation of politicians into a separate, and increasingly professional cadre, as lamented by Peter Oborne in his book "The Triumph of the Political Class". It will also require the deconstruction and simplification of what he state chooses to do. In order to be robust enough to face the periodic crises of the unexpected, if it is to retain the values of freedom and openness, democratic society will need to focus on the uncertain reality, rather than the comforting myths.


This is a challenge that few understand- but our government and society will need to get a whole lot smarter, if it is to survive at all.

6 comments:

Newmania said...

Now that is what I would call Conservatism.I like this piece and I have been blogging along similar lines , as always with you C the problem is you are in entirely the wrong Party. Hayek is a neo Liberal but has been reviled by the interventionist Liberal Party . Furthermore hubristic certainty that they and only they have a brilliant new idea is the Liberal Calling card.

Still well,done i do belive you are actually starting to "Get It"

I wrote last night ...



Part of my inchoate ‘philosophy’ is small c conservatism .Unimpressed by shiny new ideas,I am implacably attached to what provably works, however improbable the invention of say , The House of Lords , might appear. This sort of conservativism has recently acquired contemporary resonance despite its antiquity.
Kieron O Hara, in his superb, “The Conservative Tradition” ,traces a lineage from sceptical classical philosophers like Phyrrus and Sextus ,through to Michel; Eyquem de Montaigne (1533-92). This Renaissance thinker had views on human presumption that lead him to castigate those who thought they knew best how to govern .He said : "It is easy enough in a nation to generate contempt for its ancient customs : no man has ever tried to do so without reaching his goal; but for replacing the conditions you have ruined with better ones , many who have tried to do that have come to grief” We might broaden the point by saying the wisdom, in a system, is not necessarily known to any individual .Those who claim to be qualified to redesign it (Social planners we would say) are therefore to be resisted .
In the 20th century Hayek also denied the efficacy of central planning ,but valued low level decisions if they were short term with expert local knowledge.“ The shipper who earns his living by using otherwise empty of half filled journeys ..or the arbitrageur who gains from local differences in commodity prices - are all performing eminently useful functions based on special knowledge in the fleeting moment not known to others ". Information , inaccessible to bureaucrats , is in millions of Mayfly judgements and for Hayek the sum of all this was price .Forget please the Neo Liberalism that ensues ,just hold in your mind the impossibility of a sufficiently responsive planner achieving an equivalent optimum.
Today I bought the “New Scientist “ and the arresting front cover exclaimed, “ Hands Off , Why things work better when you let go !”. Naturally my curiosity was piqued and sure enough Dirk Helbing ( a Swiss Physicist ) has discovered something oddly familiar. "In many situations it is better to give up some of our controls . Often the system discovers better outcomes unlike anything our minds would imagine …” uh huh… !
For example General Motors were not able to maximise the use of their paint booths . The random arrival of trucks and unpredictable maintenance resisted a programmed assignment. The solution was to give the machines simple rules and let them ”bid” for the jobs .The results looked peculiar but General Motors saved $1million in paint alone at one plant
This healthy respect for complex unpredictability has been directed at Traffic lights .At the moment lights are pre-programmed by engineers to alter their phases at given times . Helbing gave the lights simple operating rules and left them to organise themselves with sensors relaying information about traffic flow what other lights are doing .. Again the results can look very odd, but journey times , in Dresden , were reduced by 'devolving decisions to the lights'
Boris Johnson is surely the small c conservative non pareiil . He has launched a consultation on scrapping the congestion zone extension, and today we read ,congestion in the capital generally is back at its old levels Journey times are actually longer. There has been no easing of congestion in the Western zone since the charge was rolled out ( says TFL) , and Boris has encouragingly described Ken`s baby as a “blunt instrument ..”. In a package of measures he promises to re-phase traffic lights i.e. leave them green longer .Not bad Boris but what about consulting Dirk Helbing and adopting a truly conservative and yet brilliantly modern solution . Scrap the whole thing and let the lights and drivers decide.

Cicero said...

Newmania- please next time could you link rather than quote.

I still find it rather bizarre that you keep trying to tell me what I "really" am. I am a Liberal. I support Liberalism and therefore the Liberal Democrats. So do most of the Lib Dems I know. I have done so for 30 years. I know, of course that there are Liberals amongst the Conservatives too, but there are also a larger number of people trying to impose their Conservative world view on the rest of us. Even the number of Liberal Democrats I disagree with do not try to do that. They debate in the proper Liberal fashion, which is why we tend to stay members of the same party. People don't join the Lib Dems because of any hopes of climbing the greasy pole, but rather because they genuinely believe in our Liberal ideology and our Liberal party.

Newmania said...

I suppose it would be a sort of small death to admit how perverse this self deception really is.
We Conservatives understand the value of myth and I respect your fantasy if help you to live better

( Nutty though it is )

Cicero said...

What part of Hayek's Essay entitled "Why I am not a Conservaive" at the back of the "Constitution of Liberty" can possibly support your thesis that "Liberals are "really" Conservatives?

Scott said...

What the two of you have missed is that you are both right. Ciscero- you've correctly identified the nature of the open system. As (mostly) fractal, it does what all natural fractal systems do - conserve energy and operate at maximum efficiency. If you construct the rules for it's operation- just like a tree doesn't have fixed distances for the next branch to grow, but rather good fixed rules under which a new branch should be started - then it should essentially operate on autopilot. You're right to spurn pressure to redesign the essential system from any place on the political spectrum, since it is doomed to failure on it's face.

But it's natural state is to operate in a conservative fashion. That means:
- Not consuming more than it expending, implying balanced budgets
- Not using scarce resources inefficient wastefully, implying willful removal of external influence to the maximum degree possible (think of a tree growing branches away from a shady spot, or roots towards water)
- Preparing for worst case scenario, which implies a system that can survive sustained stress.

Newmania: You need to understand that this conservatism only works if the system is well designed. The greatest failure of much conservatism, in my mind, is a lack of pragmatism (although this depends on your definition of conservatism). Again, think of the fractal of a tree - While a tree might allow some leaves to die that are too shady will also simply grow in the opposite direction to make up for it. That means that if it's cheaper to fund welfare than a prison, then do so. Even if that means violating socially conservative principles. Individual, family, community, municipality, area, region, political division, country. These are like leaves, stems, small branches, large branches, daughter trunks, and trunks. What current liberalism identifies is that community is currently undermined by inorganic forces. and without small branches, the leaves don't work very well and the tree dies. Conservative tenets often argue that the leaves are more important than the leaf stems. They're not, and that thinking needs to go away. People that promote the organic linkage between families and communities need to be supported. This is why liberalism need to learn that religion, all religion, can be an important organic community force in this open system. After all, everyone is fundamentally subservient to a creator, be that a God, science and the fundamental complexity of the universe, the god of the self, or the wonder of earth's creation. Social conservatives need to realize that liberals aren't godless- they are just 'god-alternate'.

Probably the best examples of where these viewpoints all meet is environmental policy, since it tracks back to the primary goal of a conservative system - survival.

A Conservative would say that a balanced budget is important. An environmentalist would say that it is an imperative, and therefore energy austerity is critical. A conservative would say that hoarding energy is important for security. An environmentalist would say you don't poop where you eat. All of these things are valid.

Anyways, my $0.02. I was looking for conversation on fractals and political open systems. Obama's campaign is designed as a fractal system, and it appears to be incredibly successful at reaching the organizational leaves. I don't have any comment on the election other than I think the best-designed, organic system has the best chance of success, just like in nature.

kiki said...
This comment has been removed by a blog administrator.