Gloom and Doom

In my first posting here a few days ago, I alluded to the belief held by many scientists (Richard Dawkins is an obvious example) that the cosmos is essentially a random process and has no overall order or purpose. I noted then that I disagree with that view and also that many scientists have come to question it.

There are two main reasons, I think, why this random model came to be prevalent. The first is Darwinism, which holds that species evolve as a result of random genetic mutations. The second is physics, which models matter as “particles” that interact randomly. I could also throw in the concept of entropy, a consequence of the second law of thermodynamics, which Victorian physicists first theorized would lead eventually to the “heat death” of the universe.

All of these things together produced an astonishing pessimism among the intellectual elite of the late 19th century, epitomized by Matthew Arnold’s marvelously gloomy 1867 poem “Dover Beach”:

Ah, love, let us be true
To one another! for the world, which seems
To lie before us like a land of dreams,
So various, so beautiful, so new,
Hath really neither joy, nor love, nor light,
Nor certitude, nor peace, nor help for pain;
And we are here as on a darkling plain
Swept with confused alarms of struggle and flight,
Where ignorant armies clash by night.

As I indicated the other day, I think the emphasis on randomness (as well as the gloom) is overwrought, to say the least. Even granting that, for example, atoms in an unconfined space move and interact randomly, as soon as they do interact, various natural laws and forces (gravity, the strong and weak nuclear forces, etc.) come into play to create structure. Similarly, even if a new, more “advanced” species emerges as a result of a random genetic mutation, that species’ existence (and success) increases the likelihood that a future mutation will produce an even more advanced and successful species.

An analogy: Suppose a rainstorm forms directly above the Continental Divide. The individual raindrops will fall randomly around the divide. But as soon as each drop hits the ground, gravity and geology kick in, and the pathways those drops take from that point on are predictable: They will flow either to the Atlantic Ocean or the Pacific (leaving out evaporation, absorption, being swallowed by grizzly bears, etc.).

Similarly, the widely held Big Bang model in cosmology holds that the initial state of the universe was a hot fog of more or less randomly distributed particles. But natural laws have gradually drawn matter into recognizable patterns or structures – galaxies, clusters of galaxies, stars, planets, etc.

Another way of putting it: The proponents of randomness claim the natural state of matter (and energy) is “equilibrium” – basically a more or less uniform distribution. That state may be disturbed occasionally by random events (stars colliding and so on), but it always returns eventually to equilibrium.

In contrast, researchers in recent decades (Nobel Prize laureate Ilya Prigogine, for example) have noted and analyzed the many instances of systems that are, as they say, “far from equilibrium” – that is, they exhibit self-sustaining and self-reproducing non-randomness. Today, there’s lots of interesting work being done on studying complex systems and creating models that take account of the wholeness of objects instead of breaking them down atomistically; one place to see some of that work is the Web site of the Center for Integral Science.

Despite these developments, mainstream economists continue to insist that financial markets are well-modeled as a random process. The leading proponent of this view is Burton G. Malkiel, a Princeton economics professor whose book “A Random Walk Down Wall Street” remains the Bible of this viewpoint. In essence, Malkiel claims financial markets tend toward equilibrium because all investors (the “particles” in this model, characterized as “rational agents” who seek to “maximize their personal good”) have access to the same information. Occasional “shocks” caused by the introduction of new information, or other “inefficiencies,” are quickly damped out as investors adjust. As a result, fluctuations in the markets are random and unpredictable, so it’s impossible to “beat the market” through timing or other strategies.

It’s a nice, neat, self-consistent theory, but in the four decades since Malkiel first published the book, there has been abundant research showing that the random walk doesn’t actually provide a good model of market behavior. A search for the phrase “random walk” in the Research Papers in Economics online IDEAS database produces one study after another showing that the random walk hypothesis fails when applied to a wide range of financial markets.

Even so, random walk theory continues to be taught in economics classes. (Malkiel himself remains unrepentant, or maybe he just isn’t keeping up with the literature; I interviewed him in the early 1990s and asked him whether any of the then-emerging criticisms of his theory from the perspective of nonlinear dynamics/“chaos theory” had given him pause, and he responded by talking about the so-called January Effect, which has nothing to do with nonlinearity.) One result is that when the stock market jumps 900 points one day and plummets 400 the next, everyone seems astounded, and we hear the so-called experts “explaining” these moves in terms of that day’s news headlines.

The alternative – or “an” alternative, at least – would be to acknowledge that financial markets exhibit some sort of orderly structure. That’s a claim that has long been made by a variety of non-academic observers and analysts (Elliott Wave Theory is a notable example). If true, the identification of the details of that order would have potentially enormously profitable implications. So far, if anyone has figured it out completely, they aren’t telling.