“I am not superstitious, but I avoid situations in which I continually lose.” -Barry Greenstein, Ace on the River

I want to get a few terms defined that I’m going to use in subsequent posts.  Specifically, I want to talk about the way predictive methods and markets interact.  My experience is that there are three basic classes of interactions:

• Naive Markets: When a market is “naive” with respect to a given prediction method, it means that method consistently correctly predicts the market, or alternately consistently incorrectly predicts it (and thus could be trivially reversed to predict it).  This does not mean that the predictions are 100% accurate, but rather they are consistently accurate with a frequency higher than you would expect due to randomness and that consistency persists over time.  If you find a naive predictor/market combination, this is the ideal basis for a trading system – since the predictor works consistently over time, your system will continue to be viable as long as the naivety persists.
  
  Naive markets are, at some level, an unnatural state.  Generally they occur when either the market or prediction method are new, and as a result there is little public scrutiny  directed at them.  Naive markets also frequently have barriers to trading them – they’re only available over the counter (OTC) or have limited liquidity or difficult to meet capital requirements.  These barriers to entry can prolong the naive state.
• Hostile Markets: Naive market/predictor combinations have a limited lifespan.  Eventually people become aware of the efficacy of the predictor and more people start to use it, diluting trading profits.  Then it becomes profitable to synthetically generate the conditions the trigger the predictor and trade against it.  For a detailed example of this scenario paying out, read The Rise and Fall of Trend Following and this follow-on.
  
  Hostile markets have some odd properties.  When a naive market becomes hostile, by implication it means that a population of previously very successful traders are now losing money.  This is where the quote I began this article with comes into play – eventually these former winners will get tired of the newly hostile market and leave.  At this point, there’s no money to be made in being hostile, and those traders constructing the hostile environment will cease to profit or even begin to lose themselves.  Thus hostile markets tend to go through phases:
• Phase 1: The market is indistinguishable from a naive market on the short term.  Traders looking at insufficient history start trading the predictor on this market, and make money as long as this phase persists.
• Phase 2:  As the presence of people trading the naive predictor increases, it becomes profitable for other traders to falsely trigger that predictor.  The market becomes actively hostile.
• Phase 3: All the false triggers make the naive approach unprofitable.  The traders who enetered the market in Phase 1 take losses and leave the market.
• Phase 4: With the naive traders gone, creating false predictor signals is no longer profitable.  The hostile traders start leaving the market.  Thus we return to Phase 1.

The  result, as you might expect, is a sort of cyclical behavior as the market moves between being accurately predicted by a given method and being actively hostile to that method.

• Random/Efficient Markets: The logical response to the situation above is to employ a sort of meta-game strategy.  Chart the effectiveness of a given predictor, and only use it when its success rate is good and/or improving.  Such a meta-predictor can of course be improved by increasing the speed with which it detects changes in the results of the underlying predictor. It should be noted that the trading system development process is itself an example of this sort of meta-analysis since it mostly consists of testing predictors to see if they work.
  
  The result of such meta-analysis is a sort of arms race, speeding up the cycles between naivete and hostility.  As the cycle time starts approaching one trade, the market behaves in a way that is effectively random.  The predictor (and associated meta-strategies) have lost their punch.
  
  It should also be noted that a market can appear random in the context of a given predictor when that predictor is simply completely ineffective.  For example, were I to predict market movements on the basis of ocean tides, I would expect the market movement to appear random relative to the predictor just because the predictor isn’t in any way predictive.
  
  Also note that there is nothing to be gained by speculating in efficient markets. If you can’t predict market movement with some level of accuracy, you will just end up paying commissions for no benefit if you try to trade that market.

Much academic debate has been directed at the question of the extent to which various markets operate in each of these three modes.  There are some good arguments for the existence of naive markets, even in big well-scrutinized places.  For example, Robert Shiller has produced compelling data showing that future stock returns are predicted by current earnings – his “Shiller PE” metric explores this.  Shiller also has produced data that amply shows housing prices trend very consistently (at least until recently – it may be that housing moved from naive to hostile in the 2000s).

There is a whole raft of literature arguing that most if not all markets are efficient.  This research is broadly collected under the heading of the “efficient market hypothesis” although no one can quite agree on what exactly that hypothesis is.

Fundamentally, I believe the efficient market literature is mostly wrong.  I believe it confuses hostile markets for efficient ones.  This is easy to do – if sampled over a long enough time frame (multiple cycles) the predictor appears to be neither effective nor ineffective on a hostile market.  Simple statistical tests miss the cyclic behavior.  I believe what the efficient market literature has really proven is that most markets, for most predictors, are not naive.  But by failing to distinguish between hostility and randomness, this literature is of no use to the would-be trader.

As to why I believe hostile markets are the typical state (as oposed to efficient), I have two reasons:

1. My testing continually finds major markets to be hostile with respect to certain predictors (but by no means all predictors)
2. Meta-analysis is hard.  Most traders don’t understand the things I’m writing about here.  As such, there’s not the critical mass of meta-strategy aware capital needed to move markets from hostile to efficient.  In contrast, there’s an endless supply of new naive capital that can be extracted via hostile methods.  This is in effect the “there’s a sucker born every minute” theory.  Thus hostility can persist, and continue to be profitable, essentially indefinitely.

Food for thought…