Get Diversified

Okay, stats time, folks. I wanted to give you an update on Robert Pardo's fascinating new book, The Evaluation and Optimization of Trading Strategies, which updates his 1991 bible of trading system development, Design, Testing and Optimization of Trading Systems. Pardo addresses an important question that has been raised by a couple of readers: Is it okay to have different parameter values for each market I'm trading with my COTs Timer system, based on the Commitments of Traders reports? Is this not overfitting?

Pardo says not only are differing parameter values usually needed to account for differing market conditions, but they are also advisable because they add portfolio diversification.

Overfitting, for those of you who don't know, is bad. Real bad. It's the bane of trading systems. This common problem happens when "excessive attention is paid to creating a curve that matches every twist and turn in the data," as Pardo puts it. "Overfitting occurs in the trading strategy development process when the trader adds rule after rule and parameter after parameter to the strategy and falsely boosts its trading performance beyond that which is likely to occur in future trading." I think that's a pretty good definition.

My system uses the same parameters in almost every setup. (The exceptions are a couple of setups that combine the signals from two setups.) It buys and sells when the net futures and options percentage-of-open-interest position of a group of traders hits a certain number of standard deviations above or below a moving average that in the past has led to reliable, market-beating profits. The other parameter is the number of weeks I wait after the signal to execute the trade.

The question is: Can you vary those standard deviation, moving average and trade delay values for each market? I've always believed different parameter values are needed to account for widely varying personalities I've observed in each COTs market dataset. A single set of values would probably produce a system with very poor chances of reliable profitability across many markets. But some readers have disagreed.

As well, Curtis Faith wrote in his excellent book, Way of the Turtle, that a single set of parameter values is needed for all the markets to be traded. Why? He argued there may not be enough historic data to come up with differing setups that are statistically robust in each market. My response has been that that's not the case with the Commitments of Traders futures and options data, which goes back to 1995 for most markets. Most of the recently updated trading setups I've found have at least 100 trades. That is generally considered more than enough for statistical reliability. (Thirty trades is usually seen as enough for a system to be theoretically reliable.) My second argument is this: if it's okay, and in fact advisable, to regularly re-optimize your parameter values every few years to account for new market conditions, why would it not be okay to optimize your values for an entirely new market?

Here's what Pardo says about all this (p. 38): "All markets have their own unique personalities. A trading strategy may perform well in one market with one set of values and poorly in another with those same values. I do not believe that one set of parameters should necessarily be sufficient for every market in which the strategy is traded. In my experience, such a situation is rare. In fact, I tend to prefer different values for a trading model for different markets, in that it offers an additional dimension of portfolio diversification. Optimization will identify the best set of parameter values for each market." Stay tuned for more thoughts from this book.