Wednesday, 19 March 2008

Get Diversified

Okay, stats time, folks. I wanted to give you an update on Robert Pardo's fascinating new book, The Evaluation and Optimization of Trading Strategies, which updates his 1991 bible of trading system development, Design, Testing and Optimization of Trading Systems. Pardo addresses an important question that has been raised by a couple of readers: Is it okay to have different parameter values for each market I'm trading with my COTs Timer system, based on the Commitments of Traders reports? Is this not overfitting?

Pardo says not only are differing parameter values usually needed to account for differing market conditions, but they are also advisable because they add portfolio diversification.

Overfitting, for those of you who don't know, is bad. Real bad. It's the bane of trading systems. This common problem happens when "excessive attention is paid to creating a curve that matches every twist and turn in the data," as Pardo puts it. "Overfitting occurs in the trading strategy development process when the trader adds rule after rule and parameter after parameter to the strategy and falsely boosts its trading performance beyond that which is likely to occur in future trading." I think that's a pretty good definition.

My system uses the same parameters in almost every setup. (The exceptions are a couple of setups that combine the signals from two setups.) It buys and sells when the net futures and options percentage-of-open-interest position of a group of traders hits a certain number of standard deviations above or below a moving average that in the past has led to reliable, market-beating profits. The other parameter is the number of weeks I wait after the signal to execute the trade.

The question is: Can you vary those standard deviation, moving average and trade delay values for each market? I've always believed different parameter values are needed to account for widely varying personalities I've observed in each COTs market dataset. A single set of values would probably produce a system with very poor chances of reliable profitability across many markets. But some readers have disagreed.

As well, Curtis Faith wrote in his excellent book, Way of the Turtle, that a single set of parameter values is needed for all the markets to be traded. Why? He argued there may not be enough historic data to come up with differing setups that are statistically robust in each market. My response has been that that's not the case with the Commitments of Traders futures and options data, which goes back to 1995 for most markets. Most of the recently updated trading setups I've found have at least 100 trades. That is generally considered more than enough for statistical reliability. (Thirty trades is usually seen as enough for a system to be theoretically reliable.) My second argument is this: if it's okay, and in fact advisable, to regularly re-optimize your parameter values every few years to account for new market conditions, why would it not be okay to optimize your values for an entirely new market?

Here's what Pardo says about all this (p. 38): "All markets have their own unique personalities. A trading strategy may perform well in one market with one set of values and poorly in another with those same values. I do not believe that one set of parameters should necessarily be sufficient for every market in which the strategy is traded. In my experience, such a situation is rare. In fact, I tend to prefer different values for a trading model for different markets, in that it offers an additional dimension of portfolio diversification. Optimization will identify the best set of parameter values for each market." Stay tuned for more thoughts from this book.


Anonymous said...

I hadn't heard of Stephen Vita before but have added it to my blog list, are you a subscriber to his site or just the free newsletter?

A quick question, given what has happened today with the commodities, are you tempted to close out your long positions early? or go short now?

I had problems getting the silver short ETF so instead went short with the HGD on Tuesday morning.


Alex Roslin said...

Congrats on your trade, Silverharp. You're up big-time! I subscribe to Stephen's site, but his free blog has some good stuff too. One of the most useful market blogs out there IMHO.

No silver short ETF so far, but there is one in the application process at the SEC right now, so perhaps soon.

As for me, I hadn't even thought of heading out early. A move like today often leads to a counter snap in the other direction. But that's not actually my reasoning. If you're going to follow a trading system, you have to follow it like a religion or there's no point.

Here's a couple of good Pardo quotes on that: "It is the following of the well-tested trading strategy with exactitude, and without exception, that is the greatest assurance of long-term trading success. Conversely, deviation from such a well-tested trading strategy will most often lead to trading loss."

And this: "Even the discretionary trader trades with a plan and systematically determines entries, exits, trade size, and so forth. Trading without a well-defined plan has the same likely outcome as doing anything without a plan - failure."


Anonymous said...

Your site is interesting but you don't have a trading system. You have anecdotal data that you've cleaned from recent COT reports. If you can't back test this over at least three types of economic cycles, it's utterly coin flipping prone to failure.

Alex Roslin said...

Hi Anonymous,

Thanks for your comment. I believe the various bears, bulls and trading ranges since 1995 would constitute a good number more than three different economic cycles in most of the markets studied.