Monday 31 March 2008

Filled!

Just updated my Portfolio page to reflect trades as of today. These are based on new signals for the open this morning from my trading setups based on the Commitments of Traders reports issued weekly by the U.S. Commodity Futures Trading Commission. Speaking of which, I've also posted several news items on my News page about the White House's latest plans to save the economy and crack down on the financial markets, which include merging the CFTC with other agencies. I'll be following that with great interest, including how it relates to the COTs reports. Good luck this week!

Friday 28 March 2008

Hello, Silver! S&P 500, Gold Flipflop

How about that! A nutso week capped off by an equally nutso Commitments of Traders report. What to make of this mess of data! Nothing like a new COTs report to get me all riled up. Check this out:

- Silver, I'm baaaack! The "smart money" commercial traders have cut their net short futures and options position back from its recently bearish extremes. I've got a bullish signal for execution on Monday's open. What do you think about that, silver bugs?

- My gold bullion setup is still in its flipflopping mode like something out of Cirque du Soleil. It's now back to bullish. That's four consecutive new signals in four weeks. What this means: the setup says to go long next week, then short April 7, then back to long April 14. Should qualify again for the reduced active trader commission really soon at this rate.

- My trading setup for the S&P 500 is also trying to get in on the flipping around fun. It's just gone back to bullish. That setup uses a three-week delay and already gave a bearish signal for execution Monday. So this means I'll be short three weeks and go back long on April 21.

- My setup for the Russell 2000 has also gone back to bullish. It has no trade delay, which means my short position that I put on Monday gets sold this coming Monday and I go back to being long.

For more details, check my Latest Signals page.

Housekeeping: You might notice on the Latest Signals table that I've reorganized how I post signal information a little. I realized there might have been some confusion about what the current and next signals are, especially since some setups have somewhat long trade delays. In the "Execution Date for Next Pending Signal," I'm just going to post the next upcoming signal date (red for a bearish signal, green for bullish and black for cash). If there is an additional pending signals, I will enter it only when it becomes the most immediate next one so as to avoid confusion. As well, the "Current Signal in Force" column will refer to the signal in effect on the date this table was last updated (which is in the top left-hand corner). I'll include some notes to this effect on that table soon.

Also, you might have noticed I'm getting more frequent signals these days in some of my setups. That's because most of my newly revised setups based on the COTs data are using shorter moving-average periods - generally between four and 16 weeks - than the older ones. You'll notice, for example, that most of these setups had more trades (see the "Winning/Losing Trades" column). I chose these setups because they are more responsive to the data and gave what I believe to be more reliable past profits. Very often, the setup I chose was far from being the most profitable setup in a market. What was key, however, was its statistical robustness. I'm still revising some of the setups and starting to do some walk-forward testing on them, to see how they performed on out-of-sample data. Thanks for your patience as I go through these vital refinement steps. I think they'll pay off handsomely.

Lust for Lustre

Just posted an Op/Ed piece I wrote about gold bugs for yesterday's Montreal Gazette newspaper here, titled "Lust for Lustre." Yikes, gold and silver down sharply in the pre-open. See you back here later today with my update on this afternoon's Commitments of Traders release.

Tuesday 25 March 2008

Testing: How Much is Enough?

How much is enough? That's one of the key questions of trading system design: how much historic data is enough to give reliable results? Think of it this way: if you tested a trading system on two years of a bull market, what happens when the market turns down? The answer: probably not anything good. One reader with this on their mind recently wrote in with this comment: "If you can't back test this over at least three types of economic cycles, it's utterly coin flipping prone to failure." I answered in the comment space, but because this is an important issue I wanted to devote a post to it.

I'm going to refer again to Robert Pardo's great new book, The Evaluation and Optimization of Trading Strategies, which updates his 1991 bible of trading system development, Design, Testing and Optimization of Trading Systems. He addresses this issue extensively. What he says is quite interesting. He says there is no fixed period of time a system should be backtested over, so long as the period includes at least one of each of the major market types: bull, bear, congested and cyclic. As well, it should include lots of varying volatility periods. He says 10 years of data is sufficient. One especially interesting point he makes is that a longer test period, while incorporating more variety of market conditions and likely more robust going forward, will tend to be less responsive (i.e., less profitable) to current price action. It will also need less frequent reoptimization. Conversely, a shorter time frame will be more responsive but less adaptable to dramatic changes in market conditions. I tend to lean toward the longer test period, since it's often hard to know until it's too late when conditions have changed. But one alternative approach would be to include a price indicator that does that for you. That could allow a more responsive system.

Finally, Pardo writes about a key test for a trading strategy that is relevant to the length of the test window: ensuring the strategy uses no more than 10 percent of the available degrees of freedom (or datapoints). This includes the "overhead" number of weeks before you get your first signal. I've written about this not long ago in this post. For example, if we have 665 weeks of data (or available degrees of freedom) and a trading system with 12 trading rules, this means we can't use a moving average rule that exceeds 48 weeks [(12+48)*10+60]. To read more about all this, see pages 130-1, 140-4 and 171 of Pardo's book. Good luck this week!

Monday 24 March 2008

Filled!

I guess I must have had too relaxing a holiday weekend because I didn't realize until this afternoon that the markets were open today despite Easter. What would Jesus say?! So I had a busy little period this afternoon checking the markets and timing some trades to reflect a veritable avalanche of new signals, including bearish for the Russell 2000, gold and copper and cash for the NASDAQ 100. Ah, the memories of day-trading! Among other things, I was interested to look for breakouts from this morning's opening range based on the 15-minute charts. Lots of action today! I've just updated my portfolio page with my new positioning. In the end, I decided to dump my XGD Canadian iShares Gold Fund, even though my trading setup for that remains in bullish mode, and pick up some leveraged HBD Horizons BetaPro Gold Bullion Bear Plus trading in Toronto. The reason: my gold setup is a little more robust than my XGD/HUI Gold Bugs Index setup, most likely because the former is more directly tied to the Commitments of Traders data that underlies both setups. (There is no COTs data for XGD or HUI, so I used the gold COTs data to find my XGD/HUI setup.) Good luck this week!

Friday 21 March 2008

New Bearish Signals; Only One in Five Equities Still Bulllish

Sweet Mother of God. What an insane week that was. Where do I start? Silver? Jesus. Never would have wished that on anyone, including those unfortunate souls who wrote in not so long ago to gently chide my silver bearish signal. Banks? They're acting more hopped-up than the nuttiest dot-com stock back in the day. And if you think that's nutso, wait 'till you check out my Latest Signals, which I just updated with, er, you guessed it, my latest signals. That would be from this afternoon's special Good Friday edition of the Commitments of Traders data, issued weekly for free by the nice people at the U.S. Commodity Futures Trading Commission.

Here's what we got:

- Bearish signal for my trading setup for the Russell 2000 after the "smart money" commercial traders bailed out of their net long futures and options positions. Sayonara, RUT. This setup works with no trade delay, meaning for the open of trading next week.

- Going to cash in my setup for the NASDAQ 100 index. This, after the "dumb money" small traders inexplicably boosted their net long position to bullish extremes this week. Risky move, guys! No trade delay on this one either. That means of all five of my equity setups, only the Nikkei is presently still bullish.

- Gold. What can I say? This one's flipping around like a fish on the dock. Now we're back to bearish. This setup works with a two-week delay, so at this point I've got three pending signals: (1) a bearish signal to be executed this Monday, March 24, which I will hold for one week until... (2) a bullish signal that came with the March 11 COTs report, to be executed March 31, and finally from today's signal, (3) another bearish signal, to be executed April 7. Meanwhile, my setup for the HUI Gold Bugs Index and XGD iShares Canadian Gold Fund is still in the bullish column. Kind of unusual, but that's what the data says. I have to think about what to do with my XGD holding through all this. The gold setup has slightly better statistical robustness scores, so I might sell it and go short gold bullion for next week, but not sure. I don't like not following all signals, but when they conflict like this I have to use some discretion based on the larger portfolio.

- My setup for crude oil has gone back to bullish. With the four-week trade delay, that means execution for the open of Monday, April 21.

Lots to think about. Hope you have a great holiday weekend, and good luck next week!

Wednesday 19 March 2008

Get Diversified

Okay, stats time, folks. I wanted to give you an update on Robert Pardo's fascinating new book, The Evaluation and Optimization of Trading Strategies, which updates his 1991 bible of trading system development, Design, Testing and Optimization of Trading Systems. Pardo addresses an important question that has been raised by a couple of readers: Is it okay to have different parameter values for each market I'm trading with my COTs Timer system, based on the Commitments of Traders reports? Is this not overfitting?

Pardo says not only are differing parameter values usually needed to account for differing market conditions, but they are also advisable because they add portfolio diversification.

Overfitting, for those of you who don't know, is bad. Real bad. It's the bane of trading systems. This common problem happens when "excessive attention is paid to creating a curve that matches every twist and turn in the data," as Pardo puts it. "Overfitting occurs in the trading strategy development process when the trader adds rule after rule and parameter after parameter to the strategy and falsely boosts its trading performance beyond that which is likely to occur in future trading." I think that's a pretty good definition.

My system uses the same parameters in almost every setup. (The exceptions are a couple of setups that combine the signals from two setups.) It buys and sells when the net futures and options percentage-of-open-interest position of a group of traders hits a certain number of standard deviations above or below a moving average that in the past has led to reliable, market-beating profits. The other parameter is the number of weeks I wait after the signal to execute the trade.

The question is: Can you vary those standard deviation, moving average and trade delay values for each market? I've always believed different parameter values are needed to account for widely varying personalities I've observed in each COTs market dataset. A single set of values would probably produce a system with very poor chances of reliable profitability across many markets. But some readers have disagreed.

As well, Curtis Faith wrote in his excellent book, Way of the Turtle, that a single set of parameter values is needed for all the markets to be traded. Why? He argued there may not be enough historic data to come up with differing setups that are statistically robust in each market. My response has been that that's not the case with the Commitments of Traders futures and options data, which goes back to 1995 for most markets. Most of the recently updated trading setups I've found have at least 100 trades. That is generally considered more than enough for statistical reliability. (Thirty trades is usually seen as enough for a system to be theoretically reliable.) My second argument is this: if it's okay, and in fact advisable, to regularly re-optimize your parameter values every few years to account for new market conditions, why would it not be okay to optimize your values for an entirely new market?

Here's what Pardo says about all this (p. 38): "All markets have their own unique personalities. A trading strategy may perform well in one market with one set of values and poorly in another with those same values. I do not believe that one set of parameters should necessarily be sufficient for every market in which the strategy is traded. In my experience, such a situation is rare. In fact, I tend to prefer different values for a trading model for different markets, in that it offers an additional dimension of portfolio diversification. Optimization will identify the best set of parameter values for each market." Stay tuned for more thoughts from this book.

Monday 17 March 2008

Setup Housekeeping

I've just updated my Latest Signal table to remove a couple of trading setups I no longer think are useful - those for OIH (Oil Services Holders) and XEG (Canadian Energy iShares). The problem with those two was the data started only in 2001, which I don't believe gives us enough varied market conditions for reliable testing. A second consideration is that those two setups were based on the Commitments of Traders data for crude oil, but OIH and XEG often don't move the same as crude (much less so than gold stocks vs. gold for example).

I've also temporarily taken my COTs Composite Agriculture Index offline. This one was based on the data from four of my agriculture-related setups (wheat, soybeans, corn and sugar). It gave me signals for the DBA PowerShares Agriculture Fund. The problem with the DBA setup was it used a 104-week moving average, which my research suggests may be too long for good statistical robustness in this dataset. The reason for you stats junkies: Robert Pardo's book on trading system development says your trading rules and moving average period together shouldn't use up more than 10 percent of your available degrees of freedom (including the "overhead" number of weeks before you get your first signal). In this dataset, we have about 510 weeks of data from the Goldman Sachs Agriculture Subindex (which I used to calculate the setup results). That means the trading rules plus moving average period shouldn't total more than 46. I'll update the index after I've revised those four agriculture setups in the coming weeks. Good luck this week!

Friday 14 March 2008

Gold Back to Bullish, Natural Gas Commercials Get Longer

Another totally insane week! I've just updated my Latest Signals table with the word from my trading setups based on the weekly Commitments of Traders reports. Some highlights:

- My gold trading setup, based on trading on the same side as the large speculators, has flipped back to bullish after a week in the bearish camp. This setup works with a two-week trade delay, so that means I'll buy something to short gold for the week of March 24, then go back to long. One-week trades are rare in this system, but they do sometimes happen, particularly it seems when there's market indecision. Or is it indigestion. One note of caution: the "dumb money" gold small traders once again increased their net long position as a percentage of the total interest. The wrong-way crowd now stands just a hair from triggering a bearish signal for my setups for the HUI Gold Bugs Index and XGD iShares Canadian Gold Fund.

- My five-year Treasury setup has gone from cash to bullish (meaning it thinks the yield will fall).

- Large speculators in 3-month Eurodollars, whom I trade alongside in my setup for the BKX U.S. Bank Index, maintain their fairly solid bullish bias. This means the setup thinks the recent maniacal volatility for the sector will resolve itself on the upside.

- Despite the recent vertiginous rise in the price of natural gas, commercial traders in natural gas futures and options significantly bumped up their net percentage-of-open-interest position today, meaning they appear to believe still more gains are coming. I've been holding a leveraged natural gas ETF since the signal went bullish in January, so I don't mind. But really now. Incredible.

Not much else that's new! Slow week. Have a great weekend and good luck to us all next week!

Thursday 13 March 2008

Revised Dow Jones Industrials Setup: Still Bearish

I've slightly revised my Dow Jones industrials trading setup based on the Commitments of Traders reports issued weekly by the U.S. Commodity Futures Trading Commission. These are the reports that detail what large trading firms are doing in the futures and options markets. My new Dow Jones industrials setup is still based on fading the large speculators, but with slightly different parameter values than the previous setup. It still went bearish with the Jan. 29 COTs report. It has slightly lower profit results, a Sharpe ratio of 2.6, which is slightly lower than the old's one score of 2.7, but better confidence intervals for profitability and beating the market. Overall, I think it's more robust. Good luck the rest of this week.

Monday 10 March 2008

Thinking About Robert Pardo's New Book

Not long ago I got a copy of The Evaluation and Optimization of Trading Strategies, Robert Pardo's revised version of his classic 1991 book on trading system development, Design, Testing and Optimization of Trading Systems. It's giving me lots of ideas on how to optimize my trading system based on the Commitments of Traders reports. I've already implemented a couple of basic concepts, like limiting testing to setups with at least 25 trades and excluding setups that use a moving average period of more than 35 to 45 weeks. (The latter is because a setup won't be reliable if it uses more than 10 percent of the available degrees of freedom based on the total weeks in the dataset.) But my main goal is to develop some walk-forward testing for my system based on Pardo's ideas. Stay tuned. Highly recommended book!

Friday 7 March 2008

S&P 500, Gold Setups Turn Down, But New RUT Setup Still Bullish

So is it a bear or not? Just checked the long-term weekly charts and the major indexes - SPX, NYA, TSX composite, FTSE, Nikkei - are all telling the same story: they're broken down, rallies have fizzled, and they're making new lows - the definition of a change of trend. So what's does today's Commitments of Traders data say about this? I just updated my Latest Signals page with the latest news.

Important new signal from my trading setup for the S&P 500: bearish. This setup works by trading on the same side as the "smart money" commercial traders in futures and options when they hit extremes of positioning. Well, they just got extremely negatory. That setup works with a three-week trade delay, which means I'll execute the trade for the open of Monday, March 31.

Also of interest: My setup for gold has now joined my silver setup in going bearish. The gold setup works by trading on the same side as the large speculators, believe it or not. They're not always the dumb money, it turns out. It actually depends a lot on what timeframe you look at them. The large specs tend to be trend followers, so they're only wrong at extremes and only if they stay in too late. Looks like they're bailing out of gold. This setup has a two-week delay, so execution is Monday, March 24.

My gold stocks setup for the HUI Gold Bugs Index and iShares Canadian Gold Fund (XGD) - based on fading the small traders - remains in bullish mode for now. It's important to note this setup has no trade delay, so it could flip to bearish to coincide with the gold setup. As it happens, the "dumb money" crowd has been getting increasingly exuberant for the past seven weeks straight, and it's now getting close to the extreme point that could trigger a sell. Stay tuned. I'm still long XGD for now.

Also, my 30-year Treasury setup has gone to cash. This setup combines my best signals from the large specs and commercials, and when they don't agree - like now - it goes to cash. The commercials have just gone to bearish on the bond (bullish the yield), while the large specs remain bullish the bond, as they've been since last August.

Housekeeping: I've also updated the Latest Signals table with a setup I like slightly better for the Russell 2000. This one is based on the commercial traders, not the small traders. Unlike the old setup, which flipped to bearish a few weeks ago, but with a seven-week trade delay for execution in April, the new setup is squarely in the bullish column right now.

Have a good weekend, and see you back here next week with more updates.

Thursday 6 March 2008

New Combo NASDAQ 100 Setup Bullish, New Heating Oil Setup Bullish for Early April

Just updated my Latest Signals page table with some new trading setups I'm looking forward to trading. First, I took another look at my NASDAQ 100 setup based on the Commitments of Traders reports. (In case you didn't know, these are the little-known government numbers that tell us how the big boys are positioned in the derivatives markets.) I wondered what would happen if I combined the signals for my best setups for the small traders and commercial traders, and sure enough, a great new setup was born. The Sharpe ratio is a pretty swinging 4.1, compound annual ground rate is 27.6 percent, Robust Sharpe is 3.4 and regressed annual return is 27.0 percent. The confidence intervals are on the table. Sweet numbers! That setup has been bullish since the Feb. 5 COTs report.

Also new on that table is my revised setup for heating oil. That one is based on trading in the same direction as the large speculators. They're not always the dumb money, it turns out. The results are also pretty nice. Sharpe 1.8, CAGR 36.4%, Robust Sharpe 2.2 and RAR 43.9%. It works with a six-week trade delay and just flipped to bullish for execution on the open of Monday, April 7. See you back here Friday afternoon with signals from the new data.

Monday 3 March 2008

New Setups: Platinum Bullish as of March 10, Nikkei Called Bottom Feb. 11

Just updated my Latest Signals page with details from my new trading setups for Japan's Nikkei Stock Average and platinum. Like all my other setups, these two are based exclusively on the weekly data released free in the Commitments of Traders reports of the U.S. Commodity Futures Trading Commission. Unlike trading based on market prices or fundamentals, which often stop working after a while as traders arbitrage away their edge, the setups on this website are some of the few in existence based just on the COTs data.

The Nikkei setup went long with the Dec. 11 Commitments of Traders report, and the setup uses an eight-week trade delay, so that means execution was on the open of Monday, Feb. 11. My chart shows that's a day or two after EWJ Japan iShares put in what appears to be an interim bottom. Nice. My platinum setup went bullish with the Jan. 8 COTs report, which, with an eight-week trade delay, means execution for next Monday, March 10. Good luck this week!

Filled! Long Banks

Just updated my portfolio to reflect sale of my SKF UltraShort Financials fund and going long UYG Ultra Financials a few minutes after the open this morning. (Got caught up in answering readers' comments, and I forgot to enter the trade right on the open.) The trade was based on my BKX Bank Index setup flipping back to long a week ago Friday. That setup is based on trading off the 3-month Eurodollars futures and options positioning, as reported weekly by the U.S. Commodity Futures Trading Commission. See my Latest Signals page for the full list of setups based on this fascinating government data.