S&P 500, Nikkei Setups Pass Monte Carlo Test

Figured out a way to do Monte Carlo testing on my trading setups in Excel. Phew, that was a learning experience. I've mostly tested it so far on the S&P 500. Happy days. A good number of my top trading setups based on the Commitments of Traders reports scored well above the 95-percent mark that signifies I can be confident their results weren't some kind of freak anomaly of luck, but rather a genuine market effect. A number actually scored better than the 99-percent mark. (That basically means that the average return was better than 99 percent of the returns when I scrambled the S&P 500 price data 6,000 times.) Same for my best trading setup for the Nikkei. It scored better than 99.3 percent of the 6,000 randomized returns. Unfortunately, my brand new S&P 500 setup I just announced Monday isn't one of them. It scored only at the 93.6-percent mark - not bad but not good enough for me to risk money on it. [UPDATE DEC. 5: Oops, made a mistake there. Just took another look at something I did wrong, and my current SPX setup does indeed score better than 95 percent on the Monte Carlo test. (In fact, its average weekly profit was better than that of 98.8 percent of randomly generated market runs.) Sorry about the screw-up. Still need to evaluate all the setups against each other as described below.]

I've got just one more step I want to do before I choose my best setup: a new test I just thought of combining the Monte Carlo test with my walk-around test. The idea of the walk-around test is to vary all the parameter values and see how this impacts the results. The worse those results, the greater the risk of data-mining bias (i.e., a setup that won't work well in actual trading). I check 16 variations of each setup's parameter values and come up with a "walk-around efficiency" score to see how much the results are impacted. You can see this in Column Y on my latest signals table. Curtis Faith talks about a somewhat similar technique in his book Way of the Turtle. He calls it "parameter scrambling." What I still want to do is take all these "walked-around" setups and do Monte Carlo tests on them, too. That would produce a "walk-around Monte Carlo efficiency" score. I'd add this to the measures I use to discard useless setups and choose the best ones. Should have something to announce in a few days. Incidentally, I've been thinking of creating a new webpage that talks about all these testing steps - kind of an amateur's primer on trading system development. Hope it can help someone else out there.