and whipsaws. The noise channel
breakout system shows how a filter
can improve the performance
of intraday breakout trading.
bars (nhi) by an amount greater than or equal to f, buy at market. For example, if n = 20 and f = 2 (points), you would go long when price moved 2 points above the highest high of the last 20 bars.
In addition, when short, and when calculating the highest high price (hhp), it cannot be higher than the previously calculated hhp as previous highs are dropped out of the lookback window.
Otherwise, a situation can occur where there is a higher hhp without the price filter f being hit. Therefore, when short the stock, the hhp can only stay the same or go lower. It cannot go higher.
Sell rule: If price crosses below the lowest low price of last n bars (nlo) minus an amount greater than or equal to f, sell at market. In addition, when long and when calculating the lowest low price (llp), it cannot be lower than the previous calculated llp as previous lows are dropped out of the lookback window. Again, to avoid the situation where a lower llp occurs without the price filter f being hit, when long the stock, the llp can only stay the same or go higher. It cannot go lower.
Exit rule: Close the position five min-utes before the NYSE close (no trades are carried overnight).
Testing the strategy
The “walk-forward testing” approach will be used to test this strategy because of the volatile nature of intraday stock prices. Intraday price dynamics are con-stantly changing because of economic surprises, events and trader sentiment.
Also, the time of year — such as the sea-son, holidays, vacation time, etc. — affects the character of intraday markets.
As a result, tests performed on intraday data three months ago may no longer be representative of today’s intraday price action. For more information on walk-forward testing and how it was used for this strategy, see “Proper system test-ing,.”
The best parameters will be defined as those values that generate the best net profits combined with the minimum drawdown and minimum largest losing trades. In addition, the results should be
stable — i.e., the profits, wins and drawdowns should not change much as the parameters move by a small amount away from their opti-mum values. In other words, the system
43 www.activetradermag.com • September 2001 • ACTIVE TRADER
TABLE 1 OPTIMUM PARAMETER VALUES FOR TEST DATA
Start date End date nhi nlo f
2/21/01 3/23/01 8 4 1
2/28/01 3/30/01 8 4 1
Performance summary for noise channel breakout system: IBM, five-minute bars from Feb. 28 to March 30. Statistics based upon trading 1,000 shares of IBM.
TABLE 2A TEST PERIOD 1
TABLE 2B TEST PERIOD 2
Performance summary: All trades
Total net profit ($): 13,890 Open position P/L ($): 0
Gross profit ($): 39,260 Gross loss ($): -25,370
Total no. of trades: 48 Percent profitable (%): 54
Number winning trades: 26 Number losing trades: 22
Largest winning trade ($): 5,940 Largest losing trade ($): -2,060 Average winning trade ($): 1,510 Average losing trade ($): -1,153.18 Ratio avg. win/avg. loss: 1.309 Avg. trade(win & loss) ($): 289.38
Max. consec. winners: 4 Max. consec. losers: 3
Avg. no. bars in winners: 39 Avg. no. bars in losers: 21 Max intraday drawdown ($): -8,470
Profit factor: 1.547 Max. no. contracts held: 1
Performance summary: All trades
Total net profit ($): 10,490 Open position P/L ($): 0
Gross profit ($): 35,500 Gross loss ($): -25,010
Total no. of trades: 47 Percent profitable (%): 47
Number winning trades: 22 Number losing trades: 25
Largest winning trade ($): 5,940 Largest losing trade ($): -1,840 Average winning trade ($): 1,613.64 Average losing trade ($): -1,000.40 Ratio avg. win/avg. loss: 1.613 Avg. trade(win & loss) ($): 223.191
Max. consec. winners: 3 Max. consec. losers: 3
Avg. no. bars in winners: 39 Avg. no. bars in losers: 26 Max. intraday drawdown ($): -9,660
Profit factor: 1.419 Max. no. contracts held: 1
Source: TradeStation by TradeStation Group Inc.
Performance summary for noise channel breakout system: IBM, five-minute bars from Feb. 21 to March 23. Statistics based upon trading 1,000 shares of IBM.
performance using an nhi of 10 bars should be similar to that using nine bars or 11 bars. Also, in choosing the “best”
parameters, we considered only those results with four or less maximum con-secutive losses.
Test results
Table 1 shows the optimum parameter values for the test window described in
“Proper system testing.” The nhi was eight bars, the nlo was four bars and f was 1 point. Tables 2a and 2b show test results using these parameters.
Table 3 summarizes the combined performance of the two out-of-sample data segments from March 26 to April 6.
This performance represents what would have happened in real time if you used the system parameters found in the test section (not including slippage and commissions). By comparison, the same nhi and nlo values tested without any filter resulted in a loss of $1,150.
Table 4 is a trade-by-trade summary
from March 26 to April 6. The trades in this time period are the out-of-sample trades
generated from the optimized parameters from the two test sections of Feb 21 to
ACTIVE TRADER • September 2001 • www.activetradermag.com 44
Combined walk-forward out-of-sample performance summary for the noise channel breakout system: IBM five-minute bars from March 26 to April 6.
Statistics based upon trading 1,000 shares of IBM.
TABLE 3 OUT-OF-SAMPLE RESULTS
Performance summary: All trades
Total net profit ($): 8,390 Open position P/L ($): 0
Gross profit ($): 14,460 Gross loss ($): -6,070
Total no. of trades: 16 Percent profitable (%): 50
Number winning trades: 8 Number losing trades: 8
Largest winning trade ($): 4,000 Largest losing trade ($): -1,350 Average winning trade ($): 1,807.50 Average losing trade ($): -758.75 Ratio avg. win/avg. loss: 2.382 Avg. trade(win & loss) ($): 524.38
Max. consec. winners: 5 Max. consec. losers: 3
Avg. no. bars in winners: 54 Avg. no. bars in losers: 37 Max. intraday drawdown ($): -4,480
Profit factor: 2.382 Max. no. contracts held: 1
Source: TradeStation by TradeStation Group Inc.
This trade summary for the out-of-sample test (five-minute bars, March 26 to April 6) of the noise channel breakout system shows the strategy actually worked better on the short side than the long side.
TABLE 4 TRADE-BY-TRADE SUMMARY
Entry Entry Buy Entry Exit Exit Exit # bars P&L ($) P&L (%) Max. Time Max. Time
Date time or price date time price in profit drawdown
sell trade ($)
3/26/01 10:20 Sell 93.75 3/26/01 15:55 94.52 67 (770) -0.82% 0 10:20 (1,620) 10:35
3/27/01 10:15 Buy95.59 3/27/01 15:55 99.59 68 4,000 4.18% 4,300 15:50 0 10:15
3/28/01 9:40 Sell 97.92 3/28/01 15:55 94.50 75 3,420 3.49% 3,420 12:00 (380) 9:40 3/29/01 10:05 Buy96.05 3/29/01 15:05 94.90 60 (1,150) -1.20% 950 10:30 (1,160) 11:20
3/29/01 15:05 Sell 94.90 3/29/01 15:55 94.88 10 20 0.02% 390 15:15 (500) 15:45
3/30/01 9:40 Buy96.70 3/30/01 13:05 96.20 41 (500) -0.52% 800 11:55 (1,190) 10:00 3/30/01 13:05 Sell 96.20 3/30/01 15:55 96.25 34 (50) -0.05% 220 13:15 (840) 14:35 4/2/01 9:40 Buy97.75 4/2/01 10:55 96.40 15 (1,350) -1.38% 350 10:05 (1,350) 10:55 4/2/01 10:55 Sell 96.40 4/2/01 15:55 94.50 60 1,900 1.97% 2,600 15:40 (1,300) 11:40
4/3/01 10:00 Sell 93.00 4/3/01 15:55 90.50 71 2,500 2.69% 2,740 15:40 0 10:00
4/4/01 9:45 Buy92.00 4/4/01 13:50 92.00 49 0 0.00% 1,900 11:20 (1,890) 10:30
4/4/01 13:50 Sell 92.00 4/4/01 15:55 91.85 25 150 0.16% 380 14:00 (500) 14:20
4/5/01 9:40 Buy95.68 4/5/01 15:55 98.15 75 2,470 2.58% 3,040 15:25 (10) 9:40
4/6/01 9:40 Sell 97.30 4/6/01 11:55 98.24 27 (940) -0.97% 550 11:15 (940) 11:55 4/6/01 11:55 Buy98.24 4/6/01 12:35 97.30 8 (940) -0.96% 1,660 12:05 (940) 12:35 4/6/01 12:35 Sell 97.30 4/6/01 15:55 97.67 40 (370) -0.38% 300 12:35 (1,960) 13:55
March 23 and Feb. 28 to March 30.
Figure 1 is a five-minute chart with the noise channel superimposed, as well as some of the buy and sell signals from the Table 4 trade-by-trade summary.
Breaking down the numbers With respect to average winning and los-ing trades, drawdowns and profit factor, the out-of-sample performance (Table 3) was better than the test sample perform-ance (Tables 2a and 2b) The better per-formance of the out-of-sample section could have been coincidental, but it does indicate that four weeks of test data was enough to capture the intraday price dynamics of this stock.
W
hen testing any trading strategy, the important point is how well it will perform on price data it has not been optimized on — that is, out-of-sample data. In short, without out-of-sample testing, it’s nothing more than a hope and a prayer to believe that system per-formance in the future will be anywhere near the optimized performance.For example, it’s possible to take a trading strategy with four independent variables, or parameters, and with hind-sight, find the values for each of them that give the best (optimized) results on a specific historical period — say, the last three years (using daily price data).
However, these optimized parameter values have, in essence, been cherry-picked for this particular data period (a process known as “curve-fitting”), and are unlikely to per-form as well on other historical test periods, or in actual trad-ing in the future.
A walk-forward testing procedure was applied to the noise channel breakout system as follows: Five-minute bars from a period of four weeks from the start of the test period — Feb.
21 to March 23 — were chosen and system parameter values were found through optimization on this intraday data seg-ment. In other words, the “best-performing” system parame-ters (e.g., number of days in lookback period, noise filter value) were determined by testing a range of values for each.
At this stage of system development, the only thing indicated by the optimum values in the test portion is that the data has been “curve-fitted” as best it can with this system. Without further testing on out-of-sample data, there is no way to tell if the system will work in the future.
These parameter values were then applied to an out-of-sample data period following the test segment (March 26 to March 30). This walk-forward process was repeated by moving the test data window forward one week, to Feb. 28 to March 30, and again finding the parameters values through optimiza-tion on this new data. These optimized parameter values are then applied to the next out-of-sample five-minute intraday data window (April 2 to April 6). An important (but unspoken) point in walk-forward testing is that if you cannot get good results in the out-of-sample data segments, real-time system performance will be random.
Almost any period of historical prices can be curve fitted easily to give the false illusion of future profitability.
However, these performance measures in no way reflect how a system will perform on price data it has not been optimized on. Only out-of-sample testing — that is, testing on price data the system parameters were not originally derived from — can determine if a system is robust and has a chance of perform-ing well in real tradperform-ing.
Despite these facts, many market pundits still make the unproven claim that statistics generated solely from optimized buy and sell trades in the test section (the initial period of price data)have value in predicting whether or not the system will perform well in the future. Nothing could be further from the truth. The only thing the statistics from the test section tell you is how well you have curve-fitted the data in the test section. As a matter of fact, using optimization, it’s almost impossible not to get an excellent fit with great statistical results.