QF206 Week 11. Part 2 Back Testing Case Study: A TA-Based Example. 1 of 44 March 13, Christopher Ting

Size: px

Start display at page:

Download "QF206 Week 11. Part 2 Back Testing Case Study: A TA-Based Example. 1 of 44 March 13, Christopher Ting"

Abel Williams
5 years ago
Views:

1 Part 2 Back Testing Case Study: A TA-Based Example 1 of 44 March 13, 2017

2 Introduction Sourcing algorithmic trading ideas Getting data Making sure data are clean and void of biases Selecting a software package to back test Conducting paper testing Transitioning to production and actual trading 2 of 44 March 13, 2017

3 Ideas from Books: Some Examples Algorithmic Trading: Winning Strategies and Their Rationale Inside the Black Box: The Simple Truth About Quantitative Trading Mechanical Trading Systems Pairing Trader Psychology with Technical Analysis Pairs Trading Trading Systems and Methods- 5th Edition The Science of Algorithmic Trading and Portfolio Management 3 of 44 March 13, 2017

4 Ideas from Online Blogs The Whole Street Quantivity Quantitative Trading (Ernest Chan) Quantopian Quantpedia ETF HQ Elite Trader Forums Wealth Lab Nuclear Phynance Wilmott Forums 4 of 44 March 13, 2017

5 Ideas from Academics arxiv SSRN Journal of Investment Strategies Journal of Computational Finance Mathematical Finance 5 of 44 March 13, 2017

regional brokerage firm with 41 branch offices Served as Research Director/Economist for 2 other brokerage firms Wrote a daily

farm families, ranchers, and producers in other industries and their brokers and bankers how to hedge the risk of production with

6 George C. Lane, Stochastic Oscillator Professional trader for 50+ years Originator of Stochastic Indicator Floor Broker for 10 years Owned a regional brokerage firm with 41 branch offices Served as Research Director/Economist for 2 other brokerage firms Wrote a daily market letter/hot line for 16 years Authored 4 courses for Commodities Education and 1 for the Stock Market Trained thousands of farm families, ranchers, and producers in other industries and their brokers and bankers how to hedge the risk of production with futures Since 1953, has taught thousands of market analysts, money managers and individual investors in 30+ states and abroad how to use technical analysis to trade the markets more effectively Source: 6 of 44 March 13, 2017

7 Case Study of Stochastic Indicator At day time t, look back n candlesticks to compute HL HL n (t) = (Price(t) - Lowest n )/(Highest n - Lowest n ) 100 Price(t) is the closing price of time t. Lowest is the lowest price of the immediate past n periods, i.e., t-1, t-2,,t-n. Highest is the highest price of the immediate past n periods, i.e., t-1, t-2,,t-n. 7 of 44 March 13, 2017

8 QF day Stochastic Week 11 Date High Low Highest High (14) Lowest Low (14) Current Close Oscillator 1 23-Feb Feb Feb Feb Mar Mar Mar Mar Mar Mar Mar Mar Mar Mar % Mar % Mar % Mar % Mar % Mar % Mar % Mar % Mar % Mar % Mar % Mar % Mar % Mar % 28 1-Apr % 29 5-Apr % 30 8 of 44 6-Apr March , Christopher 56.76% Ting

9 Trading Rule Based on Stochastic Indicator Select m days to look back for trading signal. If HL n (t) > HL n (t - 1),, HL n (t - m), buy at time t+1 opening If HL n (t) < HL n (t - 1),, HL n (t - m), sell at time t+1 opening 9 of 44 March 13, 2017

10 Questions? What is the intuition behind this HL strategy? Which asset classes are suitable? Shorting a stock is not easy and costly. HL trading rules may work better for currencies and futures Frequency of trade? 10 of 44 March 13, 2017

11 Back-Test Strategies for Stochastic Oscillator There are two parameters, n and m. Let say the number of look-back periods n is fixed at 4. Back test to find an m 4 that maximizes the frequency of wins. Next, fix n at 5. Back test to find an m 5 that maximizes the frequency of wins. Next, fix n at 6. Back test to find an m 6 that maximizes the frequency of wins. And so on until n is of 44 March 13, 2017

12 Test Statistic (1) Based on a back-test on some finite sample of data, we compute a certain statistical measure called the test statistic. For concreteness, let s say the test statistic is the average daily return of a trading strategy in that period. We suppose that the true average daily return based on an infinite data set is actually zero. This supposition is called the null hypothesis. We suppose that the probability distribution of daily returns is known. This probability distribution has a zero mean, based on the null hypothesis. 12 of 44 March 13, 2017

13 Test Statistic (2) Based on this null hypothesis probability distribution, we compute the probability p that the average daily returns will be at least as large as the observed value in the back-test (or, for a general test statistic, as extreme, allowing for the possibility of a negative test statistic). This probability p is called the p-value, and if it is very small (let s say smaller than 0.01), that means we can reject the null hypothesis, and conclude that the back-tested average daily return is statistically significant. 13 of 44 March 13, 2017

14 When Not to Back-Test a Strategy Example 1: A strategy that has a back-test annualized return of 30% and a Sharpe ratio of 0.3, and a maximum drawdown duration of two years. Example 2: A long-only crude oil futures strategy returned 20% in 2007, with a Sharpe ratio of 1.5. Example 3: A simple buy-low-sell-high strategy picks the 10 lowest priced stocks at the beginning of the year and holds them for a year. The back-test return in 2001 is 388%. Example 4: A neural net trading model that has about 100 nodes generates a back-test Sharpe ratio of of 44 March 13, 2017

15 When Not to Back-Test a Strategy Example 5: A high-frequency E-mini S&P 500 futures trading strategy has a back-test annual average return of 200% and a Sharpe ratio of 6. Its average holding period is 50 seconds. Can we really back-test a high-frequency trading strategy? The performance of a high-frequency trading strategy depends on the order types used and the execution method in general. Furthermore, it depends crucially on the market microstructure. Even if we have historical data of the entire order book, the profit from a high-frequency strategy is still very dependent on the reactions of other market participants. 15 of 44 March 13, 2017

16 Par 2 Quantitative Trading Strategies Data for Back Testing 16 of 44 March 13, 2017

Getting Data Quandlbeta A B http://eareview.

17 Getting Data Quandlbeta A B C D 17 of 44 March 13, 2017

18 Motivation To carry out long-term historical analysis of futures, it is necessary to concatenate various individual futures contracts. Contracts can be concatenated using a variety of roll-date rules. It is vitally important to use a roll rule and price adjustment that corresponds to your use case (trading, back-testing, technical analysis, risk management, economic forecasting). Otherwise, your analysis will be misleading. 18 of 44 March 13, 2017

19 Continuous Contract History 19 of 44 March 13, 2017

20 Calendar-Weighted Adjusted Prices, Roll On Last Trading Day CBOT Wheat Futures #1 (W1) Backwards Ratio Adjusted Prices, Roll on First of Month 20 of 44 March 13, 2017

21 Main Problem with Futures Data Expiry date => roll effect Let s say the closing price of the front contract on date T-1 is p(t-1 ), and the closing price of this same contract on date T is p(t). let s say the closing price of the next nearby contract (also called the back contract) on date T is q(t ). Suppose T is the last trading day, also the rollover date for constructing the continuous futures price. You are long the front contract, so you should sell this contract at p(t), and then buy the next contract at q(t). 21 of 44 March 13, 2017

22 Different P&L and Return! What s the P&L and return of this strategy on T + 1? The P&L is p(t) p(t-1 ), The return is (p(t ) p(t-1 ))/p(t-1 ). But using the continuous price series, since T is he roll-over day, At T-1 the price is p(t-1 ) At T, the price is q(t) The P&L is q(t ) p(t-1 ) instead The return is (q(t + 1) p(t ))/p(t ) instead. The P&L and return from the continuous price series is wrong! 22 of 44 March 13, 2017

23 Solution? To prevent this error, you can typically back-adjust the data series to eliminate the price gap, so that the P&L on T is p(t ) p(t - 1). This can be done by adding the number q(t ) p(t ) to every price p(t) on every date t on or before T, so that the price change and P&L from T-1 to T is correctly calculated. 23 of 44 March 13, 2017

24 Continuous Futures Illustrated F T-2 F T-1 F T G T G T+1. G 2T-2 G 2T-1 G 2T H 2T H 2T+1 Assume that T is the last trading day of F month The active continuous series is F T-2 F T-1 G T G T+1. G 2T-1 H 2T Daily price change at roll date: G T F T-1 But actual daily P&L of a long position on roll day is F T F T-1 Sell F month contract and Buy G month contract to maintain a long position. 24 of 44 March 13, 2017

25 Adjustment for P&L Calculation Add G T F T to every price on every date on or before T. So the P&L of using the adjusted continuous futures price becomes (F T + G T F T ) (F T -1 + G T F T ) = F T F T-1 Of course, to take care of every rollover, you need to apply this back adjustment multiple times, as you go back further in the data series. The prices may turn negative in the distant past! 25 of 44 March 13, 2017

26 Problem Solved? Not quite! Check out what the return is at T Given this adjusted price series, the return is If you back-adjust to make the P&L calculation correct, you will leave the return calculation incorrect. The spread adjustment G T F T does not work for return calculation! Reason: Based on the adjusted futures prices, the daily return r T for date T is r T = F T + (G T F T ) F T 1 + (G T F T ) F T 1 + (G T F T ) F T F T 1 F T 1 26 of 44 March 13, 2017

27 Adjustment for Return Calculation Every daily price is multiplied by the ratio G T on every F T date on or before T. For every new roll date, adjustment must be made again. G G T F T T 1 F F T T F T 1 = G F T F T 1 T 1 F T But then the P&L calculation will be incorrect. You really can t have both. As long as you want the convenience of using a continuous contract series, you have to choose one performance measurement only, either P&L or return. 27 of 44 March 13, 2017

28 Use Individual Contracts Back-test your strategy on the various individual contracts, taking care of the rollover buying and selling yourself, then both P&L and return can be correctly calculated simultaneously. This subtlety in picking the right back-adjustment method is more important when you have a strategy that involves trading spreads between different contracts. If your strategy generates trading signals based on the price difference between two contracts, then you must choose the price back-adjustment method; otherwise, the price difference may be wrong and may generate a wrong trading signal. 28 of 44 March 13, 2017

29 How about Spreads? When a strategy involves calendar spreads, this back adjustment is even more important. This is because the calendar spread is a small number compared to the price of one leg of the spread, so any error due to rollover will be a significant percentage of the spread and very likely to trigger a wrong signal both in backtesting and in live trading. However, if your strategy generates trading signals based on the ratio of prices between two contracts, then you must choose the return back-adjustment method. 29 of 44 March 13, 2017

30 Inter-Market Spread Time! There is one general detail in back-testing inter-market spreads that should not be overlooked. If the contracts are traded on different exchanges, they are likely to have different closing times. So it would be wrong to form an inter-market spread using their closing prices. This is true also if you try to form a spread between a future and an ETF. The obvious remedy of this is to obtain intraday bid-ask data so that synchronicity is assured. The other possibility is to trade an ETF that holds a future instead of the future itself. For example, instead of trading the gold future GC (settlement price set at 1:30 p.m. ET) against the gold-miners ETF GDX, we can trade the gold trust GLD against GDX instead. Because both trade on Arca and their closing prices are set at the same 4:00 p.m. 30 of 44 March 13, 2017

31 Tick-by-Tick Data Analysis and Directional Trading Strategies Intra-Day Order Flows 31 of 44 March 13, 2017 Christopher Ting

32 Order Flow and its Measures In essence, order flow suggests the net direction of the market. When there were more buy (sell) market order than sell market order, the market direction would be typically up (down). How to measure order flows? Net trade sign: total number of buyer-initiated trades less the total number of seller-initiated trades Net trade volume: aggregate size of buyer-initiated trades less the aggregate size of seller-initiated trades 32 of 44 March 13, 2017 Christopher Ting

33 Order Flow in Trading In trading, order flow is an important concept. It is the overall trade direction at any given period of time. Ex post, order flow can be inferred from the trade direction. A trade is said to be buyer-initiated if the trade took place at the ask price or higher. Trade sign is +1 Conversely a trade is seller-initiated if the trade occurred at the bid price or lower. Trade sign is of 44 March 13, 2017 Christopher Ting

34 Motivation: Stat Arb at Real Time Can the order flow be known ex ante? If order flow can be forecasted, then the trade direction going forward can be anticipated! Positive order flow suggests the market is likely to go up. Negative order flow suggests the market is likely to go down. 34 of 44 March 13, 2017 Christopher Ting

35 What Exactly is Order Flow? Order flow essentially is a measure of overall trade direction over a period of time, say 5 minutes Over x trades, say 50 trades Over y contracts traded, say 500 contracts 36 of 44 March 13, 2017 Christopher Ting

36 How to Infer Trade Direction or Sign? Very simple: First find the best bid and ask quotes before a trade occurs. If the price is at the best bid, then it is a seller-initiated trade. Trade direction is -1 If the price is at the best ask, then it is a buyer-initiated trade. Trade direction is +1 What if the transaction occurs at the midpoint of a bid and ask quotes? The trade direction is 0 What if the trade occurs nearer to the bid than to the ask? The trade direction is -1 The key idea: Relative proximity to the best bid and ask 37 of 44 March 13, 2017 Christopher Ting

37 How to Back Test? Gather as many as possible high-quality tick-by-tick data, say 500 days, from Bloomberg. How to collect Bloomberg tick data systematically? Compute the order flows according to sign, signed volume, and signed dollar volume. Separate the data into 400 days and 100 days. Conduct the momentum strategy I on 400 days to find the optimal (x, y) combinations with x and y ranging from 5 to 30, resulting in = 676 combinations. Conduct the out-of-sample test on 100 days of data. 38 of 44 March 13, 2017 Christopher Ting

38 Order Flows Signed trades number of positive trades less number of negative trades Signed volumes positive volume less negative volume Signed $ volumes positive dollar volume less negative dollar volume 39 of 44 March 13, 2017 Christopher Ting

39 Example SIMSCI Futures Sign -5 Signed volume -8 Time Price Indicator Vol Sign Signed Vol Signed $ Vol A B A Y Y A B Y Y A B Y Y A B Y A Y Y A Y of 44 March 13, 2017

40 Quantitative Trading Idea Momentum I Entry If the order flow sign over the last x trades is positive, buy l lot at the ask immediately. If the order flow sign over the last x trades is negative, sell l lot at the bid immediately. Exit If the order flow sign over the last x+y trades is negative, sell back 1 lot. If the order flow sign over the last x+y trades is positive, buy back 1 lot immediately. No position to be carried over night, i.e., square off the position before the trading session ends. 41 of 44 March 13, 2017 Christopher Ting

41 Entry, Exit, Re-Entry, Exit, Re-Entry, Exit,. x y x y x y x What is the intuition? Positive order flows give rise to positive price change. Negative order flows produce negative pride change. Which is correct? Must include own trade in the order flow calculation. Don t include own trade in the order flow calculation. 42 of 44 March 13, 2017 Christopher Ting

42 Another Idea Compute the order flows over a trading day for frontmonth SIMSCI futures. If the order flow measured by signed dollar volume exceeds a certain threshold, Buy if the signed dollar volume is negative; Sell if the signed dollar volume is positive. Enter the market at opening and exit the market at closing. 43 of 44 March 13, 2017 Christopher Ting

43 Summary Order flow is a measure indicative of the net amount of buying and selling activities Buyer-initiated trades are transactions at the ask prices Seller-initiated trades are transactions at the bid prices Order flow measured in number the number of buyer-initiated trades less the number of seller-initiated trades Order flow measured in (dollar) volume the (dollar) volume of buyer-initiated trades less the (dollar) volume of seller-initiated trades When order flows are positive (negative), prices are higher (lower) 44 of 44 March 13, 2017 Christopher Ting

Algorithmic Trading Session 4 Trade Signal Generation II Backtesting. Oliver Steinki, CFA, FRM

Algorithmic Trading Session 4 Trade Signal Generation II Backtesting Oliver Steinki, CFA, FRM Outline Introduction Backtesting Common Pitfalls of Backtesting Statistical Signficance of Backtesting Summary