Machine Learning in High Frequency Algorithmic Trading

Transcription

1 Machine Learning in High Frequency Algorithmic Trading I am very excited to be talking to you today about not just my experience with High Frequency Trading, but also about opportunities of technology based disruption in finance. Disruption When I joined HFT back in 2005, it didn t have a name. It was just trading. At most a hundred people were working on our data-driven style of trading. What was unique to us was that, we did not try to emulate star traders. We approached it as a pure learning-from-data exercise. Later we realized that we were disrupting two HUGE business lines, (a) the fat commission brokerage business that was making billions in annual revenues. (b) the traders at banks and funds who were making even more. But we didn t know that. Over the last ten years, while my strategies have made more than a billion dollars, those businesses are almost gone. Stack for HFT On the outset, HFT is about buying in one market and quickly selling in another market for a small profit. And we do this a million times a day. The trick is to very quickly cancel all our bids when one of them gets filled. The stack we have for this has been engineered around speed. The exchange API coded packet is received via 10G lines, and a nanosecond L2/L3 switch. Then it is parsed via FPGA and sent to applications via PCIe directly to user space bypassing the kernel. There are three components of the trading app: 1. Machine learning algorithms 2. Execution algorithms 3. Order routing server All are written in very low level C++. The inter process communication is all in user space, via an LMAX polling sort of architecture, where if we miss a packet, it is fine to move on, but it is not fine to wait. All this allows us to send the trade back to the switch in under six microseconds deterministically.

2 What s the trade? The real trade is about trading with people who don t care about executing orders smartly, who feel like Oh wow Facebook is a great stock and I want to buy it now. They don t have the time or resources to think about Is it a good idea to buy it now? Should I wait?. Who are these people? It s us, normal real money investors who don t have time for this. We have a job and family and the responsibility to self-manage our portfolio. We are easy targets. HFT firms make a spread in the market and keep collecting the difference between the bid price and the ask price while we trade. An HFT firm would for instance, buy at $117 when someone wants to sell a share of FB and then sell at $118 when someone wants to buy a share of FB. How can Machine Learning help? Okay, now I am going to get technical and I need you to pay attention.

3 - The best way to play blackjack is to bet on each hand an amount of money that is proportional to the reward / risk of the trade. (Kelly Breiman bet sizing) - If I am buying at $117, my expected reward from the action is (my_predicted_price - $117) - The risk = expected range of the stock in the time it takes for me to close out the trade. - Hence to maximize the reward, I would want to buy when the predicted price is much higher than $ To minimize the risk I would want to buy when I know it will not take me much time to close out the position. - Hence we used machine learning to figure out the predicted price and the time it will take me to close out the trade. Just to give you a credible example, look at the link below of a huge court case between Citadel s HFT group that made 2 Billion in 2009 and two ex-leaders of the team who quit to start their own company. In the documents they refer to weighted midpoint, which is a form of expected price, and they say that this knowledge is worth billions. df What is weighted midpoint? If 10 people are buying at $117 and 40 people are selling at $118, the midpoint price would be $117.5 but weighted midpoint price would be $ It is just a mathematical formulation that is likely to yield a price that is in average closer to the price the stock will have thirty seconds from now. This is exactly the sort of feature engineering we were doing for many years. Combining different strategies We try to predict the returns of the stock in the next thirty seconds. We take indicators like weighted midpoint, that would try to infer the fair price from the orders placed in the market book.

4 We made features based on actual trades that have happened recently. So if the last few trades have been happening at $118, perhaps that is where the price is going to. These indicators are called Trade Imbalance indicators or Order Flow Imbalance indicators. We have also found time-series indicators useful. One of the most useful set of indicators are lead-lag indicators. For instance, if Crude Oil drops suddenly, then it is safe to safe that Exxon Mobil stock will drop too. How do we combine these indicators... Most of us use a Linear Regression model. Why? Why not anything more advanced? Questions from the audience : 1. Do you think Machine Learning is more useful in HFT now or when you started in 2005? 2. Why do you think Linear Models are more popular than NonLinear models? 3. Are methods like sentiment-analysis useful? 4. How can I get started on HFT trading now? 5. What are the costs like for getting market data? 6. If HFT is counter productive for a real money investor, then why do we still keep it around? How can we remove HFT? 7. Do you learn fixed rules about trading, or do your rules change?

5 8. Is it truly possible to have completely automated trading? Doesn t the trader constantly need to steer it in a certain direction? Why finance? Finance is so boring! Let s switch gears a little bit. My aim here is not just to talk about my journey. I want you guys to do something. I want you to recognize that there is a lot of work to be done in finance and a lot of change to be brought by innovation. Well, from a monetary perspective there is a lot of opportunity. Like a friend of mine was telling me have you ever wondered why all analytics companies work mostly in healthcare and finance? Because there is a ton of money in these industries. But that also means that there is a lot that is wrong, a lot that needs fixing. For instance most of us use more machine learning while finding driving directions on an app than we use in the entire year to manage our money. That needs to change. -- Gaurav, Head of Strategy Development at QPlum gchak@qplum.co