Real-Time Market Data Technology Overview

Transcription

1 Real-Time Market Data Technology Overview Zoltan Radvanyi Morgan Stanley

2 Session Outline What is market data? Basic terms used in market data world Market data processing systems Real time requirements Designing performance of real-time market data processing software Hardware based market data processing systems Q&A

3 Financial markets Marketplaces to trade financial products like stocks, currencies, etc. Example: Stock exchanges ( ( )

4 Financial instruments The things that are traded on a financial market ie. bought by buyers / sold by sellers. Examples: Stock exchanges: stocks (

5 Orders and trades Order: An order made by someone to buy or sell some financial instruments. Buy order: someone s intention to buy a defined number of instruments at a defined / below a defined Sell order: someone s intention to sell a defined number of instruments at a defined / above a defined An order represents a market position of a trader Trade: The event when a buyer buys some instruments (e.g. stocks) from a seller at an agreed

6 Order book The list of buy/sell orders investors entered for a financial instrument. Buy side: contains the buy orders Sell side: contains the sell orders Order books are maintained electronically by the exchanges

7 Order book example buy orders Buy size Buy Individual orders

8 Order book example buy orders Buy size Buy Prices in descending order from the top to the bottom

9 Order book example buy orders Buy size Buy Price levels

10 Order book example buy orders Full order book Buy size Buy order 1 entry Aggregated order book Buy size Buy entry Sizes are aggregated

11 Order book example sell orders Sell size Sell Individual orders

12 Order book example sell orders Sell size Sell Prices in ascending order from the top to the bottom

13 Order book example sell orders Full order book Sell size Sell Aggregated order book Sell size Sell

14 Order book example aggregated buy + sell side together Morgan Stanley Best buy = highest to buy Best sell = lowest to sell Buy size Buy Sell Sell size

15 Trades A trade happens when a buy order s is equal to or greater than the lowest sell or a sell order s is equal to or less than the highest buy

16 Trades - example Buy size Buy Sell Sell size

17 Trades - example Someone entered an order to buy 3000 pieces of stock for a of 122 EUR Buy size Buy Sell Sell size There are 2000 stocks available for 122 EUR on the sell side

18 Trades - example 1000 stocks remained unfilled on the buy side at 122 EUR Buy size Buy Sell Sell size All stocks at 122 EUR got filled on the sell side The new best sell is now 125 EUR Result: 2000 stocks got traded at trade of 122 EUR.

19 Market data Market s (trade s) ( Order books

20 Market data Market data is important for traders to know what is going on the markets (

21 Market data distribution Morgan Stanley Market data processing engines Network Message distribution network Traders & trading engines Exchange

22 Market data messages Describe market events like changes Generated by exchanges in real-time Each message describes an event for one instrument (e.g. company stock) or a group of instruments Distributed to consumers via different types of network connections (TCP/IP, UDP multicast) Message formats are specific to an exchange

23 Market data messages - example A market data message describing one level of the order book of one instrument may have the below structure: MessageType (2 bytes) InstrumentName (4 bytes) OrderBookLevel (2 bytes) SellPrice (8 bytes) SellSize (8 bytes) BuyPrice (8 bytes) BuySize (8 bytes) Specification struct OrderBookMessage { char MessageType[ 2 ]; char InstrumentName[ 4 ]; char OrderBookLevel[ 2 ]; char SellPrice[ 8 ]; char SellSize[ 8 ]; char BuyPrice[ 8 ]; char BuySize[ 8 ]; }; Example representation in a C program 12 OTPB Real message

24 Market data messages protocol stack Market data messages TCP Market data messages UDP (mainly multicast) IP Data link layer Physical layer

25 Real-time: low data latency & high throughput Latency: Time it takes for data to arrive at traders from the exchange Measured in microseconds Ideally less than 50µs Throughput: The average number of messages processed in a unit of time. Measured in messages/second Market peeks around 1 million messages/sec Goal: keep latency as low and throughput as high as possible

26 Why real time? Algorithmic trading: trading computers (engines) automatically entering orders on the market Computers can react to market events much faster than human traders Latency matters! The time between a new order appears on the market, trading decision is made and action is taken may happen below µs High frequency trading Arbitrage: taking advantage of a difference between two or more markets

27 Algorithmic trading example - Arbitrage Buy size Buy Sell Sell size Market1 Buy size Buy Sell Sell size Market2

28 Algorithmic trading example - Arbitrage I am going to buy 30 stocks for 100 EUR on Market1 and sell all of them for 125 EUR on Market2 Buy size Buy Sell Sell size Market1 Buy size Buy Sell Sell size Market2

29 Algorithmic trading example - Arbitrage Buy 30 stocks for 100 EUR Buy size Buy Sell Sell size Market1 Buy size Buy Sell Sell size Market2

30 Algorithmic trading example - Arbitrage I have now 30 stocks, let s sell it quickly! Buy size Buy Sell Sell size Market1 Buy size Buy Sell Sell size Market2

31 Algorithmic trading example - Arbitrage Sell 30 stocks for 125 EUR Buy size Buy Sell Sell size Market1 Buy size Buy Sell Sell size Market

32 Algorithmic trading example - Arbitrage I made a profit of 25 EUR per stock Buy size Buy Sell Sell size Market1 Buy size Buy Sell Sell size Market2

33 Market data processing engines Morgan Stanley Market data processing engines Network Message distribution network Traders & trading engines Exchange

34 Market data processing engines Purpose: Receive market data messages in the exchange s message format Extract information from the messages Create messages in the firm s internal message format containing the extracted information and send them to the message distributor network Challenge: How to process the incoming messages to gain low latency and high throughput

35 Market data processing engines Two main categories of solutions: Single threaded message processing Multithreaded message processing

36 Single threaded solution Incoming messages are processed sequentially Can be used where message rate is low (some thousand messages/sec) and where latency is not so critical (data is only used by human traders) Easy to maintain and debug For reasonable performance code must be micro-tuned parsing of one message should not take more than µs

37 Multithreaded solution using worker threads Messages for the same financial instrument have to be processed in the same order as the messages come in Can be implemented with a pool of worker threads consuming messages from dedicated input queues Messages for the same financial instrument always have to be put in the same queue

38 Multithreaded solution using worker threads A 1 A 2 B 1 A 3 C 1 A 4 A 5 D 1 B 2 B 3 D 2... An instrument s messages will always be processed by the same thread e.g. messages for instrument A will always be put in Thread1 s queue Thread1 Thread2 Thread3 Thread4

39 Issues using worker thread approach An instrument may be more actively traded than an other one (e.g. OTP vs. TVK) uneven distribution of update rates for different instruments leads to uneven distribution of workload among worker threads Assigning messages to threads happens on a message basis without having a higher level view of the work that needs to be done Hard to define the number of worker threads Does not scale well with high number of CPUs

40 Issues using worker thread approach Test conditions: 16 core box (Intel Nehalem CPUs) running 64 bit RedHat4 OS Concurrency level - how many threads were simultaneously busy and for how long during the runtime of the process Thread activity dark green shows active threads Worker threads Time

41 Task based parallelism using Intel TBB threading library Morgan Stanley No need for managing system threads directly Developer only deals with creating / grouping logical tasks to execute The library deals with the execution of tasks ensuring high level of parallelism Task stealing: load balancing algorithm to increase CPU utilization. When a thread has finished with the tasks in its queue it can steal tasks from an other thread.

42 Task based parallelism using Intel TBB threading library Morgan Stanley Algorithm of processing incoming messages using TBB: Split up a block of incoming messages to groups which can be processed in parallel: messages for one instrument will get into the same group Messages in one group will be processed sequentally (in their chornological order) but the different groups will be executed in parallel One logical task will be responsible for processing one message group Let the TBB library schedule the execution of the tasks

43 Task based parallelism using Intel TBB threading library Morgan Stanley A 1 A 2 B 1 A 3 C 1 A 4 D 1 A 5 B 2 E 1 F 1 E 2... Thread1 Thread2 Thread3 Thread4

44 Task based parallelism using Intel TBB threading library Morgan Stanley A 1 A 2 A 3 A 4 B 1 B 2 C 1 D 1 E 1 E 2 F 1... Task1 Task 2 Task 3 Task 4 Task 5 Task 6 TBB scheduler Thread1 Thread2 Thread3 Thread4

45 How does solution using TBB perform? Concurrency level Thread activity Worker threads Time

46 Tools for performance analysis Intel VTune -Thread Profiler Intel VTune Sampling and Call Graph Intel Purify/Quantify Valgrind Internally developed tools for latency and throughput measurements

47 FPGA accelerated low-latency market data processing FPGA - Field-Programmable Gate Array Configuration: Hardware Description Language (HDL) Wide range of applications Can perfectly used for market data processing Flexible High throughput with low latency by eliminating the operating system's network stack Constant latency, irrespective of throughput ~ 4µs Very expensive

48 Thank you for your attention! Q&A