Real-Time Text Analytics for Event Detection in the Financial World

Transcription

1 Real-Time Text Analytics for Event Detection in the Financial World Volker Stümpflen April 2015 Gaining value from Big Data Winner

2 Information Delay - A Big Data Problem Markets are driven by news (and sentiments) Hacked AP tweet Loss of S&P 500 alone totaled $136.5 billion within six minutes Cost of a second: $380 million

3 Use Case Baader Bank AG A leading German investment bank, market maker, sales trader Missed relevant information in real-time financial instruments N traders and analysts news p.d. ~4 bn sentences p.a. From stocks to derivatives Increasing Decreasing time for increasing information Is constant and small From news agencies to social media channels Strongly increasing 3

4 Market Moving Event Types From news corpus Big Data analytics Reuters 2008 news corpus S&P 500 companies Price change >= 1% in less than 1 minute Event Rel Freq. CDS Price Move" 1" Analyst Forecast" 1" Business Climate Change" 1" CEO Search" 1" Company Forecast" 1" Customer Problems" 1" Debt Financing" 1" Equity Financing" 1" Fraud Investigation" 1" Government Decision (no bailout)" 1" Incorporation Change" 1" Legal Settlement" 1" M&A" 1" Restructuring" 1" Supply Chain" 1" Trading Halt" 1" Asset Liquidation" 2" Stocks Fall (Peers)" 2" Dividend Change! 3! Broker Rating! 9! Quarterly Results! 10! 4

5 Three Information Classes Common events E.g. U.K. Economy Grew 0.6% in Fourth Quarter, Revised From 0.5% Structured and accesible with simple RegEx Immediately in the market Grey Swans E.g. The Swiss National Bank scrapped the cap on the Franc. Expected market movers but individual event is less frequent Form together approx. 50% of all events Black Swans E.g. Zombie virus epedemic Unexpected market movers and typically catastrophes 5

6 The Way We Look at Events Information is always connected Economy consist of Entities like companies, people, products, locations, catastropes,... Events occur if entities doing something with each other The simplest event is Entity A is doing something with Entity B Complex events are Superpositions of simple events And/or indirect effects of simple events (guilty by association) 6

7 A Recent Example Cap on franc crapped FXCM had serious losses 7

8 Networks The Most Natural Way to Look At It 8

9 Networks With Predicate Argument Structures Swiss National Bank scrapped the cap on the franc = 225 million francs FXCM owe Customers 9

10 Wanted Software that... extracts PAS out of raw unstructured text is fast (PAS after few ms per sentence) has high precision for agent, patient, beneficiary is easy to extend for domain-specific language e.g. *EHEALTH SEES YEAR ADJ. EPS 34C-41, EST. 38C is inherently multi-lingual Nothing available did all of that so we built our own! 10

11 Preprocessing The SNB scrapped the cap on the franc. Markets are stunned. Sentence Splitting Sentence 1 Sentence 2 Tokenization The SNB scrapped the cap on the franc Markets are stunned Part-of-speech tagging DT NNP VBD DT NN IN DT NN NNS VB P The SNB scrapped the cap on the franc VBN Markets are stunned noun verb 11

12 Known Concept Identification Customers owe FXCM approx. 225 million francs. to owe owe/owes/owing/owed/owed Takes arg0, arg1, arg2 FXCM Forex Capital Markets Ltd NYSE: FXCM Currency Value Value: Currency: Swiss Franc Includes named entity recognition (NER) Uses machine learning techniques, Context Free Grammars (CFGs) and pattern matching Easy to extend 12

13 Chunk Parsing DT The NN decision IN of DT the NNP SNB RB greatly VBD surprised DT the NNS markets Chunking TOKEN TOKEN TK TK TK TOKEN TOKEN TK TOKEN Start: 1-token chunks NOUN CHUNK NOUN CHUNK VERB CHUNK NOUN CHUNK Apply CFG A chunk is (working definition) a sequence of consecutive tokens grouped by some notion of syntactic or semantic function or dependency. 13

14 Mixed Semantic and Syntactical Analysis NNP Forex NNP Capital NNP Markets NNP Ltd Chunking TOKEN TOKEN TOKEN TK KNOWN CONCEPT NOUN CHUNK FXCM Forex Capital Markets Ltd NYSE: FXCM Semantic concepts that are recognized during chunking are attached to special Known Concept chunks Subsequent CFGs can recursively check for is a company etc. 14

15 Detecting Predicate-Argument Structures DT The NN decision IN of DT the NNP SNB RB greatly VBD surprised DT the NNS markets TOKEN TOKEN TK TK TK TOKEN TOKEN TK TOKEN Start: 1-token chunks Chunking C CONCEPT NOUN CHUNK NOUN CHUNK VERB CHUNK NOUN CHUNK Known concepts Apply CFG agent patient The decision of the SNB surprised the markets detected PAS 15

16 Implemented in Scala/Akka A horizontally scalable real-time solution SRV 1 SRV 2 ElasticSearch DB ElasticSearch DB AMQP Fetcher fetch Frontend Reuters AMQP ClusterNode ClusterNode failover Bloomberg AMQP fetch Frontend Baader AMQP Fetcher Services ClusterNode ClusterNode Services Internet SRV 3 ElasticSearch DB SRV 4 ElasticSearch DB 16

17 Applications 17

18 Real-Time Event Detection 18

19 Mood Propagation Networks Systemic Mood Inferring sentiment channels Calculating positive and negative sentiment flow Similar to metabolic networks in biology Sony Google Motorola legal action Samsung Apple XYZ Microsoft Foxconn China ACTING COMPANY NEGATIVE RELATION RECEIVING COMPANY Apple sues Samsung Rare Earths in Australia LOCATION OF RELATION Arbitrary example 19

20 Systemic Mood The Fukushima Example Activation of renewable energy companies Before 3/11/2011: No interest in renewable energies After 3/11/

21 Conclusion Real-time news analytics system PAS pipeline based on machine learning techniques, Context Free Grammars (CFGs) and pattern matching Native transformation into associative network Utilized e.g. to infer sentiment propagation Benefits for Baader Bank AG Comprehensive news analytics Smart market moving events Reduced completely the losses due to missed news Substantial increased trading profit 21

22 Contact Dr. Volker Stümpflen T M v.stuempflen@clueda.com Clueda AG Elsenheimerstraße 59 D Munich Clueda AG 22