UNDERSTANDING ML/DL MODELS USING INTERACTIVE VISUALIZATION TECHNIQUES

Transcription

1 UNDERSTANDING ML/DL MODELS USING INTERACTIVE VISUALIZATION TECHNIQUES Chakri Cherukuri Senior Researcher Quantitative Financial Research Group 1

2 OUTLINE Introduction Applied machine learning in finance Case studies Twitter Sentiment Analysis Learning option prices using deep learning tools Yield Curve dimensionality reduction (PCA vs Autoencoder) Conclusion 2

3 APPLIED MACHINE LEARNING IN FINANCE 3

4 STRUCTURED DATA SETS Task Features Labels Machine Learning Technique Time series prediction Past returns, market conditions Future returns LSTM Illiquid asset pricing Asset characteristics Market price Boosted Trees/Random Forests Trading Strategies Market conditions Strategy to invest in Boosted Trees/Random Forests Dimensionality Reduction Yield Curve Yield curve PCA/Autoencoder Exotic option pricing Deal/market parameters Price Neural nets 4

5 UNSTRUCTURED DATA SETS Task Object detection from satellite images Abstractive summarization of news articles for quick consumption News/twitter sentiment for stocks, commodities etc. Entity embeddings for companies, news, documents Deep Learning Model Conv nets RNN, attention based models NLP models (Word embeddings + Nets) LSTM/RNN 5

6 TWITTER SENTIMENT ANALYSIS 6

7 NEWS/TWITTER SENTIMENT News & social sentiment from the raw news story or tweet Unstructured Highly time-sensitive Story-level sentiment Company-level sentiment Sentiment score can be used as a trading signal 7

8 RUSSELL 2000 STOCKS 8

9 TWITTER SENTIMENT CLASSIFICATION Problem statement: Predict the sentiment (negative, neutral, positive) of a tweet for a company Ex: $CTIC Rated strong buy by three WS analysts. Increased target rom $5 to $8. : Positive Three way classification problem Input: raw tweets Output: sentiment label {negative, neutral, positive} 9

10 METHODOLOGY We are given labeled train and test data sets Train classifier on training data set Predict labels on test data and evaluate performance 10

11 ONE-VS-REST LOGISTIC REGRESSION Train three binary classifiers for each label Model 1: Negative vs. Not Negative Model 2: Neutral vs. Not Neutral Model 3: Positive vs. Not Positive Get probabilities (measures of confidence) for each label Output the label associated with the highest probability 11

12 CLASSIFIER PERFORMANCE ANALYSIS Look at misclassifications Confusion matrix Understand model predicted probabilities Triangle visualization Fix data issues 12

13 TRIANGLE VISUALIZATION Model returns 3 probabilities (which sum to 1) How can we visualize these 3 numbers? Points inside an equilateral triangle Negative / Neutral Not sure Very positive 13

14 PERFORMANCE ANALYSIS DASHBOARD Use the dashboard to: Analyze misclassifications (using confusion matrix) Improve model by adding more features (by looking at model coefficients) Fix data issues (using triangle and lasso) 14

15 ANALYZE MISCLASSIFICATIONS 15

16 ANALYZE MISCLASSIFICATIONS 16

17 ANALYZE MISCLASSIFICATIONS 17

18 USE LASSO TO FIND DATA ISSUES 18

19 USE LASSO TO FIND DATA ISSUES 19

20 DEEP LEARNING TOOLS 20

21 NEURAL NETWORK WIZARD Graphical tool to build, train and diagnose deep learning models Real time plots during the training process: Loss/Accuracy curves Distributions of weights/biases/activations at each layer Diagnostic plots: Analysis of residuals (for regression) / Confusion matrix (for classification) Partial dependencies Conditional residual plots/histograms 21

22 NETWORK PARAMETERS 22

23 NETWORK ARCHITECTURE 23

24 LOSS AND ACCURACY CURVES 24

25 DISTRIBUTIONS OF WEIGHTS/BIASES/ACTIVATIONS 25

26 PARTIAL AND CONDITIONAL DEPENDENCIES Training dataset Conditioned on S=70 Conditioned on S=80 Conditioned on S=120 S T sigma moneyness S T sigma moneyness S T Sigma moneyness S T sigma moneyness Training dataset Conditioned on T=1 Conditioned on T=.5 Conditioned on T=2 S T sigma moneyness S T sigma moneyness S T Sigma moneyness S T sigma moneyness

27 DIAGNOSTIC PLOTS 27

28 YIELD CURVE DIMENSIONALITY REDUCTION 28

29 YIELD CURVE PRIMER Bonds have a fixed maturity (1M, 3M, 10Y) and pay coupons Examples of bonds treasury bonds, corporates, muni etc. Yield Curve: Plot of bond yields against maturities Adjacent points on the yield curve move together (correlated) 29

30 U.S. TREASURY YIELD CURVE 11 tenors/maturities Typically upward sloping Different shapes Pre-crisis Post-crisis Current 30

31 YIELD CURVE DYNAMICS Yield for each tenor (point on the yield curve) changes every day Problem: How to model the changes in the yield curve driven by 11 correlated variables? Any parsimonious representation possible? 31

32 PRINCIPAL COMPONENT ANALYSIS (PCA) PCA can be used to: Reduce dimensionality Retain as much variance in the dataset as possible Typically first few (3-5) PCA factors enough to explain almost all the variance 32

33 PCA OVER DIFFERENT TIME PERIODS PCA factors vary with time periods Interval Selector Quickly select different time intervals Perform stats on the selected time slices (using callbacks) 33

34 YIELD CURVE PCA: CRISIS 34

35 YIELD CURVE PCA: AFTER CRISIS 35

36 YIELD CURVE PCA: CURRENT 36

37 YIELD CURVE PCA: CURRENT 37

38 DIMENSION REDUCTION: AUTOENCODERS tanh relu linear Compressed feature vector 38

39 PCA VS AUTOENCODER PCA Autoencoder 39

40 DIMENSION REDUCTION: AE VS PCA 40

41 CONCLUSION Abundance of financial data Abundance of already existing models/techniques ML/DL techniques provide new ways of modeling financial data Interactive visualization tools help us better understand and interpret these models 41

42 RESOURCES Widget libraries used to build the applications: ipywidgets: bqplot: (and other custom widgets) ML/DL libraries scikit-learn: tensorflow: keras: Tech at Bloomberg: 42