Online Adaboost-Based Parameterized Methods for Dynamic Distributed Network Intrusion Detection

Transcription

1 Onlne Adaboos-Based Parameerzed Mehods or Dnamc Dsrbued Nework Inruson Deecon Wemng Hu, Jun Gao, Yanguo Wang, and Ou Wu (Naonal Laboraor o Paern Recognon, Insue o Auomaon, Chnese Academ o Scences, Beng ) {wmhu, gao, gwang, wuou}@nlpr.a.ac.cn Sephen Mabank (Deparmen o Compuer Scence and Inormaon Ssems, Brkbeck College, Male Sree, London WC1E 7HX) smabank@dcs.bbk.ac.uk Absrac: Curren nework nruson deecon ssems (NIDS) lack he adapabl o he requenl changng nework envronmens. Furhermore, nruson deecon n he new dsrbued archecures s now a maor requremen. In hs paper, we propose wo onlne Adaboos-based nruson deecon algorhms. In he rs algorhm, a radonal onlne Adaboos process s used where decson sumps are used as weak classers. In he second algorhm, an mproved onlne Adaboos process s proposed, and onlne GMMs are used as weak classers. We urher propose a dsrbued nruson deecon ramework, n whch a local parameerzed deecon model s consruced n each node usng he onlne Adaboos algorhm. A global deecon model s consruced n each node b combnng he local paramerc models usng a small number o samples n he node. Ths combnaon s acheved usng an algorhm based on parcle swarm opmzaon (PSO) and suppor vecor machnes (SVM). The global model n each node s used o deec nrusons. Expermenal resuls show ha he mproved onlne Adaboos process wh GMMs obans a hgher deecon rae and a lower alse alarm rae han he radonal onlne Adaboos process whch uses decson sumps. Boh he algorhms ouperorm exsng nruson deecon algorhms. I s also shown ha our PSO and SVM-based algorhm eecvel combnes he local deecon models no he global model n each node: he global model n a node can handle he nruson pes whch are ound n oher nodes, whou sharng he samples o hese nruson pes. Index erms: Nework nrusons, Dnamc dsrbued deecon, Onlne Adaboos learnng, Parameerzed model 1. Inroducon Nework aack deecon s one o he mos mporan problems n nework normaon secur. Currenl here are manl rewall, NIDS/NIPS (nework-based nruson deecon and prevenon ssems) and UTM (uned hrea managemen) lke devces o deec aacks n nework nrasrucure. NIDS/NIPS deec and preven nework behavors whch volae or endanger nework secur. Bascall, rewalls are used o block ceran pes o rac o mprove he secur. NIDS and rewalls can be lnked o block nework aacks. UTM devces combne rewall, NIDS/NIPS and oher capables ono a sngle devce o deec smlar evens as sandalone rewalls and NIDS/NIPS devces. Deep Packe Inspecon (DPI) [52] adds analss on he applcaon laer, and hen recognzes varous applcaons and her conens. DPI can ncorporae NIDS no rewalls. I can ncrease he accurac o nruson deecon, bu s more me-consumng, n conras o radonal package header analss. Ths paper ocuses on nvesgaon o NIDS. The curren praccal soluons or NIDS used n ndusr are msuse-based mehods whch ulze sgnaures 1

2 o aacks o deec nrusons b modelng each pe o aack. As pcal msuse deecon mehods, paern machng mehods search packages or he aack eaures b ulzng proocol rules and srng machng. Paern machng mehods can eecvel deec he well-known nrusons. Bu he rel on he mel generaon o aack sgnaures, and al o deec novel and unknown aacks. In he case o rapd proleraon o novel and unknown aacks, an deense based on sgnaures o known aacks becomes mpossble. Moreover, he ncreasng dvers o aacks obsrucs modelng sgnaures. Machne learnng deals wh auomacall nerrng and generalzng dependences rom daa o allow exrapolaon o dependences o unseen daa. Machne learnng mehods or nruson deecon model boh aack daa and normal nework daa, and allow or deecon o unknown aacks usng he nework eaures [60]. Ths paper ocuses on machne learnng-based NIDS. The machne learnng-based nruson deecon mehods can be classed as sascs-based, daa mnng-based, and classcaon-based. All he hree classes o mehods rs exrac low level eaures and hen learn rules or models whch are used o deec nrusons. A bre revew o each class o mehods s gven below. 1) Sascs-based mehods consruc sascal models o nework connecons o deermne wheher a new connecon s an aack. For nsance, Dennng [1] consruc sascal proles or normal behavors. The proles are used o deec anomalous behavors whch are reaed as aacks. Caberera e al. [2] adop he Kolmogorov-Smrnov es o compare observaon nework sgnals wh normal behavor sgnals, assumng ha he number o observed evens n a me segmen obes he Posson dsrbuon. L and Mankopoulos [22] exrac several represenave parameers o nework lows, and model hese parameers usng a hperbolc dsrbuon. Peng e al. [23] use a nonparamerc cumulave sum algorhm o analze he sascs o nework daa, and urher deec anomales on he nework. 2) Daa mnng-based mehods mne rules whch are used o deermne wheher a new connecon s an aack. For nsance, Lee e al. [3] characerze normal nework behavors usng assocaon rules and requen epsode rules [24]. Devaons rom hese rules ndcae nrusons on he nework. Zhang e al. [40] use he random ores algorhm o auomacall buld paerns o aacks. Oe e al. [4] propose an algorhm or mnng requen emses (groups o arbue value pars) o combne caegorcal and connuous arbues o daa. The algorhm s exended o handle dnamc and sreamng daa ses. Zanero and Savares [25] rs use unsupervsed cluserng o reduce he nework packe paload o a racable sze, and hen a radonal anomal deecon algorhm s appled o nruson deecon. Mabu e al. [49] deec nrusons b mnng uzz class assocaon rules usng genec nework programmng. Pangrah and Sural [51] deec nrusons usng uzz logc, whch combnes evdence rom a user s curren and pas behavors. 3) Classcaon-based mehods consruc a classer whch s used o class new connecons as eher aacks or normal connecons. For nsance, Mukkamala e al. [30] use he suppor vecor machne (SVM) o dsngush beween normal nework behavors and aacks, and urher den mporan eaures or nruson deecon. Mll and Inoue [31] propose he TreeSVM and ArraSVM algorhms or reducng he necences whch arse when a sequenal mnmal opmzaon algorhm or nruson deecon s learn rom a large se o ranng daa. Zhang and Shen [7] use SVMs o mplemen onlne nruson deecon. Kaack e al. [5] propose 2

3 an algorhm or nruson deecon based on he Kohonen sel organzng eaure map (SOM). Specc aenon s gven o a drec labelng o SOM nodes wh he connecon pe. Bvens e al. [26] propose an nruson deecon mehod n whch SOMs are used or daa cluserng and mul-laer percepron (MLP) neural neworks are used or deecon. Herarchcal neural neworks [28], evoluonar neural neworks [29], and MLP neural neworks [27], have been appled o dsngush beween aacks and normal nework behavors. Hu and Hewood [6] combne SVM wh SOM o deec nework nrusons. Khor e al. [55] propose a dchoomzaon algorhm n whch rare caegores are separaed rom he ranng daa and cascaded classers are raned o handle boh he rare caegores and oher caegores. Xan e al. [32] combne he uzz K-means algorhm wh he clonal selecon algorhm o deec nrusons. Jang e al. [33] use an ncremenal verson o he K-means algorhm o deec nrusons. Hoglund e al. [34] exrac eaures descrbng nework behavors rom aud daa, and use he SOM o deec nrusons. Sarasamma e al. [18] propose a herarchcal SOM whch selecs deren eaure subse combnaons ha are used n deren laers o he SOM o deec nrusons. Song e al. [35] propose a herarchcal random subse selecon-dnamc subse selecon (RSS-DSS) algorhm o deec nrusons. Lee e al. [8] propose an adapve nruson deecon algorhm whch combnes he adapve resonance heor wh he concep vecor and he Mecer-Kernel. Jrapummn e al. [21] emplo a hbrd neural nework model o deec TCP SYN loodng and por scan aacks. Eskn e al. [20] use a k-neares neghbor (k-nn)-based algorhm and a SVM-based algorhm o deec anomales. Alhough here s much work on nruson deecon, several ssues are sll open and requre urher research, or example: Nework envronmens and he nruson ranng daa change rapdl over me, as new pes o aack emerge. In addon, he sze o he ranng daa ncreases over me and can become ver large. Mos exsng algorhms or ranng nruson deecors are olne. The nruson deecor mus be reraned perodcall n bach mode n order o keep up wh he changes n he nework. Ths reranng s me consumng. Onlne ranng s more suable or dnamc nruson deecors. New daa are used o updae he deecor and are hen dscarded. The ke ssue n onlne ranng s o manan he accurac o he nruson deecor. There are varous pes o arbues or nework connecon daa, ncludng boh caegorcal and connuous ones, and he value ranges or deren arbues der greal rom {0, 1} o descrbe he normal or error saus o a connecon, o [0, 107] o spec he number o daa bes sen rom source o desnaon. The combnaon o daa wh deren arbues whou loss o normaon s crucal o manan he accurac o nruson deecors. In radonal cenralzed nruson deecon, n whch all he nework daa are sen o a cenral se or processng, he raw daa communcaons occup consderable nework bandwdh. There s a compuaonal burden n he cenral se and he prvac o he daa obaned rom he local nodes canno be proeced. Dsrbued deecon [36], whch shares local nruson deecon models learned n local nodes, can reduce daa communcaons, dsrbue he compuaonal burden, and proec prvac. Oe e al. [4] consruc a novel dsrbued algorhm or deecng oulers (ncludng nework nrusons). Is 3

4 lmaon s ha man raw nework daa sll need o be shared among dsrbued nodes. There s a requremen or a dsrbued nruson deecon algorhm o make onl a small number o communcaons beween local nodes. In hs paper, we address he above challenges, and propose a classcaon-based ramework or he dnamc dsrbued nework nruson deecon usng he onlne Adaboos algorhm [58]. The Adaboos algorhm s one o he mos popular machne learnng algorhms. Is heorecal bass s sound, and s mplemenaon s smple. Moreover, he AdaBoos algorhm correcs he msclasscaons made b weak classers and s less suscepble o over-ng han mos learnng algorhms. Recognon perormances o he Adaboos-based classers are generall encouragng. In our ramework, a hbrd o onlne weak classers and an onlne Adaboos process resuls n a parameerzed local model a each node or nruson deecon. The paramerc models or all he nodes are combned no a global nruson deecor n each node usng a small number o samples, and he combnaon s acheved usng an algorhm based on parcle swarm opmzaon (PSO) and SVMs. The global model n a node can handle he aack pes whch are ound n oher nodes, whou sharng he samples o hese aack pes. Our ramework s orgnal n he ollowng was: In he Adaboos classer, he weak classers are consruced or each ndvdual eaure componen, boh or connuous and caegorcal ones, n such a wa ha he relaons beween hese eaures can be naurall handled, whou an orced conversons beween connuous eaures and caegorcal eaures. New algorhms are desgned or local nruson deecon. The radonal onlne Adaboos process and a newl proposed onlne Adaboos process are appled, o consruc local nruson deecors. The weak classers used b he radonal Adaboos process are decson sumps. The new Adaboos process uses onlne Gaussan Mxure Models (GMM) as weak classers. In boh cases he local nruson deecors can be updaed onlne. The parameers n he weak classers and he srong classer consruc a paramerc local model. The local paramerc models or nruson deecon are shared beween he nodes o he nework. The volume o communcaons s ver small and s no necessar o share he prvae raw daa rom whch he local models are learn. We propose a PSO and SVM-based algorhm or combnng he local models no a global deecor n each node. The global deecor whch obans normaon rom oher nodes obans more accurae deecon resuls han he local deecor. The remander o hs paper s organzed as ollows. Secon 2 nroduces he dsrbued nruson deecon ramework. Secon 3 descrbes he onlne Adaboos-based local nruson deecon models. Secon 4 presens he mehod or consrucng he global deecon models. Secon 5 shows he expermenal resuls. Secon 6 summarzes he paper. 2. Overvew o Our Framework In he dsrbued nruson deecon ramework, each node ndependenl consrucs s own local nruson 4

5 deecon model accordng o s own daa. B combnng all he local models, a each node, a global model s raned usng a small number o he samples n he node, whou sharng an o he orgnal ranng daa beween nodes. The global model s used o deec nrusons a he node. Fg. 1 gves an overvew o our ramework whch consss o he modules o daa preprocessng, local models, and global models. Dsrbued node 1 Nework connecons Feaure exracon and daa labelng Onlne Adaboos learnng Dsrbued node N Nework connecons Feaure exracon and daa labelng Onlne Adaboos learnng Local model 1 Local model N PSO and SVM algorhm Global model 1 Classcaon resuls o new connecons Global model N Classcaon resuls o new connecons Fg. 1. Overvew o he nruson deecon ramework 1) Daa preprocessng: For each nework connecon, hree groups o eaures [9] whch are commonl used or nruson deecon are exraced: basc eaures o ndvdual TCP (ransmsson conrol proocol) connecons, conen eaures whn a connecon suggesed b doman knowledge, and rac eaures compued usng a wo-second me wndow [14]. The exraced eaure values rom a nework connecon orm a vecor x = (x1, x2,, x D ), where D s he number o eaure componens. There are connuous and caegorcal eaures, and he value ranges o he eaures ma der greal rom each oher. The ramework or consrucng hese eaures can be ound n [9]. A se o daa s labeled or ranng purposes. There are man pes o aacks on he Inerne. The aack samples are labeled as -1, -2,... dependng on he aack pe, and he normal samples are all labeled as +1. 2) Local models: The consrucon o a local deecon model a each node ncludes he desgn o weak classers and Adaboos-based ranng. Each ndvdual eaure componen corresponds o a weak classer. In hs wa, he mxed arbue daa or he nework connecons can be handled naurall, and ull use can be made o he normaon n each eaure. The Adaboos ranng s mplemened usng onl he local ranng samples a each node. Aer ranng, each node conans a paramerc model whch consss o he parameers o he weak classers and he ensemble weghs. 3) Global models: B sharng all he local paramerc models, a global model s consruced usng he PSO and SVM-based algorhm n each node. The global model n each node uses he normaon learned rom all he local nodes usng a small number o ranng samples n he node. Feaure vecors o new nework connecons o he node are npu o he global classer, and classed as eher normal or aacks. The resuls o he global model n he node are used o updae he local model n he node and he updaed model s hen shared 5

6 b oher nodes. 3. Local Deecon Models The classcal Adaboos algorhm [37] carres ou he ranng ask n bach mode. A number o weak classers are consruced usng a ranng se. Weghs, whch ndcae he mporance o he ranng samples, are derved rom he classcaon errors o he weak classers. The nal srong classer s an ensemble o weak classers. The classcaon error o he nal srong classer converges o 0. However, he Adaboos algorhm based on olne learnng s no suable or neworks. We appl onlne versons o Adaboos o consruc he local nruson deecon models. I s proved n [38] ha he srong classer obaned b he onlne Adaboos converges o he srong classer obaned b he olne Adaboos as he number o ranng samples ncreases. In he ollowng, we rs nroduce he weak classers or nruson deecon, and hen descrbe he onlne Adaboos-based nruson deecon algorhms Weak classers Weak classers whch can be updaed onlne mach he requremen o dnamc nruson deecon. We consder wo pes o weak classer. The rs pe consss o decson sumps or classng aacks and normal behavors. The second pe consss o onlne GMMs whch model a dsrbuon o values o each eaure componen or each aack pe Decson sumps A decson sump s a decson ree wh a roo node and wo lea nodes. A decson sump s consruced or each eaure componen o he nework connecon daa. For a caegorcal eaure, he se o arbue values C s dvded no wo subses C and C n wh no nersecon, and he decson sump akes he orm: 1 h ( x) 1 x x C n C (1) where x s he arbue value o x on he eaure. The subses C and C n are deermned usng he ranng samples: or an arbue value z on a eaure, all he ranng samples whose arbue values on are equal o z are ound; he number o aack samples n hese samples s more han he number o normal samples, hen z s assgned o C, oherwse, z s assgned o C n. In hs wa, he alse alarm rae or he ranng samples s mnmzed. For a connuous eaure, he range o arbue values s spl b a hreshold v, and he decson sump akes he orm: 1 h ( x) 1 x x v. (2) v The hreshold v s deermned b mnmzng he alse alarm rae or he ranng samples. 6

7 The above desgn o weak classers or nruson deecon has he ollowng advanages: The decson sumps operae on ndvdual eaure componens. Ths rs deals naurall wh combnaon o normaon rom caegorcal arbues and rom connuous arbues, and second deals wh he large varaons n value ranges or he deren eaure componens. There s onl one comparson operaon n a decson sump. The compuaon complex or consrucng he decson sumps s ver low, and onlne updang o decson sumps can be easl mplemened when new ranng samples are obaned. The lmaon o he above desgn o weak classers s ha he decson sumps do no ake no accoun he deren pes o aacks. Ths ma nluence he perormance o he Adaboos mehod Onlne GMM For he samples o each aack pe or he normal samples, we use a GMM o model he daa on each eaure componen. Le c 1, 1, 2,, M be a sample label, where +1 represens he normal samples and -1, -2, -M represens deren pes o aacks where M s he number o aack pes. The c GMM model on he h eaure componen or he samples wh label c s represened as: ( ), ( ), ( ) K 1 (3) c c c c where K s he number o GMM componens ndexed b, and, and represen he wegh, mean and sandard devaon or he correspondng componen. Then, he weak classer on he h eaure s consruced as: c 1 p( x ) p( x ) or c 1, 2,..., M h ( x) M 1 oherwse 1 1 where 1/M s used o wegh he probables or aack pes n order o balance he mporance o he aack samples and he normal samples. In hs wa, he sum o he weghs or all he pes o aack samples s equal o he wegh o normal samples, and hen he alse alarm rae s reduced or he nal ensemble classer. The radonal wa or esmang he parameer vecors (4) c, usng he K-means cluserng and he expecaon-maxmzaon (EM) algorhm [41], rom he ranng samples, s me-consumng and s no suable or onlne learnng. In hs paper, we use he onlne EM-based algorhm [44] o esmae hese parameers. Gven a new sample ( x, ) where x s he eaure vecor o he sample and {1, 1, 2 }, he parameer vecors are updaed usng he ollowng seps: Sep 1: Updae he number N () o he ranng samples belongng o he h componen o he GMM b: x () 1 T ( x) () (5) 0 else 7

8 N ( ) N ( ) ( x) (6) where he bnar uncon ( x) deermnes wheher (x, ) belongs o he h componen, and he hreshold T depends on condence lms requred. Updae () b: Sep 2: Calculae he ollowng equaon: K 1 () K k1 N (). (7) N ( k) ( ) p( x ( ), ( )) ( x). (8) where descrbes he relaon beween (x, ) and he GMM o. I 0, hen go o Sep 3; else ( 0 means ha here s no relaon beween (x, ) and he GMM o ), go o Sep 5 or seng a new Gaussan componen or (x, ). Sep 3: Updae he sum A () o he weghed probables or all he npu ranng samples belongng o he h componen b: ( ) ( ) p( x ( ), ( )) ( x) / (9) A ( ) A ( ) ( ) (10) where () s he probabl ha (x, ) belongs o he h componen o he GMM. Sep 4: Updae () and () b: Ex. A ( ) ( ) () ( ) ( ) x (11) A ( ) A ( ) A ( ) ( ) ( A ( ) ( )) ( ) Sep 5: Rese a Gaussan componen n ( ) ( ) ( x ( )). (12) A ( ) A ( ) b: arg mn( ( )) (13) N ( ) N ( ) 1 A ( ) A ( ) 1 (14) () x. (15) () where s used o nalze he sandard devaon o he componen. 8

9 The erms ( ) / A ( ) n (11) and ( A ( ) ( )) ( ) / A ( ) n (12) are he learnng raes o he algorhm. The are changed dnamcall: geng smaller wh he growh o he ranng samples. Ths behavor o he learnng raes ensures ha he onlne EM algorhm acheves he balance beween he sabl needed o keep he characerscs o he model learned rom he prevous samples and necess o adapng he model as new samples are receved. The compuaonal complex o he onlne GMM or one sample s OK ( ). Whle he onlne GMM, whch can be used o model dsrbuons o each pe o aack, has a hgher complex han decson sumps, he complex or he onlne GMM s sll ver low as K s a small neger. So, he onlne GMM s suable n onlne envronmens Onlne ensemble learnng To adap o onlne ranng n whch each ranng sample s used onl once or learnng he srong classer, onlne Adboos makes he ollowng changes o he olne Adaboos: All he weak classers are nalzed. Once a sample s npu, a number o weak classers are updaed usng hs sample, and he wegh o hs sample s changed accordng o ceran crera such as he weghed alse classcaon rae o he weak classers. In he ollowng, we rs appl he radonal onlne Adaboos algorhm o nruson deecon, and hen we propose a new and more eecve onlne Adaboos algorhm Tradonal onlne Adaboos Oza [38] proposes an onlne verson o Adaboos, and gves a convergence proo. Ths onlne Adaboos algorhm has been appled o he compuer vson eld [42]. When a new sample s npu o he onlne Adaboos algorhm, all he weak classers (ensemble members) are updaed n he same order [10]. Le { h } 1,..., D be he weak classers. Ths onlne Adaboos algorhm s oulned as ollows: Sep 1: Inalze he weghed couners sc and sw o hsorcal correc and wrong classcaon resuls or he -h weak classer sc sw h as: 0, 0 (=1, 2,, D). Sep 2: For each new sample (x, ) Inalze he wegh o he curren sample as: 1. For = 1, 2,, D (a) Randoml sample an neger k rom he Posson dsrbuon: Posson( ). (b) Updae he weak classer h usng he sample (x, ) k mes (c) I he weak classer h correcl classes he sample (x, ),.e. h ( x) sgn( ), hen he weghed correc classcaon couner sc s updaed b ; sc sc 9

10 he approxmae weghed alse classcaon rae s updaed b: sw sc sw (16) he wegh o he sample (x, ) s updaed b: else 1 2(1 ) (17) sw s updaed b ; sw s updaed usng (16); sw he wegh o he sample (x, ) s updaed b: (d) The wegh The wegh or he weak classer Sep 3: The nal srong classer s dened b: 1. (18) 2 h s: 1 log. (19) relecs he mporance o eaure or deecng nrusons. D D 1 H( x) sgn h ( x) sgn log h ( x) 1 1. (20) The derences beween he olne Adaboos algorhm and he onlne Adaboos algorhm are as ollows: In he olne Adaboos algorhm, he weak classers are consruced n one sep usng all he ranng samples. In he onlne Adaboos algorhm, he weak classers are updaed graduall as he ranng samples are npu one b one. In he olne Adaboos algorhm, all he sample weghs are updaed smulaneousl accordng o he classcaon resuls o he opmal weak classer or he samples. In he above onlne Adaboos algorhm, he wegh o he curren sample changes as he weak classers are updaed one b one usng he curren sample. In he olne Adaboos algorhm, he weghed classcaon error or each weak classer s calculaed accordng o s classcaon resuls or he whole se o he ranng samples. In onlne Adaboos, he weghed classcaon error or each weak classer s esmaed usng he weak classer s classcaon resuls onl or he samples whch have alread been npu. The number o weak classers s no xed n he olne Adaboos algorhm. In he onlne Adaboos, he number o weak classers s xed, and equal o he dmenson o he eaure vecors. When onl a ew ranng samples have been npu, he onlne ensemble classer s less accurae han 10

11 he olne ensemble classer. As he number o ranng samples ncreases, he accurac o he onlne ensemble classer graduall ncreases unl approxmaes o he accurac o he olne ensemble classer [43]. The lmaon o he radonal onlne Adaboos algorhm s ha or each new npu sample, all he weak classers are updaed n a predened order. The accurac o he ensemble classer depends on hs order. There s a endenc o over- o he weak classers whch are updaed rs A new onlne Adaboos algorhm To deal wh he lmaon o he radonal onlne Adaboos, our new onlne Adaboos algorhm selecs a number o weak classers accordng o a ceran rule and updaes hem smulaneousl or each npu ranng sample. Le S and S be, respecvel, he numbers o he normal and aack samples whch have alread been npu. Le be he number o he samples, each o whch s correcl classed b he prevous srong classer whch has been raned beore he sample s npu. Le sc be he sum o he weghs o he npu samples ha are correcl classed b he weak classer h. Le sw be he sum o he weghs o he npu samples ha are msakenl classed b h. Le C be he number o he npu samples whch are correcl classed b h. All he parameers S, S,, sc, sw, and C are nalzed o 0. For a new sample ( x, ), {1, 1, 2, }, he srong classer s updaed onlne b he ollowng seps: Sep 1: Updae he parameer S or S b: Inalze he wegh o he new sample b: S S 1 1. (21) S S 1 else ( S S ) / S ( S S ) / S 1 else (22) where ollows he change o S and S, n avor o balancng he proporon o he normal samples and he aack samples o ensure ha he sum o he weghs o he aack samples equals o he sum o he weghs o he normal samples. Sep 2: Calculae he combned classcaon rae or each weak classer h b: sw (1 ) sgn( ) h ( x) (1 ) sgn( ) h ( x) sc sw (23) where s a wegh rangng n (0,0.5]. The rae classcaon rae o combnes he hsorcal alse h or he samples npu prevousl and he resul o curren sample (x, ). The rae s more eecve han, as gves h whose h or he s hgh more chance o be updaed and hen ncreases he deecon rae o h. Dene mn b: 11

12 mn mn (24) {1,2,... D} The weak classers h 1,..., are ranked n he ascendng order o D and hen he weak classers are represened b { h, h,, h }, r {1,2,, D}. r1 r2 r D Sep 3: The weak classes whose combned classcaon raes r are no larger han 0.5 are seleced rom { h, h,, h }. Each o hese seleced weak classers h r s updaed usng (x, ) n he r1 r2 r D ollowng wa: Compue he number P o eraons or h r b: P Ineger ( Pexp( ( mn ))) (25) where s an aenuaon coecen and P s a predened maxmum number o eraons. Repea he ollowng seps (a) and (b) P mes n order o updae h r : (a) Onlne updae h r usng (x, ). (b) Updae he sample wegh, I sgn( ) h ( x), hen r sc, and sw : C C 1 sc sc r r. (26) 1 2 2(1 r ) ( s decreased ) I sgn( ) hr, hen sw sw r r 1 2. (27) 2 r ( s ncreased ) Sep 4: Updae C, sc and sw or each h o he weak classers whose combned classcaon raes are larger han 0.5 ( 0.5 ): sgn( ) h ( x), hen C C 1 sc sc (28) else. sw sw 12

13 Sep 5: Updae he parameer : I he curren sample s correcl classed b he prevous srong classer whch has been raned beore he curren sample (x, ) s npu, hen 1. Sep 6: Consruc he srong ensemble classer: Calculae he ensemble wegh The * s normalzed o * o h b * 1 C log (1 )log. (29) b: The srong classer H(x) s dened b: * D. (30) 1 * D H( x) sgn( h ( x)). (31) 1 We explan wo pons or he above onlne Adaboos algorhm: The aenuaon coecen conrols he number o weak classers whch are chosen or urher updang. When s ver small, P s large and all he chosen weak classers are updaed usng he new sample. When s ver large, P equals 0 s large. As a resul, or ver large, onl he weak classers wh he small are updaed. The ensemble wegh, whch s calculaed rom and log( C / ), s deren rom he one n he radonal Adaboos algorhm. The erm log( C / ) whch s called a conrbuor acor represens he conrbuon rae o h o he srong classer. I can be used o une he ensemble weghs o aan beer deecon perormance Adapable nal sample weghs We use he deecon rae and he alse alarm rae o evaluae he perormance o he algorhm or deecng nework nrusons. I s necessar o pa more aenon o he alse alarm rae, because n real applcaons mos nework behavors are normal. A hgh alse alarm rae wases resources, as each alarm has o be checked. For Adaboos-based learnng algorhms, he deecon rae and he alse alarm rae depend on he nal weghs o he ranng samples. So we propose o adus he nal sample weghs n order o balance he deecon rae and he alse alarm rae. We nroduce a parameer r (0,1) or seng he nal wegh o each ranng sample: Nnormal Nnruson r Nnormal Nnormal Nnruson (1 r) Nnruson or normal connecons. (32) or nework nrusons 13

14 where N normal and N nruson are approxmaed usng he numbers o normal samples and aack samples whch have been npu onlne o ran he classer. The sums o he weghs or he normal samples and he aack samples are ( N N ) r and ( N N ) (1 r) respecvel. Through adusng normal nruson normal nruson he value o he parameer r, we change he mporance o normal samples or aack samples n he ranng process, and hen make a radeo beween he deecon rae and he alse alarm rae o he nal deecor. The selecon o r depends on he proporon o he normal samples n he ranng daa, and he requremens or he deecon rae and he alse alarm rae n specc applcaons Local parameerzed models Subsequen o he consrucon o he weak classers and he onlne Adaboos learnng, a local parameerzed deecon model s ormed n each node. The local model consss o he parameers w o he weak classers and he parameers or consrucng he Adaboos srong classer: {, }. The d w d parameers or each decson sump-based weak classer nclude he subses C and C n or each caegorcal eaure and he hresholds v or each connuous eaure. The parameers or each GMM-based weak c classer nclude a se o GMM parameers w { 1, 2,..., D; c 1, 1, 2,...}. The parameers o he srong classer or he onlne Adaboos algorhm nclude a se o ensemble weghs d { 1,2,..., D} or he weak classers. The parameers n he decson sump-based weak classers depend on he derences beween normal behavors and aacks n each componen o he eaure vecors. The parameers n he GMM-based weak classers depend on he dsrbuons o he deren pes o aacks and normal behavors n each componen o he eaure vecors. The parameers n he srong classer depend on he sgncances o ndvdual eaure componens or nruson deecon. The local deecon models capure he dsrbuon characerscs o observed mxed-arbue daa n each node. The can be exchanged beween he deren nodes. Compared wh he non-paramerc dsrbued ouler deecon algorhm proposed b Oe e al. [4], where a large amoun o sascs abou requen emses need o be shared among he nodes, he paramerc models are no onl concse o be suable or normaon sharng, bu also ver useul o generae global nruson deecon models. 4. Global Deecon Models The local paramerc deecon models are shared among all he nodes, and combned n each node o produce a global nruson deecor usng a small number o samples le n he node (mos samples n he node are removed, n order o adap o changng nework envronmens). Ths global nruson deecor s more accurae han he local deecors whch ma be onl adequae or specc aack pes, due o he lmed ranng daa avalable a each node. Kler e al. [11] develop a common heorecal ramework or combnng classers usng he produc rule, he sum rule, he max rule, he mn rule, he medan rule, and he maor voe rule. I s shown ha he sum rule 14

15 has a beer perormance han ohers. Some researchers use mul-classers b combne he oupu resuls o all he classers no a vecor, and hen usng a classer, such as SVM or ANN, o class he vecors. The combnaon o he local nruson deecon models has wo problems. Frs, here ma be large perormance gaps beween he local deecon models or deren pes o aacks, especall or new aack pes whch have no appeared prevousl. So, he sum rule ma no be he bes choce or combnng he local deecors. Second, some o he local models ma be smlar or a es sample. I he resuls o he local models or he es sample are combned no a vecor, he dmenson o he vecor has o be reduced o choose he bes combnaon o he resuls rom local models. To solve he above wo problems, we combne he parcle swarm opmzaon (PSO) and SVM algorhms, n each node, o consruc he global deecon model. The PSO [12, 13] s a populaon search algorhm whch smulaes he socal behavor o brds lockng. The SVM s a learnng algorhm based on he srucural rsk mnmzaon prncple rom sascal learnng heor. I has a good perormance even he se o he ranng samples s small. We use he PSO o search or he opmal local models and he SVM s raned usng he samples le n a node. Then, he raned SVM s used as he global model n he node. B combnng he searchng abl o he PSO and he learnng abl o he SVM or small sample ses, a global deecon model can be consruced eecvel n each node. The sae o a parcle used n he PSO or one o he A nodes s dened as X ( x1, x2,, xa), x {0,1}, where x 1 means ha he -h local model s chosen, and x 0 means he -h local model s no chosen. For each parcle, a SVM classer s consruced. Le L be he number o local deecon nodes chosen b he parcle sae X. For each nework connecon sample le n he node, a vecor 1 2 ( r, r,, r ) L s consruced, where r s he resul o he -h chosen local deecon model or he sample (correspondng o (20) and (31)): D r h ( x) (33) 1 where D s he dmenson o nework connecon eaure vecors. These resuls are n he range [-1, 1]. All he vecors correspondng o he small number o samples le n he node, ogeher wh he arbuons (normal connecons or aacks) o he samples, are used o ran he SVM classer. Each parcle has a correspondng ness value whch s evaluaed b a ness model ( X ). Le ( X ) be he deecon rae o he raned SVM classer or parcle, where he deecon rae s esmaed usng anoher se o samples n he node. The ness value s evaluaed as: where s a wegh rangng beween 0 and 1. A L ( X) * ( X ) (1 )log (34) A For each parcle, s ndvdual bes sae l S whch has he maxmum ness value s updaed n he n-h PSO eraon usng he ollowng equaon: 15

16 S X ( X ) (S ) (35) l l, n, n l S else The global bes sae g S n all he parcles s updaed b: S g l arg max ( S ) (36) l S Each parcle n he PSO s assocaed wh a veloc whch s moded accordng o s curren bes sae and he curren bes sae among all he parcles: l g V, n 1 F wv, n c11 ( S X, n) c22 ( S X, n) (37) where w s he nera wegh whch s negavel correlaed wh he number o eraons; c 1 and c 2 are acceleraon consans called he learnng raes; 1 and 2 are relavel ndependen random values n he range [0,1]; and F () s a uncon whch connes he veloc whn a reasonable range: FV ( ) V V V max. (38) V max else The saes o he parcles are evolved usng he updaed veloc : X X V. (39), n1, n, n1 In heor, PSO can nd he global opmum aer sucenl man eraons. In pracce, a local opmum ma be obaned, due o an nsucen number o eraons. PSO has a hgher probabl o reachng he global opmum han graden descen approaches or parcle lerng approaches. The SVM and PSO-based uson algorhm s oulned as ollows: Sep 1: Inalzaon: The parcles X,01 are randoml chosen n he parcle space, where s he l number o parcles. S X,0. A SVM classer s consruced or each parcle, and he deecon rae ( X,0) o he SVM classer s calculaed. The ness value ( X,0) s calculaed usng (34). The global bes sae Sep 2: The veloces Vn, 1 1 Sep 3: The parcles saes are updaed usng (37). X n, 1 1 S g s esmaed usng (36). n 0. are evolved usng (39). Sep 4: The SVM classer s consruced or each parcle, and he deecon rae ( X n, 1) s calculaed. The ness values ( n, 1) S are l X are calculaed usng (34). The ndvdual bes saes 1 updae usng (35). The global bes sae g S s updaed usng (36). g Sep 5: nn 1. I ( S ) max_nness or he predened number o eraons s acheved, hen he parcle evoluon process ermnaes and he SVM classer correspondng o g S s chosen 16

17 as he nal classer he global model n he node; oherwse go o Sep 2 or anoher loop o evoluon. When ceran condons are me, nodes ma ransm her local models o each oher. Then, each node can consruc a cusomzed global model usng a small se o ranng samples randoml seleced rom he hsorcal ranng samples n he node accordng o he proporon o varous knds o he nework behavors. Once local nodes gan her own global models, he global models are used o deec nrusons: or a new nework connecon, he vecor o he resuls rom he local models chosen b he global bes parcle s used as he npu o he global model whose resul deermnes wheher he curren nework connecon s an aack. We explan wo pons: A unorm global model can be consruced or all he local nodes. Bu, besdes he parameers o local models, small ses o samples should be shared hrough communcaons beween he nodes. The compuaonal complex o he PSO or searchng or he opmal local models deermnes he scale o nework or our global model. The compuaonal complex o he PSO s 2 2 O( IA ) where I s he number o eraons, and s he number o he ranng samples. 5. Expermens We use he KDD (Knowledge Dscover and Daa Mnng) CUP 1999 daase [14, 15, 16, 45] o es our algorhms. Ths daase s sll he mos rusul and credble publc benchmark daase [53, 54, 55, 56, 57, 58, 59] or evaluang nework nruson deecon algorhms. In he daase, 41 eaures ncludng 9 caegorcal eaures and 32 connuous eaures are exraced or each nework connecon. Aacks n he daase all no our man caegores: DOS: denal-o-servce; R2L: unauhorzed access rom a remoe machne, e.g. guessng password; U2R: unauhorzed access o local super-user (roo) prvleges; Probe: survellance and oher probng, e.g. por scannng. Each o he our caegores conans some low-grade aack pes. The es daase ncludes some aack pes ha do no exs n he ranng daase. The numbers o normal connecons and each caegor o aacks n he ranng and es daases are lsed n Table 1. Table 1. The KDD CUP 1999 daa se Caegores Tranng daa Tes daa Normal DOS R2L U2R Probng Ohers Toal In he ollowng, we rs nroduce he perormances o our onlne learnng-based nruson deecon 17

18 algorhms: one wh decson sumps and he radonal onlne Adaboos process, and he oher wh onlne GMMs and our proposed onlne Adaboos process. Then, he perormance o our PSO and SVM-based dsrbued nruson deecon algorhm s evaluaed Local models For he onlne local models, he abl o handle mxed eaures, he eecs o parameers on deecon perormance, and he abl o learn new pes o aacks are esmaed n successon. Fnall, he perormances o our algorhms are compared wh he publshed perormances o he exsng algorhms. 1) Handlng mxed eaures Table 2 shows, respecvel, he resuls o he classer wh decson sumps and he radonal onlne Adaboos and he classer wh onlne GMMs and our onlne Adaboos, when onl connuous eaures or boh connuous eaures and caegorcal eaures are used respecvel, esed on he KDD ranng and es daa ses respecvel. I s seen ha he resuls obaned b usng boh connuous and caegorcal eaures are much more accurae han he resuls obaned b onl usng connuous eaures. Ths shows he abl o our algorhms o handle mxed eaures n nework connecon daa. Table 2. The resuls obaned b usng onl connuous eaures or boh connuous and caegorcal eaures Algorhms Decson sump + Tradonal onlne Adaboos Onlne GMM + Our onlne Adaboos Feaures Tranng daa Tes daa Deecon False alarm Deecon False alarm rae rae rae rae Onl Connuous 98.68% 8.35% 90.05% 13.76% Connuous + Caegorcal 98.93% 2.37% 91.27% 8.38% Onl Connuous 98.79% 7.83% 91.33% 11.34% Connuous + Caegorcal 99.02% 2.22% 92.66% 2.67% 2) Eecs o parameers on deecon perormance In he ollowng, we llusrae he eecs o mporan parameers,.e. he adapable nal wegh parameer r n (32), he daa dsrbuon wegh coecen 1/M n (4), he parameer n (23), he parameer n (29), and he aenuaon coecen parameer n (25). Table 3. The resuls o he algorhm wh decson sumps and he radonal onlne Adaboos when adapable nal weghs are used r Tranng daa Tes daa Deecon rae False alarm rae Deecon rae False alarm rae % 2.38% 90.83% 8.42% % 0.78% 90.69% 2.29% % 0.77% 89.80% 1.92% % 0.73% 89.66% 1.88% % 0.16% 88.97% 0.37% Table 3 shows he resuls o he algorhm wh decson sumps and he radonal onlne Adaboos when he adapable nal weghs ormulaed b (32) are used. When r s ncreased he deecon rae and he alse alarm rae are decreased. An approprae seng o r s 0.3, as wh hs value o r, he alse alarm rae o 2.28% s much smaller han he alse alarm rae o 8.42% obaned b choosng r as 0.25, whle an accepable deecon rae o 90.69% s sll obaned. As shown n Table 2, when an dencal nal wegh s used or each ranng sample n he radonal onlne Adaboos algorhm usng decson sumps as weak classers, a hgh deecon 18

19 rae o 91.27% s obaned on he es daa, bu he alse alarm rae s 8.38%, whch s oo hgh or nruson deecon. Aer adapable nal weghs are nroduced, alhough he deecon rae s slghl decreased, he alse alarm rae s much decreased, producng more suable resuls or nruson deecon. Ths llusraes ha adusable nal weghs can be used eecvel o balance he deecon rae and he alse alarm rae. Table 4 shows he resuls o he algorhm wh onlne GMMs and our onlne Adaboos when he daa dsrbuon wegh 1/M n (4) and adapable nal weghs are used or no used. I s seen ha when he parameer M s no used, he alse alarm rae s 54.15%, whch s oo hgh or nruson deecon. When he parameer M s used, he alse alarm rae alls o 4.82%. When adapable nal weghs are used (r s se o 0.35) he alse alarm rae urher alls o 1.02%, whle an accepable deecon rae s mananed. Table 4. The resuls o our onlne Adaboos algorhm based on onlne GMMs wh or whou adapable nal weghs and he daa dsrbuon wegh 1/M Parameers Whou he daa dsrbuon wegh 1/M Wh 1/M and whou adapable nal weghs Wh 1/M and adapable nal weghs Tranng daa Tes daa Deecon rae False alarm rae Deecon rae False alarm rae 99.86% 48.85% 97.49% 54.15% 99.22% 8.10% 91.74% 4.82% 98.91% 2.65% 90.65% 1.02% Parameers n (23) and n (29) nluence he deecon resuls. Table 5 shows he deecon raes and alse-alarm raes o he algorhm wh onlne GMMs and our onlne Adaboos when 0 or 0.1 and 1 or 0.8. From hs able, he ollowng wo pons are deduced: When s se o 0, he alse alarm rae s ver hgh. Ths s because he weak classers are chosen onl accordng o he alse classcaon rae Adopng he measure, whou akng no accoun he curren sample. o he combned classcaon rae balances he eecs o hsorcal samples and he curren sample, and hen resuls n more accurae resuls. When s se o 1, he deecon rae s oo low. Ths s because under hs case onl s used o wegh he weak classers,.e. ever weak classer s weghed n he same wa as n he olne Adaboos and he radonal onlne Adaboos. In he onlne ranng process, weghng ever weak classer onl usng weak classers based onl on causes over-ng o some weak classers. Namel, he ensemble weghs o conrbuor acors elds he beer perormance. are no accurae. Weghng weak classers b combnng Table 5. The resuls o he algorhm wh onlne GMMs and our onlne Adaboos when 0 or 0.1 and 0.8 or 1 Deecon rae False alarm rae % 77.66% 0.1, % 0.60% 0.1, % 1.69% and 19

20 The aenuaon coecen n (25) s mporan or he perormance o our onlne Adaboos algorhm. Table 6 shows he deecon raes and he alse alarm raes o our onlne Adaboos algorhm based on GMMs, when ranges rom 10 o 50. When {20,25,30}, he updang mes P s beer chosen or he weak classers and hus good resuls are obaned. In ac, when s se o 25, he deecon rae reaches 91.15% wh a alse-alarm rae o 1.69%. Ths s a preerable resul compared wh hose obaned when s se o 10 or 50. Table 6. The deecon resuls o our onlne Adaboos algorhm based on onlne GMMs wh varng Deecon rae False alarm rae % 12.87% % 1.17% % 1.69% % 1.26% % 0.37% % 0.34% 3) Learnng new aack pes The abl o learn eecvel rom new pes o aacks and hen correcl deec hese pes o aacks s an aracve and mporan characersc o our onlne Adaboos-based nruson deecon algorhms. Fg. 2. Onlne learnng wh decson sumps and he radonal onlne Adaboos or guess-passwd aacks Fgs. 2 and 3 show he onlne learnng process o he ensemble classer wh decson sumps and he radonal onlne Adaboos wh respec o wo aack pes, guess-passwd and hpunnel, whch have no appeared beore n he ranng daa. In he gures, he horzonal coordnae ndcaes he number o samples o guess-passwd or hpunnel, whch have appeared n he learnng process; and he vercal coordnae ndcaes he class label predced b he deecor or a sample beore akes par n he learnng process, where 1 means ha he sample s msakenl classed as a normal connecon, and 1 means ha s correcl classed as a nework aack. I s seen ha a he begnnng o learnng or new pes o aacks, he classer requenl classes msakenl samples o hese new aack pes as normal nework connecons. Aer some samples o hese pes o aacks ake par n he onlne learnng o he classer, hese new aack 20

21 pes are much more correcl deeced. These wo examples show he eecveness o onlne learnng o he algorhm wh decson sumps and he radonal onlne Adaboos or new aack pes. Fg. 3. Onlne learnng wh decson sumps and he radonal onlne Adaboos or hpunnel aacks Fgs. 4 and 5 show he onlne learnng process o he ensemble classer wh onlne GMMs and our onlne Adaboos or he wo aack pes, guess-passwd and hpunnel, whch have no appeared beore n he ranng daa. I s seen ha aer some samples o guess-passwd and hpunnel occur n he onlne learnng, hese wo pes o samples are hen deeced much more correcl b he ensemble classer. Compared wh he classer wh decson sumps and he radonal onlne Adaboos, he classer wh onlne GMMs and our onlne Adaboos learns he new pes o aacks more quckl and accurael. Ths s because he classer wh onlne GMMs and our onlne Adaboos has more powerul learnng abl based on modelng he dsrbuons o each pe o aacks. Fg. 4. Onlne learnng wh onlne GMMs and our onlne Adaboos or guess-passwd aacks Fg. 5. Onlne learnng wh onlne GMMs and our onlne Adaboos or hpunnel aacks 4) Comparsons 21

22 We rs compare he deecon resuls obaned rom par o he labeled KDD CUP 1999 daase, hen he resuls obaned rom he whole labeled KDD CUP 1999 daase. Some nruson deecon algorhms are esed on a smaller daase ha conans onl hree common aack pes, namel Nepune, Saan, and Porsweep, n order o examne he algorhm s abl o deec specc pes o aack. The samples n he daase are randoml seleced rom he KDD CUP 1999 daa se. Table 7 lss he numbers o he ranng and es samples o all he pes n hs smaller daase. Table 8 shows he deecon resuls o he algorhm based on hbrd neural neworks [21], he algorhm based on he herarchcal SOM [18], he algorhm wh decson sumps and he radonal onlne Adaboos, and he algorhm wh onlne GMMs and our onlne Adaboos, all esed on he smaller daa se. I s seen ha he deecon rae o our algorhm wh decson sumps and he radonal onlne Adaboos or he specc pes o aacks s 99.55%, whch s close o he hghes deecon rae or he wo algorhms n [18] and [21]. The alse alarm rae o hs algorhm s onl 0.17%, whch s much less han he means n he alse alarm rae ranges or he algorhms n [18] and [21]. Our algorhm wh onlne GMMs and our onlne Adaboos even obans more accurae resuls. I s shown ha our onlne Adaboos-based algorhms are eecve or hese hree pes o aacks. Table 7. A smaller daa se Caegores Tranng daa Tes daa Normal Nepune Saan Porsweep Toal Table 8. The deecon resuls or some specc aacks Algorhms Deecon rae False alarm rae Hbrd neural neworks [21] 90%-99.72% 0.06%-4.5% Herarchcal SOM [18] 99.17%-99.63% 0.34%-1.41% Our algorhm (decson sumps + radonal onlne Adaboos) 99.55% 0.17% Our algorhm (onlne GMM + our onlne Adaboos) 99.99% 0.31% We also compare our onlne Adaboos-based algorhms wh oher recenl publshed algorhms or nruson deecon. The algorhms are esed on he whole labeled KDD CUP 1999 daase. Table 9 lss he deecon resuls o he ollowng algorhms: he cluserng-based algorhm n [3], he k-nn-based algorhm n [20], he SVM-based approach n [20], he algorhm based on SOMs n [5], he genec cluserng-based algorhm n [17], he herarchcal SOM-based algorhm n [18], he bagged C5 algorhm n [19], he olne Adaboos-based algorhm n [39], he Mercer kernel ART algorhm n [8], our algorhm usng decson sumps and he radonal onlne Adaboos, and our algorhm usng onlne GMMs and our onlne Adaboos. The Mercer kernel ART algorhm n [8] and our algorhms are onlne learnng algorhms and he ohers are olne learnng algorhms. From he able, he ollowng useul pons are deduced: Overall, he deecon accuraces o he onlne Adaboos-based algorhms are comparable wh hose o 22

23 he oher algorhms. Compared wh he olne algorhms, our algorhms no onl gan sasacor deecon raes whle keepng low alse alarm raes, bu also adapvel mod he local models n onlne mode. Ths adapabl s ver mporan or nruson deecon applcaons. The deecon accuraces o he onlne Adaboos-based algorhms are comparable wh he deecon accuraces o he Adaboos-based algorhm n bach mode. Our onlne Adaboos-based algorhms ouperorm he onlne algorhms n [8]. In parcular, lower alse alarm raes are obaned. The deecon accurac o he algorhm usng decson sumps and he radonal onlne Adaboos s lower han he algorhm usng onlne GMMs and our proposed onlne Adaboos. The reason or hs s ha he algorhm based on GMMs models onlne he dsrbuon o each pe o aacks and also updaes he weak classers n a more eecve wa, whch obans more accurae weghs or he weak classers. Table 9. Comparson beween algorhms esed on he KDD CUP 1999 daa se Olne Onlne Algorhms Deecon rae False alarm rae Cluserng [3] 93% 10% K-NN [20] 91% 8% SVM [20] 91%-98% 6%-10% SOM [5] 89%-90.6% 4.6%-7.6% Genec cluserng[17] 79.00% 0.30% Herarchcal SOM [18] 90.94%-93.46% 2.19%-3.99% Bagged C5 [19] 91.81% 0.55% Olne Adaboos [39] % % Mercer kernel ART [8] % % Our algorhm(decson sumps+radonal Adaboos) 90.13% 2.23% Our algorhm(onlne GMMs+our Adaboos) % % Moreover, Brugger and Chow [50] have assessed he perormance o he Snor algorhm, a pcal paern machng-nids, on he DARPA 1998 daase whch s he daase o he raw daa o he KDD CUP 1999 daase. I s shown ha Snor has a ver hgh alarm rae. The resuls o he Snor algorhm are less accurae han hose o he machne learnng algorhms lsed n Table 9. Our onlne Adaboos-based algorhms are mplemened on a Penum IV compuer wh 2.6GHZ CPU and 256M RAM, usng MATLAB 7. The mean ranng me or our algorhm wh decson sumps and he radonal onlne Adaboos s onl 46s and ha or our algorhm wh onlne GMMs and our onlne Adaboos s 65 mnues, usng all 494,021 ranng samples. Ths s an emprcal demonsraon ha he compuaonal complex o our algorhm wh decson sumps and he radonal onlne Adaboos s relavel low, due o ver smple operaons or he decson sumps. The compuaonal complex o our algorhm wh onlne GMMs and our onlne Adaboos s moderae. In [46], he leas ranng mes requred or he SOM and mproved compeve learnng neural nework are, respecvel, 1057s and 454s, onl usng 101,000 samples or ranng. 23

24 The bagged C5 algorhm [19, 47] ook a b more han a da on a machne wh a wo-processor ulra-sparc2 (2~300Mhz) and 512M man memores. The RSS-DSS algorhm [35] requres 15 mnues o nsh he learnng process on a 1 GHz Penum III lapop wh 256M RAM. In [4], 8 mnues are aken or processng he ranng daa n an egh-node cluser, where each node has dual 1GHz Penum III processors and 1GB memor, runnng Red Ha Lnux7.2, whle 212 mnues are aken or he algorhm n [48]. The mean ranng me or he olne Adaboos s 73s [39] PSO and SVM-based global models The proposed dsrbued nruson deecon algorhm s esed wh 6 nodes. The KDD CUP 1999 ranng daase s spl no sx pars and each par o daa s used o consruc a local deecon model n a deecon node. In hs wa, he sze o ranng daa n each node s small, wh he resul ha accurae local nruson deecors canno be ound n some nodes. In each node, a global model s consruced usng he sx local models. To smulae a dsrbued nruson deecon envronmen, he our knds o aacks: nepune, smur, porsweep and saan n he KDD CUP 1999 ranng daase are used or consrucng local deecon models, as samples o hese our knds ake up 98.46% o all he samples n he KDD ranng daase. Table 10 shows he ranng ses used or consrucng he global models n he 6 nodes. I s seen ha he szes o he ranng ses are comparavel small. Node 1 has no he porsweep and saan aack samples; Node 2 has no he saan samples; and Node 3 has no he porsweep aack samples. Table 11 shows he es se used n all he nodes. Table 10. The ranng ses n he 6 nodes Nodes Aack pes Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Nepune smur porsweep saan Nomal Table 11. The es se n each node nepune smur porsweep saan Normal In he expermens, he nera wegh w n (37) vares rom 1.4 o 0.4, accordng o he ollowng equaon: Ter Ier w ( w 0.4) 0.4 (40) Ter where Ter s he predened maxmum number o eraons and Ier s he curren eraon number. Tables 12 and 13 show, usng our algorhm wh decson sumps and he radonal onlne Adaboos and our algorhm wh onlne GMMs and our onlne Adaboos respecvel, he deecon raes and alse alarm raes o he local models and hose o he PSO and SVM-based global models, as well as hose obaned b combnng he local models usng he SVM algorhm, he sum rule, and he maor voe rule. From he ables, he ollowng hree pons are deduced: I s seen ha, overall, our PSO and SVM-based combnaon algorhm greal ncreases he deecon 24

25 rae and decreases he alse alarm rae or each node aer he local models wh onl a small number o parameers are shared. In node 2, he deecon rae o he PSO-SVM mehod s slghl less han he deecon rae o he local model, bu he alse alarm rae o he PSO-SVM mehod s much less han he alse alarm rae o he local model. In node 4, he alarm rae o he PSO-SVM mehod s slghl more han he alarm rae o he local model, bu he deecon rae o he PSO-SVM mehod s hgher han he deecon rae o he local model. Thus, he PSO-SVM mehod s more accurae han he local models n nodes 2 and 4. Our PSO and SVM-based algorhm s superor o he SVM algorhm, he sum rule and he maor voe rule or combnng local deecon models. The perormance dspares beween deren local models ensure ha he sum rule and he maor voe rule are no suable or dsrbued nruson deecon. The SVM ma no nd he opmal local model combnaon o mprove he perormance. Through dnamcall combnng a small poron o he local models o oban he global model, our PSO and SVM-based algorhm eecvel chooses he opmal local model combnaon, reduces he me consumpon or deecng nrusons, and hen acheves a beer perormance. Table 12. The resuls or he 6 local deecon nodes usng our algorhm wh decson sumps and he radonal onlne Adaboos Mehods Perormance Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Average Local model Deecon rae 99.48% 99.96% 99.98% 97.01% 97.13% 99.96% 98.92% False alarm rae 24.07% 9.64% 18.45% 0.42% 0.49% 8.28% 10.23% Deecon rae 99.59% 99.70% 99.67% 99.78% 99.65% 99.74% 99.69% PSO-SVM False alarm rae 0.38% 0.44% 0.43% 0.44% 0.41% 0.48% 0.43% SVM Deecon rae 99.76% 99.78% 97.13% 99.78% 99.78% 97.13% 98.89% False alarm rae 0.44% 0.44% 0.42% 0.44% 0.45% 0.42% 0.44% Deecon rae 99.96% 99.96% 99.96% 99.96% 99.96% 99.96% 99.96% Sum rule False alarm rae 1.98% 1.98% 1.98% 1.98% 1.98% 1.98% 1.98% Maor voe Deecon rae 99.96% 99.96% 99.96% 99.96% 99.96% 99.96% 99.96% False alarm rae 7.95% 7.95% 7.95% 7.95% 7.95% 7.95% 7.95% Table 13. The resuls or 6 local deecon nodes usng our algorhm wh onlne GMMs and our onlne Adaboos Mehods Perormance Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Average Local model Deecon rae 96.19% 33.66% 33.9% 99.72% 99.82% 33.77% 66.18% False alarm rae 0.35% 0.39% 0.56% 0.34% 0.37% 0.40% 0.40% Deecon rae 99.61% 99.64% 99.89% 99.84% 99.84% 99.83% 99.78% PSO-SVM False alarm rae 0.3% 0.31% 0.39% 0.35% 0.34% 0.33% 0.34% SVM Sum Maor voe Deecon rae 97.46% 99.76% 99.85% 99.83% 99.84% 99.85% 99.43% False alarm rae 0.31% 0.35% 0.36% 0.33% 0.34% 0.34% 0.34% Deecon rae 99.76% 99.76% 99.76% 99.76% 99.76% 99.76% 99.76% False alarm rae 0.37% 0.37% 0.37% 0.37% 0.37% 0.37% 0.37% Deecon rae 33.79% 33.79% 33.79% 33.79% 33.79% 33.79% 33.79% False alarm rae 0.34% 0.34% 0.34% 0.34% 0.34% 0.34% 0.34% As saed n he las paragraph o Secon 4, a unorm global model can be consruced or all he local nodes, small ses o samples are shared among he nodes, sacrcng he prvac o raw daa. We also es he unorm global model usng he daase o he our knds o aacks: nepune, smur, porsweep and saan n he 25

26 KDD CUP 1999 se. The ranng se used or consrucng he global model onl conans 4000 randoml chosen samples, and he es se or he global model conans samples o he our pes o aacks and all he normal samples. Table 14 shows he aack pes used o ran local models n each node. I s seen ha each node has onl one or wo pes o aack. Tables 15 and 16 show, usng our algorhm wh decson sumps and he radonal onlne Adaboos and our algorhm wh onlne GMMs and our onlne Adaboos respecvel, he deecon raes and alse alarm raes o he local models n each node and hose o he PSO and SVM-based global model, as well as hose obaned b combnng local models usng he sum rule and he SVM algorhm. I s seen ha our PSO and SVM-based algorhm greal ncreases he deecon accurac n each node and our PSO and SVM-based algorhm obans more accurae resuls han he sum rule and he SVM algorhm. Table 14. The aack pes used o ran he local models n each node Local models Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Tpes o aack Nepune Smur Porsweep Saan nepune, smur porsweep, saan Table 15. The resuls o he unorm global model based on our algorhm wh decson sumps and he radonal onlne Adaboos when some samples are shared among nodes Deecon models Deecon rae False alarm rae Node % 0.12% Node % 0.22% Local models Node % 0.1% Node % 0.01% Node % 0.32% Node % 0.11% Global model (PSO-SVM) 99.99% 1.35% Sum rule 9.10% 0.01% SVM 98.98% 1.41% Table 16. The resuls o he unorm global model based on our algorhm wh onlne GMMs and our onlne Adaboos when some samples are shared among nodes Deecon models Deecon rae False alarm rae Node % 0.08% Node % 0.01% Local models Node % 0.17% Node % 0.01% Node % 0.21% Node % 1.80% Global model (PSO-SVM) 99.99% 0.37% Sum rule 26.37% 0.01% SVM 99.99% 0.39% 26

27 The dsrbued ouler deecon algorhm n [4] has a deecon rae o 95% and a alse alarm rae o 0.35% or he KDD CUP 1999 ranng daa se, however, he algorhm n [4] requres a daa preprocessng procedure n whch he aack daa are packed no burss n he daa se. An nruson s marked as deeced a leas one nsance n a burs s lagged as an ouler. Furhermore, he hgh communcaon cos and he need o share a large number o he orgnal nework daa conne he applcaon o he algorhm n [4] o nework nruson deecon. In our proposed algorhm, no assumpons are made abou burss o aacks n he ranng daa and no prvae daa are shared among he nodes, and he onl communcaon s he concse local model parameers. 6. Concluson In hs paper, we have proposed onlne Adaboos-based nruson deecon algorhms, n whch decson sumps and onlne GMMs have been used as weak classers or he radonal onlne Adaboos and our proposed onlne Adaboos respecvel. The resuls o he algorhm usng decson sumps and he radonal onlne Adaboos have been compared wh he resuls o he algorhm usng onlne GMMs and our onlne Adaboos. We have urher proposed a dsrbued nruson deecon ramework, n whch he parameers n he onlne Adaboos algorhm have ormed he local deecon model or each node, and local models have been combned no a global deecon model n each node usng a PSO and SVM-based algorhm. The advanages o our work are as ollows: 1) Our onlne Adaboos-based algorhms successull overcome he dcules n handlng he mxed-arbues o nework connecon daa. 2) The onlne mode n our algorhms ensures he adapabl o our algorhms o he changng envronmens: he normaon n new samples s ncorporaed onlne no he classer, whle mananng hgh deecon accurac. 3) Our local parameerzed deecon models are suable or normaon sharng: onl a ver small number o daa are shared among nodes. 4) No orgnal nework daa are shared n he ramework, so ha he daa prvac s proeced. 5) Each global deecon model mproves consderabl on he nruson deecon accurac or each node. Reerences [1] D. Dennng, An nruson deecon model, IEEE Trans. on Soware Engneerng, vol. SE-13, no. 2, pp , Feb [2] J.B.D. Caberera, B. Ravchandran, and R.K. Mehra, Sascal rac modelng or nework nruson deecon, n Proc. o Modelng, Analss and Smulaon o Compuer and Telecommuncaon Ssems, pp , 2000 [3] W. Lee, S.J. Solo, and K. Mork, A daa mnng ramework or buldng nruson deecon models, n Proc. o IEEE Smposum on Secur Prvac, pp , Ma [4] M.E. Oe, A. Ghong, and S. Parhasarah, Fas dsrbued ouler deecon n mxed-arbue daa ses, Daa Mng and Knowledge Dscover, vol. 12, no. 2-3, pp , Ma [5] H.G. Kaack, A.N. Zncr-hewood, and M.T. Hewood, On he capabl o an SOM based nruson deecon ssem, n Proc. o Inernaonal Jon Conerence on Neural Neworks, vol. 3, pp , Jul [6] P.Z. Hu and M.I. Hewood, Predcng nrusons wh local lnear model, n Proc. o Inernaonal Jon Conerence on Neural Neworks, vol. 3, pp , Jul [7] Z. Zhang and H. Shen, Onlne ranng o SVMs or real-me nruson deecon, n Proc. o Advanced Inormaon Neworkng and Applcaons, vol. 2, pp , [8] H. Lee, Y. Chung, and D. Park, An adapve nruson deecon algorhm based on cluserng and kernel-mehod, n Proc.o Inernaonal Conerence on Advanced normaon Neworkng and Applcaon, pp , [9] W. Lee and S.J. Solo, A ramework or consrucng eaures and models or nruson deecon ssems, ACM Transacons on Inormaon an Ssem Secur, vol. 3, no. 4, pp , Nov [10] A. Fern and R. Gvan, Onlne ensemble learnng: an emprcal sud, n Proc. o Inernaonal Conerence on Machne Learnng, pp , [11] J. Kler, M. Hae, R.P.W. Dun, and J. Maas. On combnng classers, IEEE Trans. on Paern Analss and Machne Inellgence, vol. 20, no.3, pp , March [12] J. Kenned, Parcle swarm opmzaon, n Proc. o IEEE Inernaonal Conerence on Neural Neworks, Perh, pp , [13] Y. Sh and R.C. Eberhar, A moded parcle swarm opmzer, n Proc. o IEEE Inernaonal Conerence on Evoluonar Compuaon, Anchorage, USA, pp.69-73, [14] S. Soo e al. The hrd nernaonal knowledge dscover and daa mnng ools compeon, The Unvers o Calorna Avalable: hp://kdd.cs.uc.edu/daabases/kddcup99/kddcup99.h-ml. 27

28 [15] S. Mukkamala, A.H. Sung, and A. Abraham. Inruson deecon usng an ensemble o nellgen paradgms, Nework and Compuer Applcaons, vol. 28, no. 2, pp , Aprl [16] R. Lppmann, J.W. Hanes, D.J. Fred, J. Korba, and K. Das, The 1999 DARPA o-lne nruson deecon evaluaon, ACM Trans. on Inormaon and Ssem Secur, vol. 34, no. 4, pp , Oc [17] Y.G. Lu, K.F. Chen, X.F. Lao, and W. Zhang. A genec cluserng mehod or nruson deecon, Paern Recognon, vol. 37, no. 5, pp , Ma [18] S.T. Sarasamma, Q.A. Zhu, and J. Hu. Herarchcal Kohonenen ne or anomal deecon n nework secur, IEEE Trans. on Ssems, Man and Cbernecs, Par B: Cbernecs, vol. 35, no. 2, pp , Aprl [19] B. Pahrnger. Wnnng he KDD99 classcaon cup: bagged boosng, SIGKDD Exploraons, vol. 1, no. 2, pp , [20] E. Eskn, A. Arnold, M. Prerau, L. Porno, and S. Solo, A geomerc ramework or unsupervsed anomal deecon: deecng nrusons n unlabeled daa, n Applcaons o Daa Mnng n Compuer Secur, Chaper 4. D. Barbara and S. Jaoda (edors) Kluwer, 2002 [21] C. Jrapummn, N. Waanapongsakorn, and P. Kanhamanon. Hbrd neural neworks or nruson deecon ssems, n Proc. o Inernaonal Techncal Conerence on Crcus/Ssems, Compuers and Communcaons, pp , Jul [22] J. L and C. Mankopoulos, Novel sascal nework model: he hperbolc dsrbuon, IEE Proceedng on Communcaons, vol. 151, no. 6, pp , Dec [23] T. Peng, C. Lecke, and K. Ramamohanarao, Inormaon sharng or dsrbued nruson deecon ssems, Journal o Nework and Compuer Applcaons, vol. 30, no. 3, pp , Aug [24] M. Qn and K. Hwang, Frequen epsode rules or nerne anomal deecon, n Proc. o IEEE Inernaonal Smposum on Nework Compung and Applcaons, pp , [25] S. Zanero and S.M. Savares, Unsupervsed learnng echnques or an nruson deecon ssem, n Proc. o ACM Smposum on Appled Compung, pp , [26] A. Bvens, C. Palagr, R. Smh, B. Szmansk, and M. Embrechs, Nework-based nruson deecon usng neural neworks, n Proc. o Arcal Neural Neworks n Engneerng, vol. 12, New York, pp , Nov [27] J.J.M. Bonaco, A.M. Cansan, A.C.P.L.F. De Carvalho, and E.S. Morera, Neural neworks appled n nruson deecon ssems, n Proc. o IEEE Inernaonal Jon Conerence on Neural Neworks, vol. 1, pp , [28] C. Zhang, J. Jang, and M. Kamel, Inruson deecon usng herarchcal neural neworks, Paern Recognon Leers, vol. 26, no. 6, pp , [29] S.J. Han and S.B. Cho, Evoluonar neural neworks or anomal deecon based on he behavor o a program, IEEE Trans. on Ssems, Man, and Cbernecs-Par B, vol. 36, no. 3, pp , June [30] S. Mukkamala, G. Janosk, and A. Sung, Inruson deecon usng neural neworks and suppor vecor machnes, n Proc. o Inernaonal Jon Conerence on Neural Neworks, vol. 2, pp , [31] J. Mll and A. Inoue, Suppor vecor classers and nework nruson deecon, n Proc. o Inernaonal Conerence on Fuzz Ssems, vol. 1, pp , [32] J. Xan, F. Lang, and X. Tang, A novel nruson deecon mehod based on clonal selecon cluserng algorhm, n Proc. o Inernaonal Conerence on Machne Learnng and Cbernecs, vol. 6, pp , [33] S. Jang, X. Song, H. Wang, J. Han, and Q. L, A cluserng-based mehod or unsupervsed nruson deecons, Paern Recognon Leers, vol. 27, no. 7, pp , Ma [34] A.J. Hoglund, K. Haonen, and A.S. Sorvar, A compuer hos-based user anomal deecon ssem usng he sel-organzng map, n Proc. o Inernaonal Jon Conerence on Neural Neworks, vol. 5, pp , [35] D. Song, M.I. Hewood, and A.N. Zncr-Hewood, Tranng genec programmng on hal a mllon paerns: an example rom anomal deecon, IEEE Trans. on Evoluonar Compuaon, vol. 9, no. 3, pp , June [36] S. Parhasarah, A. Ghong, and M.E. Oe, A surve o dsrbued mnng o daa sreams, Daa sreams: models and algorhms, C. C. Aggarwal, Eds. New York: Sprnger-Verlag, Nov [37] Y. Freund and R. E. Schapre, A decson-heorec generalzaon o on-lne learnng and an applcaon o boosng, Journal o Compuer and Ssem Scences, vol. 55, no. 1, pp , Augus [38] N. Oza. Onlne Ensemble Learnng, PhD hess, Unvers o Calorna, Berkele, [39] W.M. Hu, W. Hu, and S. Mabank, Adaboos-based algorhm or nework nruson deecon, IEEE Trans. on Ssems, Man, and Cbernecs, Par B: Cbernecs, vol. 38, no. 2, pp , Aprl [40] J. Zhang, M. Zulkernne, and A. Haque, Random-oress-based nework nruson deecon ssems, IEEE Trans. on Ssems, Man, and Cbernecs, Par C: Applcaons and Revews, vol. 38, no. 5, pp , Sep [41] A.P. Dempser, N.M. Lard, and D.B. Rubn, Maxmum lkelhood rom ncomplee daa va he EM algorhm, Journal o he Roal Sascal Soce, Seres B (Mehodologcal), vol. 39, no. 1, pp. 1-38, [42] H. Grabner and H. Bscho, On-lne boosng and vson, n Proc. o IEEE Conerence on Compuer Vson and Paern Recognon, pp , [43] N. Oza and S. Russell, Expermenal comparsons o onlne and bach versons o baggng and boosng, n Proc. o ACM SIGKDD Inernaonal Conerence on Knowledge Dscover and Daa Mnng, San Francsco, Calorna, pp , [44] Y. Le, X.Q. Dng, and S.J. Wang, Vsual racker usng sequenal Baesan learnng: dscrmnave, generave and hbrd, IEEE Trans. on Ssems, Man and Cbernecs, Par B: Cbernecs, vol. 38, no. 6, pp , Dec [45] J. McHugh, Tesng nruson deecon ssems: a crque o he 1998 and 1999 DARPA nruson deecon ssem evaluaons as perormed b Lncoln laboraor, ACM Trans. on Inormaon and Ssem Secur, vol. 3, no. 4, pp , Nov [46] J.Z. Le and A. Chorban, Nework nruson deecon usng an mproved compeve learnng neural nework, n Proc. o Second Annual Conerence on Communcaon Neworks and Servces Research, vol. 4, pp , Ma [47] C. Elkan. Resuls o he KDD99 classer learnng cones, SIGKDD Exploraons, vol. 1, no. 2, pp , [48] S. Ba and M. Schwabacher, Mnng dsance-based oulers n near lnear me wh randomzaon and a smple prunng rule, n Proc. o ACM SIGKDD Inernaonal Conerence on Knowledge Dscover and Daa Mnng, pp , [49] S. Mabu, C. Chen, N. Lu, K. Shmada, and K. Hrasawa, An nruson-deecon model based on uzz class-assocaon-rule 28

29 mnng usng genec nework programmng, IEEE Trans. on Ssems, Man, and Cbernecs, Par C: Applcaons and Revews, vol. 41, no. 1, pp , Jan [50] S.T. Brugger and J. Chow, An assessmen o he DARPA IDS evaluaon daase usng Snor, Techncal Repor CSE , Unvers o Calorna, Jan [51] S. Pangrah and S. Sural, Deecon o daabase nruson usng a wo-sage uzz ssem, Inormaon Secur, Lecure Noes n Compuer Scence, vol. 5735, pp , [52] D. Smallwood and A. Vance, Inruson analss wh deep packe nspecon: ncreasng ecenc o packe based nvesgaons, n Proc. o Inernaonal Conerence on Cloud and Servce Compung, pp , Dec [53] S. Mabu, C. Chen, N. Lu, K. Shmada, and K. Hrasawa, An nruson-deecon model based on uzz class-assocaon-rule mnng usng genec nework programmng, IEEE Trans. on Ssems, Man, and Cbernecs. Par C: Applcaons and Revews, vol. 41, no. 1, pp , Jan [54] C. Oe and C. Sormann, Improvng he accurac o nework nruson deecors b npu-dependen sackng, Inegraed Compuer-Aded Engneerng, vol. 18, pp , [55] K.-C. Khor, C.-Y. Tng, S. Phon-Amnuasuk, A cascaded classer approach or mprovng deecon raes on rare aack caegores n nework nruson deecon, Appled Inellgence, vol. 36, pp , [56] B. Zhang, A heursc genec neural nework or nruson deecon, n Proc. o Inernaonal Conerence on Inerne Compung and Inormaon Servces, pp , Sep [57] C.-F. Tsa, J.-H. Tsa, and J.-S. Chou, Cenrod-based neares neghbor eaure represenaon or e-governmen nruson deecon, n Proc. o World Telecommuncaons Congress, pp. 1-6, March [58] J. Gao, W. Hu, X. Zhang, and X. L, Adapve dsrbued nruson deecon usng paramerc model, n Proc. o IEEE/WIC/ACM Inernaonal Jon Conerences on Web Inellgence and Inellgen Agen Technologes, vol. 1, pp , Sep [59] P. Prasenna, A.V.T. RaghavRamana, R. KrshnaKumar, and A. Devanbu, Nework programmng and mnng classer or nruson deecon usng probabl classcaon, n Proc. o he Inernaonal Conerence on Paern Recognon, Inormacs and Medcal Engneerng, pp , March [60] K. Reck, Machne learnng or applcaon-laer nruson deecon, Dsseraon, Fraunhoer Insue FIRST & Berln Insue o Technolog, Berln German, Acknowledgmens Ths work s parl suppored b NSFC (Gran No ), he Naonal 863 Hgh-Tech R&D Program o Chna (Gran No. 2012AA012504), he Naural Scence Foundaon o Beng (Gran No ), and The Proec Suppored b Guangdong Naural Scence Foundaon (Gran No. S ). Wemng Hu receved he Ph.D. degree rom he deparmen o compuer scence and engneerng, Zheang Unvers n From Aprl 1998 o March 2000, he was a posdocoral research ellow wh he Insue o Compuer Scence and Technolog, Pekng Unvers. Now he s a proessor n he Insue o Auomaon, Chnese Academ o Scences. Hs research neress are n vsual survellance, and lerng o Inerne obeconable normaon. Jun Gao has a B.S. n Auomaon Engneerng rom BeHang Unvers, Beng, Chna, and a Ph.D. n Compuer Scence rom Insue o Auomaon, Chnese Academ o Scences. Hs maor research neress nclude machne learnng, daa mnng, and nework normaon secur. Yanguo Wang receved he BSc degree n normaon and compung scence rom Sun Ya-sen Unvers, Guangzhou, Chna, n In 2008, he receved he MSc degree a he Naonal Laboraor o Paern Recognon, Insue o Auomaon, Chnese Academ o Scences. He s currenl a proec manager n he Insue o Inrasrucure Inspecon, Chna Academ o Ralwa Scences, Beng. Hs research neress nclude paern recognon, compuer vson, and daa mnng. Ou Wu receved he BS degree n Elecrcal Engneerng rom X an Jaoong Unvers n He receved he MS degree and PhD degree boh rom paern recognon and machne nellgence rom Naonal Laboraor o Paern Recognon (NLPR), Insue o Auomaon, Chnese Academ o Scences n 2006 and 2011, respecvel. Now, he s an Asssan Proessor n NLPR. Hs research neress nclude normaon lerng, daa mnng, and web vson compung. 29

30 Sephen Mabank receved a BA n Mahemacs rom Kng's college Cambrdge n 1976 and a PhD n compuer scence rom Brkbeck college, Unvers o London n Now he s a proessor n he School o Compuer Scence and Inormaon Ssems, Brkbeck College. Hs research neress nclude he geomer o mulple mages, camera calbraon, vsual survellance ec. 30