Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation. Hongliang Yan 2017/06/21

Transcription

1 nd he class wegh bas: weghed maxmum mean dscrepancy for unsupervsed doman adapaon Honglang Yan 207/06/2

2 Doman Adapaon Problem: Tranng and es ses are relaed bu under dfferen dsrbuons. Tranng (Source) DA Tes (Targe) ehodology: Learn feaure space ha combne dscrmnaveness and doman nvarance. mnmze source error + doman dscrepancy Fgure. Illusraon of daase bas. []hps://cs.sanford.edu/~jhoffman/domanadap/

3 axmum ean Dscrepancy (D) represenng dsances beween dsrbuons as dsances beween mean embeddngs of feaures An emprcal esmae D ( s, ) sup E [ (x )] E [ (x )] 2 s x ~ s x ~ H H D (, ) (x ) (x ) 2 Ds D j H N j

4 ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j

5 ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j s s 2 wc Ec wc Ec H c c ( (x )) ( (x )) s w and w N N c c c c

6 ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j Effec of class wegh bas should be removed: hanges n sample selecon crera s s wc Ec wc Ec c c 2 H ( (x )) ( (x )) s w and w N N c c c c

7 ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j Effec of class wegh bas should be removed: hanges n sample selecon crera s s wc Ec wc Ec c c 2 H ( (x )) ( (x )) s w and w N N c c c c Fgure 2. lass pror dsrbuon of hree dg recognon daases.

8 ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j Effec of class wegh bas should be removed: hanges n sample selecon crera s s wc Ec wc Ec c c 2 H ( (x )) ( (x )) s w and w N N c c c c 2 Applcaons are no concerned wh class pror dsrbuon

9 ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j Effec of class wegh bas should be removed: hanges n sample selecon crera s s wc Ec wc Ec c c 2 H ( (x )) ( (x )) s w and w N N c c c c 2 Applcaons are no concerned wh class pror dsrbuon D can be mnmzed by eher learnng doman nvaran represenaon or preservng he class weghs n source doman.

10 Weghed D an dea: reweghng classes n source doman so ha hey have he same class weghs as arge doman Inroducng an auxlary wegh c for each class c n source doman D (, ) (x ) (x ) 2 Ds D j H N j w w s c c c s s 2 wc Ec wc Ec H c c ( (x )) ( (x ))

11 Weghed D an dea: reweghng classes n source doman so ha hey have he same class weghs as arge doman Inroducng an auxlary wegh c for each class c n source doman D (, ) (x ) (x ) 2 Ds D j H N j s s 2 wc Ec wc Ec H c c ( (x )) ( (x )) w w s c c c D (, ) (x ) (x ) 2 w Ds D s y j H N j w E s c c w E c c c c 2 H ( (x )) ( (x ))

12 Weghed DAN. Replace D wh weghed D em n DAN[4]: mn (x, ; W) D ( D, D ) W s s l l y l s l { l,..., l } L [4] Long, ao Y, Wang J. Learnng Transferable Feaures wh Deep Adapaon Neworks[J]., 205.

13 Weghed DAN. Replace D wh weghed D em n DAN[4]: mn (x, ; W) D ( D, D ) W s s l l y l s l { l,..., l } L mn (x, ; W) D ( D, D ) W, s s l l y l, w s l { l,..., l } L [4] Long, ao Y, Wang J. Learnng Transferable Feaures wh Deep Adapaon Neworks[J]., 205.

14 Weghed DAN. Replace D wh Weghed D em n DAN[4]: mn (x, ; W) D ( D, D ) W s s l l y l s l { l,..., l } 2. To furher explo he unlabeled daa n arge doman, emprcal rsk s consdered as sem-supervsed model n [5]: L mn (x, ; W) (x, ; W) D (, ) N s s l l ˆ N y y l, w Ds D W,{ yˆ j} j, N j l { l,..., l } L [4] Long, ao Y, Wang J. Learnng Transferable Feaures wh Deep Adapaon Neworks[J]., 205. [5] Amn, assh-reza, and Parck Gallnar. "Sem-supervsed logsc regresson." Proceedngs of he 5h European onference on Arfcal Inellgence. IOS Press, 2002.

15 Opmzaon: an exenson of E[6] Parameers o be esmaed ncludng hree pars,.e., The model s opmzed by alernang beween hree seps : E-sep: Fxed W, esmang he class poseror probably p( y c x ) g(x, W) j j j p( y c x ) j j W,,{ yˆ } N j of arge samples: j [7] eleux, Glles, and Gérard Govaer. "A classfcaon E algorhm for cluserng and wo sochasc versons." ompuaonal sascs & Daa analyss 4.3 (992):

16 Opmzaon: an exenson of E[6] Parameers o be esmaed ncludng hree pars,.e., The model s opmzed by alernang beween hree seps : E-sep: Fxed W, esmang he class poseror probably -sep: p( y c x ) g(x, W) j j j p( y c x ) W,,{ yˆ } of arge samples: Assgn he pseudo labels { yˆ } N j j on arge doman: c 2 updae he auxlary class-specfc weghs for source doman: wˆ w s where wˆ ( yˆ ) N c c c c c j j j j j N j yˆ arg max p( y c x ) j j j ( x) s an ndcor funcon whch equals f x = c, and equals 0 oherwse. c [7] eleux, Glles, and Gérard Govaer. "A classfcaon E algorhm for cluserng and wo sochasc versons." ompuaonal sascs & Daa analyss 4.3 (992):

17 Opmzaon: an exenson of E[6] Parameers o be esmaed ncludng hree pars,.e., The model s opmzed by alernang beween hree seps : -sep: Fxed { ˆ } N and, updang W. The problem s reformulaed as: y j j mn (x, ; W) (x, ; W) D ( D, D ) W N s s l l y y l, w s j l { l,..., l } L W,,{ yˆ } j N j The graden of he hree ems s compuable and W can be opmzed by usng a mn-bach SGD. [7] eleux, Glles, and Gérard Govaer. "A classfcaon E algorhm for cluserng and wo sochasc versons." ompuaonal sascs & Daa analyss 4.3 (992):

18 Expermenal resuls omparson wh sae-of-he-ars Table. Expermenal resuls on offce-0+alech-0

19 Expermenal resuls Emprcal analyss Fgure 3. Performance of varous model under dfferen class wegh bas. Fgure 4. Vsualzaon of he learned feaures of DAN and weghed DAN.

20 Summary Inroduce class-specfc wegh no D o reduce he effec of class wegh bas cross domans. Develop WDAN model and opmze n an E framework. Weghed D can be appled o oher scenaros where D s used for dsrbuon dsance measuremen, e.g., mage generaon

21 Thanks! Paper & code are avalable Paper: hps://arxv.org/abs/ ode: hps://ghub.com/yhldh/wd-affe