nd he class wegh bas: weghed maxmum mean dscrepancy for unsupervsed doman adapaon Honglang Yan 207/06/2
Doman Adapaon Problem: Tranng and es ses are relaed bu under dfferen dsrbuons. Tranng (Source) DA Tes (Targe) ehodology: Learn feaure space ha combne dscrmnaveness and doman nvarance. mnmze source error + doman dscrepancy Fgure. Illusraon of daase bas. []hps://cs.sanford.edu/~jhoffman/domanadap/
axmum ean Dscrepancy (D) represenng dsances beween dsrbuons as dsances beween mean embeddngs of feaures An emprcal esmae D ( s, ) sup E [ (x )] E [ (x )] 2 s x ~ s x ~ H H D (, ) (x ) (x ) 2 Ds D j H N j
ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j
ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j s s 2 wc Ec wc Ec H c c ( (x )) ( (x )) s w and w N N c c c c
ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j Effec of class wegh bas should be removed: hanges n sample selecon crera s s wc Ec wc Ec c c 2 H ( (x )) ( (x )) s w and w N N c c c c
ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j Effec of class wegh bas should be removed: hanges n sample selecon crera s s wc Ec wc Ec c c 2 H ( (x )) ( (x )) s w and w N N c c c c Fgure 2. lass pror dsrbuon of hree dg recognon daases.
ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j Effec of class wegh bas should be removed: hanges n sample selecon crera s s wc Ec wc Ec c c 2 H ( (x )) ( (x )) s w and w N N c c c c 2 Applcaons are no concerned wh class pror dsrbuon
ovaon lass wegh bas cross domans remans unsolved bu ubquous D (, ) (x ) (x ) 2 Ds D j H N j Effec of class wegh bas should be removed: hanges n sample selecon crera s s wc Ec wc Ec c c 2 H ( (x )) ( (x )) s w and w N N c c c c 2 Applcaons are no concerned wh class pror dsrbuon D can be mnmzed by eher learnng doman nvaran represenaon or preservng he class weghs n source doman.
Weghed D an dea: reweghng classes n source doman so ha hey have he same class weghs as arge doman Inroducng an auxlary wegh c for each class c n source doman D (, ) (x ) (x ) 2 Ds D j H N j w w s c c c s s 2 wc Ec wc Ec H c c ( (x )) ( (x ))
Weghed D an dea: reweghng classes n source doman so ha hey have he same class weghs as arge doman Inroducng an auxlary wegh c for each class c n source doman D (, ) (x ) (x ) 2 Ds D j H N j s s 2 wc Ec wc Ec H c c ( (x )) ( (x )) w w s c c c D (, ) (x ) (x ) 2 w Ds D s y j H N j w E s c c w E c c c c 2 H ( (x )) ( (x ))
Weghed DAN. Replace D wh weghed D em n DAN[4]: mn (x, ; W) D ( D, D ) W s s l l y l s l { l,..., l } L [4] Long, ao Y, Wang J. Learnng Transferable Feaures wh Deep Adapaon Neworks[J]., 205.
Weghed DAN. Replace D wh weghed D em n DAN[4]: mn (x, ; W) D ( D, D ) W s s l l y l s l { l,..., l } L mn (x, ; W) D ( D, D ) W, s s l l y l, w s l { l,..., l } L [4] Long, ao Y, Wang J. Learnng Transferable Feaures wh Deep Adapaon Neworks[J]., 205.
Weghed DAN. Replace D wh Weghed D em n DAN[4]: mn (x, ; W) D ( D, D ) W s s l l y l s l { l,..., l } 2. To furher explo he unlabeled daa n arge doman, emprcal rsk s consdered as sem-supervsed model n [5]: L mn (x, ; W) (x, ; W) D (, ) N s s l l ˆ N y y l, w Ds D W,{ yˆ j} j, N j l { l,..., l } L [4] Long, ao Y, Wang J. Learnng Transferable Feaures wh Deep Adapaon Neworks[J]., 205. [5] Amn, assh-reza, and Parck Gallnar. "Sem-supervsed logsc regresson." Proceedngs of he 5h European onference on Arfcal Inellgence. IOS Press, 2002.
Opmzaon: an exenson of E[6] Parameers o be esmaed ncludng hree pars,.e., The model s opmzed by alernang beween hree seps : E-sep: Fxed W, esmang he class poseror probably p( y c x ) g(x, W) j j j p( y c x ) j j W,,{ yˆ } N j of arge samples: j [7] eleux, Glles, and Gérard Govaer. "A classfcaon E algorhm for cluserng and wo sochasc versons." ompuaonal sascs & Daa analyss 4.3 (992): 35-332.
Opmzaon: an exenson of E[6] Parameers o be esmaed ncludng hree pars,.e., The model s opmzed by alernang beween hree seps : E-sep: Fxed W, esmang he class poseror probably -sep: p( y c x ) g(x, W) j j j p( y c x ) W,,{ yˆ } of arge samples: Assgn he pseudo labels { yˆ } N j j on arge doman: c 2 updae he auxlary class-specfc weghs for source doman: wˆ w s where wˆ ( yˆ ) N c c c c c j j j j j N j yˆ arg max p( y c x ) j j j ( x) s an ndcor funcon whch equals f x = c, and equals 0 oherwse. c [7] eleux, Glles, and Gérard Govaer. "A classfcaon E algorhm for cluserng and wo sochasc versons." ompuaonal sascs & Daa analyss 4.3 (992): 35-332.
Opmzaon: an exenson of E[6] Parameers o be esmaed ncludng hree pars,.e., The model s opmzed by alernang beween hree seps : -sep: Fxed { ˆ } N and, updang W. The problem s reformulaed as: y j j mn (x, ; W) (x, ; W) D ( D, D ) W N s s l l y y l, w s j l { l,..., l } L W,,{ yˆ } j N j The graden of he hree ems s compuable and W can be opmzed by usng a mn-bach SGD. [7] eleux, Glles, and Gérard Govaer. "A classfcaon E algorhm for cluserng and wo sochasc versons." ompuaonal sascs & Daa analyss 4.3 (992): 35-332.
Expermenal resuls omparson wh sae-of-he-ars Table. Expermenal resuls on offce-0+alech-0
Expermenal resuls Emprcal analyss Fgure 3. Performance of varous model under dfferen class wegh bas. Fgure 4. Vsualzaon of he learned feaures of DAN and weghed DAN.
Summary Inroduce class-specfc wegh no D o reduce he effec of class wegh bas cross domans. Develop WDAN model and opmze n an E framework. Weghed D can be appled o oher scenaros where D s used for dsrbuon dsance measuremen, e.g., mage generaon
Thanks! Paper & code are avalable Paper: hps://arxv.org/abs/705.00609 ode: hps://ghub.com/yhldh/wd-affe