Online Encoding Algorithm for Infinite Set

Transcription

1 Ole Ecodg Algorthm for Ifte Set Natthapo Puthog, Athast Surarers ELITE (Egeerg Laboratory Theoretcal Eumerable System) Departmet of Computer Egeerg Faculty of Egeerg, Chulalogor Uversty, Pathumwa, Bago, 0330, Thalad Emal: Abstract I ths paper we preset ole ecodg ad decodg algorthms for a fte put alphabet. The well-ow ecodg algorthm that explctly costructs the optmal codes s proposed by Huffma. Ths algorthm s clearly approprate for ole ecodg because the whole put data are ot ow before startg the process. We propose a algorthm that ca be ecoded whle the put data performed a ole mode. Theoretcal result shows that the tree obtaed from the algorthm satsfes a code legth property of a optmal soluto. Moreover, expermetal results are also show that the average code legth s close to the oe resulted from Huffma algorthm ad the etropy of put data. Key words: Ole ecodg algorthm, fte Huffma codes, recursve replacemet method.. Itroducto I may research domas, ecodg s a mportace problem. For stace, Golomb [9] proposed ru-legth ecodg usg the varable legth cocept for some lossless mage compresso. I 95, Huffma [] troduced the well-ow ecodg algorthm whch uses bary tree patter process for ecodg each put character. Durg the process, bottom-up techque s appled to buld the tree. Some exteded verso of Huffma s algorthm are developed the case that all characters have dfferet cost, see detal [6]. It s clear that the Huffma s ecodg algorthm caot apply to a ole ecodg problem whch the umber of dfferece characters are uow (.e., a ew character ca be appeared durg the process.) I order to solve the problem, tree structures are studed by may researchers. Some optmal prefx-free trees are studed [5, 7, 8, 0]. Oe remarable wor s a optmal fte prefx-free tree structure proposed by Gol ad Ma 004 (see detal [3]). Ladlaw [4] also troduced a algorthm for costructg codes for fte sets usg recursve replacemet method to costruct fte tree. I ths wor, we focus a ole ecodg problem. A top-dow techque whch s the reversg of Huffma s algorthm s appled for costructg a fte set of prefx-free codes usg a optmal prefx-free fte tree structure combg wth the recursve replacemet method. The tree s show to be approprate for the probablty dstrbuto. The paper s orgazed as follows: Secto recalls the Huffma s ecodg algorthm ad the cocept of fte tree costructo. Secto 3 descrbes the ole ecodg problem. Moreover, a ole ecodg ad decodg algorthm for fte set are preseted. Some theoretcal results are also demostrated. Some expermetal results comparg betwee our ole ecodg algorthm for fte set ad Huffma s algorthm are show Secto 4. Secto 5 s the cocluso of the paper.. Basc deftos We troduce some basc deftos used ths wor, startg from the cocept of etropy. The classcal varable legth ecodg algorthm proposed by Huffma s descrbed. The the cocept of recursvely fte tree costructo s also cluded ths secto.

2 . Etropy Ths secto descrbes a geeral method for measurg the structure of the source s cosdered. The cocept of etropy s troduced for smplfyg a fucto of a probablty dstrbuto. Let p be the probablty of gettg formato a, the the average code legth for each symbol a s descrbed by p log (/ p ). Form ths t follows that the average code legth over the whole alphabet should be = p log (/ p ).. Huffma ecodg algorthm Huffma [] has proposed a algorthm for geeratg codes order to reduce the sze of the specfed cotext. The frequecy of characters s a mportat factor to ecode the characters. Gve a alphabet, a fte set of characters, deoted by Σ where Σ={a, a, a 3,, a }. Let P={p, p, p 3,, p } be the set of probabltes of occurrece of each character a gve cotext such that p deotes the probablty of a. The problem s to geerate codes whch are strgs of 0 ad wth the mmum average legth. Let S={c, c, c 3,, c } be the soluto (c be the code of a ). The average sze of the soluto L av s computed by L av = = p l, where l s the legth of c. All geerated codes from the algorthm preserve the prefx-free property (.e., for ay two dfferet codes, oe must ot be the prefx of the others.) All codes ca be represeted by oe sgle bary tree structure. The Huffma ecodg algorthm use bottom-up techque to costruct the bary tree for mmzes the average code legth. It starts from the least frequecy character; the most frequecy character should be ecoded by the possble shortest legth code. Each character s assged to be a leaf ode of the tree. A code of a character correspods to the path from the root to that leaf ode. Fgure shows the problem for ecodg A, B, C, ad D wth probablty 0., 0., 0.3, ad 0.4 respectvely. For stace, the code of C s 0. Fgure. Codes represeted usg a bary tree.3 A fte tree costructo The problem to geerate a fte tree s studed ths secto. Sce the Huffma s algorthm assgs all characters to be leaf odes of a bary tree, we are terested the costructg of a fte bary tree whch leaf odes ca be occurred ay levels. The wor of Ladlaw [4] proposed that the umber of leaf odes each level s defed by a geeratg fucto; ths techque s called a recursve replacemet method. I prcple, a geeratg fucto G(x) s descrbed by a recursve defto where G(0) must be zero. I each level of costructg the tree, odes are separated to two parttos, oe for termal (leaf) part ad other for otermal (geeratg) part. Let N be the umber of odes at the level. It s clear that N = N +. The t s ot the case that all geeratg fucto ca be selected. Ladlaw also studed the characterstc of such geeratg fuctos. It s demostrated that the average code legth ca be computed usg a geeratg fucto assocated to the tree. Fgure. Tree geerated by f L = f L-

3 The bary tree Fgure s geerated by the geeratg fucto as follow: f L = fl, where f L s the umber of termal odes at level L, L, ad f 0 = 0, f =. 3. Ole ecodg ad decodg I ths secto, we start wth the descrpto of the ole ecodg problem. The bary tree structure ad the property of optmal tree wll be studed ad some theoretcal results wll be show. Fally, code terchagg, ole ecodg ad decodg algorthms wll be troduced. Fgure 3. Ole ecodg ad decodg process Fgure 3 shows the process of ole ecodg ad decodg of fte put alphabet set. Gve a cotext whch s a sequece of characters the uow put alphabet, the ole ecodg process starts by sequetally ecodg character, through the ecodg ut, character-by-character, from the frst character of the cotext utl the last character. The decodg process s performed by the same maer. 3. Ifte alphabet Sce the ecodg process performs a ole mode, the comg cotext ca be cosdered as a sequece of ow ad uow characters. Ths mples that the umber of dfferet characters ad ts frequecy are also uow. Ths s the reaso that a put alphabet s cosdered as a fte set. 3. The property of optmal tree The obectve of ecodg algorthm s to geerate the optmal soluto whch ts average code legth s mmal. That s to mmze m L = p l. av = Theorem shows that the optmal ecodg tree must satsfy the code legth property. Theorem Let S be the optmal soluto for the ecodg problem. Let c ad c be codes S ad let p ad p be ther probablty respectvely. If p p, the legth(c ) legth(c ). Proof: The proof of the theorem s to show that p p legth(c ) legth(c ). Suppose that S s the optmal soluto ad codes c, c are elemets S, correspoded to character a ad a. Let p p. It s proved that legth(c ) legth(c ). By the cotradcto techque, let legth(c ) > legth(c ). Sce S s the optmal soluto, the mmal average code legth s legth( c ) p. = Let = legth(c ), ad m = legth(c ) the the average code legth ca be rewrtte as =,, legth( c ) p + p + mp. Cosder the soluto that c ad c are codes for a ad a. The average code legth results from ths soluto s =,, legth( c ) p + mp + p. It s clear that the average code legth of ths soluto s less tha the average code legth of the soluto S. Ths cotradcts that S s the optmal soluto. That s the theorem s true. I our wor, the bary tree s geerated usg a recursve replacemet method wth respectve to the followg costat fucto, f ( 0) = f () = f () = 0 f ( L) = log, for L 3, where s the umber of characters ad L s the level the tree. We are focused the cotext whch s a Eglsh text fle, ad the the umber of characters s approxmated by 6. The bary tree structure s geerated as llustrated by Fgure 4. Defto Let c ad c be codes for characters Σ where p ad p be ther probablty respectvely. A ecodg tree s sad to satsfy the code legth property f p p the legth(c ) s less tha or equal to legth(c ).

4 c b buffer ext edf eddo ed Lemma shows that the tree obtaed from the terchagg algorthm s satsfed the code legth property. Fgure 4. A bary tree All odes a bary tree ca be separated to two parts, o-termal (brachg) odes ad termal odes (leaf). Each termal ode s represeted by a trple ode = <a, frequecy(a ),c > Tree. The parameter a s the put character, frequecy(a ) s the frequecy of character a, ad the code c s exactly the path from the root to that termal ode, ode. Termal odes wll be sequetally assged startg from the top level. 3.3 Code terchagg algorthm The cocept of code terchagg process s to preserve the code legth property of a bary tree the ole ecodg ad decodg algorthms. I the case of, oce a put character comes to the ole ecodg process, f t s the ew oe, a ew code must be assged to the character ad ts frequecy s creased by ; otherwse the bary tree must be adusted f t does ot preserve the code legth property (.e., character s frequecy s more tha other oe the tree but ts code legth s loger tha that oe.) Ths ca be performed by terchagg ts code wth the shorter code that has less frequecy. The process of ole decodg rus the same maer. The modfcato of the soluto bary tree ca be descrbed by the followg algorthm. Algorthm: Iterchagg put: Usatsfed tree a (put character) c a (code of a) p a (ts frequecy) output: Satsfed tree beg for each ode b:legth(c b ) < legth(c a ) do f p b < p a buffer c a c a c b Lemma The ecodg tree obtaed from the algorthm always satsfes the code legth property. Proof: To prove the lemma, t s to show that the average legth code computed from the output s less tha or equal to the average legth code correspoded to the put. Let a a a 3 a - be a ecoded cotext ad let c, c, c 3,, c - be codes of a, a, a 3,,a - respectvely. Let the put tree satsfy the code legth property ad let the average code legth be legth( c ). = Suppose that a ew comer (a ) arrves, ts probablty (p ) must be creased. I the case that there exts a character a such that p < p but legth(c ) < legth(c ), the put tree s usatsfed the property. That s the average code legth becomes L = ( legth( c)) + legth( c ). = By applyg the terchagg algorthm, the tree must be adusted ad the ew average code legth s L = ( legth( c )) + legth( c ). = Sce legth(c ) < legth(c ), that s L < L. The the proof s completed. 3.4 Ecodg & decodg algorthms The ole ecodg ad decodg algorthms are preseted ths secto. Durg the ole ecodg ad decodg process, character s frequecy ca be chaged deped o ts character ad the bary tree the ole ecodg ad decodg process must be always preserved the code legth property by code terchagg algorthm. The ole ecodg algorthm s show as follows: Algorthm: Ole Ecodg put: a a a 3 a 4 (a sequece of characters)

5 output: c c c 3 c 4 (a sequece of bary code) beg Tree (empty set) (curret put character) 0 (umber of termal odes Tree) whle (ot-ed-of-data) do f (a = x <x, f x, c x > = ode Tree) f x f x + c c x call Iterchagg algorthm else + ew ode Tree Tree ode ode <a,, path from root to ode > edf + eddo ed By the same techque, the decodg algorthm s as follows: Algorthm: Ole Decodg put: c c c 3 c 4 (a sequece of bary code) output: a a a 3 a 4 (a sequece of characters) beg Tree (empty set) (curret put code) 0 (umber of termal odes Tree) whle (ot-ed-of-data) do f (c = y <x, f x, y> = ode Tree) f x f x + a x call Iterchagg algorthm else + ew ode Tree Tree ode *** ode <a,, path from root to ode > edf + eddo ed *** Whe the code of ew character arrves, the character a must be trasmtted together wth ts code. Theorem The ecodg tree geerated by the proposed ecodg algorthm satsfes the code legth property. Proof: From the lemma, t s clear that the tree satsfes the code legth property. For a example of the ole ecodg ad decodg algorthms, Fgure 5 llustrates the bary tree before ad after ecodg the character A. It s see that the frequecy of A s creased ad t s greater tha the frequecy of D but the code legth of A s loger tha the code legth of D (.e., the tree does ot satsfy the property). After sedg code of A, 000, the tree must be adusted by terchagg code of A ad D. Now, the code of A s chaged to 0. By the same way, the decodg tree must be adusted after recevg the code of A, 000, order to mata the property. Fgure 5. The frequecy of put A s creased ad the property does ot preserved, the the tree s adusted (.e., code terchagg betwee A ad D.)

6 4. Expermetal results Some expermets o Eglsh alphabet are studed ths secto. Moreover, the results obtaed from our algorthm ad the results from the classcal Huffma algorthm are compared wth the etropy of the text. The geeratg fucto used the expermets s defed as follow: f ( 0) = f () = f () = 0 f ( L) = log, for L 3, where s the umber of characters ( = 6). I our expermetato, Eglsh text fles (fle 5) are used to be the data tests. Each fle cotas more tha 000 characters where the probablty dstrbuto of all characters s llustrated by Fgure 6. Ths s also smlar to the study []. Table shows the average code legth computed from our algorthm, Huffma algorthm ad the etropy of each fle. Percetage A D G J M P S V Y Alphabet Fgure 6. Average of character s frequecy related wth Table. Average code legth Fle: Our algorthm Huffma Etropy Avg Table. The comparso result betwee our algorthm ad Huffma algorthm. The results of average code legth from our algorthm are close to the average code legth of Huffma algorthm. The average sze usg our algorthm s oly.038 tmes of the etropy whle the optmal soluto s.0 tmes seeg Table. Avg/Etropy Our algorthm Huffma Table. Etropy comparso o our algorthm. 5. Cocluso We preset a ovel ole ecodg algorthm usg a fte bary tree geerated by a recursve replacemet method to solve the ecodg problem. Our bary tree s geerated by a costat fucto. Theoretcal result demostrates that our tree also satsfes the code legth property of the optmal bary tree. Expermetato has bee performed o Eglsh alphabet fles show that the average code legth s close to the oe geerated from Huffma algorthm. Moreover, t s also obtaed that the average code legth s approxmated.038 tmes of the etropy of the texts. Oe obvous drecto for future should be studed s the relatoshp betwee tree structures ad ther approprate probablty dstrbutos. 6. Refereces [] D.A. Huffma. A method for the costructo of mmum redudacy codes,. I Proc. IRE 40, volume 0, pages 098-0, September 95. [] H. Beer ad F. Pper, Cpher Systems, Wely- Iterscece, 98. [3] K.K. Ma ad M.J. Gol, Algorthms for Ifte Huffma-Codes*, I Proceedgs of the Assocato for Computer Machery 004. [4] M.G.G. Ladlaw, The Costructo of Codes for Ifte Sets, I Proceedgs of SAICSIT 004, pp [5] M. Gol, C. Keyo, ad N. Youg, Huffma Codg wth Uequal Letter Costs, Proceedg of the 34 th ACM Symposum o Theory of Computg (STOC00), (May 00) [6] M. Gol ad G. Rote, A dyamc programmg algorthm for costructg optmal prefx-free codes for uequal letter costs, IEEE Tras. Iform. Theory, vol. 44, pp , Sept [7] M. Gol ad H. Na, Optmal prefx-free codes that ed a specfed patter ad smlar problems: the uform probablty case (Exteded Abstract)*, IEEE Tras. Iform. Theory, 00. [8] M. Gol ad N. Youg, Prefx Codes: Equprobable words, uequal letter costs, SIAM Joural o Computg, 5(6):8-9, December 996. [9] S.W. Golomb, Ru Legth Ecodgs, IEEE Trasactos o Iformato Theory, IT- (July 966) pp [0] S.L. Cha ad M. Gol, A Dyamc Programmg Algorthm for Costructo Optmal -eded Bary Prefx-Free Codes, IEEE Trasactos o Iformato Theory, 46(4) (July 000) pp