IMPROVING SAS DATA ACCESS - AN EVOLUTIONARY EXPERIENCE

Transcription

1 Applicaions Developmen IMPROVING SAS DATA ACCESS - AN EVOLUTIONARY EXPERIENCE (From Inpu Saemens o Daa Forms) Bernd E. Imken, Paened Medicine Prices Review Board, Governmen of Canada, Oawa, Canada Absrac: During a recen SAS Users Group presenaion in Oawa he audience was asked o idenify how hey inpu/edi/view daa. 50% said hey use he Daa sep and Proc Prin, anoher 45% said hey used Proc FSEDIT, FSVIEW or VIEWTABLE, 5% said hey developed cusom AF screens. Ou of his 5%, only one user said ha hey have experimened wih Daa Forms and Daa Tables. The purpose of his paper is o idenify he srenghs and weaknesses of each approach and in paricular o show why organizaions such as he Paened Medicine Prices Review Board (PMPRB) benefi so significanly by using he more recenly developed SAS faciliies for user ineracion. This paper describes how he Board evolved over he las 10 years. You may find ha your organizaion is currenly a some poin on his same developmen ime line. - deails mus include prices for hospials, wholesales and pharmacies and he associaed price currency. Oher Daa - Includes exchange raes on a monhly basis for all counries being moniored. - Consumer Price Index daa including forecased and acual values. The daa base currenly conains over 8 million observaions. Each daa se is fully indexed o improve Inroducion The PMPRB was esablished as a judicial agency by he Canadian Governmen o ensure ha prices of paened medicines are no excessive. Nearly $4 Billion of paened medicines are sold each year in Canada. There are, approximaely 70 manufacures markeing nearly 1,000 paened medicines wihin Canada. The PMPRB SAS Daa Base Legislaion in Canada requires all manufacures o repor he following o he PMPRB: Figure 1 Logical View of he PMPRB Daa Base performance. You may access any par of he daa base eiher wih a screen or a repor in less han 2 seconds. Produc Daa: - generic name, brand name - dae produc inroduced on he marke - paen number, paen daes, paen holders - manufacure name, address - oher daa such as, usage, ingrediens ec. Canadian Sales Daa: - deails mus include package size, price per package, and quaniy sold - sales broken down by geography (e.g. Province) - cusomers, including, hospials, wholesalers and pharmacies Inernaional Price Daa Ex-facory prices for Germany, Sweden, UK, Ialy, France, Swizerland and he USA The PMPRB Nework Configuraion Figure 2 Simplified View of he PMPRB Nework

2 Applicaions Developmen Figure 2 displays he PMPRB nework configuraion. The Board runs boh Novell and NT servers. All SAS processing is on he NT server (Penium Pro 200). This server runs SAS 6.12 (Server Version) and provides he users wih boh SAS/Share and SAS /Connec Sofware. Each user has a Penium 233 wih 64 Meg Ram. The Original PMPRB Approach (1987) The PMPRB originally accessed daa by programming simple daa seps and repors. An example is displayed below. ec. The search was quickly begun for an alernae soluion. The Middle Era ( ) During his period of ime he quaniy of daa was rapidly increasing. The old approach became increasingly slower. The lack of indexing (version 6.04 and before) caused such poor performance ha a search for oher daa base soluions was commenced, in 1992, using an exernal consuling agency. A compeiion was held beween SAS and oher vendors such as Paradox, DBASE, Access ec. Lae in 1992, he programmers devised a mehod of creaing indexed daa ses conaining he KEY and only he observaion number. These files could be searched easily and unique observaion numbers could be rerieved quickly, which could hen be passed on o he daa se wih he POINTER= opion. The performance problem was resolved. In 1993, SAS was officially declared as he winner of he compeiion and he developmen eam was requesed o coninue developmen using SAS as he ool for i s mission criical sysems. Nex, o improve user inerface programmers suggesed he acquisiion of SAS/FSP o faciliae he design of screens and menus. If he daa se already exised hen: Figure 3 The use of he Inpu Saemen and Proc Prin Tweny percen of he daa received a he PMPRB was in he form of Lous\Excel spreadshees. The PMPRB quickly adoped he INFILE saemen, o impor ASCII files, and, adoped Proc DBF o read Lous files. During he firs wo years of operaions hese echniques worked well. Daa inpu personnel repored direcly o he SAS programers, since direc ineracion was frequenly required. The approach proved o be an expensive use programming resources. PROC FSEDIT daa=sasuser.sales screen=sasuser.salesscr ; RUN; If he daa se did no exis and he developer waned o creae a new one, hen he DATA= was replaced wih NEW=. The process could no be much simpler. SAS no only produced defauls screens, bu also allowed you o do ADDs, FINDs, WHERE, NEXT SCREEN ec. by simply making clickable selecions from he PMENU bar. A he ime faciliies as simple as his did no exis in he oher daa base producs such as DBASE or Paradox. The programmer soon developed many kinds of cusomized screens - i was simply a maer of yping MOD on Command line o access he faciliies o cusomize a defaul screen. Oher advanages and disadvanages also came o ligh: Advanages: 8 Jus BASE SAS required 8 Inexpensive Disadvanages ; Use of delimiers (e.g. INFILE esin delimier= ; ) ; No daa verificaion ; Spacing imporan ; Muliple seps ; No user friendly, mus know SAS Using his approach, he PMPRB found ha here were many inheren problems which arose resuling in incorrec keypunching, losing daa records, raining inpu personnel Figure 4 Using he Defaul FSEDIT Screen

3 Applicaions Developmen Screen modificaions allowed he developers o pain he screen wih field descripions, iles ec., making i much easier for users o adop. Now users only had o hi TAB or ENTER o navigae from one field o anoher, wihou having o worry abou columns, delimiers or he oher resricions hey had wih he INPUT saemen used previously. (Noe: The daa se mus be opened firs as DSID, his can be done using he SCL OPEN funcion, corresponding observaions may hen be jumped o using eiher WHERE or FETCHOBS, or any number of oher SCL funcions) Large quaniies of daa required developing a dual enry sysem o improve he accuracy. A FSVIEW screen allowed each daa enry person he abiliy of enering one observaion per row, jus like in a normal spreadshee. Once he A daa se was enered, he corresponding B se was enered by anoher individual. When boh were done an auomaic record by record comparison was performed idenifying inconsisencies even a a field level. Having such sringen qualiy conrol is a necessiy since he daa could in he fuure be used as evidence in a judiciary ruling. The use of FSEDIT/FSVIEW especially in a SAS/AF environmen proved o have many more advanages, bu, also, we became aware of cerain disadvanages. Figure 5 Capabiliies of MOD The procedure also allowed he developers o specify many opions for each inpu field, including field color, proeced fields, non-display fields, ec. This proved o be useful o he developers, reducing field validaion requiremens significanly. A grea deal of he iniial checking code could now be easily eliminaed. In addiion, cusomized screen processing could easily be added o each screen such ha calculaions could be performed in fron of he users each ime he ener buon was pressed. Also background colors could be se, opions o ADD or DUPLICATE could be se, and search/sring parameers could be idenified. Nex, he programmers began using cusomized MENU selecions. Proc PMENU proved o be powerful and ye easy o use, giving he developers far more conrol wihin he applicaion. Advanages: 8 Powerful 8 Reasonable error checking, min max ec. 8 Ease of use for daa inpu 8 Good color conrol on screen 8 Works well in SHARE mode Disadvanages ; Characer based screen does no permi graphics ; Images no allowed ; No push buons, radio buons ec. ; No up o dae wih compeiion (ACCESS ec) ; Lile conrol over record locking ; Canno easily blend full screen look wih FSView (spreadshee appearance) Below can be seen an acual FSEDIT applicaion develop using FSEDIT a he Board. The firs screen shows some of he basic daa colleced on each medicine. In 1994, SAS/AF was acquired allowing he programmers o develop cusom MENU screens, as well as screens for HELP and LIST. PROGRAM screens produced in SAS/AF proved o be even more powerful iniially. The sysem could now be designed for an end user who had virually no knowledge of SAS or SAS coding. The inpu personnel were now moved o wihin he operaional unis, much closer o he end users. This allowed ime o be freed up for he developmen eam. One oher significan breakhrough of his ime was discovering ha he screen developed in FSEDIT could also be used o display informaion from more han one daa se using a number of simple SCL saemens. You can add variables from oher daa ses o he screen by creaing a new screen variable rade = gevarc(dsid,varnum(dsid,'rade')); Figure 6 One of he major PMPRB FSEDIT Screens This screen shows only daa for Jan97 o Dec98. Subsequen screens show, in increasing deail, daa going

4 Applicaions Developmen back o In oal here are 6 screens conaining nearly 3000 SAS variables. Noe he users can navigae hrough he screens using he cusomized MENU. This screen had nearly 4,000 lines of SCL code allowing online updae. A full REFRESH could ake place in less han 2 seconds. Alhough he users were very pleased wih his design, he developmen eam found i onerous o consanly have o add variables o he screen o accommodae daa base growh. This was caused by hisorical records having o be mapped o SAS ARRAYS for presenaion on he screen. All of his was required because scroll down faciliies did no exis for FSEDIT screens - nor could one easily populae a FSVIEW screen over an FSEDIT screen. As a resul he screen became more and more difficul o mainain. An alernae soluion was required. The Inermediae Sage Beginning wih SAS Version 6.10 and coninuing wih SAS Version 6.11 he PMPRB noiced changes which were happening indicaing a move o a graphically based environmen. O Figure 8 Moving o Viewable Figure 9 Defaul DaaForm bviously SAS was working exensively o allow graphical user inerfaces which were compeiive o Microsof Access and oher relaional daa base producs curren on he marke a he ime. The Curren Period ( ) The PMPRB sared developing graphical objec saring in 1997 wih SAS Version Firs he developmen eam learned ha hey could develop menus far beer han hose allowed by PROGRAM enries by using SAS/AF FRAMES. This echnique proved o be very easy o implemen and soon all MENUs had disappeared in favor of icons ec. Nex he eam discovered ha he graphical inerface provided by a FRAME could also be applied o PROGRAM enries. The users much preferred o use radio selecion boxes, lis boxes, inpu objecs ec. Conrol over muli-sized fons allowed he developer o bring emphasis o various secions of he screen. I was a please o develop FRAMES using CONTROL BOXES, IMAGE TEXT ec. - he variey a ha ime seemed almos endless. Figure 7 Evoluionary changes The firs sign was he inroducion of FRAMES in SAS/AF. This quickly ook over from PROGRAMS and MENUS. I was now possible o develop radio boxes, lis boxes ec.. I was also he firs sep owards OBJECT ORIENTED programming. Evidence appeared in oher areas in SAS developmen which demonsraed ha heir was a concered effor by SAS o conform more o he new graphical echniques. One of he firs pieces of evidence was he move o VIEWTABLE. Immediaely one could see he use of buons, supporing muli-line labels raher han he characer based profile of FSVIEW. Also when one clicked o view he daa one observaion a a ime hey saw he firs Daa Forms being presened. Wihin a shor period of ime mos PROGRAM enries had also been replaced wih FRAME enries, however he FSEDIT screens remained. The developmen eam discovered ha developing FSEDIT screens was sill faser han having o develop a new FRAME were every objec had o be idenified separaely. AF FRAMES had o be linked wih a daa se using more complex SCL code - each screen field was reaed as a separae objec. One could no navigae up or down hrough a daa se wihou having o program SCL code. AF frames were a grea subsiue for PROGRAMS bu could no easily replace he speed, ease of developmen and defaul characerisics of FSEDIT. Finally, wih he inroducion of SAS Version 6.10 his problem appeared o be saring o be resolved wih Daa Forms (Experimenal in 6.10) and Daa Tables as objec wihin he SAS/AF environmen. The developmen eam eagerly sared o conver i s mos elaborae FSEDIT screen o DATA FORM lae in Early developmen proved o be difficul especially for a produc sill in he Experimenal sage. Documenaion was scarce a he ime and he Board found iself relying heavily on SAS Insiue in Cary for addiional suppor. Some of hese problems were gradually resolved as SAS Version 6.12 was inroduced. Finally now Daa Tables can be displayed on he same screen as a Daa Form and linked accordingly.

5 Applicaions Developmen The screen displayed a he op of he nex page, shows he muli-page Daa Form which was developed o service he applicaion. (Figure10) The DATA FORM shown has four pages, selecable by clicking on he fold a he op of he page. The DATA FORM iself is shown posiioned on a TAB OBJECT newly available wihin SAS/AF. Oher pages on he daa form use radio buons and check boxes o improve user accessibiliy. Boh Daa ables are independenly scrollable wih heir own verical scrollbars. The icons on he righ permi he user o obain a prinou, a graphic presenaion, and in addiion permis furher drill down, since he daa represened in he Daa Tables is of a summary naure. Wha have been he major improvemens for he PMPRB? Easier o navigae for users - Scroll bars allow users improved access Scrollable objecs reduce mainenance - In he previous FSEDIT version of he screens, he developers would have o add new screen variables for new yearly periods Improved drill-down capabiliy - can use buons and icons for represenaions A his poin in ime he PMPRB is sill makes boh he FSEDIT and Daa Form echnologies available. Each has i s own meris. Conclusion Each of hese inerfaces discussed in his paper has i s own place in he SAS world - i depends oally on wha he user is rying o accomplish, and how much experience and ime is available o he user. Figure 10 The new DaaForm appraoch being used a he PMPRB Above he TAB objec you will also noice oher fields which are redundan o he daa forms and ables locaed on he TAB objec. Noice he use of: differen fon sizes and syles labels associaed direcly wih a daa region icons o prin ou resuls for he enire screen on one page icons o allow he user o navigae wih he click of he mouse. When he user presses he hird TAB a screen is displayed showing wo Daa Tables which correspond or are linked o he Daa Form which was on he firs TAB. BUT WHY GO THROUGH ALL THIS EXTRA WORK!!! Easier for users o operae Reduced SAS end user raining required Improved conrol over wha he user sees and does Beer conrol over developmen Keep up wih ools like Oracle Access ec. bu everyhing sill in one environmen Excellen performance resuls In he graphic below I have ried o summarize he relaive aspecs of using one approach versus he oher. A small poll was conduced of he various SAS developers and users o see how hey posiioned each echnique. The common commens which were rendered wih he poll are included below he chars. Figure 12 Figure 11 View of period & Yearly sales on anoher Tab COST: Mos users did no know abou SAS sofware coss. Developers recognized ha only

6 6 Many 6 4" 6 Useful 6 For 6 Marge 6 Derek Applicaions Developmen Base SAS was required for he INFILE approach. Using FSEDIT required he addiional purchase of SAS/FSP. Using DATA FORM required purchasing SAS/AF, as well as modules such as SAS/GRAPH if graphics were o be used. TIME TO DEVELOP: The INFILE saemen is frequenly one of he firs hings learned. FSEDIT required he developer a shor ime o become familiar bu reasonably sophisicaed screens could be developed in 1-2 hours. Daa Form developers indicaed hey had o learn far more abou SCL, objec programming ec. This was no recommended as an approach unless he developer had a srong background in AF and SCL. CONTROL: End users recognized ha he real ime feedback given o verificaion and conrol was valuable especially using FSEDIT and DATA FORM. Developers ended o give DATA FORMs more weigh as a conrol mechanism because hey could use objec such as check boxes and radio buons ec. EASE OF USE: Mos commens here were received from end users. Daa Form performed bes - easier o use because of he graphic inerface. Users like differing fon sizes, and graphical elemens such as buons, boxes ec. They also indicaed i was easier o navigae around - like having a FSEDIT screen and a SPREADSHEET all on he screen a one ime. For hose of you who have no experimened wih FSEDIT and DATA FORMS I srongly advise you o give hem a ry. Once managemen and he end users see sysems developed/cusomized in his way heir level of saisfacion increases subsanially. References SAS on he WEB aricles exis on such hings as conrolling ables, muli-line labels ec. Objec-Orienaed Programming wih he SAS Sysem hick manual publish las year by SAS for all FRAME developmen work experience SCL users - very powerful 23 Annual SUGI Proceeding Scerbo and Alan Wilson, Tables and Views and Forms: Oh My! Morgan and Mike Province, A Bag of FSEDIT Tricks Soluion@Work CD Rom from SAS Acknowledgemen I would like o give special hanks o: APPEARANCE: Boh end users and developers liked like he look of he DATA FORM screens I looks beer. IT looks more like wha we are used o in Windows 95". DATA FORM was veiwed as being more inuiive. The developers however had very srong opinions on he amoun of SAS raining required for developmen. Inroducory developers could handle INFILE easily. FSEDIT was used by eiher advanced inroducory programmers or inermediae one, DATA FORMs were only used by he experienced programmers, frequenly also wih advanced raining in AF, SCL ec. Ann Carpener Rocke Lynn P. Leone from SAS Insiue, Cary, Norh Carolina,for he assisance and advice hey have given me in developing Daa Forms & Tables for he PMPRB. Conac Informaion Bernd E. Imken Chief, Informaion Sysems Division Paened Medicine Prices Review Board 333 Laurier Avenue Wes Oawa, Onario K1P 1C1 Phone Fax bimken@pmprb-cepmb.gc.ca Figure 13