Issues and methods for practical distributed data processing applications-i

Transcription

1 Issues and mehods for pracical disribued daa processing applicaions-i by MAURICE BLACKMAN Arhur Andersen & Co. Houson, Texas and HUGH RYAN Arhur Andersen & Co. Chicago, Illinois ABSTRACT This paper presens mehods for one of wo key aciviies in he creaion of pracical disribued daa processing (DDP) sysems for business compuing. The firs aciviy, discussed in his paper, is o selec a configuraion of hardware and sofware o suppor he implemenaion of a muliapplicaion plan. The second, discussed in a companion paper, is o selec daa disribuion and manipulaion approaches for one applicaion wihin he limis se by he resuls of he firs aciviy. To esablish a jusificaion for he mehods, he paper selecs a definiion of DDP, discusses he alernaives o DDP for reaching he objecives of he enerprise, and idenifies he design issues o be solved or avoided in a pracical sysem for a commercial esablishmen. 785

2

3 Issues and Mehods for Pracical DDP Applicaions-I 787 INTRODUCTION Disribued daa processing (DDP) seems o have been seleced as he compuing indusry's newes panacea for business compuing. DDP has very aracive characerisics, accompanied by some significan echnical challenges for he pioneer. This siuaion would be accepable if he decision o use disribued processing were always made by compuing professionals. Unforunaely, he specacular improvemens in he price/performance raings of small compuers has led o a significan number of aricles in he business press 1-4 ha promoe he use of small compuers in DDP configuraions. As a consequence, he iniiaive in using DDP is aken by he chief execuive, and he MIS direcor is frequenly faced wih a direcive o "implemen disribued processing." Principal Tasks In our experience and ha of our colleagues, here are wo aciviies crucial o a successful insallaion. The firs is he preparaion of a plan or sraegy for he disposiion of funcions, daa, and hardware and he idenificaion of he necessary supporing sofware. The second is he disposiion of daa and he associaed choice of approaches o he iming and synchronizaion of processing for each specific applicaion wihin he consrains se by he choices of he prior plan. The wo aciviies fi ino he conex of a complee mehod for sysems developmen. 5 They draw upon daa developed in earlier asks and provide inpu o asks ha follow. The wo asks are discussed because hey resul in key design decisions ha conrol he impac of logical problems and echnical gaps ha we see in he presen environmen. Each of he wo papers discusses one of he wo aciviies. The firs is called sraegy selecion and he second daa design. A mehod for collecing, organizing, and analyzing relevan daa for each is presened. The mehods, inended o be responsive o he curren sae of he ar in DDP producs, are aimed a minimizing presen risk for he commercial user of compuing. As logical challenges are overcome and more sophisicaed producs are inroduced, he mehods will evolve. Issues There are wo groups of issues ha can make DDP implemenaion a formidable ask: logical puzzles ha spring from he very naure of allocaing processing beween several processors 6 and gaps in he available echnology7 ha make some approaches more cosly han ohers. The second group of issues will lose heir significance as he gaps are filled, bu he firs group may persis. In any even, he MIS direcor needs o adop an approach o DDP ha avoids boh groups in order o succeed in creaing a DDP environmen wihin consrains of cos and ime. Definiion There are many inerpreaions of he erm disribued daa processing. 6,8,9 To place his conribuion in conex, we define DDP as follows: A daa processing echnique ha provides access o compuing power for end users by means of muliple processors ineracing hrough he planned exchange of daa over communicaion lines. The definiion is explici, because DDP sysems have many sources of variaion: The processing capabiliy of he devices a he remoe locaion The exen o which daa are disribued o erminal locaions The discipline used o communicae The number of erminal locaions The frequency of he processing a erminal and cenral locaions The frequency wih which he processes are synchronized The compaibiliy of he devices used for implemenaion Our definiion varies from ohers in cerain key respecs. I is more resricive han some in implying sorage capabiliy and hence daa residence a each node. I is less resricive in no being limied o online or real-ime ineracion beween processors and in no implying any dynamic allocaion of asks. We wish o sress he definiion quesion for wo reasons. Firs, a clear definiion helps he DP professional explain how a direcive o "use DDP" has been inerpreed. Second, he definiion can be used as a emplae or profile for comparison wih a proposed configuraion. A conforming configuraion may suffer from some of.he logical problems or gaps in echnology ha exis. The implemener of a conforming configuraion will be aler o hese issues and ake seps o avoid hem. STRATEGY SELECTION User Objecives Our earlier definiion hined a one of he key objecives of he organizaion examining DDP: o provide access o com-

4 788 Naional Compuer Conference, 1982 puing power for he end user. Disribued means ha some funcion of he daa processing deparmen is reassigned o a person closer o he mainsream of he business. There are many means by which ha funcion disribuion may be achieved, including online erminals and decenralized compuers. The many variaions of DDP fill he middle ground. I is he qualiy of direc access, independen of he daa processing deparmen, ha is he principal objecive of he requeser or commissioner of D D P. We herefore ake he objecive of sraegy selecion o be he configuraion of a combinaion of hardware and sofware componens ha delivers access while opimizing an objecive funcion addressing performance, developmen cos, operaing cos, delivery dae, and ease of use. General Approach The sraegy selecion aciviy can be decomposed ino a sequence of asks, as follows: Idenificaion of applicaions Idenificaion of disribued funcions Analysis and selecion of a configuraion of hardware and daa communicaions faciliies Selecion of sandards for daa disribuion Developmen of a caalogue of sofware componens Selecion of producs raegy selecion occurs during a planning phase. The resuls mus sand for five, en, or even more years. The seleced sraegy mus be responsive o requiremens ha ilre imperfecly defined and likely o change. The selecion approach mus herefore rely heavily on he experience, knowledge, and judgmen of he selecors. The seps here presened are precise in heir idenificaion and sequence bu will be subjecive in heir execuion. Applicaion Idenificaion If a sraegy is o be durable, he selecors mus have a good appreciaion of he applicaions o be implemened. From he poin of view of he user, compuing services are more convenien if hey are accessible hrough a single work saion. From he poin of view of he enerprise, produc acquisiion is more economical if negoiaed for he long erm. For such reasons, his firs ask idenifies as many poenial uses for he configuraion as possible. Useful ools for his purpose are models of general processing and daa srucure requiremens specific o he indusry o which he enerprise belongs. These chars idenify he componens of a oal informaion sysem o serve all he principal business funcions for ransacion processing, reporing, and operaional and sraegic planning. The componens are conneced o show how informaion flows beween hem. The chars are personalized o an indusry o show is unique requiremens and o help communicae wih he execuives who mus deermine which sysems are likely candidaes for auomaion. The chars form a useful checklis o help ensure ha all requiremens are idenified. Characerisics of he enerprise discovered by inerview and research, such as size, growh, and markes sough, help o develop a plan wih a sequence and ime frame ha reflecs he enerprise's prioriies and resources. (See Figure 1.) Funcion Disribuion Candidae funcions capable of disribuion a his ime are he generic funcions of a daa processing deparmen raher han he specific funcions of single applicaions. This ask ses limis on he maximum disribuion in order o idenify needed sofware suppor in he configuraion and o esablish overall quaniies and capaciy of equipmen. Table I gives examples of generic funcions in hree groups: operaions, developmen, and managemen. TABLE I-Generic daa processing funcions Operaions Developmen Managemen Inquiry Idenificaion Nework design Daa enry Jusificaion Capaciy plan Validaion Daa design Equipmen acquisiion Correcion Screen/repor Sofware selecion File analysis Process design Sandards Prining Programming Nework operaion Posing Tesing Performance repor Conversion Daa adminisraion Decisions abou he dispersion of funcions are responsive o characerisics of he enerprise and are made according o simple guidelines. Operaions In general, operaing funcions are prime candidaes and are considered firs. If operaing funcions are no dispersed, he dispersion of any oher funcions is very unlikely. Operaions funcions are dispersed when he local business uni has auonomy in is own day-o-day operaions, when is opera-. ions are on differen business cycles from hose of is paren or headquarers, and when is operaions efficiency or effeciveness can be improved by rapid access o he resuls of compuing. The only common excepion would be a lack of he necessary skills or working condiions o operae compuer peripherals. Developmen Developmen funcions are also good candidaes, for he reasons commonly cied for any form of end user paricipaion in developmen: knowledge of he business, responsiveness of he design o user needs, and enhanced accepance of he sysem. In addiion, if he sies considered for funcion dispersion are widely separaed, cenralized developmen becomes difficul as a resul of communicaions difficulies. Dispersed developmen is almos a necessiy if he managemen syle of he enerprise favors local auonomy as shown by a

5 INFORMATION REQUIREMENTS PLANNING CHART PROPERTY AND CASUALTY BUSINESS PROCESSES IIMIIET II'IIDIH/CT ITRATEGY POLICY ADMINISTRATION STRATEGY fuiid ffjallalaleit BlRATHY STRATi PI.MII o ENVIRONMENTAL FORECAST o MARKET IDENTIFICATION o NEW PRODUCT PLANNING GROWTH PROJECTIONS PRICING POLICIES POLICY PROCESSING ALTERNATIVES LONG RANGE CAPACITY PlANNING o MEMBER ADMINISTRATION POLICIES SERVICES LEVEL POLICIES FUND MANAGEMENT POLICY o ASSff UTILIZATION AND MIX INVESTMENT STRATEGY TIlAllIACTION PIIOCEIIINI OPERA' PI.MIIIIII COIITIIOL IIEPCIIIMI f I 1 CDIII'fTTTIVE DATACCIUECTION o COMPffITIVE PRICES o PROMOTIONAL ACTIVITY o MARKff SHARE PIIDDUCT IIIAIIET I'INIIIIII o MARKET DEFINITION o PRODUCT DEFINITION o NEW PRODUCT RElfASE PLANS SALES FORECASTING o PROMOTION PlANNING o REGULATORY APPROVALS UlEI/lAIIIETIIII ElCfEIIIEIICE o MARKET PENffRATION o GROSS SElL o PROMOTION PERFQfMANCE o COMMISSION o AGENCY PERFQfMANCE o COMPffITIVE ANALYSIS o PROMOTIONAL ACTIVITY 1 POLICY ADMINISTIIATION PlANNIN8 TIME/SERVICE STANDARDS PRODUCTIVITY STANDARDS DATA PRIVACY REQUIREMENTS CLAIM COST STANDARDS CLAIM FREQUENCY STANDARDS FIELD OFFICE PlANNING PERSONNEL LEVEL PlANNING -'- PIIODIICT EXI' IIISICE UNIIElIWIUTIII8 CONTIIOL POlICYHOLDEIl SEIIVICE CDNTIIOL TIME/SERVICE PERFORMANCE o PRODUCT PERFORMANCE o CLAIM FREQUENCY ANALYSIS CQMPlAINT ANALYSIS ANALYSIS o LEGAL/REGULATORY o POlICYHQLDER INQUIRY ANALYSIS o legal I REGULATORY COMPliANCE o IN FORCE REPORTING COMPliANCE o IN FORCE ANALYSIS CANCELLATION ANALYSIS o IN FORCE ANALYSIS o EXCEPTION ANALYSIS o CROSS SELL PRODUCTIVITY o EXCEPTION ANALYSIS o PROCESSING ACCURACY UNDERWRITING EXPENSE ANALYSIS I f f f T 1 I I POlICY IOUCITATION ""EMIUM MAlNTEJIAIICE MAINTENAIICE ol'ulicy ISSUE o PREMIUM RECEIVABlE o SOLICIT A TlON o ENDORSEMENTS ACCOUNTING o SOLICITATION orating o PAYMENT FEE ACCOUNTING RESPONSE o MVRIINSPECTION REPORTS o BILLING STATEMENTS o POTENTIAL o POLICY RENEWAL o DIVIDEND ACCOUNTING CUSTOMER o PREMIUM ACCOUNTING o COMBINED STATEMENTS MAINTENANCE o REINSURANCE o REFUNDS o ADVERTISING o QUOTE o SSA INTEREST o WORKLOAD REPORTING o WORKLOAD REPORTING CLAIM. CONTIIDL TIME/SERVICE PERFORMANCE o A TIORNEY CONTROL o ADJUSTQR CONTROL o COMPLAINT ANALYSIS o AVERAGE PAYMENT o PROCESSING ACCURACY o CLAIMS EXPENSE ANALYSIS f l CLAIM IAIIITBIAIICE o CLAIM ACCOUNTING CLAIM PAYMENTS o CLAIM EXPENSE PAYMENTS SALVAGE o SUBROGATION o REINSURANCE o LITIGATION o WORKlOAD REPORTING J I fuiid IIAIIMEIIEIT I'INIIIIII PORTFOLIO PlANNING o LIQUIDITY PlANNING o INVESTMENT PlAIINING FUlDlIIMIAIaIfIIT,----- o FUND PERFQfMAIfCE o RISK EVALUATION o PORTFOLIO ANALYSIS POITfOUO/.VElTIIBIT. o PORTFOlIO ACCOUNT o REVENUE RECEIVABlE o MATURITY SCHEDULING o FUI.D MAINTENANCE FUND ACCOUNTING TO: GENERAL LE06ER TQ: STATISTICAL REPORTING f/l f/l s:: ('1) f/l p:> ::s 0.. ('1) S- O 0.. f/l 0' ""'I p:> n (:;' j j '""C > "0 'E.. (:;' 0' ::s 1 Figure I-Indusry sysem schemaic-deail -...) 00 \0

6 790 Naional Compuer Conference, 1982 profi cener policy or if he producs, markes, and businesses of he unis are differen. As wih operaions, he principal impedimen o dispersion will be a lack of available skills. I should be noed ha he dispersion choices being made here do no imply ha all applicaions will be equally dispersed. The objecive is o deermine he degree of dispersion under he mos favorable condiions so as o uncover sofware and equipmen needs. Managemen Managemen funcions are leas likely o be dispersed, because he resources being managed are hose shared by all users. Their proper selecion and adminisraion requires ha he needs of all groups be blended and ha faciliies capable of working ogeher be obained. Condiions for he dispersion of managemen funcions are he dispersion and local conrol of mos of he asses of he enerprise and he dispersion of oher service funcions such as accouning, personnel adminisraion, and research and developmen. Such an enerprise is really several disinc enerprises needing only occasional conac. Hardware and Daa Communicaions Funcion disribuion, alhough subjecive and judgemenal in is mehods, is performed early, because is oucome may limi he hardware opions considered. If only he simples operaing funcions are disribued, hen an online cenralized soluion is indicaed. On he oher hand, if exensive disribuion of funcions occurs, many sraegies across he dispersedaccess specrum are possible. This ask may be subdivided ino he following smaller asks: Selecing access syles and modes Idenifying access locaions Configuring peripherals a each locaion Deermining he processing/communicaions sraegy Configuring daa sorage Access modes Access syles and modes include choices of online or bach access echniques and such special requiremens as OCR, MICR, microfiche, or badge readers. The disribued work saions' complemen of devices can be deduced by examining he likely inpu and oupu media of each applicaion. Locaions The access locaions o be given direc service mus be reviewed for insances of very low volume no jusifying equipmen. Lack of volume may be a emporary condiion. Plans for nework expansion a a laer ime may be an imporan inpu o he more precise seps discussed laer in his secion. Peripheral configuraion The configuraion of peripherals a each sie is he firs precise sep. The differen ypes of peripherals need o be reaed separaely. Mehods for esablishing keyboard screen and prining requiremens o saisfy random and scheduled demand are well esablished from online sysem experience. 5 Requiremens for specialized peripherals mus be considered carefully when uni capaciy and cos are boh high. Those forms of peripheral are more usually cenralized for all bu he larges sies. Processing/communicaions sraegy We now come o he crux of he hardware quesion: should processing be cenralized or disribued? The analysis consiss of compuing one-ime and operaing coss for alernaive soluions and comparing hem. The variey of alernaivessimple online erminals wih a value-added nework, smar erminals wih a mulidrop leased line nework, inelligen erminals (sand-alone or clusered) wih dialed WATS lines, minicompuers and mainframes wih auo dial/auo answeris usually oo many. A echnique of sampling, say hree alernaives, followed by successive refinemen, is preferred. The cos analysis mus recognize ha increased dispersed inelligence reduces daa ransmission volume and allows he use of iow-ariff periods for ransmission if he remoe and cenral processing cycles can be decoupled. The analys mus also consider geographically inermediae dispersion. There may be a case for sharing processors beween several remoe sies. This is an example of refining a sraegy once a degree of disribuion has been deermined o be advanageous. Daa sorage The final sep is o esimae daa sorage requiremens if a sraegy of physical disribuion has been seleced. This sep is performed before more precise allocaion of daa wihin applicaions, since ha canno be done wih cerainy unil designing each in urn. A his ime we are concerned only wih gross needs. Daa sorage capaciy is needed for developmen as well as producion, for programs as well as daa, for sysem sofware as well as applicaions, for backup as well as mainline daa, and for inefficiencies of use. In shor, raher large esimaes are indicaed in preference o raher small. Daa Disribuion Sraegy The final sage is o consider wheher o place any limiaions on he applicaion designer so far as he complexiy of daa arrangemens is com:erned. Our principal arge is o decide wha forms of daa synchronizaion are allowed and wha forms, if any, of daa direcory are required. Our firs ask is o deermine wheher maser daa cenralizaion is indicaed. Under curren echnology consrains, daa needing o be up o dae for all locaions mus be cenral-

7 Issues and Mehods for Pracical DDP Applicaions-I 791 ized. Oher indicaors include any suggesing he need for full-funcion daabase managemen sysem sofware. Such sofware needs he capaciy and skills of a cenral sie. As a rule of humb for sraegic planning purposes, he designer should avoid daa disribuion configuraions ha require online posing wihin one commimen uni a more han one node. The designer should be wary even of online posing a a node oher han he one a which a ransacion originaes. Paper II explains he background for his guideline. 10 Sofware Configuraion Once he decisions abou funcion, hardware, and daa disribuion are made, he generic componens of sofware mus be idenified. The key componens are hose associaed wih he fac of disribuion: Daa communicaion, including he appropriae online or offline proocols Message-rouing logic for bringing daa ino he cener and puing i ou again, including disribuion lis inerpreaion logic, logical address o physical line mapping, alernae rouing, ec. Daa locaion logic in he case of spli daa files, including direcory mainenance Daa conversion logic o map daa characers, fields, records, files, and daabases from one forma o anoher Developmen aids in each node ype expeced o suppor local developmen Program and dicionary disribuion logic if one sie is developing on behalf of ohers Remoe job reques submission and accepance and many ohers. This lis shows how he funcion, daa, and hardware disribuion decisions have consequences in he complexiy of he environmen. To complee his ask wih any reliabiliy, he designer mus undersand he sofware componens necessary o he funcions of operaing, applicaion developmen, and resource monioring. Each of hese divides and subdivides unil a porfolio of componens is developed ha may be used as a checklis. Vendor Selecion A he end of he sraegy developmen, he designer has sufficien informaion o consider he producs necessary for creaing he arge environmen. This sep will have been in view hroughou he developmen of he sraegy, since limiaions in producs available are he reasons for some of he rules of humb suggesed. The principal componens o be obained will be he processing sysems of he differen nodes and he communicaions subsysem beween hem. By far he mos producive rule a his ime is o obain processors a all nodes from he same vendor so ha much of he poenial sofware complexiy can be subconraced o he vendor. Many vendors have defined and implemened nework archiecures of richness sufficien o suppor cooperaive working among muliple processors. The decision o use differen vendors will ineviably involve he enerprise in he definiion of is own archiecure and he sofware o implemen i. SUMMARY The objecive of his paper was o describe a sequence of seps for developing a DDP sraegy. The seps have been described only o show heir purpose. The maerial is offered o help idenify he componens of he sraegy and he dependencies beween hem. The resuling sraegy is a high-level plan adequae for cosing, for vendor selecion, and for commencing he developmen of any necessary sofware. In Paper II, we consider he deail of daa design for an applicaion wihin he consrains of a sraegy. REFERENCES 1. Brancaelli, J. "Office of he Fuure." Texas Flyer, 10 (1981), pp Van Rensselaer, C. "Cenralise? Decenralise? Disribue?" Daamaion, 25 (1979), pp Wihingon, F. G. "Coping wih Compuer Proliferaion." Harvard Business Review (May-June 1980), pp Buchanan, J. R., and Linowes, R. G. "Undersanding Disribued Daa Processing." Harvard Business Review (July-Augus 1980), pp "Mehod!1." Arhur Anderson & Co., Chicago, Kohler, W. H. "A Survey of Techniques for Synchronizaion and Recover in Decenralized Compuer Sysems." ACM Compuing Surveys, 13 (1981), pp Tozer, E. "Is Disribued Daa Base Ye Maure Enough To Consider?" 22nd GUIDE Conference, May Loren, H. "Disribued Processing: An Assessmen." IBM Sysems Journal, 18, pp Scherr, A. L. "Disribued Daa Processing." IBM Sysems Journal, 17, pp Blackman, M., and Ryan, H. "Issues and Mehods for Pracical Disribued Daa Processing Applicaions-II." AFIPS, Proceedings of he Naional Compuer Conference (Vol. 51), 1982.

8