Applications of uncertainty analysis applied to the conceptual design of space systems

Transcription

1 Application of uncertainty analyi applie to the conceptual eign of pace ytem Myle A. Walton * an Daniel E. Hating Maachuett Intitute of Technology, Cambrige, MA 1 Nomenclature U l_urv U l_nap U h_urv U T r w k Q Q D S D σ σx,y ρ C D C non_recur = utility of low latitue urvey miion = utility of low latitue naphot miion = utility of high latitue urvey miion = total Utility = return of an architecture (Total Utility/Lifecycle Cot) = invetment weighting for architecture coniere = rik averion coefficient = covariance matrix = ownie cale emi-variance covariance matrix = cale emi-variance = tanar eviation = covariance = correlation = cot of iverification = non-recurring cot of evelopment Abtract One of the mot ignificant challenge in conceptual eign i managing the traepace of potential architecture chooing which eign to purue aggreively, which to keep on the table an which to leave behin. Thi paper preent the application of a framework for managing a traepace of architecture not through traitional effectivene meaure like cot an performance, but intea through a quantitative analyi of the embee uncertainty in each potential pace ytem architecture. Cot an performance in thi approach remain central theme in eciion making, but * Reearch Aitant, MIT Department of Aeronautic an Atronautic, Member AIAA Profeor, MIT Department of Aeronautic an Atronautic, Fellow AIAA Maachuett Ave, Bl , Cambrige, MA

2 uncertainty erve a the focal lene to ientify potentially powerful combination of architecture to explore concurrently in further eign phae. Introuction Conitent with the complexitie of a pace ytem, conceptual eign i plague with uncertaintie from ource both ientifiable an conceale. It i the job of thoe involve in conceptual eign to wae through the uncertainty that efine the problem an arrive at eciion an architecture that, within the current level of available information, reflect the better alternative. It clear that in uncertain environment, optimality i omething of a myth. Thi of coure i why eign i part art in aition to part cience. However, the implitic aumption of certainty of conition, even at the embryonic tage of eign, can yiel etrimental concluion. Often intractable problem, ue in large part to uncertainty in the ytem an it the environment, are relegate to abtraction of the real problem that rely on the accuracy of current etimate. Thi paper lay the framework for a new way of looking at the proce of exploring potential pace ytem architecture through the len of uncertainty, that ha the potential to change the way people think about early conceptual eign an the election of eign to purue. Deciion criteria uch a cot, performance an cheule are the tanar when it come to eciion making in pace ytem eign. Thee meaure, quantifie uing anything from back of the envelope etimation to expert opinion to intene computation an moeling, typically erve a the bai of the information provie to the eciion maker. The mechanim to calculate information, like cot, cheule an performance, ha been taught in a number of book on the eign of pace ytem in aition to the inutrial practice exercie at each contractor an continue to be the ubject of a large boy of reearch. In contrat, metho of accounting for uncertainty in preiction in pace ytem eign have been far le publihe. No metho ha been preente, a of yet, that aggregate the type an ource of uncertainty that are typical of a pace ytem an emontrate an approach 2

3 to manage uch information. Thi paper preent uch an approach an goe further to evelop a framework in which to explore the implication of uncertainty in ifferent architecture. Figure 1 preent a conceptual eign flow with the incluion of the propoe uncertainty analyi framework. Lying between concept generation an concept election, the uncertainty analyi approach woul provie ueful information to the eciion maker in preparation for electing architecture to purue. Coinciing with the viion for uncertainty to be a central eciion criterion in the conceptual eign of pace ytem, o too mut the uncertainty analyi be a central component of the conceptual eign proce. The uncertainty analyi location, a ecribe, woul be early enough in conceptual eign to poitively influence eciion, while at the ame time it location woul be late enough, o that the problem bounarie are rawn an ource of uncertainty can be ientifie, aee an quantifie. Information about externalitie i collecte in the nee analyi an concept generation of. pace ytem. Uner the uncertainty analyi approach, further information on external ource of uncertainty woul alo be tappe. Thi paper preent the practical implementation of the uncertainty analyi approach. For a ecription of the theory of the approach, ee Ref There are three cae invetigate, a pace bae raar pace ytem, a pace bae broaban communication ytem, an a pace bae ionopheric mapping miion. Thee three cae repreent the three overarching egment of pace ytem, namely military, commercial an civil (cience) miion, a hown in Figure 2. Further, the technology an conceptual architecture in each of the architecture iffer ignificantly. Thee ifference provie complementary implementation cenario for the uncertainty analyi approach that provie the reaer with a broaer viion of how the approach coul work in practice. The firt ection in each cae ecribe the overall miion a well a the conceptual eign moel ecription. All cae were moele uing the generalize information network analyi (GINA) metho, a ecribe by Shaw 2. The next ection focue on quantifying the architectural uncertainty 3

4 embee in each architecture, while the thir ection ecribe the application of portfolio theory to the iniviual cae. Finally each cae i cloe with inight an concluion that each provie about the pecific miion a well a the uncertainty analyi approach. The primary purpoe of each cae i not to ecribe the iniviual miion an moeling approach of each in epth. For thi information, reference have been provie. Intea, it i to emontrate the applicability of the uncertainty analyi approach to the broa cla of problem that each cae repreent. TechSat 21 Miion an Moel Decription TechSat 21, hort for Technology Satellite of the 21t Century, i a program aime at puhing the bounarie on the current approach to atellite ytem evelopment. It novelty lie in concept at both the architectural, ytem, ubytem an component level. The mot obviou feature of the TechSat 21 architecture i the eparture from the traitional monolithic atellite eign of the Miltar an Defene Support Program atellite. Unlike thoe ytem, TechSat 21 employ collaborate cluter of atellite in what i hope to be a more flexible, extenible, better performing an le cotly architecture. Uing a cluter of formation flying atellite, a ynthetic aperture can be create whoe propertie for a variety of miion ranging from pace bae raar to groun moving target inication. Of coure becaue thi i a non-traitional architecture there i ignificant uncertainty aociate with many apect of the concept propoe. It therefore provie a goo example of the uncertainty analyi approach applie to a highly complex, high technology, an envelope-puhing problem. To conuct a ytem analyi of the potential architecture that coul be employe to accomplih the TechSat 21 miion, bounarie were etablihe a to what concept woul be evaluate. The ifferent architectural characteritic that were coniere are preente in Table 1. In the GINA terminology, thee characteritic are calle the eign vector an a combination of the ix eign variable contitute a eparate architecture. 4

5 GINA Moel The TechSat 21 GINA moel evelope in the MIT Space Sytem Lab wa eential to completing thi cae tuy. 3 The broa architectural concept for TechSat 21 conit of a et of collaborative, formation flying pacecraft in low earth orbit that coul perform multiple miion ranging from ynthetic aperture raar to groun moving target inication to ignal interception. The egmentation of the TechSat 21 GINA moel i preente in Figure 3. The initial moule of the imulation moel are the input of the Deign Vector, a previouly ecribe, an the Contant Vector. The Contant Vector repreent thoe variable that for the enumeration of the traepace are hel contant. By oing o, architecture can be equitably compare. Once the Deign Vector an the Contant Vector have been initialize, the imulation procee with the Contellation Moule. Thi moule prouce the orbital characteritic for the pace egment that make it poible to later ae the performance of the architecture. The Raar Moule quantifie the variou technical performance meaure in a raar context. Thee inclue: probability of etection, minimum etectable velocity of a groun target, an area earch rate. The Payloa Sizing moule ue the input of the Deign an Contant Vector to moel an appropriate payloa antenna for a given architecture. Uing the Payloa Moule output, the Satellite Bu Moule eign an appropriate configuration an ize all ubytem to atify the payloa requirement in term of power an ma, a well a other conition of the Deign an Contant Vector. Once the atellite an their payloa have been moele, the launch equence i etermine by the Launch Moule. The Operation Moule efine the operational requirement for the ytem in term of people, groun tation, etc. The final moule, the Sytem Moule, uing output from the previou moel a input, generate outcome meaure for each architecture, uch a total lifecycle cot a well a cot per function meaure. 3,4 5

6 Moel Reult The GINA moel wa evaluate for thouan of potential architecture an variou outcome meaure were generate to provie input to eciion maker on potential architecture to purue in further eign exercie. Thee meaure inclue performance meaure like: probability of etecting a given target, the availability of the ytem, minimum etection velocity, ignal to noie ratio, an area earch rate. Cot meaure are alo generate from the imulation incluing launch, eign an evelopment, operation an total lifecycle cot. Although all outcome meaure are of interet to the eciion maker, the primary performance eciion criteria choen wa probability of etection, while the primary cot eciion criteria i lifecycle cot. Figure 4 preent the moel reult for 3000 architecture in the TechSat 21 traepace. Each point in the chart repreent a ingle architecture whoe compoition i efine by a unique eign vector. Knowing the primary eciion criteria a Probability of Detection an Lifecycle Cot, the Pareto optimal front can be foun for the traepace by ientifying non-ominate architecture. A nonominate architecture i one whoe performance cannot be urpae without higher cot. Figure 5 preent the Pareto optimal front, a calculate by Jilla uing heuritic earch metho. 3 The Pareto optimal eign vector value are hown in Table 2. All the architecture in the table ha an altitue of 800km, 6 plane an 42 cluter of pacecraft each having 4 atellite. The reult preente above were mae uing eterminitic aumption an calculation, but what kin of uncertainty i aociate with each architecture election an what i an appropriate mean by which uncertainty can be manage an quantitative trae-off can be mae? By applying the uncertainty analyi approach, it i hown that there i a conierable amount of uncertainty aociate with each architecture, that it can be quantifie an that portfolio theory provie a central framework in which the uncertainty of the traepace can be manage. 6

7 Uncertainty Quantification The firt tep in quantifying embee architectural uncertainty i to boun the ource of uncertainty appropriately. The poible ource of uncertainty that affect the architecture outcome mut firt be ientifie an the eigner mut ecie which will be inclue in the analyi. There are two primary reaon to not inclue all ource of uncertainty in practice. The firt i that the analyi woul quickly become intractable an the econ reaon i that there are ome ource of uncertainty whoe effect woul be either very ifficult to moel or have little impact on the architectural uncertaintie. The uncertainty categorization evelope i preente in Table 3. Thi claification help to both encompa the variou type of uncertainty an guie eigner probing for potential uncertaintie an alo erve a a framework for ialogue. Once the ource are ientifie, each ource ha to be aee for incluion in the analyi an if inclue, quantifie. After the ientification, aement an quantification of iniviual ource of uncertainty, the ame GINA imulation moel previouly evelope are ue to quantify embee architectural uncertainty through uncertainty propagation. Thi propagation provie one mean of aggregating the iniviual ource of uncertainty an a metho to ientify contribution of iniviual ource to the final embee architectural uncertainty. Source of uncertainty TechSat 21 repreent a revolution in the evelopment of pace ytem. The program i incorporating a number of unproven technologie, architectural an operational concept. It i truly a cae of puhing the envelope. That being ai, it i not urpriing that the TechSat 21 ha a goo eal of uncertainty aociate with it. Table 4 preent the attribute an value range that were ue a potential ource of uncertainty. Thee uncertaintie were choen from the contant vector an repreent both technical uncertainty, i.e. achievable fale alarm rate an moel uncertainty, i.e. tram cot enity for the TechSat 21 miion. 7

8 Embee architectural uncertainty After the iniviual ource of uncertainty have been ientifie an quantifie, the next tep i to evelop itribution of outcome for each of the architecture. In thi cae the extreme metho of uncertainty propagation wa ue. The firt tep in the technique i to lit the extreme poibilitie a wa one in Table 4. A ingle tate-bet, wort or expecte- i electe an incorporate the reult into the contant vector. Thi vector i then ue for each of the architectural imulation programme an reult are capture in an outcome vector for each architecture that inclue characteritic uch a performance meaure uch a probabilitie of etection, coverage an cot meaure uch a evelopment, operating an total life cycle cot. Next, a new tate i choen-bet, wort, expecte an the imulation i repeate for each of the architecture that are being invetigate an the outcome vector i ave. Thi proce of electing a contant vector i repeate until outcome meaure have been generate for all tate. Thi uncertainty quantification can be one for each of the architecture in the traepace, or a uggete here, an efficient traepace preproceor can be ue to evelop a ubtantially maller et of architecture from which to conuct uncertainty analyi. Figure 6 preent the reult contituting only the Pareto optimal front architecture. The prea of the wort cae from the expecte cae i noticeably larger than that of the prea of the bet cae from the expecte cae. Thi how that the uncertainty itribution for the traepace are left kewe meaning there i more ownie than upie in the architecture being coniering. Portfolio Analyi The previou ection ecribe the quantification of embee architectural uncertainty. Knowing the architectural uncertainty can help eciion maker in a number of circumtance, uch a eveloping a mitigation plan once an architecture ha been electe. Embee uncertainty, along with correlation meaure of how architecture behave uner conition of uncertainty, can provie the eciion maker with even more potential. Uing the portfolio optimization of Eq. 1, the eciion maker can create 8

9 accurate trae-off an begin to manage not the uncertainty in an iniviual architecture, but intea manage the uncertainty in a traepace of potential architecture. Figure 7 how the general characteritic of the TechSat 21 value v. uncertainty traepace. The Pareto optimal architecture that were etermine in the traitional concept exploration of utility v. cot traepace have been ue here a the potential member in any portfolio. max : r S. T. n i= 1 T w S. T. w 0 k T w w Qw 2 i = 1 Eq. 1 One of the firt inight een from the value/uncertainty traepace i that the efficient frontier i not compoe of all the Pareto optimal architecture. Intea, only a few contribute to the portfolio that contitute the efficient frontier. In all, only three of the original twelve Pareto optimal architecture contribute to memberhip along the efficient frontier. Further, the efficient frontier oe not exten beyon any iniviual architecture in the traepace an intea repreent a linear combination of only three aet. The reaon for thi i the high egree of correlation the architecture being coniere hare, i.e. all ρ i Quantifying Deciion Maker Rik Averion Once the efficient frontier ha been calculate, the next logical next tep i to etermine where the optimal trategy i for a given eciion maker. A icue in the companion paper, 1 capturing eciion maker rik averion can be relatively traightforwar through the ue of inifference curve an io-utility line. By interacting irectly with the cutomer with thi graphical technique, preference of the eciion maker can be capture an incorporate into the portfolio optimization. A previouly een, the level of averion of the eciion maker can greatly affect the optimal trategy 9

10 an thi i alo true in thi cae tuy. There are a total of 3 architecture that contitute memberhip in a portfolio omewhere on the efficient frontier an there are many combination of thoe poible. Rather than choe a ingle eciion maker averion, two eciion maker who repreent thee extreme a well a a more moerate eciion maker are preente a well a their optimal portfolio trategy that woul come from the uncertainty analyi. By uing three repreentative eciion maker, the overall enitivitie of the portfolio can be oberve an outcome compare to emontrate the aaptability of the uncertainty analyi approach to a large range of eciion maker who become involve in the evelopment of pace ytem. Aume that three-eciion maker maintain rik averion coefficient, k, value of 0.5, 2 an 3. The firt eciion maker looke at ha a rik averion coefficient of 3. The io-utility line for thi eciion maker have been overlai on the efficient frontier in Figure 8. The optimum portfolio for thi eciion maker reie in the lower left corner of the efficient frontier an conit of only a ingle architecture. Notice that the optimal trategy portfolio reie at the tangent point of the efficient frontier an the maximum utility io-utility line. The compoition of thi portfolio i hown in Table 5 an conit of only a ingle architecture Although portfolio can ugget et of aet to purue, it can alo ugget ingle aet, a in thi cae. A econ eciion maker that wa coniere wa one that ha a moerate rik averion, k=2. Thi eciion maker optimum portfolio trategy reie in the mile of the efficient frontier an conit of only a ingle aet. The relatively low rik averion eciion maker ha an optimal portfolio trategy in the upper right corner of the efficient frontier again coniting of only a ingle architecture. The compoition of the low rik averion eciion maker i hown in Table 5 an again i a ingle aet. Implication of incorporating the extenion to portfolio theory Differentiating rik from uncertainty Preente above wa the implementation of portfolio theory uing uncertainty a a urrogate for rik. Here, the impact of eparating the upie an ownie of uncertainty i explore within a 10

11 11 reapplication of portfolio optimization to icover any new inight. The firt tep i to ifferentiate the rik from the uncertainty in the itribution. The rik can be foun by focuing on the ownie emi-variance. To o o, firt ajut the variance of iniviual obervation aroun the expectation a hown in Eq. 2. Then, calculate the variance of thee new obervation error, a hown in Eq. 3. > = 0 0 if 0 E(r)) if - ( )) ( ( i i i i r r r r E r Eq [ ] = 2 )) ( ( * 2 r E r E S Downie Eq Thu creating a ownie covariance matrix a hown in Eq. 4. = 2 3 3, 2 2, 1 1, 3, , ,3 2, , ,2 1, , ,1 2 1 n n n n n n n n n n n n n Downie Q ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ Eq Finally the portfolio algorithm i implemente in the imilar manner to traitional portfolio theory, only ubtituting Q onwie for Q, a hown in Eq w :.t. 1 :.t. 2 max : 1 = = n i i Downie T T w w Q w k w r Eq Uing thi algorithm, an efficient frontier can be calculate in the ame manner performe earlier in the cae. The traepace of uncertainty an probability of etection for full variance an emi-variance i hown in Figure 9. The efficient frontier for both the full uncertainty portfolio analyi, a well a

12 the emi-variance analyi i hown in the figure. An inight to take away from thi chart i that there i more rik in the traepace than woul be perceive if uncertainty were ue a a urrogate for rik. Another obervation i that the relative poition of the architecture with repect to one another ha not change an intea, the reult from the emi-variance analyi i a imple hift to the right. Now that there i a ifferent efficient frontier, it i conceivable that eciion maker houl chooe ifferent optimal portfolio trategie. Uing the ame eciion maker previouly ue, the low, moerate an high rik averion, the effect that thi extenion provie to claical portfolio analyi are ecribe. The firt eciion maker wa the high rik averion eciion maker. Uner the efficient frontier uing emi-variance, hi optimal portfolio trategy ha remaine the ame a previouly foun, a hown in Table 6. Thi i reaonable becaue there are no le uncertain architecture to purue even though there i a higher egree of rik in the traepace. The moerate eciion maker oe have a hift in hi portfolio, a hown in Table 6. He ha hifte to the ame ingle aet portfolio trategy a the high rik averion eciion maker. The low eciion maker optimal portfolio trategy ha remaine in the upper right corner of the efficient frontier. The increae uncertainty that he i now expoe to i till not enough to ajut the low rik averion eciion maker. Obervation from TechSat 21 Thi cae provie an illutration of the uncertainty analyi approach applie to a very avance military pace ytem. The level of uncertainty in the traepace wa conierable an yet, the optimal portfolio trategie for three eciion maker were comprie of ingle architecture. Of the architecture evaluate, there wa imply not enough inepenence of architecture with repect to uncertainty for iverification poibilitie to come about. In the next two cae preente, iverification oe how up a an optimal trategy; however, thi cae point out that not all traepace contain complementary architecture, that when combine yiel more than any ingle 12

13 aet, thu making the teaching point that optimal portfolio trategie ometime conit of ingle architecture. The incluion of uncertainty analyi i illutrate the large amount of uncertainty aociate with each architecture in the traepace, thu allowing the eciion maker to bae eciion, not on eterminitic preiction, but one that are cautione by ome level of uncertainty. The uncertainty analyi further illutrate the ability to compare architecture in the traepace an unertan the relative enitivitie an trace thoe enitivitie back to ource of uncertainty. Thi traceability allow eigner to concentrate on either moeling with more reolution or builing in enough margin in their eign to accommoate the reultant poibilitie. Broaban Miion an Moel Decription Thi cae preent the ytem analyi of a pace bae broaban architecture. Thi commercial venture allow the emontration of the uncertainty analyi framework in a context that inclue apect of market uncertainty. Numerou example of the effect of market uncertainty can be een on the pace inutry, ranging from uncertaintie in launch vehicle capacity to meet the evolving nee of low earth atellite elivery to market uncertaintie that efine bankruptcie in the cae of Iriium an GlobalStar pace ytem. Where the major eciion criteria for a complex ytem i market riven, market uncertaintie houl alway be coniere The major feature of the architectural concept conit of a atellite network complemente by groun tation. While a pace ytem ha been choen to ervice thi market, the etail of the architecture have not been efine an intea have been left open for efining the traepace. Six traable parameter efine the bounarie of the traepace, a given in Table 7. GINA Moel Figure 10 ecribe the imulation flow that wa employe in thi cae tuy, bae on work by Kahitani. 5 The moel initiate with the efinition of a contant vector that contain parameter of 13

14 the eign that remain contant acro all of the architecture that are being evaluate. Example of contant in the Broaban moel are cientific contant, uch a the earth raiu, an converion factor. Other contant that are inclue in the Broaban moel are market contant uch a market ize an itribution, atellite izing ratio, an launch vehicle performance. The imulation i relatively coure in ytem eign etail, but erve a a goo cae for analyi becaue of the ue of market moel that exemplify circumtance where market uncertainty can have the riving effect on outcome. The flow of the moel begin with a relative izing of the pacecraft bae on rule of thumb an the eign vector input. For example, from the antenna power an antenna ize, the relative ma an ize of the pacecraft can be etermine from izing relationhip commonly ue in conceptual eign. 6 After the relative ize of the pacecraft i calculate, Satellite Tool Kit i ue to propagate the atellite in their iniviual orbit an capture ephemeri that can be ue in the coverage moel. The coverage moel calculate a global map of acceptable coverage that i achieve from the pace egment of the architecture, bae on probabilitie of atellite in view. The ytem capacity moel then generate the total ubcriber that the architecture being evaluate coul upport. Thi calculation i bae primarily on the link buget calculation of iniviual pacecraft umme over the contellation. The capacity of the architecture an it coverage are then compare with a market eman moel that efine the number of likely ubcriber over the coure of a given year. The launch moule then create a launching cheme bae on the orbital characteritic, a well a ma an ize characteritic of the atellite contellation. The ytem component cot are then calculate a well a the total ytem cot that i then tranforme to preent valuation. The final moule accept the input from the previou moel an generate a number of outcome meaure, i.e. profit, cot-per-billable hour, etc. 5 For the remainer of thi cae the billable hour-per ollar pent i ue a the key eciion criteria. 14

15 Moel Reult Figure 11 preent the ubcriber hour an ytem cot traepace with ot repreenting the 13 Pareto optimal architecture that were calculate uing a heuritic earch of the eign traepace. 3 Thee are the expecte outcome for the 13 architecture on the Pareto front, but of coure there i uncertainty that urroun each expectation that will be aree in the next ection. 2 From thi traepace of total ubcriber hour generate by the pace ytem an the ytem cot, a billable hour per ollar-invete metric (ubcriber hour/$) i evelope that i ue later a the ingle meaure of value for the eciion maker. Uncertainty Quantification Once the baeline GINA moel wa evelope, the uncertainty quantification approach wa initiate. The firt tep in the proce wa to ientify the potential ource of uncertainty in architecture being invetigate. Once the initial ource were ientifie an quantifie a Monte Carlo uncertainty propagation technique wa ue to evelop the embee uncertainty for each architecture. Source of uncertainty Becaue the Broaban GINA moel i relatively coare, a goo eal of the uncertainty being quantifie arie from the rule of thumb being ue in the moel imulation to generate reult. However, becaue of the commercial nature of the cae, market uncertaintie are alo introuce. Cot Uncertainty The cot moule for thi ytem ue cot etimating relationhip to tranform ma into cot for evelopment of the pacecraft. Thi erve a one ource of cot uncertainty. For example, The hitorical rule of thumb for Theoretical Firt Unit Cot per Kilogram i $84,000. A normal itribution centere aroun $84k with a tanar eviation of $10k wa ue in the imulation moel to capture the expectation an uncertainty aociate with the cot etimating relationhip. 2 A total of 17 Pareto optimal architecture were initially foun; however 4 of thee became infeaible uner the incluion of uncertainty an were exclue from further conieration. The infeaibility wa caue by launch vehicle contraint on ma that were violate for thee architecture. 15

16 Market Uncertainty The broaban ytem analyi affor the opportunity to introuce market uncertainty into application. Specifically thi market uncertainty i ariing from the etimation of three main parameter: 1.) total market ize of broaban cutomer, 2.) percent market capture for thi project, an 3.) the icount rate ue in the cah flow analyi. Thee three ource of market uncertainty erve a repreentative example of market uncertainty. Other coul have been inclue uch a uncertainty in market geographic itribution or competition cenario. Kelic invetigate a number of market uncertaintie that inclue thoe lite above in her analyi of potential pace bae broaban elivery ytem. 7 Uncertainty in total market ize i moele uing a lognormal itribution that i conitent with perviou market analyi of the broaban market potential. A lognormal itribution i ue for the obviou reaon that the market ha a lower boun of zero, but a more uncertain upper boun. The expecte market ize wa calculate on an annual bai with a ix year projection. The percent market capture i another ource of uncertainty. Even with a precie market, there i no way to know what competitor you ll have an what cutomer will prefer. Again a lognormal itribution i ue here to repreent an expecte market capture of 7.5% an the itribution aroun that. Finally, a icount rate wa ue in ome of the calculation to generate net preent value for variou architecture outcome. The icount rate uncertainty wa repreente by a normal itribution with mean of 30% with a tanar eviation of 7.5%. Although market uncertaintie exit in the Broaban cae, by no mean are market uncertaintie iolate to commercial venture. Military an civil ytem alo uffer from market uncertaintie in a number of way, ranging from competition to eman for the ytem to unknown repone from averarie. 16

17 Moel Uncertainty Becaue, the imulation moel wa relatively coure, there were a number of eign rule of thumb ue to ize feature of the architecture, incluing payloa power per unit ma, ma fraction of the payloa with repect to ry ma, fraction of ry ma in wet ma, an enity of atellite. Thee rule of thumb are bae on hitorical tren an the hope i that the previou eign tren will hol for the current ytem. Mot of thee rule of thumb have aociate with them an expecte caling factor an a tanar error. 8 The moel uncertaintie that were coniere in thi cae were the izing relationhip for payloa power per unit ma, ma fraction of the payloa with repect to ry ma, fraction of ry ma in wet ma, an the enity of the atellite. Embee Architectural Uncertainty To calculate the embee uncertainty in each architecture, the et of iniviual ource of uncertainty i built into the contant vector. The firt tep i to ample the contant vector uner conition of uncertainty. Once the contant vector i initialize, thi vector i the ue for each of the potential eign vector combination uner conieration an reult in an outcome vector for each architecture coniere. Next, a new contant vector i electe from the itribution of poible contant vector. The imulation for each eign vector combination uner conieration i repeate, reulting in a econ et of outcome obervation for each architecture evaluate. Thi proce of electing a contant vector i repeate many time until a populate itribution of outcome meaure can be generate. The number of run i only limite by the computation require an time allowe, a many imulation moel for every eign vector combination can take 5-10minute. The en reult of the uncertainty propagation i an orere et of outcome for every architecture coniere. Thi ata can be ue to create tatitical meaure of uncertainty for a ingle architecture an alo the pair-wie correlation coefficient that are neceary in portfolio optimization. Figure 12 preent a naphot of the embee uncertainty that wa calculate for each architecture on the 17

18 Pareto optimal front. The iamon repreent the expecte value of the architecture in term of ytem cot an total ubcriber hour, while the ellipe repreent the uncertainty of each architecture in both imenion. Portfolio Aement Once the embee architectural uncertaintie have been calculate, the portfolio aement can be applie. The portfolio aement provie a context in which trae-off of uncertainty an value, ubcriber hour/$, can be mae. Uing an expecte return an covariance matrix bae on 100 obervation of 13 architecture, the portfolio optimization algorithm wa applie to generate the efficient frontier. Uing an architecture portfolio analyi flight imulator, the eigner an eciion maker can ynamically explore trae-off between uncertainty an function-per-cot. Figure 13 provie a creen hot of the flight imulator. The ot inicate the current portfolio, while the weight of each architecture in the portfolio i lite on the right han ie along with the expectation of function-percot an uncertainty. An immeiate obervation from the portfolio traepace i the clear emarcation of GEO, MEO an LEO architecture along meaure of value an uncertainty. Quantifying Deciion Maker Rik Averion Once the efficient frontier ha been calculate, the next logical next tep i to etermine the optimal trategy for a given eciion maker. Rather than choe a ingle eciion maker averion, two eciion maker who repreent thee extreme are preente a well a a more moerate eciion maker an their optimal portfolio trategy that woul Aume rik averion coefficient, k, calculate for three eciion maker are 0.03, 0.1, an 1. Deciion maker with high rik averion optimal portfolio trategy The firt eciion maker looke at wa the highly rik avere eciion maker with a k value of 1. A highly rik avere eciion maker woul expect to fin themelve in the lower left han corner of the efficient frontier an that i exactly what i hown in Figure 14. The optimal invetment trategy 18

19 where the highet io-utility curve become tangent to the frontier i hown in the figure. The compoition of the optimal trategy i efine in Table 8. There were lower rik aet for which the eciion maker coul have invete, uch a the one GEO architecture on the Pareto optimal front, but thi eciion maker eire more return that the lower rik architecture coul provie. Deciion maker with moerate rik averion optimal portfolio trategy The econ eciion maker invetigate ha a k value equal to 0.1. In mot cae thi value woul not be coniere a moerate level of rik averion, but the phrae i ue here to how the relative preference to uncertainty of three eciion maker. The compoition of thi portfolio lie at a ingle architecture, a LEO architecture coniting of 40 atellite contellation each with a 2 m 2 antenna an 1 kw power. Deciion maker with low rik averion optimal portfolio trategy The thir eciion maker ha a very low level of rik averion, k=0.03. The optimal trategy for thi eciion maker, a one might expect, it reie in the upper right corner of the efficient frontier. The compoition of the portfolio i ecribe in etail in Table 8. The reaon for large LEO architecture ominating thi portfolio i that the larger the contellation of atellite an the more capacity a ytem ha to achieve ubcriber hour if the market conition are goo. However, uner avere market conition, the ytem won t achieve the ubcriber expecte an it will have require a ignificant capital invetment to contruct it. Notice that one of the aet uggete i only 2% of the portfolio. In practice 2% of an architectural invetment woul mot likely not be enough to prouce tangible benefit, o thi percentage might bet be itribute amongt the other two aet. Implication of incorporating the extenion to portfolio theory The claical implementation of portfolio theory ha been preente uing uncertainty a a urrogate for rik, but in fact, the two can be eparate, a hown below through the ue of emi-variance. Further, the low rik averion eciion maker ha a uggete optimal portfolio that conit of more 19

20 than one aet. What i the extra cot of that portfolio an how houl a cot benefit trae be mae? To fin the anwer, the correlation coefficient of the portfolio member i ue a a tarting point. Differentiating rik from uncertainty Uing the algorithm in Eq. 5, an efficient frontier can be calculate in the ame manner performe earlier in the cae. The efficient frontier for both the full uncertainty portfolio analyi, a well a the emi-variance analyi i hown in Figure 15. There i le rik in the traepace than woul be perceive if uncertainty were ue a a urrogate for rik. Uing the ame eciion maker previouly ue, the low, moerate an high rik averion, the effect that thi extenion to claical portfolio analyi woul provie are icue. The firt eciion maker wa the high rik averion eciion maker. Uner the efficient frontier uing emi-variance, hi optimal portfolio trategy ha remaine the ame a wa previouly foun in Table 9. The moerate rik averion eciion maker ha een no hift in hi optimal portfolio trategy. Although there i le perceive rik in the traepace uner the emi-variance calculation, the ame architecture i till retaine. However, the low rik avere eciion maker ha een a hift in trategy. He previouly ha a portfolio of three aet a an optimal portfolio trategy, but now ha two, namely the architecture that ha the highet value. Cot of iverification Some of the optimal portfolio trategie that have been foun in thi cae have inclue more than one aet an therefore more than one architecture to purue in eign. In orer to calculate the exact cot to iverify into a portfolio, the iniviual aet houl be cloely looke at by the eigner an eciion maker. For example, two LEO architecture with 45 inclination operating at 1kW an 2m 2 antenna an having only a mall ifference in the number of atellite in lightly ifferent plane will probably not incur twice the eign cot of a ingle architecture becaue of the commonality between the two architecture. In contrat, a two aet portfolio with a very large LEO architecture requiring 20

21 many groun tation an a two atellite GEO architecture might repreent a ignificantly higher cot to evelop than either of the two iniviually. A relative meaure of the cot of iverification i ue to juge the relative extra cot of carrying a portfolio bae on the correlation of aet in the portfolio, a ecribe in the companion paper 1. For an example, the low rik averion eciion maker uner the full uncertainty itribution woul have a cot to iverify equal to 0.5% of the cot to eign the architecture with the eign vector {LEO, 45, 5, 8, 1, 2} plu 0.1% of the cot to eign the architecture with the eign vector {LEO, 45, 7, 10, 1, 3}. Therefore the total cot to procee with the portfolio woul be the cot of eigning the majority contituent in the three-architecture portfolio plu thi aitional cot to iverify. Thi type of calculation can provie the bai for aitional conieration by the eciion maker on whether or not to procee with the portfolio trategy. Again the cot to iverify calculate here i a figure of merit an repreent a relative etimate on what the cot coul be. The actual cot to iverify will be cae pecific an houl be looke at carefully by the eigner an eciion maker. Obervation from Broaban cae tuy Thi cae emontrate the applicability of the uncertainty analyi approach to pace bae broaban communication architecture. Market an moel uncertainty were explore a primary ource of uncertainty an the cae emontrate how ignificant thee ource coul be to the overall value of a given architecture, with ome architecture maintaining 50% uncertainty. The role of ownie emi-variance focu wa alo emontrate, in contrat to a full uncertainty, an the impact that uch eparation woul have on the eciion maker optimal trategy with the ame level of rik averion wa hown. The intuitive obervation that come from the analyi uch a LEO architecture having preominantly greater uncertainty than MEO an GEO architecture i reinforcing to current peculation, but the cae provie a quantitative bae for exploring the intuition in more etail. 21

22 Moreover, an intereting final note on thi cae i how real worl ytem are acting with repect to the efficient frontier that wa evelope. The Teleeic Space Sytem ha been in evelopment for ome time. Initially conceive a a very large, LEO contellation of atellite, the ytem woul provie global broaban capability with very low latency an at a reaonable price. The original concept wa releae in 1994 a having 840 atellite at evelopment cot of $6.3B an total life cycle cot of $17.8B. In 1998, Teleeic went through a ramatic reeign from 840 atellite in LEO to 288 atellite. A wa hown in the analyi, thi hift to fewer pacecraft lowere the potential market capture of the ytem, but alo lowere the expoure to rik that the ytem woul have from the upfront evelopment cot invetment. In February 2002, a Teleeic architecture reeign wa publicly releae coniting of 12 atellite in MEO at a evelopment cot of $1B an 18 more MEO atellite eploye at a later ate to upplement coverage to achieve global capacity. Again, thi i a ownwar movement on the efficient frontier, opting for le capacity an ubcriber hour/$, but at a ignificantly lower cot than the LEO ytem. Terretrial Oberver Swarm Miion an Moel Decription The ATOS Miion (hort for the firt iteration, A, in a erie of Terretrial Oberving Swarm miion) ha the primary objective of collecting an ieminating fine meaurement of the ionophere. Thi ata woul be ue by the cience cutomer a input to a imulation moel ue for ecribing the behavior for the ionophere. An unertaning of the ionophere compoition at fine etail woul allow for more accurate preiction an mitigation of error in communication an location meaurement. Potential tactical benefit of a etaile mapping of the ionophere begin to paint a clear picture of the potential value of uch a miion beyon the pure cience of ionopheric mapping. 22

23 The ATOS Miion Given the ignificant impact of the ionophere an it variability on communication an navigation, the cience- an even broaer pace-community are interete in the accurate moeling an preiction of ionopheric ynamic. The overarching goal of the ATOS miion wa to eign a pace ytem that capture both the large cale an time-cale apect of the ionophere, a well a the etaile, mall time-cale fluctuation that are o unpreictable. More etaile time reolve meaurement of the ionophere are eential to preictive moeling of RF cintillation. One of the mot intereting feature of the ATOS moel evelope wa the ue of utility meaure to efine goone in the traepace of architecture. Intea of uing a et of performance meaure, uch a uable byte elivere or reolution an accuracy, a non-traitional approach, utility theory 9, wa applie to the problem of balancing the many et of cutomer preference that were involve in the program. Utility theory allow the eigner to capture preference of the cutomer in mathematical equation that provie for cutomer-in-the-loop trae-off, while not necearily requiring their phyical preence. Of coure, the phyical collocation of the cutomer can be clarifying an perhap more accurate an aaptive to unforeeen circumtance than utility theory, but can take a toll on the cutomer in term of time invete in the project. Five concept to accomplihing the ATOS miion were invetigate, a hown in Figure 16. The ionophere i repreente by the otte pattern in each of the five chart, while the boxe repreent the notional atellite that are part of the overall pace ytem. The in-itu approach wa the one employe in the ATOS architecture. Perhap the eaiet to conceive of all the poible meaurement cheme, the concept woul rely on irect meaurement of ionopheric conition a the iniviual atellite pae through it. The payloa woul be a paive payloa coniting primarily of Planar Langmuir Probe (PLP) to recor charge particle enitie. Thi approach ha the benefit of 23

24 having a relatively imple paive payloa that require little power, ma an recor ata that nee little pot-proceing to arrive at ueful information for the cutomer. Derive Utility Function The utility of the ATOS ytem wa erive from an architecture ability to atify three itinct ubmiion eire by the cutomer. The firt, the low latitue urvey miion, wa an equatorial region urvey that woul ientify untable region of the ionophere near the equator. The econ miion, the low latitue naphot miion, woul require the pace ytem to initiate an extenive ata collection of an untable region once the firt miion ientifie a intability. The thir miion wa to perform a high latitue urvey that woul accurately meaure relative ionopheric enity correlate with GPS-to-groun ata. The low latitue urvey miion wa to meaure the low latitue characteritic of the ionophere at a ampling rate of approximately 1Hz, a hown in Figure 17. From thi information, the cutomer moel coul be populate with the large-cale characteritic of the ionophere. The low latitue naphot ub-miion woul be important once the urvey miion ientifie a ionophere iturbance. Uing a warm of atellite, fine-cale meaurement of the anomaly woul be collecte. A hown in Figure 18, atellite at ifferent eparation itance are ueful to the cutomer, becaue at large eparation itance the overall hape an characteritic of the iturbance woul be capture, while mall eparation itance woul provie more baeline meaurement of variability within the iturbance. All thi ata woul fee into the cutomer moel an provie better preictability for future communication outage planning. The lat ub-miion i a high latitue urvey. The major charge particle concentration in the ionophere i centere about the equatorial ban an the high latitue region. Although not a ignificant a ub-miion a the low latitue miion, the high latitue miion woul provie further population of the cience communitie global ionopheric moel an preiction ability. Typically, the 24

25 high latitue region i le turbulent than the low latitue region an therefore, the cience community i only interete in the urvey miion a oppoe to the urvey an naphot miion. Figure 19 repreent the notional pace egment that coul accommoate the high latitue miion. The eirable eparation among atellite in thi miion i about 75km in the irection of longitue an latitue an 20km in the irection of altitue. In the low latitue region, the ionophere i fairly contant with altitue, thi aumption oe not hol with the high latitue miion. Further, there woul be ae value to the cience community if GPS occultation meaurement coul be taken a well to correlate the ata prouce by in-itu meaurement of the warm. From the above miion, utility function were calculate for each of the miion, a a function of each miion attribute, a notionally hown in Eq. 6, Eq. 7, an Eq. 8. For example, the low latitue urvey miion wa a function of iniviual ample obervation an the location an time of ay of each meaurement. U ( Low _ Surv) = f ( X, X 2,..., X 1 m ) Eq U Low _ Snap) = g( Y, Y,..., Y ( 1 2 m ) Eq U ( High _ Surv) = h( Z1, Z 2,..., Z n ) Eq Aitionally, a total architecture utility variable wa efine a a weighte um of the two eparate miion utilitie, a hown in Eq. 9. Although thi i a imple linear aggregation of the multiple element of utility it provie a firt look at how iea of utility coul be incorporate into the pace ytem conceptual eign proce. Conierable progre ha been mae on ubequent eign iteration that exploit the full potential of multi-attribute utility theory. U ( Total) = U ( High _ Surv) / U ( High _ Surv) + 2*( U ( Low _ Surv) + U ( Low _ Snap)) /( U ( Low _ Surv) + U ( Low _ Snap max )) max Eq 25

26 The eign vector that wa evelope to efine the traepace of architecture that woul be invetigate i preente in Table 10 an graphically in Figure 20. The eign vector conite mainly of orbital parameter, a the miion wa riven by the in-itu location of iniviual atellite throughout the miion lifetime. Sytem Analyi Moel The conceptual eign imulation moel evelope uring the ATOS eign effort were bae on heritage moel evelope uner the generalize information network analyi (GINA) metho 10, but actually went beyon the original approach by applying utility theory to capture the preference of the cutomer in the imulation. A utility function wa formulate, a ecribe earlier, an incorporate into the ytem imulation, a preente in Figure 21 to generate outcome meaure that woul enable informe trae-off of potential architecture. Moel Reult Over four thouan architecture were evaluate uing the imulation moel. After calculating the expecte outcome for thee thouan of potential architecture, the traepace wa explore along the funamental utility an cot meaure evelope in the early problem formulation. Figure 22 how how the traepace took hape in term of low latitue utility, high latitue utility an cot. Each hae quare in the chart repreent at leat one pecific architecture concept, a efine by a unique eign vector. In the figure, the two imenion of cutomer utility have been plotte an the hae quare repreent the life-cycle cot of a given architecture. The firt intuitive concluion that can be rawn from thi traepace i that utility increae with increaing cot. It i further evient though that there are ome relatively inexpenive architecture that accomplih the low latitue miion quite well, but on t perform very well in the high latitue miion. Thee type of multiimenional utility plot can be ue with the cutomer to reviit the relative importance of iniviual miion. 26

27 Exploring the traepace further, the iniviual point can be ientifie by what architecture they repreent, a well a what characteritic rive the performance outcome. Figure 23 repreent the total utility an cot preicte outcome for 1380 eign, which are repreente a iamon in the plot. By uing the total utility function in Eq. 9, overall architectural preference can begin to be oberve. For example the highet utility-per-ollar or the highet total utility architecture can be foun, a hown in the figure. Intereting to notice i that the highet utility-per-ollar architecture i eparate from a ba eign by only a few eign parameter changing. The ATOS miion analyi prouce a fairly large traepace with varying outcome meaure an there are ome very goo eign, but there are alo ome le eirable outcome eign. The horizontal ban that are forme are ue to the number of atellite in the warm, which rive the lifecycle cot of the miion. Other imenion of the eign vector rive the outcome value for total utility. Uncertainty Quantification Uncertainty invetigation i limite to the Pareto optimal front of architecture in the traepace, primarily for the reaon of computational efficiency. The et of Pareto optimal architecture inclue all non-ominate architecture. In the ATOS cae, a non-ominate architecture i one who performance cannot be improve without paying a higher lifecycle cot. The true Pareto optimal architecture in the ATOS traepace conit of only 6 architecture, however. Therefore to increae the pool of potential architecture to raw upon in the portfolio analyi, 24 aitional near Pareto optimal architecture were choen to ue in the uncertainty analyi approach. Thee 30 architecture are graphe in Figure 24. Source of uncertainty Technical Uncertainty The major technical uncertainty that wa inclue came from the mean time to failure (MTTF) for a ingle atellite in the contellation. Becaue the mean time to failure i a repreentative reliability of the 27

28 entire atellite, it i a very ifficult number to meaure. Small atellite uch a thoe preente have previouly ue 500 month a the mean time to failure. However, there are not a lot of thee itribute atellite ytem in operation, o the reliability warrante the incluion of uncertainty boun. A normal itribution with a tanar eviation of 50 month wa ue to repreent the uncertainty in MTTF. Cot Uncertainty The cot uncertainty aroe from both cot to evelop an the cot of operation. The uncertainty in the cot of evelopment of the atellite bu wa capture uing the tanar error in the hitorical cot etimating relationhip ue in the imulation moel. The evelopment cot uncertainty for the payloa wa alo inclue. The operation cot uncertainty aroe from uncertainty in the etimation of the iniviual ource that contribute, uch a the number of engineer an operator require for maintaining the ytem an the uncertainty in the cot of groun oftware an equipment. Utility Uncertainty Becaue thi cae relie on utility a the key eciion criteria, an element of utility uncertainty wa inclue in the analyi. The combine low latitue miion an high latitue miion were ifficult for the cutomer to itinguih in term of precie relative value, o a nominal value of 2:1 wa ue a the utility ratio of the combine low utility miion to the high utility miion a wa hown in Eq. 9. Intea of uing thi ratio, the relative worth of the high latitue miion over the low latitue miion wa moele a a probabilitic enity function with a mean of 2 an a tanar eviation of 1. Moel Uncertainty The moel uncertainty in the ATOS cae tuy aroe from the eigner in ability to preciely quantify ifferent apect of the ytem through mathematical formulation. Intea, eign rule of thumb or parametric relationhip are ue that are bae on hitorical obervation. Two moel uncertaintie in the cae of ATOS were the atellite enity, which i ue to calculate the erive ma an overall 28

29 tructure within the moel, an the learning curve ue to etimate prouction cot for more than one atellite. Embee Architectural Uncertainty Once the ource of uncertainty have been ientifie an each ha been quantifie an inerte into the contant vector, a Monte Carlo ampling routine i conucte with the goal of eveloping itribution of outcome for each of the architecture evaluate. Thee itribution characterize the embee architectural uncertainty an are ue to compare architecture an their repone to the variou ource of uncertainty. The en reult of the uncertainty propagation i an orere et of outcome for every architecture coniere. Thi ata can be ue to create tatitical meaure of uncertainty for a ingle architecture an alo the pair-wie correlation coefficient that are neceary in portfolio optimization. Figure 25 preent a naphot of the embee uncertainty that wa calculate for each architecture on the Pareto optimal front. The point repreent the expecte value of the architecture in term of cot an total utility, while the ellipe repreent the uncertainty of each architecture in both imenion. Normality in Architectural Ditribution Portfolio theory abtract uncertainty characteritic to imple meaure of expectation an variance that are conitent with gauian itribution. The iniviual architecture uncertainty itribution houl be invetigate to atify thi aumption that the characteritic of the uncertainty itribution can inee be capture by thee imple meaure. Normality can be tete uing tatitical meaure uch a kewne an kurtoi a well a graphical technique. Uing the Shpiro-Wilk tet for normality, the hypothei for normality coul not be rejecte for any of the architectural itribution create in the ATOS ytem analyi. Portfolio Selection Uing the quantifie uncertainty for each architecture, an knowing the correlation of outcome, portfolio theory can be applie with the goal of efining optimal invetment trategie. Uing an 29

30 expecte return an covariance matrix bae on 60 obervation of 30 architecture, the portfolio optimization algorithm wa applie to generate the efficient frontier. Figure 26 preent the efficient frontier, a erive uner the claic portfolio optimization algorithm in Eq. 1. The efficient frontier exten beyon the performance achieve by any ingle architecture, a hown in Figure 27. Thi i an important fining a it how that portfolio theory can provie more potential to the eciion maker than woul otherwie be poible with a ingle aet. The reaon for the extenion beyon any ingle architecture can be trace back to Figure 23. Notice that ome architecture achieve a very high level of low latitue value at low cot but perform the high latitue miion poorly, wherea other perform well in both low an high latitue miion. Becaue of thee two ifferent approache to achieving total value, there arie a chance to iverify uncertainty. The amount iverifie i not enormou, but it i meaurable an preent one of the firt illutration that portfolio aement in pace ytem can help eciion maker achieve higher return for a given level of uncertainty than they otherwie coul with ingle aet. Quantifying Deciion Maker Rik Averion Rather than choe a ingle eciion maker averion, two eciion maker are preente who repreent thee extreme a well a a more moerate eciion maker an the optimal portfolio trategy that each woul follow from the uncertainty analyi. By uing three repreentative eciion maker, the overall enitivitie of the portfolio can be oberve an outcome compare to emontrate the aaptability of the uncertainty analyi approach to a large range of eciion maker who become involve in the evelopment of pace ytem. Deciion maker with high rik averion optimal portfolio trategy The firt eciion maker looke at ha a rik averion coefficient, k, equal to 2. Thi i a relatively high egree of rik averion an o it i expecte that thi eciion maker optimal portfolio trategy reie in the lower left corner of the efficient frontier, an that i exactly what Figure 28 how. To ientify thi point, the efficient frontier i plotte an then overlai with io-utility curve for a given eciion 30

31 maker averion level. The three convex curve repreent the io-utility curve for a eciion maker, each increaing in utility a they move to the upper left. Therefore the optimal portfolio invetment trategy woul lie at the tangent point of the io-utility curve an the efficient frontier, a how in Figure 28. The compoition of the optimal portfolio for thi eciion maker i 48% of one architecture an 52% of another, a hown in Table 11. The two architecture that have been electe behave ifferently enough to move the curve beyon a imple linear combination of the two an provie meaurable value through iverification. A trategy ha been create that ha le uncertainty than either of the portfolio aet. Thee kin of non-intuitive ynergie can only be foun through a metho uch a the one preente here. The majority of the portfolio i occupie by the architecture that wa ientifie a having the maximum utility in the traepace in Figure 23, while the remaining portfolio inclue a much maller architecture of only 2 atellite, compare to 26, an ha a much maller cot. The two combine ynergitically in thi portfolio becaue the two architecture are achieving total utility/$ in two way. The 26 atellite architecture i achieving both the low an high latitue miion, but at a high price. The 2 atellite miion i achieving goo reult on the low latitue urvey ub-miion, but oen t have enough atellite to o a goo job at either the low latitue naphot miion or the high latitue urvey. On the other han, the 2 atellite architecture i inexpenive, thu it achieve a goo total utility/$. Becaue the approache of the two architecture are ifferent with repect to total utility/$, the effect of uncertainty on each of the architecture outcome meaure have been ifferent. Deciion maker with moerate rik averion optimal portfolio trategy The next eciion maker preente ha a more moerate rik averion coefficient, k, of 1. Thi eciion maker trategy lie in lower left half of the efficient frontier, but till relatively far from the previou eciion maker. Notice that a eciion maker with thi level of rik averion can be 31

32 accommoate in the portfolio analyi technique. However, without the creation of portfolio, the eciion maker woul have to ettle for ingle aet that meet hi rik averion criteria but achieve much lower total utility/$, or the eciion maker woul nee to accept a higher level of rik than their averion coefficient woul preict them to be comfortable with to achieve a high level of total cot/$. The ability of portfolio theory to create continuou invetment trategie i another benefit that can t be achieve with ingle aet. The compoition of the optimal portfolio i for the moerate rik avere eciion maker i preente in Table 11. Once again, the majority of the portfolio i occupie by the 26 atellite architecture a wa een with the previou eciion maker. The ret of the portfolio ha change though. Thi eciion maker i looking to get more return an willing to accept more rik, o two architecture enter the portfolio that have greater return than the architecture in the highly rik avere eciion maker cae, but alo greater uncertainty. The higher return in term of Total Utility/$ come about becaue the 4 atellite architecture, although achieving le overall miion utility, o o uing a far le proportional cot. Deciion maker with low rik averion optimal portfolio trategy Finally, a eciion maker who ha a relatively low rik averion coefficient, k, of 0.4 i invetigate. The optimal portfolio trategy lie in the upper right corner of the efficient frontier. With a relatively low level of rik averion, thi type of eciion maker i trying to get the mot value out of the ytem with relatively little worry about the rik that the olution might carry. The compoition of the optimal portfolio i preente in Table 11. Notice one architecture from the moerate rik avere eciion maker i kept, but a new architecture ha been ae a well. Thi architecture houl be familiar, a it wa calle out in Figure 23 a the bet value eign. Inee, thi architecture i have the highet total utility/$, but it alo ha the highet level of uncertainty for any of the architecture. 32

33 Implication of incorporating the extenion to portfolio theory Although claic portfolio technique were ue above, the extenion to portfolio theory, a ecribe in Ref. 1, coul alo be applie to glean any new information. The firt extenion that can be mae i eparating the rik from the uncertainty previouly ue. Thi eparation can be ueful to illutrate to eciion maker that architecture are more or le riky than their uncertainty itribution might lea to one to believe. The econ extenion that i ue i to quantify the cot of carrying a portfolio of architecture, rather than any ingle aet. Differentiating rik from uncertainty Uing thi algorithm in Eq. 5, the emi-variance cale efficient frontier with the full uncertainty itribution efficient frontier i hown in Figure 29. The efficient frontier for both the full uncertainty portfolio analyi, a well a the emi-variance analyi i hown in the figure. An important inight to take away from thi chart i that by uing the full itribution variance, rather than the ownie emivariance, the rik to a given eciion maker wa overetimate. The firt eciion maker wa the high rik averion eciion maker. Uner the efficient frontier uing emi-variance, hi optimal portfolio trategy ha change only in percentage invetment in each of the aet in hi portfolio, a hown in Table 12. The optimal portfolio now ha a higher egree of emphai on the higher return aet. The moerate eciion maker ha kept two of the previou three aet portfolio. Again, there i an overall percentage invetment increae in the higher return aet. The low rik averion eciion maker ha move hi optimal trategy to a one-aet portfolio. Although there i not appear to be a large egree of hift in the efficient frontier uing emi-variance in place of full uncertainty, there wa enough to move 17% invetment of another architecture out of thi portfolio, o that 100% of the reource coul be evote to the highet value ytem. Cot of iverification Some of the optimal portfolio trategie uggete in thi cae have inclue more than one aet an therefore more than one architecture to purue in eign. In orer to calculate the exact cot to 33

34 iverify uing a portfolio, the iniviual aet houl be invetigate by the eigner an eciion maker. For example, two 4 atellite architecture with imilar characteritic iffering only in altitue by 200km will probably not incur twice the eign cot of a ingle architecture becaue many of the imilaritie in each architecture can be eign for both. In contrat, a two-aet portfolio with a 26 atellite architecture at very large eparation itance an a two atellite warm at very mall eparation itance architecture might repreent a ignificantly higher cot to evelop than either of the two iniviually. A meaure of the cot of iverification can be ue to juge the relative extra cot of carrying a portfolio bae on the correlation of aet in the portfolio, a preente in Ref. 1. For example, the high rik averion eciion maker uner the full uncertainty itribution woul have a cot to iverify equal to 33% of the cot to eign the 2 atellite architecture. Thi high cot to iverify i bae on the low correlation that exit between the two aet, In contrat the cot to iverify of the low rik avere eciion maker woul be only 3.5% of the cot to eign the 4 atellite architecture in the portfolio. Thi lower aitional cot i ue to the higher egree of correlation that exit, Obervation from ATOS cae tuy A propoe uncertainty analyi approach ha been preente an implemente on a repreentative cientific pace miion. Thi cae provie an opportunity to implement the uncertainty analyi approach in the context of a cientific pace ytem analyi. The ATOS cae tuy emontrate the ue of the uncertainty analyi metho in a ytem exploration that centere aroun utility/$ a a funamental eciion criteria. Thi approach capture the benefit that portfolio analyi can provie to a eciion maker by creating invetment trategie for eign that achieve higher value for lower uncertainty than woul be poible with any ingle aet. Illutrate by the high rik avere eciion maker whoe optimal portfolio conite of architecture that achieve value through ifferent approache an therefore reacte 34

35 ifferently to uncertainty. Finally the focu on ownie of uncertainty wa hown to have impact on the optimal invetment trategy of ifferent eciion maker. Concluion The application of an uncertainty analyi approach ha been preente in the context of three miion clae of pace ytem. The broa applicability of the uncertainty analyi framework ha been emontrate in the eign proce where tructure moeling an imulation i one that enable the prouction of performance outcome itribution for outcome. Further, it ha been hown in ome cae that portfolio of olution lea to higher expecte value than any ingle aet can attain. Acknowlegement The author woul like to thank Profeor E Crawley an Earll Murman, an Dr. Joyce Warmkeel, Dr. Hugh McManu, Dr. Cyru Jilla, an Dr. Annalia Weigel (all of MIT) for their inight an feeback on thi reearch work. Dr. William Borer, Dr. Willam Kaliaro, Dave Ferri, Anre Girar, Aam Ro, Dan Thunnien an Branon Woo prouce much of the ATOS imulation moel. The author gratefully acknowlege the financial upport for thi reearch from the Space Sytem Policy an Architecture Reearch Conortium (SSPARC), a joint reearch program fune by the U.S. government involving the Maachuett Intitute of Technology, the California Intitute of Technology, an Stanfor Univerity. 35

36 External Source of Uncertainty Social/Market Factor Enterprie Goal Cutomer Value Regulatory Technology Nee Analyi Functional Analyi Conceptual Deign Concept Generation Uncertainty Analyi Approach Lifecycle Portfolio/Set Uncertainty Selection & Puruit Quantification Concept Selection Prelim. Deign Capture embee uncertainty of potential architecture Ientify an quantify iniviual ource of uncertainty Ue uncertainty propagation to capture embee architectural uncertainty Explore the eign pace with uncertainty perpective Ue portfolio optimization to unertan bet et to explore Figure 1: Inertion of the uncertainty analyi approach in conceptual eign Miion Name: TechSat 21 (Military) Value Meaure: Probability of Detection/$ Uncertainty Meaure: StDev(P/$) Miion Name: Broaban Sytem (Commercial) Value Meaure: Billable Hour/$ Uncertainty Meaure: StDev(BH/$) Miion Name: ATOS (Science) Value Meaure: Total Utility/$ Uncertainty Meaure: StDev(TU/$) Figure 2: Three cae tuie ummary 36

37 Table 1: Deign vector for the TechSat21 Satellite Sytem Name Decription Potential Value Altitue The operating altitue of the TechSat21 contellation km Number of Satellite The number of atellite in each cluter/warm 4-16 Number of Cluter The number of cluter/warm that comprie the contellation Number of Plane The number of orbital plane occupie by the 2-10 contellation Antenna Power Antenna Tranmiion Power 100-1kW Antenna Diameter Antenna Aperture Diameter 0.5-3m Input Contant Vector MTTF, MTTR Operation Concept Performance Requirement Subytem Mae Subytem Power Miion Lifetime. Deign Vector # S/C per Cluter # Cluter.. Contellation Altitue Aperture Diameter. Antenna Power MATLAB Moel Contellation S/C Bu Launch & Operation Benefit of GINA Raar Payloa Sytem Analyi Enable the egmentation of the problem Enable the integration of icipline Enable traepace exploration Key Output Lifecycle Cot Availability Probability of Detection Reviit Rate Reolution & MDV Enable the equitable evaluation of ifferent architecture [Shaw, 1998 Jilla, 2002] GINA enable uncertainty propagation an conitent evaluation metric Figure 3: GINA Moel Flow Chart Figure 4: TechSat 21 Traepace 37

38 Figure 5: Pareto Optimal Front for TechSat 21 Architecture Table 2: TechSat 21 Pareto Optimal Architecture an Outcome Meaure Architecture Ant Diam Ant Pow P() 90.0% 95.0% 97.0% 98.2% 98.9% 99.4% 99.7% 99.8% 99.8%-99.9% Lifecycle Cot ($B) Development Uncertainty Political Uncertainty- uncertainty of evelopment funing intability Requirement Uncertainty- uncertainty of requirement tability Development Cot Uncertaintyuncertainty of eveloping within a given buget Development Scheule Uncertaintyuncertainty of eveloping within a given cheule profile Development Technology Uncertaintyuncertainty of technology to provie performance benefit Table 3: Uncertainty Categorization Operational Uncertainty Political Uncertainty- uncertainty of operational funing intability Lifetime Uncertainty - uncertainty of performing to requirement in a given lifetime Obolecence Uncertainty uncertainty of performing to evolving expectation in a given lifetime Integration Uncertainty uncertainty of operating within other neceary ytem Operation Cot Uncertainty uncertainty of meeting operation cot target Market Uncertainty-uncertainty in meeting eman of an unknown market Moel Uncertainty 38

39 Table 4: TechSat 21 Moele Source of Uncertainty 1 Attribute Bet Value Expecte Value Wort Value Miion Lifetime 9 year 10 year 11 year Raar Freq. 10.2E9 Hz 10E9 Hz 9.8E9 Hz Ant Tran. Duty Cyc Raar Cro Section 10.2 m 2 10 m m 2 Require Range Re. 240m 250m 260m Tram Ma Den. 8 kg/m 2 6 kg/m 2 4 kg/m 2 Tram Elec. Den. 7 kg/m 2 5 kg/m 2 3 kg/m 2 Tram Cot Den 1.25E6 $/m 2 1E6 $/m E6 $/m 2 Tram Elec. Cot Den. 1.25E6 $/m 2 1E6 $/m E6 $/m 2 Stowe Depth 0.6 m 0.5 m 0.4 m # PrimexPPT Ma Primex PPT 2 kg 1.5 kg 1 kg Pow. Primex PPT 20 W 1 W 1 W #Micro PPT Ma Micro PPT 0.5 kg 0.2 kg 0.2 kg Pow Micro PPT 10 W 0.5 W 0.4 W Star Sen. Ma 2.5 kg 0.7 kg 0.5 kg Star Sen. Pow 20 W 4 W 4 W Magnometer Ma 1.2 kg 0.13 kg 0.13 kg Mangonmeter Pow 1 W 0.5 W 0.5 W # Torque Ro Ma Torque Ro 0.6 kg 0.45 kg 0.4 kg Power Torque Ro 2 W 1.3 W 1 W MB per Chip 48 MB 64 MB 96 MB Ma per Chip 1 kg 0.5 kg 0.5 kg Bat. Power Denity 45 W hr/kg 51 W hr/kg 60 W hr/kg Operator per Satellite MTTF 450 month 498 month 550 month MTTR 2 month 3 month 4 month Figure 6: Pareto Front for Three Cae of Uncertainty 39

40 Figure 7: TechSat 21 Portfolio Analyi Optimal Strategy Portfolio Figure 8: Optimal invetment trategy for high rik averion eciion maker Table 5: Compoition of TechSat 21 eciion maker trategie Deciion Percentage Architecture Deign Vector Total Uncertainty Maker Type of Portfolio {alt,at/clutr,#clutr,#plane,ant pow,ant_iam} Utility/$ High Rik 100% {800, 4, 42, 6, 1000, 2.5} Averion Complete Portfolio Value an Uncertainty Moerate Rik 100% {800,4,42,6,900,3} Averion Complete Portfolio Value an Uncertainty Low Rik 100% {800,4,42,6,900,3.5} Averion Complete Portfolio Value an Uncertainty

41 Full Uncertainty Semi- Variance Figure 9: TechSat 21 Portfolio Analyi with full uncertainty an emi-variance Table 6: Compoition of high rik avere optimal portfolio trategy uing emi-variance Deciion Maker Percentage Architecture Deign Vector Total Uncertainty Type of Portfolio {alt,at/clutr,#clutr,#plane,ant pow,ant_iam} Utility/$ High Rik 100% {800, 4, 42, 6, 1000, 2.5} Averion Complete Portfolio Value an Uncertainty Moerate Rik 100% {800, 4, 42, 6, 1000, 2.5} Averion Complete Portfolio Value an Uncertainty Low Rik 100% {800,4,42,6,900,3.5} Averion Complete Portfolio Value an Uncertainty Table 7: Deign vector for the Broaban Communication Satellite Sytem Name Decription Potential Value Altitue Altitue for a efine circular orbit LEO(1500km), MEO(20184km), GEO(35786km) Inclination The inclination of the circular orbit Satellite per Plane The number of atellite in each of the 1-8 occupie plane Number of Plane The number of orbital plane that the 1-10 atellite contellation occupie Payloa Power Downlink power from an iniviual 1kW-10kW atellite Phae Array Area Area in quare meter of the total phae array antenna area 1-5m 2 41

42 Deign Vector Contant Vector Simulation Moule Satellite Sizing Orbit analyi Coverage analyi Capacity Analyi Launch analyi Cot Analyi Performance aement Figure 10: Sytem Simulation Flow Figure 11: Commercial Broaban Sytem Pareto Optimal Front 42

43 Figure 12: Broaban traepace with the incluion of uncertainty Big LEO Little LEO MEO, GEOS Figure 13: Snaphot of the Architecture Portfolio Flight Simulator 43

44 Optimal Strategy Portfolio Figure 14: Optimal invetment trategy for high rik averion eciion maker Table 8: Compoition of Broaban high rik averion eciion maker trategy Deciion Maker Type Percentage of Portfolio Architecture Deign Vector {alt, inc, at/plane, plane, pow, ant area} Subcriber Hour/$ Uncertainty 55% {MEO, 0, 8, 1, 1, 3} High Rik 45% {LEO, 0, 7, 1, 2, 0.5} Averion Portfolio Value an Uncertainty Moerate Rik 100% {LEO, 45, 5, 8, 1, 2} Averion Portfolio Value an Uncertainty % {LEO, 45, 5, 8, 1, 2} Low Rik 2% {LEO, 45, 7, 10, 1, 3} Averion 89% {LEO, 60, 6, 10, 1, 3.5} Portfolio Value an Uncertainty Semi-variance Full Uncertainty Figure 15: Broaban portfolio analyi with full uncertainty an emi-variance 44

45 Deciion Maker Type Table 9: Compoition of Broaban high rik averion eciion maker trategy Percentage of Portfolio Architecture Deign Vector {alt, inc, at/plane, plane, pow, ant area} Subcriber Hour/$ Uncertainty High Rik Averion 49% {MEO, 0, 8, 1, 1, 3} % {LEO, 0, 7, 1, 2, 0.5} % {LEO, 30, 5, 6, 1, 4} Portfolio Value an Uncertainty % {LEO, 45, 5, 8, 1, 2} Moerate Rik Averion Portfolio Value an Uncertainty % {LEO, 45, 7, 10, 1, 3} Low Rik 76% {LEO, 60, 6, 10, 1, 3.5} Averion Portfolio Value an Uncertainty GPS UV Sening GPS Occultation Topie Souner In Situ Direct Scintillation Sening Figure 16: Approache to Meauring Ionophere Characteritic x x 1 ec 1 ec 1 ec Figure 17: ATOS Low Latitue Survey Miion 45

46 1 m 200 km Figure 18: ATOS Low Latitue Snaphot Miion 75 km GPS 20 km Figure 19: ATOS High Latitue Survey Miion Table 10: Deign vector for the ATOS Satellite Sytem Name Decription Altitue Altitue of the atellite warm Subplane per Swarm Number of ubplane in warm Satellite per Swarm Number of atellite in warm Suborbit per Swarm The number of concentric orbit in warm Subplane Yaw The yaw angle of the warm with repect to nair Separation Ditance The maximum along track eparation eparation itance V ubplane per warm (3 hown) ubplane yaw Figure 20: Graphical repreentation of ATOS Deign Vector 46

47 Reliability Environment Deign Variable, Contant Orbit Utility Spacecraft Launch Initial Launch Replenih Operation Cot Sytem A-TOS Simulation Architecture-Level Metric Figure 21: ATOS GINA Moel Moule Flow Diagram Figure 22: Low an High Utility Traepace v. Lifecycle Cot 47

48 LifeCycle Cot ($M) Ba Deign Vector 8 Sat 1 Suborbit/Plane 200 km ep 1 Subplane/Swarm Alt. Inepenent Min Util/Cot=0.2 Want high value (i.e. High Utility/Cot) Max Util/Cot = 1.7 Maximum Utility Deign Vector 26 Sat 4 Suborbit/Plane 200 km ep 2 Subplane/Swarm Alt=500km Bet-Value Deign Vector 8 Sat 2 or 4 Suborbit/Plane 3.8 or 14.1 km ep Subplane/Swarm 2.5 Total Utility Alt. Inepenent Figure 23: ATOS Cot an Utility Traepace Figure 24: (Near) Pareto Optimal Front for the ATOS Architectural Traepace 48

49 Figure 25: ATOS Utility an Cot Traepace with Uncertainty Ellipe (1StDev) Figure 26: Efficient Frontier in the ATOS traepace Figure 27: Cloer Look at the ATOS Efficient Frontier 49

50 Optimal Strategy Portfolio Deciion Maker Type High rik averion Moerate rik averion Low rik averion Figure 28: Optimal invetment trategy for high rik averion eciion maker Percentage of Portfolio Table 11: Compoition of ATOS eciion maker trategy Architecture Deign Vector {at/warm, uborb,ize,yaw,ubplane,alt} Total Utility/$ Uncertainty 52% {26,4,14.1,60,2,700} % {2,1,3.8,30,1,300} % Portfolio Value an Uncertainty % {26,4,14.1,60,2,700} % {4,2,3.8,30,1,500} % {4,1,14.1,0,1,700} % Portfolio Value an Uncertainty % {8,4,14.1,30,1,700} % {4,2,3.8,30,1,500} % Portfolio Value an Uncertainty

51 Semi-variance Variance Deciion Maker Type High rik averion Figure 29: ATOS portfolio analyi with full uncertainty an emi-variance Table 12: Compoition of ATOS high rik averion eciion maker trategy uing emi-variance Percentage of Portfolio Architecture Deign Vector {at/warm, uborb,ize,yaw,ubplane,alt} Total Utility/$ Uncertainty 64% {26,4,14.1,60,2,700} % {2,1,3.8,30,1,300} % Portfolio Value an Uncertainty Moerate 65% {26,4,14.1,60,2,700} rik averion 35% {4,2,3.8,30,1,500} % Portfolio Value an Uncertainty Low Rik 100% {8,4,14.1,30,1,700} Averion 100% Portfolio Value an Uncertainty