RMS U.S. Hurricane Model

Transcription

1 RMS U.S. Hurricane Model Presentation to Florida Commission on Hurricane Loss Projection Methodology June 1, 2005

2 Presentation Overview Hurricane model component overview Discussion of changes to model components 2005 Risk Management M ag t Solutions, S i Inc. n

3 Overview of the Hurricane Model Define Hurricane Assess Wind speed Calculate Damage L O S S Basin-wide track and parameter simulation and calibration (including extra- tropical transition) Pressure history simulation and calibration Importance sampling of simulated tracks Time-stepping wind field calculation of over-water gradient wind Directional factors for surface roughness upstream of over-land location ZIP Code data resolution Engineering model calibrated with historical claims 26 vulnerability classes based b on material, height, and occupancy Mitigation measures County inventory to infer material/height if unknown 2005 Risk Management M ag t Solutions, S i Inc. n

4

5 Pressure History Model for Each Track Use random-walk model to add pressure histories along length of each simulated track Overland filling rates follow functional form of Kaplan and DeMaria (1995) Minimum pressures constrained by sea surface temperature Calibrate model at t coastline based on historic pressure distributions along linear coastal segments 2005 Risk Management M ag t Solutions, S i Inc. n

6 Importance Sampling of Simulated Storms 100,000 years of hurricane activity for Florida is reduced to 19,067 events Number of events differs from 2003, due to changes in event definition for calculating loss costs Total rate of occurrence in model equals observed mean annual rate of occurrence of historic storms 2005 Risk Management M ag t Solutions, S i Inc. n

7 Wind Field Calculation Over Water Based on the work of Georgiou (1983; 1985), the model calculates the gradient wind field Wind field at 10 m height is calculated as a function of the gradient wind speed 2005 Risk Management M ag t Solutions, S i Inc. n

8 Estimation of Over Land Peak Gust Wind Speed Creation of a database describing the land surface in n terms of roughness length using: National Land Cover Data dataset produced by the United States Geological Survey (based on mid-1990s Landsat Thematic Mapper satellite data) Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) satellite imagery to ensure the timeliness of the data Representative roughness length is assigned to each land- cover class (some lengths updated for 2004 submission) Coefficients describing the impact of land friction are calculated by sampling the roughness lengths for local and upstream conditions based on the direction of the wind 2005 Risk Management M ag t Solutions, S i Inc. n

9 Sampling Surface Roughness South Miami # # 2005 Risk Management M ag t Solutions, S i Inc. n

10 Variable Resolution Grid Variable Resolution Grid (VRG) makes it possible to model wind at a higher resolution than ZIP Code Highest resolution cells are in areas of high exposure (major metropolitan areas) and high hazard gradient (e.g., coastal regions) Miami: ZIP Code vs. VRG V G cell c l sizes 2005 Risk Management M ag t Solutions, S i Inc. n

11 Base Vulnerability Curves 26 base curves (including residential and commercial lines) developed separately for building and contents based on a combination of: Construction class Occupancy Building height Modified based on year of construction Before 1975; 1975 to 1993; 1994 or after Additional Living Expenses (ALE) are a function of building damage and occupancy 2005 Risk Management M ag t Solutions, S i Inc. n

12 26 Base Vulnerability Curves MDR (%) Windspeed 2005 Risk Management M ag t Solutions, S i Inc. n

13 Model Uncertainty in Mean Damage Ratio The mean damage ratio (MDR) is assigned a coefficient of variation (CV) based on the distribution of historic insurance losses paid MDR (%) Peakgust (mph) 2005 Risk Management M ag t Solutions, S i Inc. n

14 Secondary Modifier Methodology Building specific attributes or mitigation measures Modeled by scaling vulnerability ity functions up or down from the unknown vulnerability curve Basic vulnerability Modified vulnerability Modified vulnerability MDR D (%) % Wind speed (mph) Wind speed (mph) Risk Management M ag t Solutions, S i Inc. n

15 Summary of Model Changes Vulnerability Update to the mobile home vulnerability curve Hazard Update of event rates to account for another year Included the fact that no events occurred in 2003 Definition of an event Changes to roughness and wind speed aggregation Updated land use classification Updated roughness lengthsl Changes in Variable Resolution Grid (VRG) aggregation to Zip 2005 Risk Management M ag t Solutions, S i Inc. n

16 Mobile Home Vulnerability Update The mobile home vulnerability update utilized data from the hurricanes of 2004, in addition to prior event data. A detailed description of the mobile home update was presented to the Professional Team, including a review of the data utilized in the update, and how the data was processed and used in the derivation of the updated vulnerability curves. The mobile home update resulted in a substantial increase for pre-1994 mobile homes but a slight decrease for post mobile homes. State-wide, the change in the mobile home output range was 66% ($500 deductible case) Other than mobile homes, no other changes were made to the vulnerability model Risk Management M ag t Solutions, S i Inc. n

17 Overall Change in Loss Costs (con( con t) Form A-9, A Figure 24(b),(c) Mobile Home Only Excluding Mobile M HomeH e 2005 Risk Management M ag t Solutions, S i Inc. n

18 Inclusion of 2003 Hurricane Activity Renormalization of Hurricane Event Rates Accounts for each new year of historic hurricane activity Annual rate (2004) = Annual rate (2003) * (103/104) Results in very minor changes in annual loss costs for all Florida counties (~1% in each county) 2005 Risk Management M ag t Solutions, S i Inc. n

19 Definition of an Event Event starts when a storm reaches category 1 wind speeds and causes loss in Florida This is implemented by defining the first point the storm is within 250 km of the Florida coast and windspeeds are at least category 1. After that point all losses are considered regardless of the strength of the storm If a storm is greater than or equal to category 1 in the Atlantic basin but weakens to less than category 1 when within 250 km of Florida the storm is not included Risk Management M ag t Solutions, S i Inc. n

20 Example Track 2005 Risk Management M ag t Solutions, S i Inc. n

21 Change in County Loss Costs Using new Event Definition Minimum Reduction -0.04% Maximum Reduction -0.40% Statewide Reduction -0.18% 2005 Risk Management M ag t Solutions, S i Inc. n

22 Update to Land Use Land Cover (LULC) data Land Use Land Cover data used to derive roughness lengths was previously based on National Land Cover Data (NLCD) provided by USGS, this data represents conditions in the early to mid 1990 s. This year, the Land Use Land Cover data was updated using ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) data. The ASTER data was obtained between May 2000 and October Risk Management M ag t Solutions, S i Inc. n

23 ASTER Classification Process Data Import Georeferencing Classification Base Layer open, grassland, forest, etc. Built-up up layer Presence of vegetation allows designation into i o low intensity t built lt- up areas For city center and dense city center areas manual classification is needed using factors such as shadow lengths, image texture, and visual inspection. Water Validation Aerial imagery Bodies of water are represented properly etc Risk Management M ag t Solutions, S i Inc. n

24 Santa Rosa County 2005 Risk Management M ag t Solutions, S i Inc. n Previous Updated Built- up p areas s

25 Close Up on Santa Rosa County Built-up up areas Previous Update 2005 Risk Management M ag t Solutions, S i Inc. n

26 Washington County 78% of Washington County Exposure is in ZIP Previous Updated 2005 Risk Management M ag t Solutions, S i Inc. n Built- up p areas s

27 Zip in Washington County 2005 Risk Management M ag t Solutions, S i Inc. n

28 Change in Loss by County (excluding Mobile Home) Largest changes for low population counties with a large percentage of value in an individual postal code. Storm set changes are between -1.0% and -1.4% for all counties. Zip aggregation changes are typically larger. Total Change Storm m Set Roughness Zip Code Aggregation Total t l Change Storm S o m Set Roughness Zip Z Code Aggregation BAKER -12% LIBERTY -15% BAY -12% MADISON -15% CHARLOTTET -21% MANATEE E - 12% % COLLIER -13% NASSAUS U - 19% % DIXIEE -18% OKALOOSA S - 15% % FRANKLIN -32% SANTA ROSA -23% HAMILTON -14% ST. JOHNS -19% JEFFERSONE F N -13% WAKULLAA A - 22% % LAFAYETTE -16% WALTONA O - 33% % LEEE -14% WASHINGTONA 32% LEVY -14% STATEWIDE -8.5% = primary contributor to change 2005 Risk Management M ag t Solutions, S i Inc. n

29 Output Range Example Masonry Owners Loss Cost for Masonry 6.0 to to to to to to 0.5 No Exposure Zip Masonry (Figure 20) County Masonry Owners Loss Cost (Form A-7, A $500 Deductible) 2005 Risk Management M ag t Solutions, S i Inc. n

30 Compliance with 2004 Standards Presentation i n to Florida Commission on Hurricane Loss Projection Methodology June 1, 2005

31 General Standards

32 Standard G-1: G Scope of the Computer Model and its Implementation Model version number: RiskLink 4.5a Updated from RiskLink 4.32a in 2003 submission Concise technical description of the model provided in the 2004 submission document Use of Advanced Spaceborne thermal Emission and Reflection Radiometer (ASTER) satellite imagery described Number of events updated to 19,067 Minor changes to improve clarity

33 Standard G-1: G Scope of the Computer Model and its Implementation (2) Flow diagram illustrating interactions among major model components No material changes in component interactions Minor revision to improve clarity of descriptions Pertinent references were consolidated, expanded, and clarified in response to disclosure 4 Other aspects of this standard remain unchanged from 2003

34 Standard G-2: G Qualifications of Modeler Personnel and Independent Experts Employee/revenue/client statistics and biographies updated Pre-2004 revenue data by market segment removed Professional credentials updated reflecting changes the model team or in employment status, and relevance to the previous or latest model generation Project and/or employment departures: Anders Brix,, Brian Owens, Andy Schaub Additions to project: Munish Arora, Arundhati Bopardikar, Prasad Gunturi,, Sherry Huang, Sridhar Iyer, Vikrant Kalhan, Sameer Khandekar,, Phillip LeGrone,, Hemant Nagpal,, Terrance Ng, William Andrew Wheeler, Pooja Sayal, Raj Singh, Jim Tomcik,, and Liang Zhang

35 Standard G-2: G Qualifications of Modeler Personnel and Independent Experts (2) Workflow diagram updated to link all personnel to business workflow phases and to include decision-making phases in technical reviews Version of the model that part time employees / consultants contributed to was added to Table 8 Dates of independent reviews in disclosure G-2.3a G added and/or clarified Forms G1-G6 G6 on Expert Certification updated with each revision cycle Due to Anders Brix departure from RMS, the final Expert Certification for the statistical standards was conducted by Rohit Mehta, Senior Engineer, M.S. Statistics Other aspects of this standard remain unchanged from 2003

36 Standard G-3: G Risk Location RMS response to this standard remains unchanged from 2003 ZIP Code data provided by a third-party developer and is of an August 2003 United States Postal Service vintage The RMS model does not use ZIP-Code centroids as proxies for exposure. If a building location is entered as a ZIP Code, then the model uses wind speeds that are exposure weighted averages of wind speeds across the ZIP-Code extent. ZIP-Code information is examined by RMS for consistency and is subject to standardized quality control testing and checking by experts employed by RMS for that purpose.

37 Standard G-4: G Units of Measurement Specific conversion factors used by the model disclosed All conversion factors related to units are consistent with 2002 Institute of Electrical and Electronics Engineers/American Society for Testing and Materials (IEEE/ASTM) standards Meteorological and wind engineering conversion factors are taken from standard meteorological and wind engineering sources (Holland 1993; Cook 1985) Other aspects of this standard remain unchanged from 2003

38 Standard G-5: G Independence of Model Components RMS response to this standard remains unchanged from 2003 The vulnerability, meteorological, and actuarial model components are theoretically sound and have each thoroughly and independently tested and calibrated They have also been tested in an integrated way to ensure that the relationships between the components are reasonable

39 Meteorological Standards

40 Standard M-1: M Official Hurricane Set Response updated to reflect the hurricane set used by RMS for Florida i a matches the Official Storm Set S t and includes all storms listed in the Report of Activities through 2003 Meteorological eorol references were w consolidated and included in response to M-1.2M Other aspects s of RMS response to t this t standard remains r unchanged from 2003 The hurricane i set used to o develop the RMS U.S... Hurricane ricane model for Florida includes both landfalling and by-passing hurricanes that produce losses in Florida. The hurricane e set used by RMS R S matche s the Official Storm Set provided by the Commission, including storms through h

41 Standard M-2: M Hurricane Characteristics Process for converting gradient winds to surface winds has been updated to provide more detail (disclosure M-2.3) M Comparison of historic and modeled multiple landfall occurrences by pair of adjacent 50 km gates was updated (Figure 6) to reflect changes in 1) modeled event definition and 2) inclusion of 2003 hurricane activity

42 Standard M-2: M Hurricane Characteristics (2) Complete reference list relevant to this standard included in response to M-2.9M Other aspects of this standard remain unchanged from 2003 Each of the methods for depicting hurricane characteristics in the RMS U.S. Hurricane model is based on currently accepted scientific literature or RMS information previously accepted by the FCHLPM.

43 Standard M-3: M Landfall Intensity Definition of an event modified to include: For the purposes of calculating losses, a storm is first considered when maximum winds reach Category 1 wind speeds and damage is caused in Florida. From that point on wind speeds and losses are calculated regardless of whether maximum winds are greater than or less than Category 1. Example provided during the model change overview presentation

44 Standard M-3: M Landfall Intensity (2) The categorization of landfall intensity of a hurricane in the model remains unchanged from 2003 The maximum wind speeds produced by the model per hurricane category are consistent with the Saffir-Simpson Simpson Scale.

45 Standard M-4: M Hurricane Probabilities Description of use of sea surface temperature data expanded for clarity Distributions of hurricane frequency by category and region have been updated to reflect the modified event definition and renormalization of stochastic event rates Other aspects of this standard remain unchanged from 2003 Modeled probability distributions for hurricane intensity, forward speed, radii for maximum winds and landfall angle are consistent with observed historical hurricanes in the Atlantic basin. The basis for developing probability distributions for each of these parameters is the record of historical hurricanes.

46 Standard M-5: M Land Friction and Weakening Response update to reflect changes in land use from the time that the National Land Cover Data (NLCD) was obtained (mid-1990s). The land use data for the RiskLink 4.5a model was updated using Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) satellite imagery. The ASTER data is based on data collected between May 2000 and April Response also updated to note model changes were made to the roughness length values for some land use classes. These roughness lengths were reviewed by Dr. Craig Miller.

47 Standard M-5: M Land Friction and Weakening (2) Figure 7: Variation of Friction Coefficients with Distance to Coast for Florida ZIP Codes was updated to reflect the above roughness updates

48 Standard M-5: M Land Friction and Weakening (3) Other aspects of land friction in this standard remain unchanged from 2003 The treatment of roughness in the RMS U.S. Hurricane model is based on Cook/ESDU methodologies Wind speeds at a location are impacted by the local and upstream surface roughness in eight different directions Approach used has been reviewed by Cook, who concluded RMS have correctly implemented well-established established scientific principles and practices to produce a high quality model for the wind speed field in U.S. Hurricanes.

49 Standard M-5: M Land Friction and Weakening (4) All aspects of the overland weakening rate in this standard remain unchanged from 2003 The wind speed decay for each storm follows the functional form of the Kaplan and DeMaria (1995) model Based on a cleaned data set Widely accepted peer-reviewed reviewed reference The RMS model allows variation in the filling rate but in the mean is consistent with Kaplan & DeMaria

50 Standard M-5: M Land Friction and Weakening (5) The exhibit in Form M-2 M 2 (Figure 10) has been updated to reflect the 104 year historical maximum one-minute wind speeds and is based on updated roughness data The exhibit in Form M-2 M 2 (Figure 11) has been updated to reflect the changes in roughness, event definition, and the inclusion of the 2003 hurricane activity Both figures are also now presented as contour plots, as required by the standard

51 Standard M-6: M Logical Relationships of Hurricane Characteristics Description of response to Form M-3 M 3 has been included to note that in our model, Rmax is also dependent on latitude (in addition to central pressure). This dependence is reflected in the ranges provided in the Form M-3 M 3 table. Other aspects of this standard remain unchanged u from f 2003 The radius to maximum winds is modeled based on historical values of radii of maximum winds. The magnitude of asymmetry increases with increasing translational speeds, all other factors being held constant. The wind speeds decrease with increasing surface roughness, all other factors held constant.

52 Vulnerability Standards

53 Standard V-1: V Derivation of Vulnerability Functions Other than mobile homes, no other changes were made to the vulnerability model from 2003 submission. A - Vulnerability functions are based on well-supported structural and wind engineering principles and detailed analyses of historical claims data B Methods used to derive vulnerability functions are theoretically sound C Secondary Modifiers to Vulnerability functions based on engineering neering understanding, damage statistics, building g codes, engineering studies, wind tunnel experiments D Construction types and characteristics are used in the derivation and application of vulnerability functions E Changes in building codes/construction practices are modelled through Year Modifiers.

54 Standard V-1: V Derivation of Vulnerability Functions (con t) F Separate vulnerability functions Derived for structures and mobile homes, Appurtenant structures use same function as main structure, but can be input separately Separate functions for contents, ALE G Minimum wind speed generates damage 50 mph peak gust ~42 mph one minute sustained Form V-1 V 1 has been submitted and reflects changes to the form as well as the mobile home vulnerability functions.

55 Standard V-2: V Mitigation Measures The following secondary modifiers are available in the model: Roof sheathing strength Roof covering Roof anchor Foundation system Wind resistance of window openings Wind d resistance of door openings Roof geometry Opening protection (shutters) The application of modifier options ons are reasonable e when applied individually and in combination Form V-2 V 2 has s been submitted i ted

56 Actuarial Standards

57 Standard A-1: A Modeled Loss Costs The track and pressure of each tropical cyclone are modeled throughout its lifetime in the Atlantic basin from genesis to decay. For the purposes of calculating losses, a storm is first considered when maximum winds reach Category 1 and damage is caused in Florida. From that point on wind speeds and losses are calculated regardless of whether maximum winds are greater than or less than Category 1.

58 Standard A-2: A Underwriting Assumptions RMS response to this standard remains unchanged from 2003 RMS uses historical loss information in the development of vulnerability functions Any adjustments, edits, inclusions, or deletions to insurance company input are based on accepted actuarial, underwriting, and statistical procedures and are documented in writing The vulnerability of property observed in historical events is assumed to be indicative of vulnerability of such property in future events where subjected to similar wind loads

59 Standard A-3: A Loss Cost Projections RMS response to this standard remains unchanged from 2003 RMS loss cost calculations do not include expense, risk loads, investment income, premium reserves, or profit margins RMS loss cost projections do not make any prospective provision for economic inflation. The RMS model does not include demand surge in its loss cost projections

60 Standard A-4: A User Inputs RMS response to this standard remains unchanged from 2003 Input data to the RMS hurricane model is explicitly provided by the user for each particular analysis. The model assumes that inputs provided by the user are reflective of actual exposures.

61 Standard A-5: A Logical Relationship to Risk RMS response to this standard remains unchanged from 2003 Loss costs do not display an illogical relation to risk, nor do they vary significantly when the underlying risk does not change significantly Loss costs are positive and non-zero for all Florida ZIP Codes Loss costs do not increase as roughness increases As illustrated in the output ranges, loss costs: do not increase as quality increases, all other factors held constant decrease as deductibles increase exhibit relationships between coverages and loss costs for each coverage that are consistent and reasonable

62 Standard A-6: A Deductibles and Policy Limits RMS response to this standard remains unchanged from 2003 Loss net of deductible and limit = ( x D) f ( x) dx + L[ 1 F( D + L) ] D+ L D where x = ground-up up loss D = deductible L = limit f (x) = p.d.f. of the ground-up up loss F (x) = c.d.f. c of f the t e ground-up up lossl s

63 Standard A-7: A Contents RMS response to this standard remains unchanged from 2003 Contents and structure loss relativities are based on an analysis of historical claims data The relationship between structures and contents losses are reasonable

64 Standard A-8: A Additional Living Expenses (ALE) RMS s s response to this standard remains unchanged from 2003 ALE losses include only hurricane related factors, are theoretically sound, and consider the time to repair the structure ALE losses are determined based upon the estimated damage to the structure ALE and structure loss relativities are based on an analysis of historical claims data and is reasonable

65 Standard A-9: A Output Ranges Differences in the updated output ranges from RMS s s 2003 submission ssi are reasonable and driven by the following f g factors: Vulnerability Update to the mobile home vulnerability curve Details presented as part of the vulnerability standards Hazard Update of event rates to account for another year Included the fact that no events occurred in 2003 Definition of an event Changes to roughness and wind speed aggregation Updated land use classification Updated roughness lengths Changes in Variable Resolution Grid (VRG) aggregation to Zip

66 Statistical Standards

67 Standard S-1: S Modeled Results and Goodness of Fit We have included charts showing an assessment of uncertainty in loss costs for output ranges. Other aspects of this standard remain unchanged from RMS uses empirical methods in model development and implementation to match stochastic storm generation to historic data, which is supported by currently accepted scientific literature. The chosen distributions have been shown to have reasonable agreement with the historic data.

68 Standard S-2: S Sensitivity Analysis for Model Output In addition to the sensitivity analyses that were shown last year, we have included charts showing the change in loss costs due to a 1% change in the following variables: Central pressure difference RMax Forward speed Other aspects of this standard remain unchanged from 2003

69 Standard S-3: S Uncertainty Analysis for Model Output In addition to the uncertainty analyses that were shown last year, we have included charts showing the change in loss costs due to changing the following variables to the 5% and 95% confidence bounds: Central pressure difference RMax Forward speed Other aspects of this standard remain unchanged from 2003

70 Standard S-4: S County Level Aggregation RMS s s response to this standard remains unchanged from 2003 The standard error of each output range at the county level of aggregation is less than 2.5% of the loss cost estimate.

71 Standard S-5: S Replication of Known Hurricane Losses We have added comparisons between observed and modeled losses for mobile homes in hurricane Charley. Other aspects of this standard remain unchanged from 2003 The RMS model is able to reliably and without significant bias reproduce incurred losses on a large body of past hurricanes, both for personal residential and mobile homes.

72 Standard S-6: S Comparison of Projected Hurricane Loss Costs RMS s s response to this standard remains unchanged from 2003 The difference between historical and modeled annual average statewide loss costs are reasonable, by established statistical expectations and norms.

73 Computer Standards

74 Standard C-1: C Documentation Computer Standards document binder Contains material addressing each standard Prepared in electronic form Organized hierarchically, following structure of standards Document consistency imposed via templates and development standards External documentation supplements documentation within source code Binder updated and expanded for RiskLink 4.5a to: Reflect model updates Reflect improvements in development processes and procedures Reflect standards updates

75 Standard C-2: C Requirements Documents and requirement types covered: RiskLink System Administration (resource, security, database interface) RiskLink DLM User Guide (user interface, functionality) Coding Standards (user interface, human factors design, source code design, implementation, database design and coding, component nt interface design) Marketing Requirements Documents (functionality)( ty Functional Specifications (user interface, functionality, data, resources, o r e, quality u y assurance) s u a c ) Project management documents (resources) Visual SourceSafe 6.0 (security) IT Security documents (security) Quality Assurance test plans (quality assurance, user interface, functionality, data)

76 Standard C-3: C Model Architecture and Component Design Control flow diagrams Illustrate key processes, branches, and loops within software Data flow diagrams Illustrate data-dependency dependency relationships between software components Component diagrams Document software module interfaces Database schema documentation Includes field type definitions Includes databases and other data files (binary and text) Model component custodians interviewed during on-site audit

77 Standard C-4: C Implementation Coding guidelines s documents Standards ds for f software e coding,, development environment setup, component design, file versioning, source control system usage Include procedures for creating and verifying data Components traceable to t code level Data flow diagrams demonstrate hierarchical decomposition d t on of processes Processes range from very high level down to processes implemented via individual subroutines in source code Data dictionary i used to t link l processes s s to source code Software components sufficiently and consistently commented Demonstrated by table of software components listing comment lines vs. total lines, and by comment review

78 Standard C-5: C Verification (1) RMS description of procedures for verification and tests performed for each software t component were expanded to reflect the t expansion of quality assurance, and in response to 2004 standards Summary of procedures for general testing Design and prototype of model modifications/additions Written specifications describe purpose, algorithm (e.g., pseudo- code, control flowcharts, or data flow diagrams), and testing plans Review of specifications by both engineers and by senior software developers Independent execution of test plans by RMS Software Quality Assurance (QA) and Engineering QA departments Code inspections,, reviews, and walk-throughs to verify code correctness Catch run time errors by embedding numerous logical assertions, exception handling mechanisms, and flag-triggered output statements in source code

79 Standard C-5: C Verification (2) Component testing Rational/IBM Enterprise package is used for analyzing and testing all components Software debuggers are extensively used to verify execution paths and calculation results Custom Tests are used to check components using range of input values Suites of test scripts run to: Check output from individual components and collection of related components Check output from logical assertions, exception handling mechanisms, and flag-triggered output statements Perform regression tests (comparing output to expected results) Aggregation tests s are performed by running the product as complete package to check all the components and data files accessed by model. Run multiple times to verify repeatability of results.

80 Standard C-5: C Verification (3) Data testing Range of testing software tools used to check databases and data files accessed by components Database packages used to check the range of values Mapping Software used to check the spatial distribution of data Rational Robot regression tests used to check values and integrity of all the data files Custom tools created in Excel, Access or C++ to perform cross checks, run statistical tests, and generate data visualization output o (e.g. graphs and charts) from datasets Vulnerability it database ab checked for combination b on of occupancy and construction classes, range a of damage ag ratios, plot of f damage ratio ios vs. peak gust, etc. e Binary i files accessed by y components s checked via software w e that t converts c s data from text to binary, binary to text, then performs a comparison of the input and output text filesf Aggregation tests performed by running the product as complete package to check all the components and data files accessed by modelm

81 Standard C-6: C Model Maintenance and Revision RMS response to this standard remains essentially unchanged from 2003, but more detail now included in submission Visual Intercept Quick Reference Guide Describes use of tool for tracking features and fixes Sample Visual Intercept incidents Demonstrate use of VI for tracking feature development Client Response System CRS knowledge base article demonstrates support documentation CRS ticket demonstrates tracking of interaction with client

82 Standard C-6: C Model Maintenance and Revision (con t) File Versioning Documents how software versioning is done Most Recent Change By Component Lists dates of code change by software component Specifications Functional, technical, and/or engineering specs prepared for all features and fixes Product Delivery Overview Illustrates maintenance / revision process in detail

83 Standard C-6: C High-level Model Revision Policy Product Feedback Technical Marketing Internal & External Research Business Case Other Projects & Constraints Marketing Committee Approved Model Revisions

84 Standard C-6: C Detailed Model Revision Policy

85 Standard C-7: C Security RMS has documented and implemented i security procedures es for access to code, data, and documentation in accordance with standard industry practices Security requirements documented and enforced by RMS Legal Department Company personnel are trained in security requirements and procedures Company personnel required to sign non-disclosure agreement as condition of their employment Physical security maintained using u locked doors, key- card access, video cameras, and security patrols

86 Standard C-7: C Security (con( con t) Servers and desktops d audited for security compliance Microsoft Visual Source Safe used to track modifications of source code. VSS maintains source code in encrypted form. A separate login required to access source code. Nature and author of all changes recorded. RMS network protected via hardware firewalls Security at client site is ensured by compiled binary files which cannot be modified without access to original source code Servers and desktops protected with Norton Antivirus software Servers backed up nightly. Off-site backups are maintained at a secure commercial facility.