Sensitivity Analyses: Capturing the Most Complete View of Risk 07.2010 Introduction Part and parcel of understanding catastrophe modeling results and hence a company s catastrophe risk profile is an understanding of the uncertainties and sensitivities associated with the modeling process. Because uncertainty is a factor in both the development of catastrophe models and in the data input by model users, companies that want to fully comprehend the financial impact of catastrophes on their books of business should test the sensitivity of different inputs and assumptions. Some inputs and assumptions can alter the risk profile of a book of business significantly perhaps leading to best or worst case scenarios while others have minimal impact. Sensitivity testing allows users to see these relativities and make decisions based on them; for example, whether it is cost-effective to collect detailed data on structural characteristics that may have a mitigating or aggravating impact on losses. Of course, knowing how these inputs and assumptions can impact a particular loss outcome does not provide the whole picture in a risk analysis. A company s business model, risk appetite, and strategic goals, as well as its position in the insurance value chain, will impact how a company handles its risk management process, including its approach to conducting analyses. In the end, however, it is clear that performing sensitivity analyses helps companies make decisions that better align with their strategic business goals Sensitivity Tests as Tools for Conceptualizing Uncertainty In general, catastrophe model users face two kinds of uncertainty, primary and secondary. Primary uncertainty deals with the uncertainty around whether or not an event will occur. Given an event has occurred, secondary uncertainty deals with uncertainty in levels of damage and loss associated with that event. Both types of uncertainty can be better understood using sensitivity testing. More specifically, through different analysis options within the AIR software, users can test the sensitivity of some of the most critical components of the model, including: event catalogs building characteristics storm surge demand surge policy terms and conditions
Despite their importance, the impact of these factors can be misunderstood during discussions between catastrophe modelers and underwriters, which can make model results hard to interpret. Knowing and using all of the capabilities of AIR s software applications can help users determine the relative influence, so that company resources can be focused on understanding and improving processes related to those factors that have the greatest impact. Testing Sensitivity to Alternate Catalogs Looking at multiple views of hazard is a best practice wellknown to the industry. Testing the sensitivity of a model s hazard component in terms of event frequency can be accomplished by running different catalogs of simulated events. For the hurricane peril, for example, companies can choose to run AIR s standard or warm sea-surface temperature (WSST) conditioned catalog. While both represent views of long-term risk, the WSST catalog is developed based on only those years since 1900 in which sea-surface temperatures were warmer than average. Not surprisingly since hurricanes are fueled by warm ocean waters the WSST catalog incorporates higher rates of tropical cyclone activity. However, the relative impact of these two catalogs varies by region. Depending on where you are most heavily exposed, your portfolio losses can vary from between 10% and 30%. And in Massachusetts and other Northeast states, the WSST catalog has virtually no impact. In the case of earthquake risk, clients can choose to run their exposure through the time-independent or the time-dependent catalog. In time-dependent models of earthquake occurrence, the probability that an earthquake will occur on a particular fault increases with the length of time elapsed since the previous event on that fault, while in time-independent models the probability of an earthquake is independent of when the last event occurred. Unlike the standard and warm sea-surface temperature catalogs, where the impact is perhaps more intuitive, the impact of running these different earthquake catalogs cannot be predicted before the analysis is run. But clearly, different assumptions around the mean earthquake recurrence interval for specific faults will have implications on modeled losses. Testing Sensitivity to Building Characteristics The sensitivity of catastrophe model results can also be tested by looking at how losses differ based on different primary and secondary building characteristics. In AIR parlance, primary building characteristics include such attributes as occupancy, construction, height and year built, while secondary building characteristics include more detailed features, such as roof-covering, glazing type and roof-to-wall connection. Of course, in developing a catastrophe model, AIR performs extensive sensitivity testing in house. Some of the results of these tests are made available to our clients in the form of lengthy tables of relativities in the vulnerability of industry exposure by construction type, year built and height. But companies can dig even deeper by creating their own notional portfolio of a single construction type, for example, and varying other parameters incrementally. Sensitivity testing can also help companies understand the impact of model change, including changes geared toward increased realism in assessing building vulnerability. The most recent release of the AIR U.S. hurricane model, for example, features an enhanced ability to account for building age at much more granular geographic and temporal levels than before. Clients can test the sensitivity of these enhancements by comparing losses when exposures are either bulk coded or or left Unknown with results when exposures are coded with their actual year of construction. Exploring the impact of secondary risk characteristics and their complex interaction is a particularly worthy exercise in light of the increasing number of states adopting mitigation credits. And as efforts toward mitigation garner more attention, information on secondary risk characteristics (presence of hurricane shutters and roof anchorage, for example) will undoubtedly become more readily available for input into catastrophe models. Sensitivity testing will play a greater role in promoting the understanding of the impacts of these vulnerability characteristics on modeled losses. Indeed, users may want to run cost-benefit analyses to determine the benefit of collecting additional data on mitigation features. Users could run a test in which all homes (or buildings) in their portfolio have not just hurricane shutters, but an entire suite of hurricane mitigation features (Figure 1). Doing so would help these companies understand the relative importance of each mitigation feature and weigh this with the cost of collecting the information. For example, it is cheaper to collect roof cladding and shutter information via drive-by inspections 2
than it is to undertake a physical inspection in the home in order to determine the presence of a feature like hurricane straps. When adjusting large amounts of claims for residential policies after a catastrophic event it is generally accepted that some of the storm surge losses inevitably end up being paid as hurricane wind claims. AIR s default assumption is that this is on average 10% of the separately modeled storm surge. CLASIC/2 can easily be used to test the sensitivity of this assumption and adjust the amount of storm surge paid as wind loss depending on the company s view of the ability of adjusters to make the distinction. Figure 1: The home in this figure exhibits several mitigation features, including roof strips, clips on an interior wall, and hurricane shutters. (Source: AIR). Testing Sensitivity to Storm Surge The AIR storm surge module is a separate, fully probabilistic component of the U.S. hurricane model, one that incorporates detailed databases of coastal elevation and geometry, tide heights, and bathymetry (water depth relative to sea level). Losses arising from storm surge are highly sensitive to an exposure s location relative to the coastline, so accurate geocoding is essential; just how sensitive and how important accurate location information is can be explored through sensitivity tests that vary how exposure information is aggregated or disaggregated geographically. Also important for robust risk assessment is an accurate accounting of storm surge policy conditions for commercial exposures1. AIR s CLASIC/2 application, which analyzes detailed, location-level exposure data, enables insurers to code which policies do and which do not cover storm surge. For those policies coded to cover damage from storm surge, 100% of the modeled storm surge losses will be included in the total loss estimates. The model s sensitivity to the accurate coding of storm surge coverage was dramatically highlighted by Hurricane Ike in 2008. After working with a number of clients, it was clear that many companies were not coding their data for storm surge and instead relied on the 10% default assumed take up rate. AIR performed sensitivity tests on several of our clients books of business and showed that when properly coding their exposures for storm surge, modeled losses were far closer to actual losses. Testing Sensitivity to Demand Surge Demand surge is the increase in costs of materials, services, and labor due to increased demand following a catastrophic event. The AIR demand surge function is regularly reviewed to incorporate updated construction material and labor cost data. However, because demand surge is a phenomenon seen only with especially large catastrophes, there are relatively few events with which to validate demand surge functions, particularly outside of the U.S. Users can test the sensitivity of their losses to demand surge assumptions by adjusting the parameters of the default demand surge function in the software. Users have the ability, for example, to adjust the triggering loss or the rate at which demand surge increases with industry losses. Testing Sensitivity to Policy Conditions Determining the sensitivity of varying policy conditions is an important underwriting tool. Companies may quickly vary the attachment and exhaustion points, participation and even region of application within CATRADER as they explore risk transfer options. Sensitivity analyses are easily performed in CLASIC/2 by adjusting policy conditions and observing the difference in modeled losses. A company may want to explore the impact on the risk profile, for example, when minimum deductibles are implemented in coastal counties. Typical residential policies provide policy limits of 10% of building limit for appurtenant structures and some companies use this as replacement value also. Sensitivity tests can be performed to determine by how much you are overstating your risk if the true replacement value of appurtenent structures is only, say, 3%. Tests can also be used to explore how various combinations of policy terms might affect tail value at risk. And the aforementioned sensitivity testing of the impact of secondary risk characteristics can be used to fashion mitigation credits. 3
Sensitivity Testing the Impact of Exposure Data Quality The impact of uncertainty in the exposure data used as input to the models cannot be overemphasized. As largeloss U.S. hurricanes in recent years have demonstrated, the reliability of model output is only as good as the quality of the input exposure data. Uncertainties or inaccuracies in building characteristics or replacement values can propagate dramatically into the estimates of losses. Indeed the impact of poor exposure data quality can dwarf the impact of even the most comprehensive model update. Since catastrophe models estimate loss by applying vulnerability functions to the replacement value before applying policy terms and conditions, accurate replacement values are essential for obtaining accurate catastrophe loss estimates. If a property s replacement value is understated by 50 percent, for example, the estimated catastrophe loss will be understated by at least that much. Accurate location, age, and construction information is also critical. The following example quantifies the impact of imprecision in the specification of construction type. Scenario: Company A is an insurer that writes residential and commercial earthquake insurance in the U.S. Midwest and West Coast and is conducting an overall business review to identify the key factors that impact its portfolio loss results. Exposure data is the first area to be analyzed. This mix would be appropriate for Company A s analysis if the company s portfolio did in fact mirror the industry mix, but in this case, it does not. Therefore, the general masonry coding may be either penalizing or rewarding Company A by over- or underestimating earthquake losses. Furthermore, Company A has an underwriting guideline that indicates it prefers reinforced masonry. In order for Company A to determine if the general masonry coding is important, the company runs a sensitivity analysis. It compares the industry mix against their specific portfolio type: reinforced masonry. Running an analysis with the exposure as originally coded (i.e., as general masonry) and comparing it with an analysis in which the exposure is coded as reinforced masonry does in fact show a significant difference. At the 1% and 0.4% exceedance probabilities (100- and 250-year return periods), the difference is on the order of 15-16%, with the exposure coded as reinforced masonry generating significantly lower losses. Based on this result, the company feels that, at least for this part of the portfolio, losses may have been over-estimated. Indeed, new tools are now available that can augment company data with highly reliable, highly detailed propertyspecific data. AIR s TruExposure is just such a tool. Model users can choose to perform extensive sensitivity testing using the augmented data or to integrate it with their own. The first input parameter to be looked at is construction class and how it is specified in the exposure data. Company A knows it underwrites a broad range of construction and occupancy types. However, they have become accustomed to using generic codes for construction material masonry, for example. When only masonry is identified, a damage function is applied that represents a weighted average of all of the masonry construction types, with weights determined by their representation in the industry exposure. This now includes adobe, rubble stone masonry, reinforced masonry, and unreinforced masonry, among others. Figure 2: Using Detailed Exposure Data Can Alter Your Risk Profile 4
Conclusion Today s catastrophe models are more sophisticated than ever before. Still, applying model results to make educated decisions is not necessarily any easier; critical to informed decision-making is a comprehensive understanding of the uncertainties and sensitivities associated with the model and the modeling process, both in terms of outputs and inputs. Using sensitivity testing to gain a better understanding of uncertainty is a best practice AIR has promoted for years, and one that is being actively promoted by various regulatory regimes, including the Actuarial Standards of Practice (ASOP 38 and 39, in particular) and Solvency II. Best of all, sensitivity testing needn t be daunting; leading companies have implemented integration solutions to automate the process. References 1 Residential exposures are not typically covered under private flood insurance, but rather under the National Flood Insurance Program. 2 For a more extended discussion of uncertainty in catastrophe modeling, see the AIR Current Understanding Uncertainty. About AIR Worldwide AIR Worldwide (AIR) is the scientific leader and most respected provider of risk modeling software and consulting services. AIR founded the catastrophe modeling industry in 1987 and today models the risk from natural catastrophes and terrorism in more than 50 countries. More than 400 insurance, reinsurance, financial, corporate and government clients rely on AIR software and services for catastrophe risk management, insurance-linked securities, site-specific seismic engineering analysis, and property replacement cost valuation. AIR is a member of the ISO family of companies and is headquartered in Boston with additional offices in North America, Europe and Asia. For more information, please visit www. air-worldwide.com. 2010 AIR Worldwide. All rights reserved. 5