AN APPROACH TO THE STUDY OF MULTIPLE STATE MODELS. BY H. R. WATERS, M.A., D. Phil., 1. INTRODUCTION

Transcription

1 AN APPROACH TO THE STUDY OF MULTIPLE STATE MODELS BY H. R. WATERS, M.A., D. Phil., F.I.A. 1. INTRODUCTION 1.1. MULTIPLE state life tables can be considered a natural generalization of multiple decrement tables in the same way as the latter can be considered a natural generalization of the ordinary mortality table. The essential difference between a multiple state model and a multiple decrement model is that the former allows for transitions in both directions between at least two of the states, see for example Haberman (1983, Fig. 2). whereas in the latter, transitions between any pair of states can be in one direction only, if they are possible at all; see for example Haberman (1983, Fig. 1). Multiple state life tables, unlike mortality tables and multiple decrement tables, are not included in the professional actuarial examination syllabuses in Britain and have only rarely been mentioned in the British actuarial literature, one example being C.M.I.R. (1979, Appendix 2). Despite this, such models have been included in the actuarial examination syllabuses of other countries for several years, for example Denmark, and would appear to have obvious actuarial applications, for example sickness insurance. Dr Haberman s paper (J.I.A. 110) on the subject of multiple state models is to be welcomed since it provides both an introduction to this subject and an interesting application. The approach to the study of multiple state models put forward by Haberman (1983, $3) is characterized by the use of flow, orientation and integration equations. For brevity we shall refer to this as the FOI-approach to multiple state models. The purpose of the present paper is to put forward an alternative approach to multiple state models. This approach uses the forces of transition, or transition intensities, between states as the fundamental quantities of the model. For brevity we shall refer to this as the TI-approach It is clearly not sensible to maintain that one particular approach to such a complex structure as a multiple state model is always to be preferred to another approach, the particular application and the data available will to a large extent determine the most suitable approach. Nevertheless, it is our opinion that in general terms and with actuarial applications in mind, the TI-approach has important advantages, both theoretical and practical, when compared to the FOI-approach In the following section we list and discuss briefly the four aspects of the study of multiple state models where, in our opinion, the TI-approach is better than the FOI-approach. In the remainder of the paper we shall illustrate the TI-approach by using it to analyse a simple example of a multiple state model. 363

2 364 An Approach to the Study of Multiple State Models 1.4. This paper contains nothing that is original. The TI-approach to multiple state models has been discussed frequently in the actuarial literature outside Britain, although there has been no recently published comprehensive account of the theory. Our illustration of the TI-approach in this paper will necessarily be superficial but we hope that the reader will be able to appreciate the important points of the approach from our illustration and will also appreciate that multiple state models could have interesting actuarial applications and that the TI-approach has important advantages when compared to the FOI-approach The author is grateful to Professors J. M. Hoem and A. D. Wilkie for many useful comments on an earlier draft of this paper. 2. THE ADVANTAGES OF THE TRANSITION INTENSITIES APPROACH 2.1. In this section we shall outline four aspects of multiple state models where, in our opinion, the TI-approach has important advantages when compared to the FOI-approach. The reader may well find that the points made in this section become more meaningful when he has studied both the remainder of this paper and Haberman (1983). The reader interested in a far more authoritative and detailed comparison of different approaches to the study of multiple state models should consult Hoem and Funck Jensen (1982) The four aspects of multiple state models we shall consider are: (a) the specification of the model (b) natural assumptions to aid computation (c) the comparison of different models (d) the estimation of probabilities or intensities and the statistical properties of the estimators. In the remainder of this section we shall consider briefly each of these aspects in turn (a) The natural setting for multiple state models is a particular type of stochastic process, namely a time continuous Markov chain with a finite state space. See Hoem and Funck Jensen (1982, 1.2). By regarding the transition intensities as the fundamental quantities, the TI-approach, as we hope to demonstrate in the following sections, keeps the true, stochastic, nature of the model and the associated assumptions in the foreground. That this is not necessarily true for the FOI-approach is demonstrated by Haberman (1983). His general model is specified in his 3 algebraically by three sets of equations. There is no indication in this specification of the stochastic nature of the model but when we reach his 6 we find that Corresponding to a (multiple state) life table model an associated finite-space, continuous-time, time-inhomogeneous Markov process can be defined. It must be stressed that in each case the underlying model is the same, it is only the approach that is different and we suggest that an approach which keeps the full nature of the model in the foreground is to be preferred to one which need not. The dangers inherent in not

3 An Approach to the Study of Multiple State Models 365 doing so are demonstrated by Haberman (1983, 6). The transition probabilities, in his notation, are defined by his formula (15). However, we find these probabilities defined again in his formula (18) and defined as a ratio of two expected values whose values, presumably, depend on the transition probabilities themselves (b) When using the TI-approach it is natural, although not necessary, to assume the transition intensities are piecewise constant functions of age whenever this is convenient, for example when estimating the intensities (see 5). The corresponding natural assumption in the FOI-approach is a set of integration equations. The integration equations used by Haberman (1983, $3) are similar to the assumption of a uniform distribution of deaths between integral ages for a mortality table. This latter assumption, even though it implies a decreasing force of mortality between integral ages, is usually regarded as acceptable for practical purposes. However, Hoem and Funck Jensen (1982, 3), while noting that data analysed using different types of assumptions often give very similar numerical results, are extremely critical of the use of integration equations, especially when, as in Haberman (1983) they appear to be a part of the specification of the model rather than just a computational aid. Hoem and Funck Jensen give references to practical studies where such assumptions have given unacceptable results and provide a simple example themselves (c) In the opinion of Hoem and Funck Jensen (1982, 1.2), human behaviour is reflected most easily in the specification of the state space and transition intensities, and the fundamental assumptions of a model specification should be made in those terms, not by means of transition probabilities, survival or transfer functions, or other derived quantities. We endorse this opinion and add as a corollary that if we wish to compare or relate two different multiple state models it is easier to do so in terms of transition intensities, i.e. use the TI-approach, than to do so in terms of transition probabilities or multiple state life tables, i.e. use the FOI-approach. A simple example will clarify this point. Suppose we have a conventional multiple decrement table and we wish to define the related single decrement tables. The easiest and most satisfactory (and perhaps only?) way to do this is by equating the relevant forces of decrement, i.e. we define a related single decrement table by requiring that for the relevant decrement α for all ages X, in the usual notation. Attempts to define the related single decrement tables using transition probabilities are, in our opinion, less then satisfactory. See for example Neill (1977, 9.4) (d) Let us suppose that, having specified a multiple state model in mathematical terms, we obtain some relevant data and we wish to use this data to estimate some of the quantities that are a part of the model, for example the transition intensities or the transition probabilities. It is apparent from the stochastic nature of the model that any estimates we derive from our data will be

4 366 An Approach to the Study of Multiple State Models subject to random variation and will have a mean value and a variance. The mean value of the estimator will be equal to the quantity being estimated, provided the estimator is unbiased. The variance of the estimator can be more difficult to ascertain but there are at least two situations where it is important to know this variance, or at least to be able to estimate it. These situations are (i) if we intend to graduate our estimates, and (ii) if we wish to test whether a set of estimates based on one set of data is significantly different to a set of estimates of the same quantities based on a different set of data. Generally speaking the TI-approach copes with this problem very easily, whereas other approaches may not. For a detailed discussion of this point see Hoem and Funck Jensen (1982, 4). We suggest that although these considerations may not be very important in some demographic applications of multiple state models, they are likely to be very important in some actuarial applications. For evidence of this see C.M.I.R. No, 4 (1979, para 2.4). 3. THE SPECIFICATION OF A MULTIPLE STATE MODEL 3.1. We shall illustrate the TI-approach by using it to analyse a very simple example and we hope that in doing so we shall clarify some of the points made earlier. In this section we shall specify the model and discuss transition intensities. In the remaining sections we shall discuss transition probabilities, parameter estimation and some practical points relating to the application of the model It must be stressed that even though we shall discuss a very simple model, our treatment of this model must necessarily be superficial; we shall mention only those points most relevant to the applications we have in mind. The reader interested in a more comprehensive treatment should consult the references quoted later Our example can conveniently be represented by the diagram in Figure 1. The boxes represent the three possible states an individual can be in at any time and the arrows indicate the possible transitions between these states. State 3 is Figure 1

5 An Approach to the Study of Multiple State Models 367 absorbing in the sense that once an individual has entered it he cannot subsequently leave it. In our applications we shall interpret state 3 as Dead. Transfer is possible from both state 1 and state 2 to either of the other two states. This is precisely the model described by Haberman (1983, Fig. 2). With Haberman s interpretation we would interpret being in state 1 as not having a specified disease Z, and being in state 2 as being ill with disease Z. To emphasize the insurance applications of the model we shall adopt a different interpretation. We shall regard the model as a mathematical model for an individual permanent health insurance (PHI) policy. We interpret being in state 1 as the policyholder is healthy, being in state 2 as the policyholder is ill, where ill means ill in accordance with the conditions of his policy although he may not be receiving any benefit since the duration of his current illness may not yet exceed the policy s deferred period, and being in state 3 as the policyholder is dead The description of the model in the previous paragraph is very informal; we now wish to specify the model more precisely. Let {S(x); 0 < x < } be a continuous time, time inhomogeneous Markov chain with a finite state space. (A useful introduction to stochastic processes of this type is provided by Chiang (1968).) The model is a stochastic process in continuous time since the time, or age, variable x moves continuously through its range. The process has a finite state space since the value of S(x) for any x must be either 1, 2 or 3. We interpret the statement S(x) = l, for example, to mean the process is in state 1 at time x, or, in our applications, the individual, or policyholder, is in state 1 at age x Now consider conditional probabilities of the form P[S(x + t) = h S(x) = g] g, h = 1, 2, 3 (1) i.e. the probability that the process is in state h at time (x + t) given that it was in state g at time x. We shall denote the above conditional probability (Note that our notation differs from that of Haberman (1983, 6); in his paper the above conditional probability is denoted.) The process is time inhomogeneous since we assume depends not only on t, the length of the time interval concerned, but also on x, the starting point of the time interval, as well as on the values of g and h. The process is Markov since we assume the value of is unchanged if we are given any information about the behaviour of the process before time x. In crude terms, the behaviour of the process at any future time (x + t) depends only on its state at the present time x and not on its history before time x In our particular example we want state 3 to be absorbing and we can specify this mathematically by assuming = 0 for all x, t 0 and h = 1, 2 (2) 3.7. Next we define the transition intensities. We assume the following limits exist for all x 0 which we denote σ x (3)

6 368 An Approach to the Study of Multiple State Models which we denote ρ x which we denote µx which we denote vx (4) (5) (6) For mathematical convenience we assume x, σ ρ x, µx, and vx are piecewise continuous functions of x. The above transition intensities correspond to the force of mortality in the ordinary mortality table. The above definitions of the transition intensities are equivalent to the following, where we have chosen σ x as an example: for fixed x 0 we have = σ t x + 0(t) (7) where O(t) is a quantity such that (8) For any x 0 the probability of two or more transitions in a time interval [x, x + t] is 0(t). 4. FORMULAE FOR PROBABILITIES 4.1. In this section we shall continue the study of the multiple state model introduced in 3. We shall assume the transition intensities are known continuous functions of x and we shall show how to derive probabilities for the model from these intensities We shall start by considering probabilities of the form P[S(x + u) = g for all u [0, t] S(x) = g] (9) for any x, t 0 and g = 1, 2. We shall denote this probability so that, for example, denotes the probability that the process remains in state 1 thoughout the time interval [x, x + t] given that it is in state 1 at time x. We can derive the following formulae for these probabilities: (10) The similarity between the above formulae and the formula for tpx in terms of the force of mortality for the ordinary life table should be noted. We shall derive (10), the derivation of (11) is similar. Let x, t 0 be fixed and let h > 0. Using the Markov property we can write (12) (11)

7 An Approach to the Study of Multiple State Models 369 The probability of two of more transitions in the time interval [x + t, x + t + h] is o(h) so the law of total probability gives Using (7) we see that (13) becomes Putting (14) into (12) and re-arranging gives Letting h 0 we obtain the differential equation (10) is the solution to (16) since it satisfies both (16) and the boundary condition Note that since the transition intensities are assumed to be known, and can be evaluated directly from (10) and (11) We can derive the following differential equations for the transition probabilities: (13) (14) (15) (16) (17) (18) (19) Similar equations hold for, and. The derivation of these formulae is similar to that of (10). Note that (18) and (19) are a pair of simultaneous differential equations with two unknown functions and. By differentiation and substitution we can obtain a second order ordinary differential equation for, or for, and the numerical solution of this equation should present no difficulties. See for example Conte and de Boor (1980) Bearing in mind the application of our model to individual PHI we shall be interested in the probability that the process is in state 2 throughout the time interval [x + t - α, x + t], where α, t 0, given that it is in state 1 at time x and we shall denote this probability (α). Clearly this probability is zero if α t. For α < t we have the formula (20) (21)

8 370 An Approach to the Study of Multiple State Models This formula has an obvious intuitive derivation and can be proved as follows. We regard x 0 and t > 0 as fixed and α > 0 as variable. By considering the state of the process at time (t α h) where h > 0 we see that Re-arranging, dividing by h and letting h 0 we see that (22) (23) It is easily seen that (21) satisfies (23) and the boundary condition 5. PARAMETER ESTIMATION 5.1. In this section we show how we could estimate the transition intensities in a practical application of the model of 3. The important points we wish to make in this section are that there is a simple and well-developed statistical theory available and that the estimates provided by this theory have very desirable properties and easily estimated variances Let us suppose we set up an observation period, for example a period of several consecutive calendar years, during which certain individuals, for example policyholders with PHI policies issued by certain offices, could come under our observation. We assume each individual represents an independent realization of the process {S(x); 0 x < } where x is the individual s age. We assume that while under observation we can observe each transition made by the individual. This last point will be discussed further in relation to PHI in the following section We shall choose an age interval, (x1, x2) say, over which we are prepared to assume the transition intensities σ x, ρ x, µx, and vx are constant with values σ, ρ, µ and v respectively. We shall observe individuals only when their exact age is between x1 and x2. Suppose an individual comes under our observation at time t1, perhaps because he effects a PHI policy at that time, perhaps because he reaches age x1 at that time or perhaps because t1 is the start of the overall observation period. For the sake of example let us assume our particular individual is in state 1 at time t1 transfers to state 2 at time t1 + t2 transfers to state 1 at time t1 + t2 + t3 and leaves our observation for some reason at time (t1 + t2 + t3 + t4). We assume that the behaviour of this individual while under observation was not affected by the way in which he came under observation and did not affect the way in which he left our observation. With this assumption we can write the likelihood of his sample path as (24)

9 An Approach to the Study of Multiple State Models Now suppose a total of N individuals come under our observation in the age range x1 to x2 in the overall observation period. The likelihood of the collection of sample paths is the product of the individual sample paths and with the help of (25) this can be written where T1 and T2 are the total observed times spent in states 1 and 2 respectively N1 is the number of observed transitions from state 1 to state 2 N2 is the number of observed transitions from state 2 to state 1 N3 is the number of observed transitions from state 1 to state 3 N4 is the number of observed transitions from state 2 to state 3. It is easily seen from (26) that the maximum likelihood estimates of σ, ρ, µ and v are (25) (26) (27) Text Text respectively and we would regard these as point estimates of σ x, ρ x, µx and vx respectively where x = (x1 + x2)/ It can be shown with the natural assumptions that T1/N and T2/N converge in probability to some positive constants τ1 and τ2 as N, that, for example, converges in distribution to N(0, σ/τ1), that no other estimators have, asymptotically, smaller variances than,, and and that these estimators are asymptotically independent of each other and also of the corresponding estimators for different age groups. An elementary treatment of maximum likelihood estimators can be found in Weatherill (1981). Professor J. M. Hoem has informed the author that a forthcoming paper by Borgan (1984) will be a useful reference for this topic Knowing the asymptotic distributions of the estimators enables us to test a graduation for goodness of fit in the same way as we would test the graduation of a mortality table, see Benjamin and Pollard (1980, Ch. 11), and test hypotheses about the observed transition intensities. For example we could test whether the results of an investigation are significantly different from the results of an earlier investigation It must be appreciated that in this section we have mentioned only some of the more important aspects of what is a large and growing body of theory. Sverdrup (1965) provides a very full discussion of this topic, Hoem (1976) provides a good review of the topic including a discussion of graduation and Hoem and Funck Jensen (1982, 4) also make some interesting points. (30)

10 372 An Approach to the Study of Multiple State Models 6. SOME PRACTICAL POINTS 6.1. In this section we shall discuss some practical points relating to multiple state models. These points will be discussed in relation to the model specified in 3 and in particular its interpretation in terms of PHI Our first point is that although the fundamental quantities in our model are the transition intensities we can easily use these to calculate transition probabilities, as explained in 4, and then to construct a multiple state life table based on the model. To do this let be the multiple state life table were x0 is some initial age and and denote the number of lives in state 1 and the number of lives in state 2 at age x respectively. Then and are arbitrary and determine an initial distribution for the process in the sense that can be regarded as the probability that a life aged x0 is in state i, i = 1 or 2. The life tables are then specified by for i = 1 or The mathematical model for PHI specified in 3 is very different from the traditional model for sickness insurance based on the Manchester Unity rates but, in their application, the two models need not be very different. The fundamental quantities in the traditional model are the quantities denoted which Neill (1977, Ch. 11) describes as the probability that an individual aged x is sick and that the duration of his current sickness is between m and (m + n). This type of probability, conditional on some initial distribution, can easily be calculated using the model of 3. For simplicity let us assume that the process is in state 1 at some initial time x0, i.e. we assume all individuals are healthy at some initial age x0. Then the probability that an individual is alive at age x given that he was healthy at age x0 is and the probability that he is sick at age x with duration of sickness between m and (m + n) given that he was healthy at age x0 is Hence the probability that an individual is sick at age x with duration of sickness between m and (m + n) given that he is alive at age x and given that he was healthy at age x0 is given by (34) divided by (33) An important feature of the quantity is that it takes no account of any information we may have concerning the individual s present state of health or his state of health at some earlier time and the quotient (34)/(33) is an attempt to recreate this ignorance by conditioning only on the individual s state of health at some earlier age x0, where x0 is possibly considerably less than x. However, this (31) (32) (33) (34)

11 An Approach to the Study of Multiple State Models 373 assumed ignorance is unrealistic in relation to PHI since one piece of information we will have is that the policyholder was healthy at the time his policy was effected. The model of 3 makes it very easy to take this sort of information into account. To illustrate this, consider the following simple example: a life aged 30 effects a 35-year PHI policy. He pays a premiun of P p.a. continuously while he is not receiving benefit and receives a benefit of B p.a. continuously during periods of illness, with no benefit payable for the first year of any sickness. The premium P can be calculated from the following formula which takes account of the information that the policyholder is healthy at age 30 Now suppose the policy has been in force for some time and we wish to calculate a valuation reserve. It is unreasonable to assume we would know whether or not the policyholder is healthy at the present time but we may assume we know whether or not the policyholder is currently receiving benefit. It is possible to derive formulae for the prospective reserve depending on whether or not the policyholder is currently receiving benefit. These formulae are given by Hoem (1969a) and are not reproduced here When discussing parameter estimation in 5 we assumed that when an individual was under observation we could observe all the transitions he made. This is an unrealistic assumption if we apply the model to PHI policies with a deferred period since we are unlikely to have any information about periods of illness of duration less than the deferred period and hence which do not result in any benefit being paid. The theoretical problems in this case are discussed in detail by Hoem (1969b) We have used the model of 3 to illustrate the TI-approach to multiple state models and the particular model was chosen for its simplicity. We have emphasized the application of this model to PHI since this is an interesting actuarial application and because this application of this particular model has been suggested as far back as Du Pasquier (1912, 1913). However, one feature of the model which may not seem very realistic is that the transition intensities are functions of attained age x only. This is not unreasonable for σ x and µx, but a more realistic model would allow the intensities of recovery and death after being ill, i.e. ρ x and vx, to depend not only on attained age x but also on the duration of the current illness. This extension of the model of 3, together with some practical suggestions for dealing with the parameter estimation difficulty outlined in the previous paragraph, have been discussed informally by Waters (1983). REFERENCES BENJAMIN, B. & POLLARD, J. H. (1980). The Analysis of Mortality and Other Actuarial Statistics. Heinemann, London. BORGAN, Ø. (1984). Maximum likelihood estimation in a parametric counting process model, with (35)

12 374 An Approach to the Study of Multiple State Models applications to censored failure time data and multiplicative models. To appear in Scandinavian Journal of Statistics. CHIANG, C. L. (1968). Introduction to Stochastic Processes in Biostatistics. Wiley, New York. C.M.I.R. No. 4 (1979). Continuous Mortality Investigation Reports. Published by the Institute of Actuaries and the Faculty of Actuaries. CONTE, S. D. & DE BOOR, C. (1980). Elementary Numerical Analysis: An Algorithmic Approach. McGraw-Hill, New York. DU PASQUIER, L. G. (1912, 1913). Mathematische theorie der invaliditatsversicherung. Mitteilungen der Vereinigung schweizerischer Versicherungsmathematiker, 7, 1-7; 8, l-153. HABERMAN, S. (1983). Decrement tables and the measurement of morbidity: I. J.I.A. 110, HOEM, J. M. (1969a). Some notes on the qualifying period in disability insurance. I. Actuarial values. Mitteilungen der Vereinigung schweizerischer Versicherungsmathematiker, 69, HOEM, J. M. (1969b). Some notes on the qualifying period in disability insurance. II. Problems of maximum likelihood estimation, Mitteilungen der Vereinigung schweizerischer Versicherungsmathematiker, 69, HOEM, J. M. (1976). The statistical theory of demographic rates: A review of current developments. Scandinavian Journal of Statistics, 3, HOEM, J. M. & FUNCK JENSEN, U. (1982). Multistate life table methodology: a probabilist critique. Appeared in Land, K. C. and Rogers, A. (eds) (1982). Multidimensional mathematical Demography. Academic Press, New York. NEILL, A. (1977). Life Contingencies. Heinemann. London. SVERDRUP, E. (1965). Estimates and test procedures in connection with stochastic models for deaths, recoveries and transfers between different states of health. Scandinavian Actuarial Journal, WATERS, H. R. (1983). A possible mathematical basis for individual permanent health insurance. Unpublished manuscript. WEATHERILL, G. B. (1981). Intermediate Statistical Methods. Chapman and Hall, London.