An advanced method for preserving skewness in single-variate, multivariate, and disaggregation models in stochastic hydrology

Transcription

1 XXIV General Assembly of European Geophysical Society The Hague, 9-3 April 999 HSA9.0 Open session on statistical methods in hydrology An advanced method for preserving skewness in single-variate, multivariate, and disaggregation models in stochastic hydrology Demetris Koutsoyiannis Department of Water Resources National Technical University of Athens Introduction Y = b V Y = b V + b V Y 3 = b 3 V + b 3 V + b 33 V 3 Due to the central limit theorem: The distribution of Y i tends to be more symmetric for increasing number of innovations For constant 3 [V i ]: 3 [Y ] > 3 [Y ] > 3 [Y 3 ] > A simple stochastic model that generates hydrological variables Y i using innovations (noise variables) V i (with Var[V i ] = ) Conversely: To obtain a constant 3 [Y i ] for increasing number of innovations, more and more skewed innovations V i are needed: 3 [V ] < 3 [V ] < 3 [V 3 ] < In finite samples the skewness cannot be arbitrarily high Theoretically the limit equals (sample size) Practically, a 3 [V i ] > 0.5 (sample size) cannot be preserved and therefore 3 [Y i ] is not preserved (Todini, 980) D. Koutsoyiannis, An advanced method for preserving skewness

2 Model: Y = az+ bv where Y: vector of variables to be generated Z: vector of variables with known values V: vector of innovations (with Var[V i ] = ) a and b: matrices of parameters (b square) Main parameter estimators: b b T = c. := 3 [V] = [b (3) ] { 3 [ ] 3 [az]}... where c := Cov[Y, Y] a Cov[Z, Z] a T (equivalently, c := Cov[Y a Z, Y a Z]) b (3) : matrix with elements the cubes of b Mathematical framework Representative for most common stochastic models in hydrology Infinite solutions if c is positive definite No solutions otherwise (inconsistent c) There exist two algorithms for determining (different solutions) b Cholesky decomposition (triangular b) Singular value decomposition (based on eigenvectors of b) The skewness of V depends on b If some element of = 3 [V] is too high then 3 [ ] will be not preserved D. Koutsoyiannis, An advanced method for preserving skewness Problem formulation Determine b from the known c = b b T so that the coefficients of skewness of V be as small as possible For c positive definite: Find the optimal solution b, leading to the smallest value of max i { i } Optimisation problem (single-objective, unconstrained) For c not positive definite: Find a solution b, leading to a small departure of b b T from c, and simultaneously a small value of max i { i } Optimisation problem (multiple-objective, or single-objective constrained) D. Koutsoyiannis, An advanced method for preserving skewness 3

3 Example A: Temporal rainfall disaggregation Consider the generation of a rainfall event with duration D = 0 h using a halfhour time resolution (k = 0 half-hour rainfall increments Y i, i =,, 0) Assume covariance structure of Y i as in the Scaling Model of Storm Hyetograph (Koutsoyiannis and Foufoula-Georgiou, Water Resources Research, 9(7), 993) Cov[Y i, Y j ] = [(c + c ) f( j i, ) k c ](D ( + ) / k ) where f(m, ) = (/) [(m ) + (m + ) ] m if m > 0 f(m, ) = if m = 0 Assume two parameter gamma distribution for Y i Parameters: c = 8.7, c = 85.8, = 0.9, = 0. Statistics of Y i : E[Y i ] =. mm, C v [Y i ] =., C s [Y i ] =.88 Single variate problem with long memory (not a typical ARMA model) Generation model Y = bvwith bb T = Cov[Y, Y] b is a matrix of parameters with size 0 0 (00 unknowns) D. Koutsoyiannis, An advanced method for preserving skewness Example A Solution : Cholesky decomposition Value Column Row Graphical view of b (after standardisation so that (bb T ) ii = ) D. Koutsoyiannis, An advanced method for preserving skewness 5

4 Example A Solution : Singular value decomposition Value Column Row Graphical view of b (after standardisation so that (bb T ) ii = ) D. Koutsoyiannis, An advanced method for preserving skewness.0 Example A Solution 3: Optimal 0.8 Value Column Row Graphical view of b (after standardisation so that (bb T ) ii = ) D. Koutsoyiannis, An advanced method for preserving skewness 7

5 Coefficient of skewness Example A Resulting coefficients of skewness for innovation variables Variable number Cholesky decomposition Singular value decomposition Optimal solution D. Koutsoyiannis, An advanced method for preserving skewness 8 Example B: Multivariate generation of monthly rainfall and runoff Multivariate generation problem with locations: variables: simultaneous monthly rainfall and runoff 3 basins: Evinos, Mornos and Yliki, supplying water to Athens, Greece Model PAR(): Y = a Z+ bv where Y X s, Z X s (s stands for subperiod, i.e., month; here s = 8 May) Characteristic statistics: Coefficients of skewness of Y i : Cross-correlation coefficients: Autocorrelation coefficients of runoff: Autocorrelation coefficients of rainfall: 0 Matrix c = Cov[Y, Y] a Cov[Z, Z] a T is inconsistent (not positive definite) D. Koutsoyiannis, An advanced method for preserving skewness 9

6 (a) Example B Different solutions of matrix b value Cholesky with adjustment for min b ii = 5 (max i { i } = 9000) (b) value 3 column ro w -0.5 Optimal for approximation of covariance constrained on skewness (max i { i } = 5.37) 3 column ro w -0.5 (c ) (d) value Optimal for approximation of covariance (max i { i } = 7.9) value 3 column ro w -0.5 Optimal for skewness (max i { i } =.57) 3 column ro w -0.5 D. Koutsoyiannis, An advanced method for preserving skewness 0 Proposed algorithm: Objective function Component : Preservation (or approximation) of covariances d := i j d ij where d := bb T c Component : Preservation of variances d* := i d ii where d * := diag(d,, d nn ) Component 3: Preservation of skewness p := ( i i p ) /p where p a large integer so that p max i { i } Combination of the three components and problem solution by minimising (b) := ( / n ) d(b) + ( / n) d * (b) + 3 (b) where n is the matrix size, and, and 3 adjustable multipliers typical values: =, = 0 3, 3 = 0 3 D. Koutsoyiannis, An advanced method for preserving skewness

7 Proposed algorithm: Optimisation procedure The matrix of derivatives of with respect to the unknown parameters b ij has a very simple expression, i.e., d / db = ( / n ) d b + ( / n) d * b 3 p p w where w is a matrix with elements w ij := j i and is a vector defined by := {[b (3) ] } (p ) This enables the use of typical nonlinear optimisation methods such as the Fletcher-Reeves Conjugate Gradient method The initial value of b could be either the Cholesky solution or even the identity matrix D. Koutsoyiannis, An advanced method for preserving skewness Evolution of solution through iterations Example B d*, d, 00 0 d d* Iteration number Final D. Koutsoyiannis, An advanced method for preserving skewness 3

8 A note on disaggregation problems The proposed technique is directly applicable to disaggregation models All-at-once disaggregation models such as Schaake-Valencia or Mejia- Rousselle may involve huge sizes of matrices with an unreasonably high number of parameters The proposed technique is strongly recommended for coupling with the Simple Disaggregation model (Koutsoyiannis and Manetas, Water Resources Research, 3(7), 99) whose parameters coincide with those of the typical multivariate PAR() model D. Koutsoyiannis, An advanced method for preserving skewness Conclusions The problem of preserving skewness in stochastic hydrologic models is directly associated to the problem of covariance matrix decomposition A new technique is presented for covariance matrix decomposition based on an optimisation framework, with the objective function being composed of three components aiming at complete preservation of the variances of variables either preservation of covariances, or optimal approximation thereof (in case of inconsistent covariance matrices) preservation of the skewness coefficients by keeping the skewness of the noise variables as low as possible The technique is implemented by a simple nonlinear optimisation algorithm based on analytically determined derivatives Applications indicate that the algorithm is quick, stable and easily applicable even in cases with as much as 00 parameters D. Koutsoyiannis, An advanced method for preserving skewness 5