Statistical Disclosure Control for Tabular Data. Tabular data protection theory. Tabular data protection: Introduction. Theory and Methods (part I)

Transcription

1 Statistical Disclosure Control for Tabular Data Theory and Methods (part I) Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 1 Tabular data protection theory Session Ia Introduction Primary Disclosure Risks Session IIa Secondary Disclosure Risks Session IIb Protection Methods Special Topics Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 2 Tabular data protection: Introduction Disclosure Risk Concepts Basic terminology of tables Protection Concepts Information Loss Concepts Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 3 1

2 Tabular data protection: Theory and methods, Introduction Disclosure Risk Concepts Basic terminology of tables Protection Concepts Information Loss Concepts Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 4 Risk of Identification ( too few units ) for units with rare combinations / uniques (=1 s, 2 s) in frequency tables No direct disclosure risk But Imagine: External knowledge may identify uniques Then: linking to other tables (of other data sources) => disclose sensitive attributes of the unique individual Example: Male, immigrant, age 80+, divorced (=unique) Receives social security through linking to social security statistics Critical: rare combinations of identifying variables like Geography (low level) Sex broad categories of age or nationality etc. Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 5 Risk of Attribute Disclosure in frequency tables: Concentration on few items of a variable with sensitive categories. Can concern individuals or groups of individuals Examples: All inhabitants of a municipality are Roman-catholic (= group attribute disclosure) Or: all except for one All immigrants are muslims there is only one immigrant (individual attribute disclosure) All high income households score on the same income size class category Problem: distribution / pattern of 0 s across the categories of the variable Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 6 2

3 Risk of Attribute Disclosure in magnitude tables: Concentration of a sensitive quantitative variable on one or two identifiable units within the same table cell. Examples: Turnover for sector mining, if there is only one mining company Or: if there are two mining companies Investment for sector mining, even if there are 20 mining companies, but only one with non-zero investment Or: only two with non-zero investment Turnover for sector mining, if there are 20 mining companies, but 99% of the turnover is concentrated on only one out of the twenty Or: on two of the twenty Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 7 Secondary Disclosure Risks: Illustrative example Earnings of University professors by Sex (Annual Earnings in Tsd. Euro) Department: Physics Insgesamt Male Female Earnings No of Profs Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 8 Secondary Disclosure Risks: Illustrative example Earnings of University professors by Sex (Annual Earnings in Tsd. Euro) Department: Physics Total Male Female Earnings No of Profs =85 => Disclosure Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 9 3

4 Secondary Disclosure Risks: Illustrative example Earnings of University professors by Sex (Annual Earnings in Tsd. Euro) Department: Physics Total Male Female Earnings No of Profs Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 10 Secondary Disclosure Risks: Illustrative example Earnings of University professors by Years of teaching experience (Annual Earnings in Tsd. Euro) Department: Physics Total and more Earnings No of Profs =3585 Impute 3585 into the by-gender table: =85 => Disclosure Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 11 Risk of Disclosure by Differencing Examples: Two tables: a) All Pensioners b) Pensioners at home Differencing => c) Pensioners in residential homes c) refers to small universe: likely to contain critical patterns of 0 s, 1 s, 2 s Two tables with one slightly different classification, defined by identical classifications otherwise: a) 0-55;56-100;more b) 0-50;51-100;more Differencing => c) c) refers to small universe: likely to contain critical concentration cases Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 12 4

5 Risk of Disclosure by Linking Example Two tables: a) non-immigrants by age (1-year age band) and sex b) 5-20 year olds by sex and type of school (all!) Summing up from b) (across school types) => c) 5-20 year olds by sex (all!) Summing up from a) (across age years) => d) 5-20 year old non-immigrants by sex Differencing c) and d) => e) 5-20 year old immigrants by sex e) refers to small population: likely to contain critical patterns of 0 s, 1 s, 2 s Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 13 Disclosure by differencing in a hierarchical table Groningen 21 Friesland x Drenthe 23 Overijssel 27 Gelderland 41 Flevoland x Utrecht 32 Noord-Holland 54 Zuid-Holland 67 Zeeland 38 Noord-Brabant 44 Limburg 39 Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 14 Disclosure by differencing in a hierarchical table Groningen 21 Friesland x Drenthe 23 Overijssel 27 Gelderland 41 Flevoland x Utrecht 32 Noord-Holland 54 Zuid-Holland 67 Zeeland 38 Noord-Brabant 44 Limburg 39 North 63 East 80 South 83 West 191 Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 15 5

6 Disclosure by differencing in a hierarchical table Groningen 21 Friesland 19 Drenthe 23 Overijssel 27 Gelderland 41 Flevoland x Utrecht 32 Noord-Holland 54 Zuid-Holland 67 Zeeland 38 Noord-Brabant 44 Limburg 39 North 63 East 80 South 83 West 191 Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 16 Tabular data protection: Introduction Disclosure Risk Concepts Identification Attribute Disclosure Disclosure by Differencing and/or Linking Basic terminology of tables Protection Concepts Information Loss Concepts Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 17 Basic terminology Table cells Refer to groups of units (enterprises, companies, households, persons, ), groups are defined by cross-combination of qualitative grouping variables Geography, NACE, Age, Sex, SPANNING variables Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 18 6

7 Basic terminology cell value = Sum of a quantitative RESPONSE VARIABLE Response variable is summed up for units of the group defining the cell Examples: Turnover, no of employees, salaries, number of cattle, etc. NOT: mean salaries, Indices, Special case: Frequency counts table cell value = cell count = Number of respective units = Sum of a (0,1) indicator variable Response variable = (0,1) indicator variable Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 19 Basic terminology Table Dimension = Number of spanning variables used to specify the table (=:N) Margins/marginal cells: Cells specified by less than N spanning variables Overall total = Domain (all units grouped together) Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 20 Basic terminology Spanning variable = Classification Special case: hierarchical classification Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 21 7

8 Example: hierarchical classification Groningen 21 Friesland 19 Drenthe 23 Overijssel 27 Gelderland 41 Flevoland 12 Utrecht 32 Noord-Holland 54 Zuid-Holland 67 Zeeland 38 Noord-Brabant 44 Limburg 39 Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 22 Example: hierarchical classification Groningen 21 Friesland 19 Drenthe 23 Overijssel 27 Gelderland 41 Flevoland 12 Utrecht 32 Noord-Holland 54 Zuid-Holland 67 Zeeland 38 Noord-Brabant 44 Limburg 39 North 63 East 80 South 83 West 191 Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 23 Example: hierarchical classification Groningen 21 Friesland 19 Drenthe 23 Overijssel 27 Gelderland 41 Flevoland 19 Utrecht 32 Noord-Holland 54 Zuid-Holland 67 Zeeland 38 Noord-Brabant 44 Limburg 39 North 63 East 80 South 83 West 191 Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 24 8

9 Hierarchical classifications Example Drinks alcoholic Soft drinks Beer Vine liquor Water Lemonade Juice red rosè white Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 25 Hierarchical tables Linked Subtables Are contained in a larger table Exist, if at least one spanning variable is hierarchical Example drinks bin recycling alcoholic drinks beer vines liquores bin recycling vines red white alcoholic drinks soft drinks bin recycling soft drinks water lemonade fruit juices. bin recycling Rows alcoholic drinks, soft drinks, and vines are contained in two subtables. The subtables are linked. rose Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 26 Table equations Row relation Column relation Example Drink bins = Alcolholic drink bins + Softdrink bins Describing hierarchical relations by indentation alcoholic_drinks beer vines white red rosé liquors soft_drinks water lemonade fruit_juices Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 27 9

10 Table equations Row relation Column relation Describing hierarchical relations Hessen by code truncation Example Bremen Germany = Bremen = Bremen City + Bremerhafen 04 = Nuts levels (district 11 in Dept. 6 in State Hessen) Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 28 Tabular data protection: Introduction Disclosure Risk Concepts Basic terminology of tables Protection Concepts Information Loss Concepts Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 29 Protection Concepts Nonperturbative Type of Method Pretabular Posttabular Perturbative Swapping, Noise Noise, Rounding, Controlled Rounding Global Recoding Cell Suppression Available in τ -ARGUS Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 30 10

11 Example: Cell suppression Primary suppressions: suppress cells that are sensitive according to used measure Region A B C D Total Harps Organs Pianos Other Total Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 31 Example: Cell suppression Primary suppressions: suppress cells that are sensitive according to used measure Region A B C D Total Harps 58 x Organs Pianos x 336 Other Total Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 32 Example: Cell suppression Secondary suppressions: suppress additional cells to prevent recalculation Region A B C D Total Harps 58 x 36 x 230 Organs Pianos 92 x 59 x 336 Other Total Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 33 11

12 Example: Rounding Conventional Rounding Example: Round to the closest multiple of 100 Region A B C D Total Harps Organs Pianos Other Total Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 34 Tabular data protection: Introduction Basic terminology of tables Disclosure Risk Concepts Protection Concepts Information Loss Concepts Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 35 Information Loss For Cell Suppression Obvious: Number of suppressed cells by level of aggregates For perturbative methods Table level measure: distribution of (relative) deviations true perturbed Cell level measure? More suggestions: see Literature Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 36 12

13 Tabular data protection: Primary Disclosure Risks Frequency tables Magnitude Tables Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 37 Primary risks models for frequency tables Spanning variables: some identifying (Region, gender, type of business, ) some sensitive (Sexual behaviour, criminal offence, (size class?), ) Sensitive cells: All spanning variables are identifying: Small cells (1 s, 2 s) ( Identity disclosure ) Some spanning variables sensitive, some identifying: Zero (and small) cells ( Attribute Disclosure ) Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 38 Attribute Disclosure in Frequency Tables Spanning variables: one sensitive remaining identifying Example: number of ship-owners Environmental offence Region Yes No Total A Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 39 13

14 Attribute Disclosure in Frequency Tables All ship-owners in region A committed an environmental Offence Conclusion: Not all respondents should score on a sensitive category (or not all, except for one) Note: Group disclosure must be evaluated in a sensible way Information on large group = statistics (?) How likely is it for the discovered attribute to be unknown before to those who can identify the respective individuals? Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 40 Attribute Disclosure in Frequency Tables Zero-cells may create disclosure risk, when categories of a sensitive variable are identifying at a higher level scores should be sufficiently spread over all categories Example: Income Income classes Region A B C D Total.A Perturbative methods introduce uncertainty about zeroes. This solves many problems of frequency tables Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 41 Tabular data protection: Primary Disclosure Risks Frequency tables Magnitude Tables Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 42 14

15 Primary risk models for magnitude tables (in business statistics) Spanning Variables: Identifying (Region,NACE, ) Problem: Cell values referring to Single Units, Dominating Units Sensitive attributes: Quantities (Turnover, Investment ) Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 43 Attribute disclosure in magnitude tables (example) Turnover (10 6 ) of instrument producing companies number of respondents Region A B C Total Harps Organs Pianos Other Total Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 44 Attribute disclosure in magnitude tables (example) Law / agreement: No sensitive information on identifyable respondents should be published Problem: Cell consisting of one contribution Piano-producing company in region C Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 45 15

16 Attribute disclosure in magnitude tables (example) How about the two piano-producing companies in region B? number of respondents Region A B C Total Harps Organs Pianos Other Total Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 46 Attribute disclosure in magnitude tables (example) How about the two piano-producing companies in region B? If they know they are the only two, they can disclose each others contribution! Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 47 Attribute disclosure in magnitude tables (example) How about the five piano-producing companies in region A? Turnover (10 6 ) of instrument producing companies number of respondents Region A B C Total Harps Organs Pianos Other Total Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 48 16

17 Attribute disclosure in magnitude tables (example) How about the five piano-producing companies in region A? Suppose: Company X: Company Y: Other three: Total: 81,000,000 5,000,000 2,000,000 each 92,000, = 87 mln is within 7.4%! Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 49 Primary risk models for magnitude tables Sensitive cells: one contribution two contributions one or more dominating contributions Need: Sensitivity measure Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 50 Primary risk models for magnitude tables Sensitivity Rules Rule Frequency-rule (1,k)-Dominance-Rule (2,k)-Dominance-Rule Definition Cell is sensitive, if there are too few contributions too few often < 3 Largest contribution > k % of cell value Largest 2 contributions > k % of cell value Concentration- Rules p%-rule (similar: (p,q)-rule) Difference between cell value and (joint) contribution of the N largest contributions < p % of the largest contribution Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 51 17

18 Primary risk models for magnitude tables Concentration Rules Note: Concentration Rules only make sense, if the size of the variable is identifying, e.g. if intruders may know who the largest respondents are. Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 52 Concentration rules (1,k)-Dominance Rule Example: Company X: Company Y: 3 others: Total: (together) ,000 < (90/100)* not sensitive according to (1,90)-Rule. But Y knows: X < = , exceeds X by only 2%! Never use (1,k)-dominance rule as only concentration rule Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 53 Concentration rules p%-rule vs. (2,k)-dominance rule Both, (2,k)-dominance-rule and p% -rule consider additional knowledge of second largest respondent p% -more efficient Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 54 18

19 Concentration rules p%-rule vs. (2,k)-dominance rule 2nd largest respondent can deduce bound for largest contribution x 1 by: X - x 2 Declare cell sensitive, if distance between this bound and the true value x 1 too small. Distance is defined as too small,if it is below p% of x 1 (p%-rule) Concept more natural! (100-k)% of X (2,k)-Dominance-Rule For k=100*100/(100+p) same safety-level it holds: Any cell which is sensitive according to the p%- Rule is sensitive according to the (2,k)-Rule as well (proof: see Statistical Disclosure Control (Wiley), pp ) Some cells are non-sensitive according to the p %-Rule, but sensitive according to the (2,k)-Rule (2,k)-Rule causes overprotection! Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 55 Concentration rules p%-rule vs. (2,k)-Dominance Rule Example p = 10, hence k=100*100/(100+p) =90,9 Example: Company X: Company Y: 3 others: Total: together Y knows: X < = , exceeds X by 15,4%. So: cell non-sensitive according to p%-rule. But X+Y = % of ( > 90.9% of ). So: cell sensitive according to (2,k)-Rule Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 56 Sarah Giessing FSO / Section - Mathematical Statistical Methods Telefon: +49 (0)611 / Telefax: +49 (0)611 / sarah.giessing@destatis.de Federal Statistical Office of Germany, C 1 Mathematical Statistical Methods Folie 57 19