Security-aware Program Transformations

Security-aware Program Transformations Massimo Bartoletti, Pierpaolo Degano, Gian Luigi Ferrari Dipartimento di Informatica, Università di Pisa ICTCS 03 p.1

Stack Inspection (1) access control mechanism based on the analysis of the execution stack (stack of method frames) a security policy maps each class to a protection domain (a named set of permissions) to check if a permission P is granted: for each frame in the call stack (starting from top) if P is not granted to the frame throw an AccessControlException if the frame is privileged return ICTCS 03 p.2

Stack Inspection (2) lazy evaluation strategy: the one shown above - slow security checks - prevents from interprocedural optimizations + no update of the security context ICTCS 03 p.3

Stack Inspection (2) lazy evaluation strategy: the one shown above - slow security checks - prevents from interprocedural optimizations + no update of the security context eager evaluation strategy: the set of granted permissions is updated at each method call + fast security checks + allows for interprocedural optimizations (in combination with security passing style) - update of the security context ICTCS 03 p.3

Program Model Java Bytecode CF A Control Flow Graph control flow + security checks no data flow conditional construct nondeterminism dynamic dispatching nondeterminism ICTCS 03 p.4

Program Model - syntax (1) n 0 : call D 1 n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 3 n 2 : call n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return ICTCS 03 p.5

Program Model - syntax (2) Method call D 1 P 0, P 1 n 0 : call n 1 : call D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 2 n 2 : call Security check n 5 : call D 3 P 2 Method return D 3 n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return ICTCS 03 p.6

Program Model - syntax (3) Interprocedural flow D 1 Entry point P 0, P 1 n 0 : call n 1 : call D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 2 Exception handling D 3 P 2 Intraprocedural flow D 3 n 2 : call n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return ICTCS 03 p.7

Program Model - syntax (4) Privileged call D 1 P 0, P 1 n 0 : call n 1 : call D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 2 n 2 : call Protection domain n 5 : call D 3 P 2 Method D 3 n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return ICTCS 03 p.8

Program Model - syntax (5) n 0 : call D 1 n 1 : call D P P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 2 D 3 P 2 Perm : D 2 P D 3 n 2 : call n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return ICTCS 03 p.9

Program Model - semantics (1) state = call stack + exception flag Example: [n 0,..., n k ], true = [n 0,..., n k ] ICTCS 03 p.10

Program Model - semantics (1) state = call stack + exception flag Example: [n 0,..., n k ], true = [n 0,..., n k ] top node ICTCS 03 p.10

Program Model - semantics (1) state = call stack + exception flag Example: [n 0,..., n k ], true = [n 0,..., n k ] an exception is active! ICTCS 03 p.10

Program Model - semantics (1) state = call stack + exception flag Example: [n 0,..., n k ], true = [n 0,..., n k ] stack inspection σ P transition relation σ, x σ, x reachability relation G σ, x when there is a trace from [], false to σ, x ICTCS 03 p.10

Program Model - semantics (2) Stack inspection [] P [ 1 ] P Perm(n) σ : n P σ P [ 2 ] P Perm(n) σ : n P Priv(n) [ 3 ] ICTCS 03 p.11

Program Model - semantics (3) Method call/return n [] [n] [ entry ] l(n) = call n n σ : n σ : n : n [ call ] l(m) = return n n σ : n : m σ : n [ ret ] ICTCS 03 p.12

Program Model - semantics (4) Security checks l(n) = check(p ) σ : n P n n σ : n σ : n [ pass ] l(n) = check(p ) σ : n σ : n σ : n P [ fail ] ICTCS 03 p.13

Program Model - semantics (5) Exception handling n n σ : n σ : n [ catch ] σ : n n σ [ propagate ] ICTCS 03 p.14

The Trace Permissions Analysis (1) for each node n, it computes the security contexts τ(n) of all σ such that G σ Security context of a state σ: {Dom(n)} Γ([]) = Γ(σ : n) = Γ(σ) {Dom(n)} if Priv(n) otherwise Set of permissions granted to a security context γ: Π(γ) = D γ Perm(D) ICTCS 03 p.15

The Trace Permissions Analysis (2) For each state σ and permission P : σ P P Π(Γ(σ)) For each solution τ and state σ : n G σ : n = γ τ(n). γ = Γ(σ : n) For the minimal solution τ and each state σ : n γ τ(n) = σ. G σ : n γ = Γ(σ : n) the minimal solution is computed in O(N) by our worklist algorithm (N is the number of nodes) ICTCS 03 p.16

The Trace Permissions Analysis (3) n 0 : call D 1 n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 3 n 2 : call n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return [] ICTCS 03 p.17

The Trace Permissions Analysis (3.1) n 0 : call D 1 {D1} n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 3 n 2 : call n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) [] [n 1 ] n 4 : return n 7 : return n 9 : return ICTCS 03 p.18

The Trace Permissions Analysis (3.2) n 0 : call {D0} D 1 {D1} n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 3 n 2 : call n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return [] [n 1 ] [n 1, n 0 ] n 7 : return n 9 : return ICTCS 03 p.19

The Trace Permissions Analysis (3.3) n 0 : call {D0} D 1 {D1} n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 {D0,D2} n 2 : call n 5 : call D 3 n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return [] [n 1 ] [n 1, n 0 ] [n 1, n 0, n 2 ] ICTCS 03 p.20

The Trace Permissions Analysis (3.4) n 0 : call {D0} D 1 {D1} n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 {D0,D2} n 2 : call {D0,D2} n 5 : call D 3 n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return [] [n 1 ] [n 1, n 0 ] [n 1, n 0, n 2 ] [n 1, n 0, n 2, n 5 ] ICTCS 03 p.21

The Trace Permissions Analysis (3.5) n 0 : call {D0} D 1 {D1} n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 3 n 2 : call {D0,D2} {D0,D2} {D0,D2,D3} n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return [n 1, n 0, n 2, n 5 ] [n 1, n 0, n 2, n 5, n 8 ] ICTCS 03 p.22

The Trace Permissions Analysis (3.5) n 0 : call {D0} D 1 {D1} n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 3 n 2 : call {D0,D2} {D0,D2} {D0,D2,D3} n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return [n 1, n 0, n 2, n 5, n 8 ] n 7 : return n 9 : return ICTCS 03 p.22

The Trace Permissions Analysis (3.6) n 0 : call {D0} D 1 {D1} n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 3 n 2 : call {D0,D2} {D0,D2} {D0,D2,D3} n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) {D0,D2} n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return {D0,D2,D3} [n 1, n 0, n 3 ] ICTCS 03 p.23

The Trace Permissions Analysis (3.7) n 0 : call {D0} D 1 {D1} n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 3 n 2 : call {D0,D2} {D0,D2} {D0,D2,D3} n 5 : call n 8 : check(p 2 ) n 3 : check(p 0 ) {D0,D2} n 4 : return {D0,D2} n 6 : check(p 1 ) n 7 : return n 9 : return {D0,D2,D3} [n 1, n 0, n 3 ] [n 1, n 0, n 4 ] ICTCS 03 p.24

Optimizations Elimination of the redundant checks Dead code elimination Method inlining Tail call elimination Fast implementation of eager stack inspection ICTCS 03 p.25

Redundant Checks Elimination (1) n 0 : call D 1 n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 1 D 3 n 2 : call n 5 : call n8: check(p2) n 3 : check(p 0 ) n 6 : check(p 1 ) n 4 : return n 7 : return n 9 : return ICTCS 03 p.26

Redundant Checks Elimination (1) n 0 : call D 1 n 1 : call P 0, P 1 D 1 P 1, P 2 D 2 P 0, P 1, P 2 D 3 P 2 D 2 D 1 D 3 n 2 : call n 5 : call n8: check(p2) n 3 : check(p 0 ) n 4 : return n 7 : return n 9 : return ICTCS 03 p.26

Redundant Checks Elimination (2) a check node n for permission P is redundant when: σ N. G σ : n = σ : n P ICTCS 03 p.27

Redundant Checks Elimination (2) a check node n for permission P is redundant when: σ N. G σ : n = σ : n P Let τ = T P = (G, Perm). For each check node n, define: Π(n) = { Π(γ) γ τ(n) } Π(n) is the set of permissions (statically) granted to n. ICTCS 03 p.27

Method inlining (1) Example 1 (before inlining) {} D 1 {P } D 2 {P } n 0 : call n 1 : call n 2 : check(p ) n 3 : return n 4 : return n 5 : return ICTCS 03 p.28

Method inlining (1) Example 1 (before inlining) {} D 1 {P } D 2 {P } n 0 : call n 1 : call n 2 : check(p ) n 3 : return n 4 : return n 5 : return G [n 0 ] [n 0, n 1 ] [n 0, n 1, n 2 ] P ICTCS 03 p.28

Method inlining (2) Example 1 (after inlining of n 1 ) {} D 1 {P } n 0 : call n 1 : nop n 2 : check(p ) n 3 : return n 4 : return n 5 : nop ICTCS 03 p.29

Method inlining (2) Example 1 (after inlining of n 1 ) {} D 1 {P } n 0 : call n 1 : nop n 2 : check(p ) n 3 : return n 4 : return n 5 : nop Ġ [n 0 ] [n 0, n 1 ] [n 0, n 2 ] P ICTCS 03 p.29

Method inlining (3) Example 2 (before inlining) {P } D 1 {} D 2 {P } n 0 : call n 1 : call n 2 : check(p ) n 3 : return n 4 : return n 5 : return ICTCS 03 p.30

Method inlining (3) Example 2 (before inlining) {P } D 1 {} D 2 {P } n 0 : call n 1 : call n 2 : check(p ) n 3 : return n 4 : return n 5 : return G [n 0 ] [n 0, n 1 ] [n 0, n 1, n 2 ] P ICTCS 03 p.30

Method inlining (4) Example 2 (after inlining of n 0 ) {P } n 0 : nop D 2 {P } n 1 : call n 2 : check(p ) n 3 : return n 4 : nop n 5 : return ICTCS 03 p.31

Method inlining (4) Example 2 (after inlining of n 0 ) {P } n 0 : nop D 2 {P } n 1 : call n 2 : check(p ) n 3 : return n 4 : nop n 5 : return Ġ [n 0 ] [n 1 ] [n 1, n 2 ] P ICTCS 03 p.31

Method inlining (5) Let ṅ n be the call candidate for inlining. We require: static dispatching, non-recursiveness: m N. ṅ m = m = n m / µ(ṅ) ICTCS 03 p.32

Method inlining (5) Let ṅ n be the call candidate for inlining. We require: static dispatching, non-recursiveness: m N. ṅ m = m = n m / µ(ṅ) original version inlining: m N. m n = m = ṅ ICTCS 03 p.32

Method inlining (5) Let ṅ n be the call candidate for inlining. We require: static dispatching, non-recursiveness: m N. ṅ m = m = n m / µ(ṅ) original version inlining: m N. m n = m = ṅ isolation of the protection domain of n : m / µ(n ). Dom(m) Dom(n ) ICTCS 03 p.32

Method inlining (6) The key idea is the following: method inlining is safe iff the outcome of the security checks is preserved. ICTCS 03 p.33

Method inlining (6) The key idea is the following: method inlining is safe iff the outcome of the security checks is preserved. let Dom(ṅ) = D, Dom(n ) = D. We define: γ if D / γ Inl ṅ(γ) = (γ \ {D }) {D} otherwise ICTCS 03 p.33

Method inlining (7) The correctness of method inlining is decided as follows: assume that a solution τ to the TP analysis is available. We assign a fresh name to Dom(n ), then we restart the worklist algorithm from ṅ. each time we reach a node l(n) = check(p ), we check that, for each context γ τ(n), P Π(γ) P Π(Inl ṅ(γ)) ICTCS 03 p.34

Method inlining (8) We define the ṅ-inlined version of a CFG G = N {n ε }, E, Priv G, Dom G as Ġ = N {n ε }, E, PrivĠ, DomĠ, where: true if Priv G (ṅ) and ṅ µ(n) PrivĠ(n) = Priv G (n) otherwise DomĠ(n) = Dom G (ṅ) Dom G (n) if ṅ µ(n) otherwise ICTCS 03 p.35

Method inlining (9) Method call l(n) = call n n n ṅ σ : n ṅinl σ : n : n l(ṅ) = call ṅ n σ : ṅ ṅinl σ : n ICTCS 03 p.36

Method inlining (10) Method return l(n ) = return ṅ m ṅ µ(n ) σ : n : n ṅinl σ : m l(n ) = return ṅ m ṅ µ(n ) σ : n ṅinl σ : m ICTCS 03 p.37

Method inlining (11) Theorem (Correctness of method inlining.) If ṅ is inlineable in G and Ġ is the ṅ-inlined version of G: σ 0, x 0 σ k, x k σ 0, x 0 ṅinl ṅinl σ k, x k where σ 0 = [], x 0 = false, and σ i = inl ṅ(σ i ) for i 0..k. inl ṅ(σ) = σ top(σ) ṅ inl ṅ(σ) = σ inl ṅ(σ : n ) = σ : n inl ṅ(σ : ṅ : n ) = σ : n ICTCS 03 p.38

Conclusions interprocedural optimizations in presence of stack inspection + based on solid static techniques (CFA) + no update of the security context + dynamic linking is possible - overhead at linking time / deoptimization TO DO: parametric permissions (ongoing work) dynamic policies (ongoing work) implementation & performance evaluation ICTCS 03 p.39

Appendix - Def. of the TP Analysis (1) T P in (n) = T P out (m, n) = T P out (m, n) (m,n) E {{Dom(n)}} if n { γ {Dom(n)} γ T P call (m) } if m n T P trans (m) if m n T P catch (m) if m n T P call (n) = {{Dom(n)}} T P in (n) if Priv(n) and T P in (n) otherwise ICTCS 03 p.40

Appendix - Def. of the TP Analysis (2) T P trans (n) = { γ T P in (n) P Π(γ) } if l(n) = check(p ) { γ T P in (n) Trans(n, {Dom(n)}) } if l(n) = call, Priv(n) { γ T P in (n) Trans(n, γ) } otherwise def Trans(n, γ) = m ρ(n). γ {Dom(m)} T P in (m) { γ T P in (n) P / Π(γ) } if l(n) = check(p ) T P catch (n) = { γ T P in (n) Catch(n, {Dom(n)}) } if l(n) = call, Priv(n) { γ T P in (n) Catch(n, γ) } otherwise Catch(n, γ) def = m ξ 1 (n). γ {Dom(m)} T P catch (m) ICTCS 03 p.41