Lecture 7 Requirements Prioritisation Risk Management 246
Lecture 7 Requirements Prioritisation Risk Management 247
Basics of Prioritisation Need to select what to implement Ä Customers (usually) ask for way too much Ä Balance time-to-market with amount of functionality Ä Decide which features go into the next release For each requirement/feature, ask: Ä How important is this to the customer? Ä How much will it cost to implement? Ä How risky will it be to attempt to build it? Perform Triage: Ä Some requirements *must* be included Ä Some requirements should definitely be excluded Ä That leaves a pool of nice-to-haves, which we must select from. 248
A Cost-Value Approach Calculate return on investment Ä Assess each requirement s importance to the project as a whole Ä Assess the relative cost of each requirement Ä Compute the cost-value trade-off: Value (percent) 30 25 20 15 10 High priority Medium priority 5 Low priority 5 10 15 20 25 30 Cost (percent) 249
A Cost-Value Approach Calculate return on investment Ä Assess each requirement s importance to the project as a whole Ä Assess the relative cost of each requirement Ä Compute the cost-value trade-off: Value (percent) 30 High priority 25 Two approaches: 20 Medium priority Ä Absolute 15 scale (e.g. dollar values) Ø Requires much domain experience 10 Ä Relative values (e.g. less/more; a little, somewhat, very) 5 Ø Much easier to elicit Low priority Ø Prioritization becomes a sorting problem 5 10 15 20 25 30 Cost (percent) 250
Hard to quantify differences Some complications Ä easier to say x is more important than y Ä than to estimate by how much. Not all requirements comparable Ä E.g. different level of abstraction Ä E.g. core functionality vs. customer enhancements Requirements may not be independent Ä No point selecting between X and Y if they are mutually dependent Stakeholders may not be consistent Ä E.g. If X > Y, and Y > Z, then presumably X > Z? Stakeholders might not agree Ä Different cost/value assessments for different types of stakeholder 251
Hierarchical Prioritisation Group Requirements into a hierarchy Ä e.g. A goal tree Only make comparisons between branches of a single node: Better train system Comparison set 1 serve more passengers minimize costs improve safety add new tracks Comparison set 2 increase train speed more frequent trains minimize operation costs minimize development costs Comparison set 3 increase safe distance Comparison set 4 clearer signalling 252
Analy&c Hierarchy Process (AHP) Source: Adapted from Karlsson & Ryan 1997 Create n x n matrix (for n requirements) Ä For element (x,y) in the matrix enter: Ø 1 - if x and y are of equal value Ø 3 - if x is slightly more preferred than y Ø 5 - if x is strongly more preferred than y Ø 7 - if x is very strongly more preferred than y Ø 9 - if x is extremely more preferred than y Ø (use the intermediate values, 2,4,6,8 if compromise needed) Ä and for (y,x) enter the reciprocal. EsJmate the eigenvalues: Ä E.g. averaging over normalized columns Ø Calculate the sum of each column Ø Divide each element in the matrix by the sum of it s column Ø Calculate the sum of each row Ø Divide each row sum by the number of rows This gives a value for each requirement: Ä giving the esjmated percentage of total value of the project 253
AHP example - es&ma&ng costs Req1 Req2 Req3 Req4 Req1 1 1/3 2 4 Req2 3 1 5 3 Req3 1/2 1/5 1 1/3 Req4 1/4 1/3 3 1 Normalise columns Req1-26% of the cost Req2-50% of the cost Req3-9% of the cost Req4-16% of the cost Result Req1 Req2 Req3 Req4 Req1 0.21 0.18 0.18 0.48 Req2 0.63 0.54 0.45 0.36 Req3 0.11 0.11 0.09 0.04 Req4 0.05 0.18 0.27 0.12 Sum the rows sum sum/ 4 1.05 0.26 1.98 0.50 0.34 0.09 0.62 0.16 254
Repeat AHP process twice: Ä Once to estimate relative value Ä Once to estimate relative cost Plot ROI graph High priority Value (percent) 30 25 20 15 10 5 x x x x x Medium priority Low priority 5 10 15 20 25 30 Cost (percent) 255
Other selection criteria Above average value Below average cost Value (percent) 30 25 20 15 10 5 x x x Above average in both cost and value x Above average cost Below average value x Relative Probability 30 25 20 15 10 5 x x Low Risk Exposure x x High Risk Exposure x 5 10 15 20 25 30 Cost (percent) 5 10 15 20 25 30 Relative Loss 256
Security Risk Management in Airline Turnaround Sector Check-in passenger information Ä Risk1: Blacklisted passenger presents fake document, gets checked-in because personnel could be bribed Ä Risk2: Attacker uses phishing email to extract passenger booking number and uses it to check-in to the flight Luggage information Ä Risk3: The personnel records values lower than actual weight of luggage and ground operations uses the information in the loading of the aircraft Ä Risk4:The personnel accepts luggage and adds contraband items to a passenger s luggage v v Fuel slip Ø Ø Risk5: A malicious insider with access to the computer that stores the fuel slip performs changes to the data contained in the fuel slip Risk6: The attacker intercepts the fuel slip, changes the data contained and sends it to the supplier Cargo assignment Ø Ø Risk7: A malicious insider with access rights performs changes to the cargo assignment document before it is sent to a service provider Risk8: An attacker hacks the airline mailing list, receives the cargo assignment, changes the data contained and sends the cargo assignment to a service provider 257 [Matulevičius et al., 2016] FDSE 2016 257
Security Risk Management in Airline Turnaround Sector Check-in passenger information Ä Risk1: Blacklisted passenger presents fake document, gets checked-in because personnel could be bribed Ä Risk2: Attacker uses phishing email to extract passenger booking number and uses it to check-in to the flight Luggage information Ä Risk3: The personnel records values lower than actual weight of luggage and ground operations uses the information in the loading of the aircraft Ä Risk4:The personnel accepts luggage and adds contraband items to a passenger s luggage v v Fuel slip Ø Ø Risk5: A malicious insider with access to the computer that stores the fuel slip performs changes to the data contained in the fuel slip Risk6: The attacker intercepts the fuel slip, changes the data contained and sends it to the supplier Cargo assignment Ø Ø Risk7: A malicious insider with access rights performs changes to the cargo assignment document before it is sent to a service provider Risk8: An attacker hacks the airline mailing list, receives the cargo assignment, changes the data contained and sends the cargo assignment to a service provider 258 [Matulevičius et al., 2016] FDSE 2016 258
Security Risk Management in Airline Turnaround Sector Check-in passenger information Ä Risk1: Blacklisted passenger presents fake document, gets checked-in because personnel could be bribed Ä Risk2: Attacker uses phishing email to extract passenger booking number and uses it to check-in to the flight Luggage information Ä Risk3: The personnel records values lower than actual weight of luggage and ground operations uses the information in the loading of the aircraft Ä Risk4:The personnel accepts luggage and adds contraband items to a passenger s luggage v v Fuel slip Ø Ø Risk5: A malicious insider with access to the computer that stores the fuel slip performs changes to the data contained in the fuel slip Risk6: The attacker intercepts the fuel slip, changes the data contained and sends it to the supplier Cargo assignment Ø Ø Risk7: A malicious insider with access rights performs changes to the cargo assignment document before it is sent to a service provider Risk8: An attacker hacks the airline mailing list, receives the cargo assignment, changes the data contained and sends the cargo assignment to a service provider [Matulevičius et al., 2016] FDSE 2016 259
Requirements Priori&za&on Why PrioriJzaJon is needed Ä Basic Trade- offs Cost- Value Approach Ä SorJng Requirements by cost/value Ä EsJmaJng RelaJve Costs/Values using AHP 260
Lecture 7 Requirements Prioritisation Risk Management 261
Risk Management About Risk Ä Risk is the possibility of suffering loss Ä Risk itself is not bad, it is essential to progress Ä The challenge is to manage the amount of risk Two Parts: Ä Risk Assessment Ä Risk Control Useful concepts: Ä For each risk: Risk Exposure Ø RE = p(unsat. outcome) X loss(unsat. outcome) Ä For each mitigation action: Risk Reduction Leverage Ø RRL = (REbefore - REafter) / cost of intervention 262
Continuous Risk Management Identify: Ä Search for and locate risks before they become problems Ø Systematic techniques to discover risks Analyse: Ä Transform risk data into decision-making information Ä For each risk, evaluate: Ø Impact Ø Probability Ø Timeframe Ä Classify and Prioritise Risks Plan Ä Choose risk mitigation actions Track Ä Monitor risk indicators Ä Reassess risks Control Ä Correct for deviations from the risk mitigation plans Communicate Ä Share information on current and emerging risks Source: Adapted from SEI Continuous Risk Management Guidebook 263
Risk Assessment Quantitative: Ä Measure risk exposure using standard cost & probability measures Ä Note: probabilities are rarely independent Qualitative: Ä Develop a risk classification matrix: Likelihood of Occurrence Very likely Possible Unlikely (5) Loss of Life Catastrophic Catastrophic Severe (4) Loss of Spacecraft (3) Loss of Mission (2) Degraded Mission Catastrophic Severe Severe Severe Severe High High Moderate Low (1) Inconvenience Moderate Low Low 264
Top 10 Development Risks (+ Countermeasures) Personnel Shortfalls Ä use top talent Ä team building Ä training Unrealistic schedules/budgets Ä multisource estimation Ä designing to cost Ä requirements scrubbing Developing the wrong software functions Ä better requirements analysis Ä organizational/operational analysis Developing the wrong User Interface Ä prototypes, scenarios, task analysis Gold Plating Ä requirements scrubbing Ä cost benefit analysis Ä designing to cost Continuing stream of reqts changes Ä high change threshold Ä information hiding Ä incremental development Shortfalls in externally furnished components Ä early benchmarking Ä inspections, compatibility analysis Shortfalls in externally performed tasks Ä pre-award audits Ä competitive designs Real-time performance shortfalls Ä targeted analysis Ä simulations, benchmarks, models Straining computer science capabilities Ä technical analysis Ä checking scientific literature 265
Risk Management Risk Management is a systematic activity Ä Requires both technical and management attention Ä Requires system-level view Ä Should continue throughout a project Techniques exist to identify and assess risks Ä E.g. fault tree analysis Ä E.g. Risk assessment matrix Risk and Requirements engineering Ä Risk analysis can uncover new requirements Ø Especially for safety-critical or security-critical applications Ä Risk analysis can uncover feasibility concerns Ä Risk analysis will assist in appropriate management action 266
Misuse cases A modeling technique use cases Ä Normal actors and wanted functionality + Ä Mis-users, harmful acts Makes it possible to discuss Ä Security requirements together with functional requirements. Ä With a technique that is Ø In normal use Ø Relatively easy to understand for end-users As with use-cases, there are two possibilities: Ä Diagrams Ä Textual descriptions 267
Misuse cases A modeling technique use cases Ä Normal actors and wanted functionality + Ä Mis-users, harmful acts Makes it possible to discuss Ä Security requirements together with functional requirements. Ä With a technique that is Ø In normal use Ø Relatively easy to understand for end-users As with use-cases, there are two possibilities: Ä Diagrams Ä Textual descriptions 268
Register customer threaten Flood system threaten Order goods threaten Get privileges Customer inlcude threaten Reveal customer Protect info mitigate Steal card info Outside Crook Submit question threaten threaten Spread malicious code Submit review mitigate Shop Clerk Screen input System Administrator 269
Security risk management process 270
1. Context and Assets Identification 2. Security Objectives Determination Description of organisation and its environment Ä sensitive activities related to information security 271 271
3. Risk Analysis 272 272
3. Risk Analysis 273
4. Risk Treatment Decisions Risk treatment decisions Avoiding risk Transferring risk Retaining risk Reducing risk Definition Decision not to be involved in, or to withdraw from a risk Sharing with another party the burden of loss for a risk Accepting the burden of loss from a risk Action to lessen the probability, negative consequences, or both, associated with a risk 274 274
5. Security Requirements Definition 275 275
5. Security Requirements Definition 276
What have we learnt today? Requirements Prioritisation Risk Management 277