Lecture 5: Performance Analysis (part 1)

Transcription

1 Lecture 5: Performance Analysis (art 1) 1

2 Tyical Time Measurements Dark grey: time sent on comutation, decreasing with # of rocessors White: time sent on communication, increasing with # of rocessors Oerations in a arallel rogram: 1. Comutation that must be erformed sequentially 2. Comutations that van be erformed in arallel 3. Parallel overhead including communication and redundant comutations 2

3 Basic Units n roblem size number of rocessors σ(n) inherently sequential ortion of comutation φ(n) ortion of arallelizable comutation κ(n, ) arallelization overhead Seedu Ψ n, Efficiency ε n, = = sequential execution time arallel execution time sequential execution time rocessors used arallel execution time 3

4 Amdahl s Law (1) Sequential execution time = σ n + φ(n) Assume that the arallel ortion of the comutation that can be executed in arallel divides u erfectly among rocessors Parallel execution time σ n + Seedu Ψ n, Efficiency ε n, σ n + σ n +φ(n) +κ(n,) σ n +φ(n) σ n + +κ(n,) + κ(n, ) 4

5 Amdahl s Law (2) If the arallel overhead κ n, is neglected, then Seedu Ψ n, σ n +φ(n) σ n + Let f be the inherently sequential ortion of the comutation, f = σ n σ n +φ(n) 5

6 Amdahl s Law (3) Ψ n, Ψ n, Ψ n, Ψ n, σ n + σ n + σ n /f σ n + σ n ( 1 f 1)/ 1/f 1 + ( 1 f 1)/ 1 f + (1 f)/ Amdahl s Law: Let f be the fraction of oerations in a comutation that must be erformed sequentially, where 0 f 1. The maximum seedu Ψ n, achieved by a arallel comuter with rocessors erforming the comutation is Ψ n, 1 f+ (1 f)/ Uer limit: as, Ψ n, 1 f+ 1 f < 1 f 6

7 Seedu vs. f Amdahl s law assumes that the roblem size is fixed. It rovides an uer bound on the seedu achievable by alying a certain number of rocessors. 7

8 Examle 1 If 90% of the comutation can be arallelized, what is the max. seedu achievable using 8 rocessors? Solution: f = 10%, Ψ n,

9 Suose σ n Examle 2 = n = n2 100 What is the max. seedu achievable on a roblem of size n = 10000? Solution: Ψ n, σ n σ n / 9

10 Remark Parallelization overhead κ(n, ) is ignored by Amdahl s law Otimistic estimate of seedu The roblem size n is constant for various values Amdahl s law does not consider solving larger roblems with more rocessors Amdahl effect Tyically κ(n, ) has lower comlexity than. For a fixed number of rocessors, seedu is usually an increasing function of the roblem size. The inherently sequential ortion f may decrease when n increases Amdahl s law (Ψ n, < 1 ) can under estimate seedu for large f roblems 10

11 Gustafson-Barsis s Law Amdahl s law assumes that the roblem size is fixed and show how increasing rocessors can reduce time. Let the roblem size increase with the number of rocessors. Let s be the fraction of time sent by a arallel comutation using rocessors on erforming inherently sequential oerations. s = so 1 s = σ n σ n + / σ n + 11

12 σ n = σ n + s = σ n + 1 s σ n + Ψ n, σ n + ) = (s+ 1 s )(σ n + σ n + = s + 1 s = + 1 s Gustafson-Barsis s law: Given a arallel rogram of size n using rocessors, let s be the fraction of total execution time sent in serial code. The maximum seedu Ψ n, achieved by the rogram is Ψ n, + 1 s 12

13 Remark Gustafson-Barsis s law allows to solve larger roblems using more rocessors. The seedu is called scaled seedu. Since arallelization overhead κ(n, ) is ignored, Gustafson-Barsis s law may over estimate the seedu. Since Ψ n, + 1 s = 1 s, the best achievable seedu is Ψ n,. If s = 1, then there is no seedu. 13

14 Examle An alication executing on 64 rocessors using 5% of the total time on non-arallelizable comutations. What is the scaled seedu? Solution: s = 0.05, Ψ n, + 1 s = =

15 Kar-Flatt Metric Both Amdahl s law and Gustafson-Barsis s law ignore the arallelization overhead κ(n, ), they overestimate the achievable seedu. Recall: Parallel execution time T n, = σ n + Sequential execution time T n, 1 = σ n + + κ(n, ) Define exerimentally determined serial fraction e of arallel comutation: e n, = σ n +κ(n,) σ n + 15

16 exerimentally determined serial fraction e may either stay constant with resect to (meaning that the arallelization overhead is negligible) or increase with resect to (meaning that arallelization overhead dominates the seedu ) Given Ψ n, using rocessors, how to determine e n,? 16

17 Since T n, Ψ n, = Therefore, 1 Ψ = T n, 1 e + T n,1 (1 e) and Ψ n, T(n, 1) T n, 1 (1 e) T n, 1 e + = e + 1 e e = 1 Ψ = = T(n,1) T(n,) 1 e + 1 e 17

18 Examle 1 Benchmarking a arallel rogram on 1, 2,, 8 rocessors roduces the following seedu results: Ψ n, What is the rimary reason for the arallel rogram achieving a seedu of only 4.71 on 8 rocessors? 18

19 Solution: Comute e n, corresonding to each data oint: Ψ n, e n, Since the exerimentally determined serial fraction e n, is not increasing with, the rimary reason for the oor seedu is the 10% of the comutation that is inherently sequential. Parallel overhead is not the reason for the oor seedu. 19

20 Examle 2 Benchmarking a arallel rogram on 1, 2,, 8 rocessors roduces the following seedu results: Ψ n, What is the rimary reason for the arallel rogram achieving a seedu of 4.71 on 8 rocessors? Solution: Ψ n, e Since the exerimentally determined serial fraction e is steadily increasing with, arallel overhead also contributes to the oor seedu. 20