Practice Note 2: Peer Review. July 2017 Draft

Transcription

1 Practice Nte 2: Peer Review July 2017 Draft

2

3 Cntents 1. Abut this Practice Nte Purpse Develpment 1 2. Peer review The value f peer review Reviews cnsidered t be peer reviews Wh can undertake a peer review? 4 3. Guidance fr peer reviewers Establishing the scpe and bjectives Liability and insurance fr peer reviewers Ethical bligatins f peer reviewers Peer review and cnfidentiality Infrming the design engineer their wrk is being reviewed Engineering judgement 6 4. Types f Peer Review Cncept r Strategic Peer Review Regulatry Peer Review Specific Peer Reviews Frensic Peer Review 8 5. Cnducting a Peer Review 9 6. References Legal infrmatin 11

4 1. Abut this Practice Nte 1.1 Purpse A peer review is an independent assessment f an engineered asset r an engineer s design, design prcess, materials chice r assessment against set criteria. Whether as the design r assessing engineer r the peer reviewer, engineers in any field may find themselves being part f a peer review. Peer review plays an increasingly imprtant rle in the engineering design prcess, as a means f managing risk, maintaining design quality and ensuring cmpliance. Peer reviews may als be requested n assessments made n existing structures r engineered assets. Sme fields f engineering, such as thse invlved with cnstructin, have well develped peer review requirements and prcesses. Other fields f engineering use peer reviews t a lesser extent. This Practice Nte prvides guidance t engineers, clients and ther interested parties n what a peer review is and hw t undertake ne. Differentiating a peer review frm ther types f review is imprtant, as is differentiating between types f peer review. The Practice Nte sets ut the varius kinds f peer review used in New Zealand with the aim f creating cmmn understanding f the terms used. This guidance is generally applicable t all fields f engineering where peer reviews are undertaken, but it des nt attempt t reflect details in every circumstance. This Practice Nte differentiates between the fllwing types f peer reviews: Cncept (r strategic) Peer Review a preliminary stage review, which takes place befre detailed design r engineering wrk begins Regulatry Peer Review t cnfirm cmpliance f a cmplex r unique aspect f a design with a statutry cde r standard Specific Peer Review t cnfirm the suitability f specific aspects f a cmpleted r partially cmpleted design r t review an assessment f a specific aspect f an existing structure r asset Frensic Peer Review usually required as a result f a failure f a cmpleted design. This Practice Nte des nt extend t: in-huse review prcesses that may be part f an engineering firm s quality assurance prcesses cmpliance reviews that may be undertaken by staff frm a Building Cnsent Authrity (BCA) t review aspects f a design slely fr cmpliance with the Building Cde secnd pinins, where an alternative design is required withut review f anther engineer s wrk cmpetence reviews, where an engineer s cmpetence is reviewed expert reviews required as part f a legal r disciplinary prcess Develpment The Institutin f Prfessinal Engineers New Zealand (IPENZ) has facilitated the develpment f this Practice Nte. This versin supersedes Practice Nte 02 (ISSN ), which was published in An IPENZ Practice Nte n the Rle f the Expert Witness is being develped. 26 July 2017 Versin 2 Draft 1

5 The Practice Nte has been prepared in accrdance with standard IPENZ Practice Nte prcedures, which include reprting n prgress t the Engineering Practice Advisry Cmmittee, peer review and general membership review. This review and reprting prcess ensures the delivery f a rbust gd practice dcument. Dcuments referred t r drawn n during the develpment f this Practice Nte are summarised n page 11. Jul 2017 Versin 2 Draft 2

6 2. Peer review A peer review is an independent assessment f an engineered asset r an engineer s design, design prcess, materials chice r assessment against set criteria. It is carried ut by anther engineer cmpetent in the subject f the review criteria. It is a prfessinal pinin based n sund engineering analysis and assumptins, gd practice, apprpriate cdes and unbiased judgement. A peer review is ften desirable in the design prcess fr cmplex r unique aspects f the design r where the impact f the design will be significant. It is nt intended as a prcess fr reslving design issues, althugh the utput can be a key surce fr such reslutin. Many engineers in certain industry areas, such as cnstructin, are finding they are increasingly engaged t carry ut peer reviews. This increasing need riginates frm regulatrs wh d nt have the necessary in-huse technical expertise, and frm clients wh seek verificatin f cmpliance and their wn quality assurance prcedures. Fr the purpses f this Practice Nte we refer t: the persn wh undertk the riginal design r assessment as the design r assessing engineer the persn undertaking the peer review as the peer reviewer the persn r rganisatin cmmissining the peer review as the client. (In many cases the client will cmmissin the peer review fr submissin t a regulatr. The client will als ften be the party wh cmmissined the riginal wrk r design.) 2.1 The value f peer review Peer review is a pwerful tl in maintaining and enhancing design quality and is an imprtant feature f a selfregulating prfessin such as engineering. Peer review helps ensure suitable prcesses, assumptins, decisins and designs have been taken r made. This helps strengthen the quality f wrk engineers d and gives New Zealanders cnfidence in the wrk f engineers. A peer review will mean an engineer s wrk will be scrutinised by anther. When design and assessing engineers willingly and actively participate in the prcess, the quality f the prfessin s verall utput remains at a high standard. Wider benefits, including reduced cnflicts and quicker cmpletin, can be achieved if the peer review is regarded as a cllabratin between designer r assessr and reviewer. In sme cases a peer review may identify situatins where design decisins culd have been made differently. The design r assessing engineer, and mst likely their engineering firm, has the chance t hear and understand alternative appraches t particular design challenges, making peer review an pprtunity fr nging prfessinal develpment. When a peer reviewer is engaged early in the design f larger and mre cmplex prjects the value f a peer review is significantly increased. Inadequate r incrrect decisins r assumptins picked up early by the peer reviewer can save significant re-design effrt. 2.2 Reviews cnsidered t be peer reviews This Practice Nte differentiates between the fllwing types f peer reviews: Cncept r Strategic Peer Review Regulatry Peer Review Specific Peer Review Frensic Peer Review. Each type f review is discussed in sectin 4. Jul 2017 Versin 2 Draft 3

7 In-huse review prcesses that may be part f an engineering firm s quality assurance prcesses, secnd pinins and cmpetence reviews are nt cnsidered t be peer reviews. 2.3 Wh can undertake a peer review? Befre agreeing t undertake a peer review, an engineer may need t cnfirm their cmpetence t the client wh wishes t engage them. This cnfirmatin can include a CV, references, assessments made by a prfessinal engineering bdy, r a prtfli f relevant experience. Ideally there shuld be n frm f dependent relatinship between the peer reviewer and the design r assessing engineer. The peer reviewer shuld declare they are independent f the design r assessing engineer and the design r assessing engineer s rganisatin, and have n financial r ther interest in the utcme f the peer review. In sme practice fields r disciplines where there are small numbers f engineers but a significant demand fr peer reviews, independence can be difficult. Design r assessing engineers may end up being the peer reviewer fr ther design r assessing engineers wh, in turn, peer review their wrk. The same situatin may ccur in certain lcatins where lcal knwledge is required, r preferred, but the pl f suitable engineers is small. In these cases, the peer reviewer shuld clearly identify any ptential cnflicts f interest prir t being engaged and cnsider their ethical bligatins t maintain hnesty, bjectivity and integrity. In sme niche areas where the necessary expertise is nt available in New Zealand the client may need t engage an verseas peer reviewer. Fr regulatry peer reviews, it is imprtant that the peer reviewer is cnsidered acceptable by the Regulatr. Fr Building Cnsent, it is advisable that the peer reviewer is apprved by the relevant Building Cnsent Authrity befre carrying ut review wrk. Jul 2017 Versin 2 Draft 4

8 3. Guidance fr peer reviewers This sectin sets ut the matters engineers in particular shuld cnsider when they are appinted t the rle f peer reviewer. Besides technical expertise, peer review relies n parties effectively agreeing the scpe and bjectives f the wrk, acting ethically and being aware f their liability. 3.1 Establishing the scpe and bjectives Peer reviews may be limited t parts f a design r may relate t an entire design r design prcess. Equally, if a pstdesign assessment r a review f such an assessment is t be carried ut then this may be limited t certain criteria related t the built structure r asset. The bjectives and scpe f a peer review need t be clearly determined and agreed by the client and peer reviewer befre the peer reviewer begins their wrk. There shuld be a cntract in place between the peer reviewer and the client, supprted by a brief setting ut what is and is nt t be cvered in the peer review. The scpe shuld ideally be fcused n cmplex and unusual aspects f the design. Keeping the scpe fcused n key aspects where pssible is the mst effective chice, especially when demand fr peer reviewers is high. Fr regulatry reviews, peer reviewers shuld be engaged, in particular, t cnsider alternative slutins used t shw cmpliance t the Building Cde. It is at the discretin f the peer reviewer t add relevant cmments ver and abve the brief supplied by the client. The peer reviewer may think it apprpriate t prvide additinal cmments n bth psitive and negative aspects f the design. Hwever, it is nt cnsidered apprpriate fr peer reviewers t cmment n persnal design r material preferences. 3.2 Liability and insurance fr peer reviewers Engineers engaged as peer reviewers shuld cnsider limiting their level f liability in cntract r in trt and make this clear in the cntract with the client. It is recmmended that engineers limit their liabilities in accrdance with industry guidelines and by discussing apprpriate levels f liability fr certain high cst prjects with their insurer. Under the Cnsumer Guarantees Act, engineers need t be aware that there is n limit f liability where the wrk relates t persnal, dmestic r husehld use. In the event f an upheld claim related t the design wrk r ther aspects f the design prcess, the peer reviewer will be liable fr relative damages apprtined by the curt. Fr regulatry peer reviews carried ut t verify Building Cde cmpliance, peer reviewers will have t cmplete a Prducer Statement (PS2 Design Review). The IPENZ/ACENZ/NZIA PS2 frm states a maximum amunt f damages payable t the BCA in relatin t the wrk dne by the peer review firm. This amunt can be altered but shuld nly be altered fr certain high cst prjects. 3.3 Ethical bligatins f peer reviewers Ethical cnsideratins can arise during peer review, particularly as the design r assessing engineer s infrmatin is being shared with the peer reviewer and there is the ptential fr differences f pinin between the design r assessing engineer and the peer reviewer. Every rule in the CPEng and IPENZ Cdes f Ethical Cnduct is relevant t all stages f peer review, including accepting peer review wrk. The IPENZ Practice Nte Engineers and Ethical Obligatins (2016) prvides guidance t IPENZ members and Chartered Prfessinal engineers n hw t interpret the relevant Cde f Ethical Cnduct. Jul 2017 Versin 2 Draft 5

9 Engineers cncerned abut their ethical bligatins as the peer reviewer (including cncerns abut health and safety and cmpetence) can cntact IPENZ. 3.4 Peer review and cnfidentiality When reviewing anther engineer s wrk the peer reviewer must agree t maintain cnfidentiality f that design wrk r any infrmatin relating t it that may be available as part f the review prcess. When cnsidering disclsure f infrmatin btained during a peer review, peer reviewers must take accunt f their ethical bligatins t maintain cnfidentiality under the Chartered Prfessinal Engineers Cde f Ethical Cnduct as well as any additinal clauses n cnfidentiality set ut in the cntract. 3.5 Infrming the design engineer their wrk is being reviewed It is n lnger cnsidered an ethical bligatin under the 2016 Chartered Prfessinal Engineers and IPENZ Cdes f Ethical Cnduct fr the peer reviewer t infrm the design r assessing engineer they are reviewing their wrk. Hwever, it is strngly recmmended that peer reviewers infrm the design r assessing engineer they are reviewing their wrk as a prfessinal curtesy, prvided that cnfidentiality bligatins are maintained. 3.6 Engineering judgement All engineering design and engineering assessment includes elements f engineering judgement a unique persnal perspective n what is apprpriate r acceptable derived frm persnal and prfessinal experiences and skills. Judgements will vary frm engineer t engineer just as persnal and prfessinal experiences vary. Cdes and standards used fr cmpliance purpses determine the minimum criteria a design must satisfy. In sme circumstances, fr example, when shwing cmpliance t the Building Cde using an Alternative Slutin, guidelines are used as a basis fr shwing cmpliance. Similarly, engineering assessments may be based n guidelines where standards r cdes dn t exist. Minimum criteria set ut in these standards r guidelines may nt be sufficient t meet the client s brader requirements fr the entire building r prcess as judged by design r assessing engineers and peer reviewers. Design r assessment engineers and peer reviewers may differ n the pint at which thse brader requirements are met. A peer reviewer may feel their rle is t err n the side f cautin t a greater extent cmpared t the design r assessment engineer. Hwever, peer reviewers are encuraged t maintain a fair and reasnable apprach when judging the design, prcess r assessment chices made by the design r assessment engineer. Jul 2017 Versin 2 Draft 6

10 4. Types f Peer Review The fllwing reviews are widely recgnised and understd as peer reviews within the design cmmunity. Sme reviews may be required at specific checkpints in the design prcess f certain prjects, while thers can be called fr when needed irrespective f the stage f the prject. It is imprtant that there is a cmmn understanding and shared expectatins between the client and the peer reviewer f the utput, the prcess and the end use f the review. 4.1 Cncept r Strategic Peer Review A cnfirmatin f directin r cncept peer review typically ccurs relatively early in the design prcess. It may be requested by the client r anther engineer t cnfirm the prpsed design r directin is apprpriate. This is mre likely t be required when the design prblem has cmplex elements. This review may be a lengthy prcess as there may be significant scial, cultural r envirnmental impacts t be analysed. The purpse f a Cncept Review is t check the verall cncept is well cnceived and understd. It will als prvide an pinin n design ptin identificatin and evaluatin, risks and the cnclusins drawn by the design engineer (r team f design engineers). The review will include cmments n the prpsed design methds and hw cmpliance with pertinent regulatins and laws and design cdes will be achieved. As a general guide, a Cncept r Strategic Review culd cver: client bjectives scial, cultural and envirnmental impacts relevant regulatry framewrk and standards design inputs and assumptins cnsidered in cncept drawings and a preliminary Design Features Reprt lad paths r prcesses prpsed analytical methds prpsed QA prcesses Risk Register health and safety cnsideratins aspects f the design that shuld be subject t further review. 4.2 Regulatry Peer Review A Regulatry Peer Review is an pinin based n reasnable grunds n whether certain aspects f the design submitted by the design engineer cmply with the pertinent regulatins, cdes and standards. It is imprtant t differentiate between a frmal Regulatry Peer Review and an in-huse cmpliance check, undertaken by a BCA, fr example. A Regulatry Peer Review is nt intended t be, and ught nt t be used as, an independent verificatin f the details f every calculatin. It shuld als nt be used as a means t verify that design dcuments, such as Prducer Statements, satisfy cntractual bligatins. Regulatry reviews may be requested r triggered by the regulatr fr several reasns including: cmplexity, unusual frm, use f new technlgy r material, r unusual risk f prject (including cmpliance via an alternative slutin that is nt well understd) the design engineer nt meeting the regulatry bdy s required level f knwledge r skills criteria in the relevant area f design cncerns abut aspects f the design r design methdlgy. Jul 2017 Versin 2 Draft 7

11 A Regulatry Peer Review may als be requested by the client as a matter f curse prir t applying fr cnsent in a cmplex r higher risk prject. Fr building cnsents, Regulatry Peer Reviews may be required fr a number f Building Cde clauses. In additin t what is cvered in a Cncept Peer Review, a Regulatry Peer Review may cver: lad assessments mdelling assumptins sftware mdel inputs and utputs design f primary and selected secndary elements drawings and specificatins perfrmance specificatins. 4.3 Specific Peer Reviews A Specific Peer Review is a technical review that can be undertaken at any part f the design prcess t review the suitability f ne r mre aspects f a cmpleted r part-cmplete design, r a review f an assessment n an already as-built structure r engineered asset. The criteria triggering a specific peer review can include cst effectiveness, buildability r cmplexity f an assessment Within the design prcess, a Specific Peer Review is usually requested by the client in the event f sme cncern abut the design r a new factr, such as a change in legislatin r a physical (gelgical) event that has arisen since the riginal design was undertaken. The scpe f the review will usually be limited t certain elements r aspects f the design. A Specific Peer Review f an assessment f an already built structure r engineered asset can be requested by the client if the structure is cmplex r there have been differences in pinin in previus assessments. The methdlgy f a Specific Peer Review is similar t a Regulatry Peer Review except it is nt necessarily being undertaken fr cmpliance purpses. In additin t what is cvered in a Regulatry Review, a Specific Peer Review may cver: ecnmic viability buildability envirnmental impact innvatin risk seismic capacity. 4.4 Frensic Peer Review A Frensic Peer Review may be part f an investigatin and is typically requested when an examinatin f the perfrmance f a cmpleted r as-built prject is needed fllwing an engineering failure r dispute. A Frensic Peer Review culd be requested by a client, a third party, a regulatry bdy r a prfessinal bdy and can be carried ut by either a single engineering expert r a panel f experts t examine the failure f the structure r built asset. The fcus f this type f peer review is mre n the likely perfrmance and cmpliance f the cmpleted wrk rather than the design analysis and calculatin. It may cnsider a repeated design that ccurs in several existing structures r built assets. The peer reviewer(s) in this type f review will nt be determining culpability fr the failure r dispute, but the review they prduce may be used as a basis fr litigatin r disciplinary actin. Jul 2017 Versin 2 Draft 8

12 5. Cnducting a Peer Review This sectin utlines the sequential steps fr establishing, undertaking and cmpleting a peer review. 1. The client identifies the need fr a peer review and the brad scpe f the review. The client then identifies a cmpetent peer reviewer, ensuring the peer reviewer is acceptable t the Regulatr where a Regulatry Peer Review is being undertaken. 2. The peer reviewer makes the client aware f any ptential cnflicts f interest. 3. The client and peer reviewer agree n a brief that sets ut: the purpse f the peer review bjectives fr the wrk (thse given t the design r assessing engineer) scpe f the peer review cst and expected timeframes fr the peer review lines f cmmunicatin between the peer reviewer and the client reprting schedule (where interim reviews are expected) any barriers that may prevent aspects f the peer review being carried ut, such as knwn limitatins in btaining necessary dcumentatin the limit f the peer reviewer s liability recipients f the review. 4. The client and peer reviewer enter int a cntractual agreement, which includes the agreed brief. The Shrt Frm IPENZ/ACENZ cntract shuld be suitable unless the peer review is cmplex. Other agreements such as the American Sciety f Civil Engineers E-581 Agreement between wner, design engineer and peer reviewers fr peer review f design may be apprpriate where the IPENZ/ACENZ SFA is nt sufficient. 5. The peer reviewer undertakes the peer review in accrdance with the brief agreed with the client. As a prfessinal curtesy, it is recmmended that the peer reviewer infrms the design engineer at this stage that they have been cntracted t undertake a peer review f the engineer s wrk. The peer reviewer may als want t ask the design r assessing engineer fr relevant dcumentatin and agree hw further requests fr infrmatin and feedback f matters shuld be managed between them. 6. During a Cncept r Strategic Peer Review, Regulatry Peer Review r a Specific Peer Review it is cmmn fr there t be cnsiderable cmmunicatin between the Peer reviewer and the design engineer. There may be little r n cmmunicatin fr a Frensic Peer Review depending n any specific directives, such as legal cnstraints. It is recmmended that the peer reviewer maintain a lg f queries and respnses between themselves and the design r assessing engineer. Maintaining a lg may be a requirement fr sme regulatry bdies. The lg shuld be maintained until each matter raised has been reslved and retained accrding t the engineer s internal QA prcedures. 7. Upn cmpletin f the peer review, the peer reviewer reprts t the client and shuld send a curtesy cpy f their reprt t the design r assessing engineer having btained the client's cnsent. (Prviding a curtesy cpy t the design engineer may nt be pssible fr a Frensic Peer Review due t legal cnstraints.) The review reprt shuld: specify wh is entitled t rely n the reprt and under what circumstances state the scpe and purpse f the reprt describe what has and hasn t been reviewed Jul 2017 Versin 2 Draft 9

13 prvide key findings include disclaimers and qualifying statements fr wrk nt undertaken matters requiring further investigatin reliance n infrmatin prvided by thers assumptins made. 8. A Prducer Statement (PS2 Design Review) shuld als be cmpleted where a regulatry review has been undertaken. Care must be taken t clearly define the scpe f the design review when cmpleting the PS2. Jul 2017 Versin 2 Draft 10

14 6. References The fllwing resurces were used in preparatin f this Practice Nte, r referred t in it: American Sciety f Civil Engineers (2011). E-581 Agreement between Owner, Design engineer, and Peer reviewers fr Peer Review f Design. Available at Cleman, J (2016). Gvt initiated inquiry int Havelck Nrth water: Beehive. Available at: Institutin f Prfessinal Engineers New Zealand (2016). Cde f Ethical Cnduct: What yu need t knw. Available at: Institutin f Prfessinal Engineers New Zealand (2014) and Assciatin f Cnsulting Engineers New Zealand. Practice Nte 1: Guidelines n Prducer Statements. Available at: Institutin f Prfessinal Engineers New Zealand and Assciatin f Cnsulting Engineers New Zealand (2016). Shrt Frm Agreement fr Cnsultant Engagement. Available at: library/dcs/default-surce/news-publicatin/engineering-practice/shrt-frm-agrmt-apr- 2016a8a300e1c0486f9aa31eff bd.dcx?sfvrsn=4 Institutin f Prfessinal Engineers New Zealand, Assciatin f Cnsulting Engineers New Zealand and New Zealand Institute f Architects (2013). Prducer Statement PS2 Design Review. Available at: Registratin Authrity fr Chartered Prfessinal Engineers (2016). Cde f Ethical Cnduct. Available at: (Part 3 f Chartered Prfessinal Engineers f New Zealand Rules (N 2) 2002) Smith, N. (2016). MBIE investigatin int building perfrmance: Beehive. Available at Wellingtn City Cuncil (n.d.). Building cntrl and peer review Wellingtn City Cuncil Guidance Dcument. Wellingtn: Wellingtn City Cuncil. 7. Legal infrmatin Practice Ntes ffer guidance t practising engineers by explring issues f imprtance t the prfessin and setting ut gd-practice methdlgies. They are written by practitiners and subject t peer review by IPENZ members. While every care is taken in their preparatin, these dcuments are nt intended t be exhaustive and are nt ffered as frmal advice. Practices, systems and advice may vary depending n individual circumstances, and practitiners must exercise their wn prfessinal skill and judgement. IPENZ accepts n liability arising frm their use and nthing in the Practice Nte binds IPENZ in determining the utcme f any future cmplaint. Practice Ntes are cpyright t IPENZ and cannt be reprinted withut permissin. The Institutin f Prfessinal Engineers New Zealand Inc. Jul 2017 Versin 2 Draft 11