RISK ASSESSMENT METHODOLOGY: LATE AND INCORRECT SMART METER TECHNICAL DETAILS

Transcription

1 This document sets out the risk assessment methodology to estimate potential Settlement inaccuracy as a result of late and incorrect Meter Technical Details (MTDs) following a Data Communications Company (DCC) serviced smart Meter (SMETS2) installation during the mass roll-out. We plan to revisit this assessment once SMETS2 Meter installs ramp up and we have sufficient data to reassess performance. Please note: This risk assessment methodology seeks to estimate impact on Settlement under the BSC. Impacts outside of BSC Settlement (e.g. Distribution Use of System charges, end consumer billing etc.) are not covered by this assessment. What is the risk? Smart Meters serviced by the DCC present a change to responsibilities for production, maintenance and submissions of MTDs. The Supplier will be responsible for configuring the tariff of a DCC serviced SMETS2 Meter and communicating the Smart Meter Configuration Details (D0367 data flow or equivalent) to the Meter Operator Agent (MOA). The MOA will then translate these details into the Non-Half Hourly (NHH) MTDs (D /D ) and submit them to the Supplier hub. A diagram outlining this process can be found in Appendix 1. The Issue 69 workgroup acknowledged the change in processes presented by DCC serviced smart Meters with the Supplier playing a new and key role. As Suppliers have not traditionally been tasked with the production and submission of MTDs and with the high volume of installs over a short period, the workgroup identified key risks related to the provision of smart MTDs. These risks were categorised as Supplier agent interface and Meter operations and received a risk rating of high and medium respectively. BSC obligations Provision of MTDs BSCP & sets out the processes for installation and/or reconfiguration of a DCC serviced smart Meter. These processes state that if the Metering System is configured remotely by the Supplier, it will provide the configuration details through the D0367 or an alternative method bilaterally agreed to the NHH MOA in time to allow the NHH MOA to distribute MTDs within 10 WDs of the Meter install/reconfigure date. Quality of MTDs Section L2.4.1 states that the Registrant of each Metering System, which in the case of Supplier Volume Allocation (SVA) is the Supplier, shall ensure that the Meter Technical Details for that Metering System will be true, complete and accurate. BSCP (e) states that for smart Meters, the Supplier is responsible for ensuring that the Meter Register Ids and/or Meter Register Descriptions in the D0367; and the MOA is responsible for ensuring that the Meter Technical Details, are reflective of the display on the Meter. Impact on Settlement Late MTDs MTDs sent outside the BSC required timeframes following the installation or reconfiguration of a DCC serviced smart Meter will result in any subsequent Meter readings relating to the new Meter/configuration failing NHH Data Collection (DC) validation. This will result in estimation of consumption for Meter reads which may have passed validation had MTDs been provided within BSC timescales. 1 Notification of Mapping Details 2 Non Half-hourly Meter Technical Details Page 1 of 11 ELEXON 2017

2 Any over or understating of Settlement consumption through estimated data will feed into the volume of unaccounted energy which is smeared across Suppliers through group correction factor. Incorrect MTDs Under certain circumstances, incorrect MTDs will result in Meter reads failing BSC validation, e.g. where the number of registers/rates provided in the Meter read does not correspond with that expected as per the MTDs. This will result in estimation of consumption which has an associated amount of inaccuracy. There are other circumstances where incorrect MTDs will not result in Meter reads failing BSC validation, e.g. where the physical registers/rates on the Meter are mapped incorrectly in the MTDs, but the number of registers and identifiers for those registers match. This will cause the Data Collector to misinterpret the reads, e.g. interpret the night register/rate as the day register/rate and vice versa. If such reads do not fail consumption level validation, an erroneous actual consumption value (AA) will enter Settlement. Any over or understatement of Settlement consumption through estimated data or erroneous AAs will feed into the volume of unaccounted energy which is smeared across Suppliers through group correction factor. Page 2 of 11 ELEXON 2017

3 RISK ASSESSMENT METHODOLOGY This section details each step in the risk assessment methodology and the assumptions or limitations we have identified with proposed mitigations, if any. As the steps for assessing late and incorrect MTDs are the same or follow similar principles, they are outlined together with any differences clearly marked. Step 1 Estimate number of smart Meters still to be installed We will use our latest quarterly Supplier Meter Registration Service (SMRS) snapshot to determine the total number of energised, metered Profile Class 1-4 sites. We will subtract from this number the volume of SMETS1 and Automatic Meter Reading (AMR) Meters currently installed, which we will determine using our database of D0150s sent over the Data Transfer Network (DTN). This will provide an estimate of the remaining Meters which will need to be replaced as part of the smart roll-out, i.e. our population at risk. The below tables provides an estimation of the number of smart Meter installs still to occur. PC Total MPANs Total SMETS1/AMR installs Remaining installs 1 23,876,395 2,890,306 20,986, ,304, ,509 4,094, ,666, ,851 1,229, , , ,598 Total 30,350,211 3,713,718 26,636,493 The identified assumptions and limitations with this step are as follows: At the time of producing this methodology (Aug-17), we have observed approx. 2m Advanced Domestic Meter (ADM) installations using our database of D0150s sent over the DTN. Some of these may be remotely convertible to be SMETS compliant and may therefore count towards a Supplier s smart Meter roll-out. However, we are unaware of the proportion that is capable of being converted remotely. The estimation of smart Meters still to be installed doesn t take into account changes in volumes associated with Profile Class 1-4 new connections and disconnections over the rollout. We do not know how many smart Meters will be installed in total taking into account consumers opting-out of the process and failures to install after taking reasonable steps. Our analysis of Meters already installed that count towards a Upon remote conversion from ADM to SMETS a revised set of MTDs will need to be produced and distributed. This presents the opportunity for the revised MTDs to be late or incorrect. We estimate our coverage of D0150s to be Page 3 of 11 ELEXON 2017

4 Supplier s roll-out (i.e. SMETS1 and AMR) is based on D0150s sent over the DTN and therefore doesn t provide a complete view of the market. in excess of 90%. Step 2 Estimate the proportion of installations that will be subject to late or incorrect MTDs Late MTDs We will develop a report on timely submission of MTDs following a NHH Meter exchange using our database of D0150s sent over the DTN. This report will assess historical performance at sending MTDs following a Meter exchange and determine the proportion of late MTDs which will be grouped into pre and post Final Reconciliation Settlement Run (RF) timescales. This historical performance will be used as the basis for estimating the proportion of late MTDs for future smart Meter installations. As per Annex C of the Master Registration Agreement Data Transfer Catalogue, a D0150 that references a removed Meter should only be sent once in response to its removal. On a Meter Point Administration Number (MPAN) basis, the report will identify the initial D0150 that references a newly installed and removed Meter, and then a comparison will be made between the submission date of the D0150 and the Meter install date to determine how many days it took for the MOA to provide the details of the new Meter. The below table provides the historical performance at sending NHH MTDs following a Meter Exchange (MEX). Band Total MEX Pct Received before SF 8,108, % Received SF-RF 456, % Received after RF 14, % Total 8,578,880 An initial methodology was assessed that utilised volumes reported through PARMS Serial NM11 Timely Sending of NHH MTDs to NHHDCs for determining historical performance at sending late MTDs. However, following an assessment of the high level figures in NM11, suspect reporting was identified. The below tables summarises the proportion of late NHH MTDs as reported through NM11 over a six month period (January to June 2017). Band Pct (NM11) Received before SF 86.38% Received SF-RF 6.97% Received after RF 6.64% Assessment of NM11 Drill Down data identified reporting anomalies which were raised with the relevant data providers. Four data providers have since confirmed reporting errors in their NM11 submissions. Of these reporting errors, two related to over reporting (particularly against the received after RF standard (7)) and two related to under reporting. The four data providers in question are currently amending their NM11 reporting to bring it in line with the BSC requirements. Page 4 of 11 ELEXON 2017

5 Due to the confirmed erroneous reporting in NM11, it was decided not to use PARMS data for the purposes of this risk assessment. Incorrect MTDs We will develop a report on the volume of back dated corrections to NHH MTDs following a Meter exchange using our database of D0149s and D0150s sent over the DTN. This report will assess historical performance at sending back dated NHH MTD corrections following a Meter install and determine the proportion of incorrect MTDs which will be grouped into pre and post RF timescales. This historical performance will be used as the basis for estimating the proportion of incorrect MTDs for future smart Meter installations. A back dated NHH MTD submission which includes a change to a key field suggests the initial submission was incorrect. The report will identify all Meter installs and record the MTDs sent immediately following the install. A comparison will then be made for any subsequent MTD submissions with the same effective from date to determine whether they provide a back dated correction to a key field. Of each back dated correction to a key field, the difference will be calculated between the receipt date and effective from date of the corrected MTD to determine the timescale that the correction was applied under. We have used the below data items within the D0149 and D0150 data flows as key fields: Item Name Standard Settlement Configuration Id Meter Id (Serial Number) Meter Register Id Meter Register Type Measurement Quantity Id Meter Register Multiplier Time Pattern Regime Data Flow D0149/D0150 D0149/D0150 D0149/D0150 D0150 D0150 D0150 D0149 The below table provides the historical performance of correcting NHH MTDs following a MEX. Band Total MEX Pct No Correction 8,495, % Correction before SF 14, % Correction SF-RF 56, % Correction after RF 11, % Total 8,578,880 The initial version of this assessment included Number of Register Digits as a key field. However, when the initial assessment was undertaken, significantly more MTDs were identified as being subject to a backed dated correction (4.69% as opposed to 0.96% referenced above). Following an investigation into the back dated corrections, it was identified that just under 80% were by a single participant amending the same data item within D0150s (Number of Page 5 of 11 ELEXON 2017

6 Register Digits). Following an internal assessment of this scenario, we determined that it was highly unlikely that this issue had presented a material impact on Settlement (as it was unlikely that the Meters had rolled over since install), and that the performance of this single participant was not reflective of wider industry performance. We therefore decided to remove this scenario from our risk assessment calculation. This resulted in the estimated Settlement impact of incorrect smart MTDs reducing by 10 times. The identified assumptions and limitations with this step are as follows: Using historical performance at sending late and incorrect MTDs to estimate the proportion of late and incorrect MTDs for SMETS2 installs assumes that performance at sending MTDs for a SMETS2 Meter will be comparable. As the SMETS2 MTD process is more complex and new participants are involved, performance may not be comparable, i.e. it could be better or worse depending on the level of automation and volumes of exceptions needing to be manually worked. Our report on late and incorrect MTDs following a NHH Meter install is based on D0150s sent over the DTN and therefore does not provide a complete view of the market. This dataset may not provide an entirely representative view of the proportion of late or incorrect MTDs in the market. Determining proportions of late and incorrect SMETS2 MTDs based on previous Meter installs does not take into account more regular changes to tariffs envisaged for SMETS2 Meters. Using historical performance at sending MTDs to determine the proportion of late and incorrect MTDs does not capture MTDs that are currently (and may always be) missing or incorrect. These unidentified issues may impact our estimation of the proportion of late and incorrect MTDs. On the initial materiality assessment, we will calculate an upper and lower estimated error banding to provide for performance increasing or decreasing by 10%. Once we see SMETS2 Meter installs ramp up, we will assess the proportion of late and incorrect MTDs for these installs and revisit our risk assessment. We estimate our coverage of D0150s to be in excess of 90%. Once we see SMETS2 Meter installs ramp up, we will assess the proportion of late and incorrect MTDs following an install or tariff change and revisit our risk assessment. Step 3 Estimate the number of Settlement Days impacted Using the reports on late and incorrect MTDs developed for Step 2, the average days late/corrected will be calculated. This average number of days will be compared against Settlement Run timescales to determine actual Settlement Days impacted, i.e. excluding the pre-initial Settlement Run (SF) proportion as there was no impact on Settlement. For MTDs received or corrected after RF, it will be determined how many of the associated Settlement Days will have undergone correction through the fluid period and how many had crystallised and will not be corrected through normal Settlement reconciliation. Page 6 of 11 ELEXON 2017

7 Late MTDs The below table provides the average days of sending late MTDs following a MEX per Settlement banding. Band Avg. days MTDs received Received SF-RF 82 Received after RF 738 Incorrect MTDs The below table provides the average days of correcting NHH MTDs following a MEX per Settlement banding. Band Avg. days MTDs corrected Correction SF-RF 140 Correction after RF 707 The identified assumptions and limitations with this step are as follows: The estimated number of impacted Settlement Days does not take into account how long it may take the DC to process the set of MTDs. Step 4 Estimate daily inaccuracy associated with estimating consumption We will develop a report on NHH consumption estimation inaccuracy using our database of D0019s 3 sent over the DTN. This report will analyse NHH estimated and actual consumption to determine an average daily error per Profile Class. On a MPAN basis, the report will compare the estimate (EAC) with the latest actual (AA), where available, to determine the difference, i.e. how close the estimate was to actual consumption. As estimation can both understate and overstate actual consumption, the gross difference will be used when determining the average error. This is to provide a view of Settlement inaccuracy, regardless of the direction, on an average basis as a result of estimating consumption. This aligns with the treatment of error for erroneously large EAC/AA reported to PAB through SR Metering System EAC/AA Data 4 The risk that NHHDCs process incorrect Meter readings, resulting in erroneous data being entered into Settlement. Page 7 of 11 ELEXON 2017

8 The below tables provides the average inaccuracy associated with estimating consumption per day for Profile Classes 1-4 based for a random sample of just under 1 million sites: Profile Class Unique MPANs Days estimated Avg. (mean) daily inaccuracy (kwh) Avg. (median) daily inaccuracy (kwh) 1 717, ,873, , ,291, , ,844, ,122 46,060, Total 964,930 1,236,070,300 As per the above, there is a large variance between the mean and median. This is due to extreme values within the dataset which are weighting the mean. These outliers could be caused by genuine cases of extreme estimation inaccuracies, Settlement issues such as erroneously large EACs/AAs or by the application of Gross Volume Correction (GVC). To exclude these extreme values from our risks assessment, as they may not reflect true estimation inaccuracy, we are taking the median as the average. This is to provide a realistic view of the inaccuracy associated with estimating consumption for a normal NHH site. The identified assumptions and limitations with this step are as follows: Our analysis of average daily estimation error is based on D0019s sent over the DTN and therefore doesn t provide a complete view of the market. This dataset may not provide an entirely representative view of average estimation error in the market. Our D0019 dataset doesn t confirm whether the consumption flows were used in Settlement, i.e. whether they were accepted by NHH Data Aggregation systems. For a comparison between the estimated (EAC) and actual (AA) consumption to be made, an actual consumption value must be available. Therefore, this reporting doesn t cover sites where the actual consumption is yet to be provided. Such sites which may be read on a less frequent basis may provide higher levels of error associated to estimation. We are assuming incorrect MTDs that result in erroneous AAs will fail consumption level validation and will be prevented from entering Settlement, i.e. through BSC validation rules such as twice the expected advance and zero/negative consumption. Therefore, we are assuming that incorrect MTDs will result in estimated consumption. We estimate our coverage of D0019s to be approximately 55%. Page 8 of 11 ELEXON 2017

9 Using historical performance at estimating consumption may not take into account recent and future changes in the accuracy of estimated consumption. As the smart Meter roll-out progresses, we can reassess the average error associated with estimating consumption and revisit our risk assessment. General assumptions/limitations The below table outlines the general assumptions and limitations we have identified and any proposed mitigations. As we don t know the installation profile of the smart Meter roll-out, the methodology provides an estimation of total error associated with smart MTD processes over the course of the roll-out. We acknowledge that there could be spikes in smart Meter install volumes that present heightened risk. This assessment doesn t take into account impacts on participant resource for chasing late or requesting corrections to SMETS2 MTDs. This methodology identifies late and incorrect NHH MTDs through the MTDs sent by the MOA at the end of the MTD transfer process. For SMETS2 processes, late or incorrect MTDs could be caused by Supplier or MOA processes. Some back dated NHH MTD corrections may present an immaterial impact to Settlement. Whilst we have limited our reporting to back dated correction of NHH MTD key fields which we feel have the potential to present a material impact, a proportion of these may still be immaterial. We are estimating error on an average basis, i.e. the sites subject to late MTDs may be subject to substantially less or more error associated to estimated consumption. Using the average system buy/system sell prices to monetise the risk may not reflect the exact impact on Parties imbalance charges. If a risk profile over the roll-out is desired, we could request access to high level figures from Supplier provided smart roll-out plans from either Suppliers themselves or Ofgem. If the monitoring of this risk suggests significant volumes of back dated NHH MTD corrections and therefore estimated consumption entering Settlement, further analysis could be undertaken on a more granular site specific basis to give a more accurate view of error. If the monitoring of this risk suggests significant volumes of late MTDs and therefore estimated consumption entering Settlement, further analysis could be undertaken on a more granular site specific basis to give a more accurate view of error. The wider PAF review work is looking at how Settlement Error should be assessed and reported. Calculation formula Settlement inaccuracy = total installs prop. late or incorrect days impacted avg. daily estimation error Page 9 of 11 ELEXON 2017

10 This calculation will output an error volume which we will monetise using the average system buy/sell prices over the last 12 months. RISK ASSESSMENT CALCULATION This section provides the high level output from the risk assessment calculation. Potential Settlement inaccuracy caused by smart MTD processes Late smart MTDs Settlement impact Lower band ( ) Upper band ( ) SF-RF (fluid error) 5.5m 6.8m After RF (crystallised error) 0.8m 0.9m Total 6.3m 7.7m Incorrect smart MTDs Settlement impact Lower band ( ) Upper band ( ) SF-RF (fluid error) 1.8m 2.2m After RF (crystallised error) 0.5m 0.6m Total 2.3m 2.8m As noted throughout this methodology, when deciding how to treat different scenarios in the risk assessment, we have sought to provide a reasonable view of potential Settlement impact. This produces a combined Settlement impact upper banding of 10.5m (as per the above). If we were looking at these risk assessments from more of a worst case perspective, it would increase the estimated Settlement impact. For example, if we: used the mean average daily estimation inaccuracy as opposed to the median, under the assumption that some extreme estimation inaccuracies may occur following a late or incorrect smart MTD; and included the volume of back dated NHH MTD corrections observed to Number of Register Digits (increasing the post-sf exception proportion from 0.79% to 4.5%), under the assumption that it could present a material impact on Settlement and other participants may be subject to a similar issue. Treating the two scenarios as per the above, the combined potential Settlement impact lower and upper bands are calculated as 73m and 89.3m respectively. This further demonstrates the level of uncertainty around this future activity. Page 10 of 11 ELEXON 2017

11 SMART METER TECHNICAL DETAILS APPENDIX 1 DCC SERVICED METER PROCESS DIAGRAM The below diagram provides an overview of Meter installs and remote read process for a DCC serviced smart Meter. Supplier requests the DCC to configure, reconfigure or read smart Meter Supplier updates the Meter Operator with new tariff details Meter Operator sends Meter Technical Details reflecting new tariff Supplier, Data Collector & Distributor XML Response D0367 D0149/D0150 DCC Supplier Meter Operator DCC confirms action Supplier provides smart Meter read to Data Collector D0010 D0052 D0205 Supplier confirms new tariff details to Distributor D0041 D0019 BSC Settlement Systems Data Aggregator Data Collector Distributor Data Aggregator performs Settlement Runs and submits data to BSC Settlement Systems Data Collector validates Meter reads provides estimated/actual consumption data to Data Aggregator Supplier confirms new tariff details to Data Collector Page 11 of 11 ELEXON 2017