Elsevier

Energy and Buildings

Volume 231, 15 January 2021, 110617
Energy and Buildings

Disaggregation of household solar energy generation using censored smart meter data

https://doi.org/10.1016/j.enbuild.2020.110617Get rights and content

Highlights

  • Solar panels can not be detected from smart meters in over 50% of cases if censoring is not considered.

  • A method is presented to detect the maximum power output of a PV panel and infer PV generation from any smart meter.

  • For the sample data, performance metrics were comparable to state-of-the-art approaches which do not consider censoring.

  • An extension demonstrates how to infer PV capacity for a neighbourhood of houses without local solar irradiance.

Abstract

Quantifying small scale domestic solar (PV) generation from energy consumption is becoming increasingly important as the install base of small solar (PV) panels rapidly grows. Unfortunately, it is often the case that the only insight into the consumption and generation of energy within a house comes from smart-meter readings. The smart meter records the amount of energy the house takes from the grid, and does not independently measure and report the local generation that might be consumed by the home, or fed back to the grid. To address this issue, we propose a novel approach to disaggregate PV generation from energy consumption that also infers installed PV capacity. This is done by disaggregating PV generation from censored smart meter readings, and specifically by finding the most likely distribution for the energy consumption and using it to infer the solar generation. We extend this approach to propose the first technique to infer PV capacity without weather data or a solar proxy, using instead only smart meter readings given a group of houses in close proximity. We evaluate the algorithm on two datasets: (i) the US Pecan Street dataset is adapted so that net energy meter readings are censored; and (ii) a constructed dataset, combining smart meter readings from UK households and solar energy generation from locations across the UK. Our results show comparable accuracy at inferring PV capacity compared to existing approaches, which cannot deal with censored readings which represent over 50% of PV panel installations in the UK.

Introduction

Solar energy (PV) generation in the UK has increased by a factor of 130 between December 2010 and December 2019, with small installations (under 10 kW) increasing in number by 4000%,1 making up a growing proportion of the grids electricity supply. Thus, a grid operator who must balance the supply and demand of electricity in real time in order to keep the grid at a stable frequency [1], [2], [3], have an increasing challenge to estimate the quantity of energy provided by these PV installations. Ideally, the location of these panels, and their capacity would be reported to the grid operator, and their generation would be measured and reported in real-time through smart meters. Unfortunately, in the UK, no reporting scheme currently operates, and the deployed base of smart meters (and those scheduled to be deployed to all UK homes in the future) do not separate local generation and consumption. Rather, they simply record the net energy taken from the grid. More significantly, they also fail to report net export to the grid, and simply report 0 kWh of consumption over each half hour, during these periods. In statistical terms, these observations are said to be censored. Thus, there is a need to develop approaches to identify homes with PV panels installed, and to estimate their real-time net contribution to overall grid generation, by disaggregating domestic energy generation using existing censored smart meter data.

In the setting where smart meters are uncensored, existing approaches often combine known physical models of PV panels with an analytical approach to identify the maximum generation of a PV panel to disaggregate the energy generated [4], [5]. Others take a supervised learning approach to the problem, using radial basis functions and wavelet kernel support vector machine that map weather metrics to a solar output [6]. Many approaches do not focus on disaggregation, but only on inferring PV generation using its relationship with weather [7], [8]. Alternative approaches have implemented solar disaggregation at a feeder or small-region level, which does not correspond directly to our work [9], [10], [11].

The above approaches do not explicitly account for censored smart meter readings. This means that they would not work in over 50% of real world settings where censoring occurs due to incorrect installations.2 As the number of PV panel installations grow, this creates a significant issue in balancing the grid. A new approach is required to deal with censored smart meter readings, otherwise PV panels will go undetected and their generation unaccounted for. Without a non-intrusive approach, costly interventions need to be taken, such as re-fitting smart meters or installing new hardware, to measure PV generation separately.

In this paper, an approach to disaggregate PV generation and energy consumption from censored smart meter readings is presented. The approach infers the maximum power input into the house that can be expected from the specific system of PV panels, this means the inverter efficiency is also accounted for. Then using a solar proxy, which is defined as the known solar generation from a local house, the PV generation can be inferred. Using a solar proxy to infer solar generation of a different house has been successfully demonstrated in other bodies of research [5]. To find the PV capacity, the most likely joint probability distribution of the PV capacity and the energy consumption for each time period across a year is found. Fig. 1 provides a visualisation of this process.

Our novel approach allows solar disaggregation to be performed in real-world situations where smart meter readings are censored. Furthermore, in line with current smart meter standards, only half-hourly readings are required for the algorithm to work. The algorithm can classify the presence of PV panels on small buildings and infer the solar generation at each time of the day. The algorithm is evaluated on the widely-used US Pecan Street dataset, and to demonstrate the viability of the approach in the UK setting a dataset is constructed combining a dataset of UK smart-meter readings and a dataset of energy generation recordings from small PV systems. Combining the two datasets is required as there are multiple publicly available datasets of small houses with labelled energy consumption, however there are currently no large scale datasets with local energy generation also recorded. Our experiments demonstrate the algorithms success at detecting houses with solar panels from censored smart meter readings and inferring their PV capacity and PV generation. This is the first approach to explicitly deal with censored smart meters and the results are comparable to other approaches, which do not address the issue of censored readings and assume net-metering. This approach will allow inference of PV capacity and generation from all energy generating customers in the UK, as the current methods rely on reporting to the feed-in tariff, inference from quarterly energy generation and national grid models which do not explicitly account for generation by individual homes.

The rest of the paper is structured as follows. Section 3 outlines the model to infer PV capacity from censored smart meter readings. Section 4 illustrates our implementation of an algorithm to infer PV capacity, which is extended in Section 5 to not require solar irradiance. Finally, Section 6 introduces the datasets, Section 7 outlines the experiments and the results and Section 8 summarises our contribution to the field and outlines future work.

Section snippets

Related work

Recent work proposes a method for disaggregating solar PV generation behind-the-meter for individual buildings using historical advanced metering infrastructure, which records the net energy consumption for a house, feeder level net energy consumption, and a solar proxy (similar to what is used in this paper), however the approach still requires net meter data, which means that in its current form it would not work when the smart meter data is censored [5]. Other work has looked at identifying

Censored solar disaggregation model

We consider a dataset of smart-meter readings, rhtd [kWh], where h[1H],t[1T],d[1N], such that H denotes the number of households, T is the number of time steps the data readings are separated into, and N is the number of days of data. A smart meter records only the energy supplied to the house from the grid. This means if behind-the-meter energy generation, ghtd [kWh], is larger than the energy consumption, xhtd [kWh], there is a censored reading, rhtd=max(0,xhtd-ghtd).

PV capacity inference with solar proxy

We present a novel algorithm to infer the PV capacity, ch, using a smart meter with censored readings, rhtd. For a given house, our algorithm aims to find the maximum power input into the home that can be expected from the installed PV panels. Notice that there is a counter-intuitive nature about the algorithm, as the PV capacity, which is our main point of interest, is found in the process of estimating the most likely distributions for energy consumption, Xhtd.

Our algorithm finds the most

PV capacity inference with only clearsky solar irradiance

We extend the above approach to the inference of PV capacity only using clearsky irradiance (as opposed to a solar proxy for each panel), under the assumption we have a group of houses in close proximity (e.g., sharing the same postcode). The clearsky (solar) irradiance, std, is the solar irradiance that would be incident on the ground if there was no atmospheric or weather interference. It is calculated using existing physical models, as it only depends on the location on Earth relative to the

Datasets

To evaluate the approach a subset of the US Pecan Street dataset and a constructed UK dataset are used.

The Pecan Street dataset provides energy data for houses with PV panels [28]. We have cleaned the data by creating censored smart meter readings for each house, as obtained from their energy consumption and generation. From this dataset, 30 houses have been identified to have solar panels and selected and the PV Capacity is in the range 2.5 kW to 10.2 kW for the Pecan Street dataset.

Alongside

Empirical evaluation

To evaluate the performance of the algorithm four metrics are used. The root mean square error (RMSE) measures the accuracy of the inferred values by taking its distance from the true value, regardless of the true PV capacity. RMSE is a useful metric in a situation when we are not interested in the contributions of individual houses and instead focus on the total prediction of PV generation,RMSE=h=1H(ĉh-ch)2H.

Note that, ĉh denotes the inferred PV capacity and ch is the real PV capacity. To

Conclusions and future work

In this paper, we have proposed the first approach to dissagregate solar (PV) generation from energy consumption given censored smart meter readings and to infer the PV capacity. To evaluate our approach, we have used an appropriate subset of the Pecan Street dataset and a custom dataset that we have created by combining data supplied from the Sheffield Solar microgen dataset and the smart meter readings from London households. We have shown that if the solar irradiance is known as a proxy from

CRediT authorship contribution statement

Joe Brown: Conceptualization, Methodology, Software, Validation, Investigation, Data curation, Writing - original draft. Alessandro Abate: Conceptualization, Methodology, Investigation, Writing - review & editing, Supervision. Alex Rogers: Conceptualization, Methodology, Investigation, Writing - review & editing, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (31)

  • M. Tabone, S. Kiliccote, E. C. Kara, Disaggregating solar generation behind individual meters in real time, in:...
  • R. Mohan, T. Cheng, A. Gupta, V. Garud, Y. He, Solar energy disaggregation using whole-house consumption signals, in:...
  • N. Sharma et al.

    Predicting solar generation from weather forecasts using machine learning

  • P. Chakraborty et al.

    Fine-grained photovoltaic output prediction using a Bayesian ensemble

  • E. Vrettos et al.

    Estimating PV power from aggregate power measurements within the distribution grid

    Journal of Renewable and Sustainable Energy

    (2019)
  • Cited by (17)

    • A deep learning approach for detecting distributed generation in residential customers

      2023, Sustainable Energy, Grids and Networks
      Citation Excerpt :

      presented a classification model to identify the DG capacity at the low voltage level and demonstrated the impact of installing DGs exceeding the capacity of small grids, which potentially can be caused if the customers does not inform DNO of the intention to install a DG source. Another area of research focused on disaggregating the DG generation and battery discharge profile from the net energy consumption measured by the utility smart meter installed at the property using smart meter data [22]. The aim of disaggregating the generation profile form the net load is to increase grid edge-observability [23] and determine the generation profile of DG so DNO can use that information during network planning.

    • Data efficient energy disaggregation with behind-the-meter energy resources

      2022, Sustainable Energy, Grids and Networks
      Citation Excerpt :

      A shortcoming of this method is that it requires three years of historical data, which may not always be available for each customer. In [20] a novel approach is proposed to perform solar disaggregation in real-world situations where the amount of solar power exported to the grid is not reported by the smart meter. To our knowledge, disaggregating solar power from net meter data in the presence of other BTM components, such as battery packs, has been studied in two recent papers only.

    • An ontology based energy management for smart home

      2021, Sustainable Computing: Informatics and Systems
      Citation Excerpt :

      It is currently structured around six major themes linked to each other (Fig. 1): (01) Smart Meters: it concerns the first step towards an intelligent network, the deployment of advanced meters associated with a first two-way telecommunications network is the cornerstone of future electricity networks. However, this first phase usually leads to low returns on investment, based on operational savings and reduction of non-technical losses [2]. ( 02) Smart Grid: it is a communicating electricity distribution network that integrates all of the new information technologies into the management of networks and connected assets.

    View all citing articles on Scopus
    View full text