Disaggregation of household solar energy generation using censored smart meter data
Introduction
Solar energy (PV) generation in the UK has increased by a factor of 130 between December 2010 and December 2019, with small installations (under 10 kW) increasing in number by ,1 making up a growing proportion of the grids electricity supply. Thus, a grid operator who must balance the supply and demand of electricity in real time in order to keep the grid at a stable frequency [1], [2], [3], have an increasing challenge to estimate the quantity of energy provided by these PV installations. Ideally, the location of these panels, and their capacity would be reported to the grid operator, and their generation would be measured and reported in real-time through smart meters. Unfortunately, in the UK, no reporting scheme currently operates, and the deployed base of smart meters (and those scheduled to be deployed to all UK homes in the future) do not separate local generation and consumption. Rather, they simply record the net energy taken from the grid. More significantly, they also fail to report net export to the grid, and simply report 0 kWh of consumption over each half hour, during these periods. In statistical terms, these observations are said to be censored. Thus, there is a need to develop approaches to identify homes with PV panels installed, and to estimate their real-time net contribution to overall grid generation, by disaggregating domestic energy generation using existing censored smart meter data.
In the setting where smart meters are uncensored, existing approaches often combine known physical models of PV panels with an analytical approach to identify the maximum generation of a PV panel to disaggregate the energy generated [4], [5]. Others take a supervised learning approach to the problem, using radial basis functions and wavelet kernel support vector machine that map weather metrics to a solar output [6]. Many approaches do not focus on disaggregation, but only on inferring PV generation using its relationship with weather [7], [8]. Alternative approaches have implemented solar disaggregation at a feeder or small-region level, which does not correspond directly to our work [9], [10], [11].
The above approaches do not explicitly account for censored smart meter readings. This means that they would not work in over 50% of real world settings where censoring occurs due to incorrect installations.2 As the number of PV panel installations grow, this creates a significant issue in balancing the grid. A new approach is required to deal with censored smart meter readings, otherwise PV panels will go undetected and their generation unaccounted for. Without a non-intrusive approach, costly interventions need to be taken, such as re-fitting smart meters or installing new hardware, to measure PV generation separately.
In this paper, an approach to disaggregate PV generation and energy consumption from censored smart meter readings is presented. The approach infers the maximum power input into the house that can be expected from the specific system of PV panels, this means the inverter efficiency is also accounted for. Then using a solar proxy, which is defined as the known solar generation from a local house, the PV generation can be inferred. Using a solar proxy to infer solar generation of a different house has been successfully demonstrated in other bodies of research [5]. To find the PV capacity, the most likely joint probability distribution of the PV capacity and the energy consumption for each time period across a year is found. Fig. 1 provides a visualisation of this process.
Our novel approach allows solar disaggregation to be performed in real-world situations where smart meter readings are censored. Furthermore, in line with current smart meter standards, only half-hourly readings are required for the algorithm to work. The algorithm can classify the presence of PV panels on small buildings and infer the solar generation at each time of the day. The algorithm is evaluated on the widely-used US Pecan Street dataset, and to demonstrate the viability of the approach in the UK setting a dataset is constructed combining a dataset of UK smart-meter readings and a dataset of energy generation recordings from small PV systems. Combining the two datasets is required as there are multiple publicly available datasets of small houses with labelled energy consumption, however there are currently no large scale datasets with local energy generation also recorded. Our experiments demonstrate the algorithms success at detecting houses with solar panels from censored smart meter readings and inferring their PV capacity and PV generation. This is the first approach to explicitly deal with censored smart meters and the results are comparable to other approaches, which do not address the issue of censored readings and assume net-metering. This approach will allow inference of PV capacity and generation from all energy generating customers in the UK, as the current methods rely on reporting to the feed-in tariff, inference from quarterly energy generation and national grid models which do not explicitly account for generation by individual homes.
The rest of the paper is structured as follows. Section 3 outlines the model to infer PV capacity from censored smart meter readings. Section 4 illustrates our implementation of an algorithm to infer PV capacity, which is extended in Section 5 to not require solar irradiance. Finally, Section 6 introduces the datasets, Section 7 outlines the experiments and the results and Section 8 summarises our contribution to the field and outlines future work.
Section snippets
Related work
Recent work proposes a method for disaggregating solar PV generation behind-the-meter for individual buildings using historical advanced metering infrastructure, which records the net energy consumption for a house, feeder level net energy consumption, and a solar proxy (similar to what is used in this paper), however the approach still requires net meter data, which means that in its current form it would not work when the smart meter data is censored [5]. Other work has looked at identifying
Censored solar disaggregation model
We consider a dataset of smart-meter readings, [kWh], where , such that H denotes the number of households, T is the number of time steps the data readings are separated into, and N is the number of days of data. A smart meter records only the energy supplied to the house from the grid. This means if behind-the-meter energy generation, [kWh], is larger than the energy consumption, [kWh], there is a censored reading, .
PV capacity inference with solar proxy
We present a novel algorithm to infer the PV capacity, , using a smart meter with censored readings, . For a given house, our algorithm aims to find the maximum power input into the home that can be expected from the installed PV panels. Notice that there is a counter-intuitive nature about the algorithm, as the PV capacity, which is our main point of interest, is found in the process of estimating the most likely distributions for energy consumption, .
Our algorithm finds the most
PV capacity inference with only clearsky solar irradiance
We extend the above approach to the inference of PV capacity only using clearsky irradiance (as opposed to a solar proxy for each panel), under the assumption we have a group of houses in close proximity (e.g., sharing the same postcode). The clearsky (solar) irradiance, , is the solar irradiance that would be incident on the ground if there was no atmospheric or weather interference. It is calculated using existing physical models, as it only depends on the location on Earth relative to the
Datasets
To evaluate the approach a subset of the US Pecan Street dataset and a constructed UK dataset are used.
The Pecan Street dataset provides energy data for houses with PV panels [28]. We have cleaned the data by creating censored smart meter readings for each house, as obtained from their energy consumption and generation. From this dataset, 30 houses have been identified to have solar panels and selected and the PV Capacity is in the range 2.5 kW to 10.2 kW for the Pecan Street dataset.
Alongside
Empirical evaluation
To evaluate the performance of the algorithm four metrics are used. The root mean square error (RMSE) measures the accuracy of the inferred values by taking its distance from the true value, regardless of the true PV capacity. RMSE is a useful metric in a situation when we are not interested in the contributions of individual houses and instead focus on the total prediction of PV generation,
Note that, denotes the inferred PV capacity and is the real PV capacity. To
Conclusions and future work
In this paper, we have proposed the first approach to dissagregate solar (PV) generation from energy consumption given censored smart meter readings and to infer the PV capacity. To evaluate our approach, we have used an appropriate subset of the Pecan Street dataset and a custom dataset that we have created by combining data supplied from the Sheffield Solar microgen dataset and the smart meter readings from London households. We have shown that if the solar irradiance is known as a proxy from
CRediT authorship contribution statement
Joe Brown: Conceptualization, Methodology, Software, Validation, Investigation, Data curation, Writing - original draft. Alessandro Abate: Conceptualization, Methodology, Investigation, Writing - review & editing, Supervision. Alex Rogers: Conceptualization, Methodology, Investigation, Writing - review & editing, Supervision.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (31)
- et al.
Disaggregating solar generation from feeder-level measurements
Sustainable Energy, Grids and Networks
(2018) - et al.
Artificial neural network estimation of global solar radiation using air temperature and relative humidity
Energy Policy
(2008) - et al.
A 24-h forecast of solar irradiance using artificial neural network: Application for performance prediction of a grid-connected PV plant at Trieste, Italy
Solar Energy
(2010) - et al.
Predicting solar energy generation through artificial neural networks using weather forecasts for microgrid control
Renewable Energy
(2018) - et al.
DeepSolar: A machine learning framework to efficiently construct a solar deployment database in the United States
Joule
(2018) - et al.
Improved satellite-derived PV power nowcasting using real-time power data from reference PV systems
Solar Energy
(2018) - A. Peruffo, E. Guiu, P. Panciatici, A. Abate, Aggregated Markov models of a heterogeneous population of photovoltaic...
- A. Peruffo, E. Guiu, P. Panciatici, A. Abate, Safety guarantees for the electricity grid with significant renewables...
- et al.
Non-Gaussian power grid frequency fluctuations characterized by Lévy-stable laws and superstatistics
Nature Energy
(2018) - D. Chen, D. Irwin, Sundance: Black-box behind-the-meter solar disaggregation, in: Proceedings of the Eighth...
Predicting solar generation from weather forecasts using machine learning
Fine-grained photovoltaic output prediction using a Bayesian ensemble
Estimating PV power from aggregate power measurements within the distribution grid
Journal of Renewable and Sustainable Energy
Cited by (17)
A deep learning approach for detecting distributed generation in residential customers
2023, Sustainable Energy, Grids and NetworksCitation Excerpt :presented a classification model to identify the DG capacity at the low voltage level and demonstrated the impact of installing DGs exceeding the capacity of small grids, which potentially can be caused if the customers does not inform DNO of the intention to install a DG source. Another area of research focused on disaggregating the DG generation and battery discharge profile from the net energy consumption measured by the utility smart meter installed at the property using smart meter data [22]. The aim of disaggregating the generation profile form the net load is to increase grid edge-observability [23] and determine the generation profile of DG so DNO can use that information during network planning.
Data efficient energy disaggregation with behind-the-meter energy resources
2022, Sustainable Energy, Grids and NetworksCitation Excerpt :A shortcoming of this method is that it requires three years of historical data, which may not always be available for each customer. In [20] a novel approach is proposed to perform solar disaggregation in real-world situations where the amount of solar power exported to the grid is not reported by the smart meter. To our knowledge, disaggregating solar power from net meter data in the presence of other BTM components, such as battery packs, has been studied in two recent papers only.
An ontology based energy management for smart home
2021, Sustainable Computing: Informatics and SystemsCitation Excerpt :It is currently structured around six major themes linked to each other (Fig. 1): (01) Smart Meters: it concerns the first step towards an intelligent network, the deployment of advanced meters associated with a first two-way telecommunications network is the cornerstone of future electricity networks. However, this first phase usually leads to low returns on investment, based on operational savings and reduction of non-technical losses [2]. ( 02) Smart Grid: it is a communicating electricity distribution network that integrates all of the new information technologies into the management of networks and connected assets.
Behind-the-Meter Solar Generation Disaggregation at Varying Aggregation Levels Using Consumer Mixture Models
2023, IEEE Transactions on Sustainable Computing